[Update 6-10-10: added new spectrum “False Interface Potentia” based on comment from David Tao. Added “End User Perception” based on comment from Erik Pupo. Removed controversial portions. full explanation at the end]
Currently, the NHIN Direct project is having a debate around what protocol to use.
NHIN Direct is supposed to enable something that “feels like email” to clinical and patient users, but in fact allows for the secure transfer of PHI over the Internet between clinical and patient parties. Essentially, a secure network for PHI messages.
But how to do that? We have to choose a technology suite in order to make that happen and it is not clear which protocol we should be choosing.
Yesterday, I listened to a call where the various implementation teams made the “case” for their particular protocol.
First let me say that I have great sympathy for these implementers, all of them have put in lots of work and thought into their particular implementation approach and all of them have a made a good case. However, Open Source is a meritocracy. We have to decide on one “best” approach for NHIN Direct and not everyone will get what they want. In competitions like this, we might have at least one very happy implementation group, but we are sure to have several approaches that are abandoned. Those people who have worked hard on implementations that will not be used are in a difficult social situation. They will inevitably feel that the group has chosen poorly, and that their work was overlooked. But at the same time the group will be asking for their help in unifying the project behind a single approach (if not a single protocol). As a developer, being in that situation and being rejected really hurts. I have been in the Open Source community long enough to have been on both the “winning” side and the “loosing side” of this several times. It is better to win.
It is critical that the larger group show its appreciation for the work of everyone, even as it rejects most of the approaches. Contributing to an Open Source project like this is expensive emotionally and financially, and just because some implementations will “loose” does not mean that there were valueless. In fact, as we abandon implementation approaches, the project members will do well to recognize the abandoned approaches as “templates for improvement” and/or “concessions to expediency”.”Right” is not really a position anyone gets to take.
To all of the implementation groups: Thank you. Your work, whether embraced or not, is impressive. As you can see later, every approach has merits not found in other approaches. This will be a tough decision.
XMPP
XMPP is basically a chat protocol.
Advantages: Chat has some fundamental advantages over email (SMTP). When a user is offline in a chat context, you can still send them messages, so chat “falls back to” a stored message system like email. But when a user is online, that information can be selectively broadcast to other users, who can then send messages knowing that the person is right there on the keyboard. XMPP was designed from the ground up to handle real-time messaging, and so it might be more appropriate if you consider situations, (i.e. surgery) where you need to transfer messages and attachments and get information back in near-real-time. In fact you might consider an XMPP system as “more compatible” with a tele-medicine approach for this reason. There are also some interesting “subscribe/broadcast” models that XMPP has built into the protocol. There are lots of solid implementations of XMPP currently in the market and the mature ones should be sufficiently configurable to handle the complex security configuration that NHIN Direct will require. So there is a big pile of existing software available and most of it is Open Source.
Disadvantages:
Not many people understand XMPP and it is not as widely used as email. It is not out-of-the-box compatible with NHIN Exchange.
Score:
- New Implementation Coding Required: almost none: 10
- Open Source implementations available: Lots: 10
- Existing experts: some but not many: 5
- Compatibility with NHIN Exchange: must be built from scratch: 1
- Future Flexibility: the protocol is the protocol, new stuff has to go on top: 3
- Mature Standard: Yes huge chat networks already running: 10
- End User Interface Familiarity: Lots of people use chat: 7
- False Interface Potential: Might make users think “this is just chat”: 2
- End User Perception: You are using chat for secure PHI exchange? Wrong perception of course, but still not good: 2
- Cool feature bonus: +5 for enabling real-time chat
Total: 55
REST
REST is web-based software design philosophy.
Advantages: REST lets you build really complex things really quickly. As proof, the REST implementation basically already has a working prototype. REST is extremely powerful and will allow rapid iteration of future versions. As we have future requirements we can build them easily.
Disadvantages: You are building a messaging infrastructure from scratch. You will have to re-invent strategies for reliability, message queuing, and countless other things that you do not realize that SMTP or XMPP are doing for you. It is not out-of-the-box compatible with NHIN Exchange.
Score:
- New Implementation Coding Required: almost everything: 1
- Open Source implementations available: Lots of good REST libraries: 10
- Existing experts: no experts in what does not yet exist: 1
- Compatibility with NHIN Exchange: must be built from scratch: 1
- Future Flexibility: Allows for very rapid iteration: 10
- Mature Standard: While REST is mature, the application design on top is not all, it will be totally new: 1
- End User Interface Familiarity: Interfaces will likely mimic email but they will be untested and totally new: 3
- False Interface Potential: Because the interfaces will be new, notions of PHI transfer can be built in: 5
- End User Perception: Essentially none, its reputation will stand on itself, and because we will do a good job, this is not a disadvantage: 10
Total: 42
IHE
IHE is a set of profiles for exchanging health information. It is designed from the ground up to handle the complexities of Health Information exchange. The relevant profiles are XDR and XDM
Advantages: Using the IHE messaging standard means that NHIN Direct users would be participating in the more advanced NHIN Exchange, they just would not be using all of its power. Merely push messaging has some fundamental limitations and fully participating in the larger NHIN Exchange will allow providers to embrace larger portions of the more advanced health information network when those features are needed. Both Mirth, Open Health Tools, and MOSS provide excellent Open Source implementations of these protocols. IHE provides a formalized mechanism, in an international environment for improving and updating the standard. The largest and biggest EHR vendors already have support for IHE.
Disadvantages: IHE is a moving target. The NHIN CONNECT project is already handling the considerable complexities of mapping to those standards and profiles even as they are finalized. There is already tension there between what CONNECT actually does and what the standards say should be done. The whole point of NHIN Direct is to provide a much simpler model of Health Data exchange than is available with NHIN Exchange. While the big EHR vendors typically have IHE compliant implementations, it is only in the last few years that they have been successfully using them to connect to each other at the Connectathon. Given that, it is something of a stretch to hold out IHE as a fully mature standard. [Update 6-10-10 section removed]
Discussion: The NHIN Exchange will have direct messaging through IHE protocols. Unless you want an AOL/Compuserve (or Twitter/identi.ca/GoogleBuzz) style messaging split between the NHIN Direct/Exchange networks, there will have to be a bride between the messaging protocols. Given that bridge, the real question becomes “Is there any reason not to use the IHE messaging as a backbone and some other protocol at the edges”. (which is essentially what the IHE team is proposing) From what I could tell the critical issue is how much data is “required” to be included as a minimum in any IHE message. Apparently, it will break down if there is not a patient id of some kind in the message (I do not mean a social security number, but something assigned locally by a computer program). This excludes people who do not have a list of patient ids to send from using NHIN Direct to send messages (they can still receive). This essentially makes EHR-integration a requirement; excluding those without a computer system at all, or who are using an antiquated practice management system whose patient id’s would be difficult to access. So the real question is do we use IHE as the core technology or merely bridge to it?
- New Implementation Coding Required: Everyone except the large vendors will have to code or adopt Open Source libraries: 5
- Open Source implementations available: Several very promising projects already in live use: 10
- Existing experts: IHE is not a well-know protocol: 3
- Compatibility with NHIN Exchange: it is identical: 10
- Future Flexibility: There is a formal process in place, but it is hardly fast it has been going for decades: 4
- Mature Standard: Still in flux: 3
- End User Interface Familiarity: Very few clinicians use IHE now, so the interfaces are immature: 3
- False Interface Potential: Although relatively young, the existing IHE messaging clients are designed to handle PHI notions: 7
- End User Perception: IHE is totally “legit” they might not be happy with it, but they will respect it: 10
Total: 50
SMTP
SMTP is the protocol behind email.
Advantages: There is a huge expanse of people who are familiar with this protocol stack. It has already been extensively used for this purpose and in this way (with S/MIME). It is the simplest known-good solution. It is the protocol to beat. Plenty of Open Source implementations. Super mature standard that has withstood decades of security scrutiny.
Disadvantages: Email is great and email sucks. It is does not have some features of XMPP, and IHE, and it is such a broad protocol that there are many options that make predictions about how it will be used difficult. It is not out-of-the-box compatible with NHIN Exchange.
- New Implementation Coding Required: Everything is done: 10
- Open Source implementations available: old and mature projects to most anything: 10
- Existing experts: You can’t swing a cat without hitting an SMTP expert: 10
- Compatibility with NHIN Exchange: must be built from scratch: 1
- Future Flexibility: the protocol is the protocol, new stuff has to go on top: 3
- Mature Standard: Is there a more mature standard anywhere?: 10
- End User Interface Familiarity: Many people familiar with many different mature interfaces: 10
- False Interface Potential: Users may make the mistake of not seeing that this is -not- just email while using traditional clients: 1
- End User Perception: they might be a little uncomfortable with “email” but if you say “secure email” that might work: 8
Total: 63
Epilogue
It should be noted that my scoring system is somewhat arbitrary. I think my assessments are fair, but you could easily get different “totals” be choosing different items to include in the comparison. What significant categories of comparison does this ignore? Suggest something in the comments and if it makes sense I will add it.
Update 6-10-10:
So several people have commented on this blog post, and I would encourage you to read all of the comments. They are uniformly well reasoned, even when I disagree with them. I will be updating the scoring with two new categories, and I wanted to explain them.
The first is my response to a comment from David Tao regarding the User Interfaces. My previous scoring is biased in favor of older mature user interfaces. David’s comments made me realize that if a mature interface gave the wrong idea to a clinician or a patient about the working of the system, that even though it was a more stable interface, it could detract from the overall impact of the UI. There is an argument to be make that a “new messaging paradigm (i.e. secure PHI) should be paired with a new interface that exposes the important differences involved”. However I cannot give IHE a “10” in that new category because the IHE messaging systems, in the context of a national exchange are still not mature, but they might still get some benefit from being tailored to PHI exchange. I think both the REST model and the IHE model, which favour newer interface designs should get points here, and IHE should beat REST because they have been testing their new designed for some time. So this is what I mean by “false interface potential” category.
The second is my response to a comment from Erik Pupo. Erik believes that a secure messaging infrastructure based on a chat protocol (XMPP) will not be taken as seriously as other protocols. This is somewhat sad, since I think this is a incorrect public perception about the value of a reliable protocol, but it is unwise for us to be trying to change that unfair public perception even as we try to encourage adoption. Its like saying “here use this tool that you do not take seriously..” not a cogent message for us. So I have added the category of ‘End User Perception” and dinged XMPP for it because of this. I think email, as a known protocol, will also have a slight disadvantage here.
Lastly, I have removed my original criticism of EHRA. Of course, I am still right about the issue 😉 but the project managers decided that the discussion was unproductive. Given that, it seemed wise to remove that content from this page so that it could remain as a “somewhat not totally subjective” resource. This post is intended to further the legitimate and focused technical debate, and not have us going around in circles about more fundamental issues that we are unlikely to agree on.
-FT
Nice analysis; took much longer to create than read.
Is it the case that XMPP requires participants to be on-line at the time information is exchanged (a la chat?) If so I am wondering if creating a scale that adds up points from several azes might be misleading. It seems to me that a protocol where people have to be on-line is a non-starter.
Of course “anything” can be expanded to do “anything.” But if we are trying to measure the stretch required in new code and new concepts to get to a concrete result, this seems to require a lot of stretch.
@Wes Rishel
XMPP does not require you to be online… when you are not online it works very much like email… your messages wait for you. But when you are online… and for only those other users you want to share your online status with… it works like chat.
Fred, I think you will probably get some major disagreement from vendors and other stakeholders on several of your scores for IHE. For example, there are plenty of Java implementations of IHE components, fully open source, which could be used for NHIN Direct. IHE is considered fairly mature, at least by vendors, and profile support is built into many major health IT products (including open source). IHE profiles also form a large core of the work done by HITSP and there are a core set of experts to draw from (although not as numerous as SMTP or XMPP).
I would suggest one additional scoring criteria (although it might be hard to capture it). A criteria such as “Level of Adoption That Could be Expected”. I can imagine that if NHIN Direct adopted XMPP you would get a rather sharp reaction from various healthcare industry stakeholders (its innovative but it brings up the question of who would go along with it).
Overall though this is a good analysis.
Fred, re the End User UI consideration, I don’t get why that’s there or how it can be scored. NHIN Direct and the Concrete Implementations have not proposed a UI, and that’s up to the “applications” (modules, whatever) that build on top of the protocol, the protocol doesn’t constrain the UI.
End users don’t use any “protocol” directly. How many e-mail and web users could describe SMTP, POP3, REST, SOAP, etc.? No one from an end-user perspective “uses” IHE either. They get data through a web service and then they display it to the user however they built their app.
EHRs or e-mail clients or web browsers would be the UI that people see. As shown in the IHE and SMTP demos yesterday, the user experience was using an inbox in Cerner or AllScripts or Epic or other EHR, or an e-mail client like Outlook or Thunderbird, or PHR like HealthVault or Lucy. Healthvault’s UI doesn’t show SMTP directly either,it’s just the web UI that Microsoft designed. I even asked Sean yesterday about the UI, and he said that the user experience was pretty close among the four options (except for some subtle differences, though he pointed out other advantages that he saw in his SMTP approach).
So I suggest that the End User UI criterion be removed, as it’s a “wash.”
Thanks,
David
Hi Fred,
Your analysis is good, and I agree that the value if the IHE approach hinges largely on the value of the IHE specified metadata. And, it’s true that the minimum conformance bar is higher than the household email client use case calls for.
However, I was able to use the MIME message-id in the SMTP->XDR step up converter as a kind of surrogate for patient id in this case, but its admittedly not a perfect ‘semantic’ fit. There are also problems with the patient id centric approach in IHE that causes additional complexity. For instance, IHE requires a single ‘patient’ per XDR transaction. So, unless the user simply attaches one CDA attachment without typing any text (as in our demo 8), the step up conversion should generate separate XDR transaction for each part when those parts do not contain structured attachments that bear same patient id. In this case just copying message id won’t work and a random UUID would be better. So while we can use converters to keep the simple easy on the NHIN Source, admittedly the backbone protocol is a bit more complex.
I thought the point someone raised about the IHE approach being a matter of placing the NHIN Exchange bridge out at the edges made sense. This same ‘step up’ approach could sit at the NHIN Exchange GW and need not be the NHIN direct backbone. Likewise the ‘step up’ could even sit in front of the Destination on the NHIN Destination HISP in order to support an XDR push into a system that has that sort of interface. So it is perhaps, in theory, a matter of ‘bridge placement’.
Regarding End User UI and the patient ID issue:
Fred, would you agree that IF/WHEN it’s possible to file accept incoming messages (SMTP, IHE, or whatever) under the correct patient automatically without requiring human intervention, that would be an advantage for the end user? I grant that’s not always possible, but isn’t it worth trying for? If a sending and receiving system can accurately resolve who the patient is (which is what EMPI patient matching and the IHE PIX/PDQ profiles address), then that reduces work for the end user. Given the busy demands on clinicians and their staff, that should count for something. So the “patient ID” metadata issue, which you count as a hurdle actually should contribute positive points to the “End User UI” criterion if you keep that criterion. The ideal approach, IMHO, would be to support patient ID in exchanges for maximal integration benefit, but to not make it mandatory for all messages. Then the “good” isn’t the enemy of the perfect, nor is the perfect the enemy of the good. Extraction of the ID by parsing various kinds of “self-describing” attachments, while it can be done, seems more complicated than just reading a predictable metadata field. I think Arien summed it up well, in response to my question, on the NHIN Direct wiki http://nhindirect.org/message/view/SMTP+Implementation/23878807
@David Tao
I certainly agree that having a system which supports advanced metadata has substantial advantages, the “lure of IHE”. I think that is definitely the right direction. But requiring a patient id to send messages excludes lots of providers. I would support any kind of hybrid proposal that would give the benefits of having a patient id. But not exclude users without EHR systems.
@Matt Potter,
I was the one that made that comment about “where we bridge”. You are the first person to directly address some of the basic questions that I had about IHE. Thanks so much for doing that.
I agree that this boils down to “bridge placement”. I like the term and will try to use it from now on.
-FT
@David Tao good points, but I disagree on some issues.
There is something to be said both for and against mature interfaces that a user is already familiar with. I remember when gmail started presenting emails a “threads” it was a tremendous fundamental improvement on the email UI experience and that idea has bubbled to most other interfaces. It was such a small powerful change and we forget how much of a difference that makes. New interfaces are hard to get “right”, it takes years sometimes for people to even recognize what “right’ is. Even then they do not agree.
From a user simplicity standpoint the ability to use existing mail clients is the simplest user training approach. However, I think there is a good argument to be made that we should abandon simplicity here. Perhaps by having simplistic and well-understood interfaces we are masking important differences between how email works and how secure PHI messaging should work. Perhaps we should be thinking “new interface new implications”
I agree that not having scores to balance this “hidden complexity” or “bad UI analogy” is problematic. But it is also unclear to me that the spectrum of IHE interfaces gets that issue “right”. I am going to add a score for “false simplicity” to try and balance this out. Obviously IHE will fair better than SMTP/XMPP here but I cannot just give it a 10. IHE messaging interfaces are still less mature than email interfaces and cannot uniformly assumed to be free of bad design…
Obviously the “totals” are pretty irrelevant since they are just the sum of my personal chosen priorities, but I hope that by adding this new spectrum at all, I acknowledge that you have a very good point here.
-FT
@Erik Pupo
Erik,
I think this is an important issue, and you are right, I had been ignoring it. I have been very critical of the notion that we should accept technical solutions because a give technology provider favors it, but that is not the same thing as how the clinical user population would accept the solution. You are right, despite the fact that XMPP offers extra cool features (that I give it a bonus for) it might be regarded as “you are using chat for secure message exchange?” That is a problem that other protocols will face less. I will add a new score to account for this, and I think that sadly XMPP will lose the most on this… since you can always say “secure email” and because people already respect email as a business solution people will take you seriously. I think this is especially tragic, since I can remember a time when email was regarded as a “toy system” and I remember people looking at me like i was crazy when I said “can you email that to me?”
I think the chat protocol will get a bad rap, but we cannot pretend that it wont, just because all of us know that it is a solid protocol.
-FT
I think the meta-data issue/discussion is off slightly. PatientID is not meta-data from a transport perspective; it is content. I have not problem with defining a content type for NHIN meta-data (does not even have to be NHIN-Direct specific) that is optional. Requiring it for each transaction will limit the use cases. For example, anonymous HIV testing, sending multiple patient data in a single message, etc. IMHO, we must NOT fall into the perpetual HIT black hole of mixing content and transport. What if we mixed HTML headers with HTTP? so much for XML over HTTP.
Also, I see not reason why REST, SMTP, and XMPP could not use the same NHIN meta-data content type as proposed by IHE but as optional. Therefore, you can provide the destination functionality demonstrated by IHE team in the videos in the other concrete implementations.
-SW
I’ve been waiting for answers to some questions to appear in the NHIN discussion.
1) In addition to small-office docs, NHIN Direct end-users will eventually include retail pharmacies, standalone clinical lab specimen collection offices, standalone imaging services, nurse practitioners, dentists, psychological counselors, chiropractic and other non-MD practitioners, visiting nurses, first responders, social various safety-net organizations, and patients plus their designated or legal custodians. Can we have a thought experiment on how to smoothly extend NHIN Direct to these users? How relevant are the protocol discussions to that end-user community? What should those users presume before accessing NHIN Direct? How simple is it in comparison to, let’s say, a cell phone?
2) At some point in most prior EHR discussions, questions get asked about end-to-end privacy and security. That applies to patients and the end-users’ privacy. For now, assume that privacy and security policies are the current HIPAA rules. Is there a privacy and security risk analysis model for NHIN Direct and NHIN Exchange, as well as the end-point they connect to? If there is not, who is responsible for creating it? Maintaining it over time? How can we communicate the model and its realization(s) in NHIN to the user communities in a way that gives them acceptable assurance?
Great work outlining the proposed protocols.
Could someone enlighten me as to why SMTP is being preferred when it has no really good structure unlike XML which is easy to program with SOAP based web services.
I know from providers I work with that SMTP is a pain to implement from a technical standpoint with assigning and maintaining certificates. Setting up NHIN email addresses for every user would be a burden as well. Furthermore SMTP is poor at handling errorts and undeliverable messages off line. Plus without good tracking is it really HIPAA compliant through all the touch points?
I understand having a lightweight protocol but handling email messages and decipehring contents would seem like an additional hassle to EHR/EMR developers and users. In addition the support costs would be higher to keep things running.
Has anyone proposed a ultra-lite IHE that is SOAP and web based but requires much simpler messaging like SMTP? It really only needs a few payload options and could inlcude TIFF/PDF option and HL7 option, a CDA/CDR option, etc. It would seem to us to be a natural fit. You have structure with XML, messaging delivery being guaranteed, full end-to-end tracking for HIPAA compliancy. SOAP and web services are based on HTTPS so that is a snap for SSL security (unlike TLS certs which cost about $80/yr extra) regardless of the chosen port. It is easy to code for and against. It just seems like a natural fit that is being left out of the discussion.
Thanks