HL7, EDI & FHIR Data Integration for Analytics & Reporting with PilotFish

This is a demonstration of the PilotFish eiConsole, a graphical integrated development environment for the rapid configuration of any Healthcare Interface.

Here we see a number of example routes that are representative of some real-life uses of the software.

• Data acquisition and normalization of Health Information Exchange (HIE) data
• Receipt in the transformation of laboratory orders status and results
• On-Premise medical equipment integration such as smart cards in a hospital that must be integrated with pharmacy, EMR and billing systems
• Acquisition of clinical and administrative data for reporting and analytics
• Integration of HL7 based hospital systems with cloud-based solution providers that may be using technology such as JSON, XML and web services

Creating a Multiple Source Data Route

Now let’s take a more in-depth look at this multiple source to reporting in analytics example. Double-clicking a route takes us to the main route grid which graphically depicts the flow of data between source and target systems.

Here we see data coming in as HL7 over LLP. We have some claims-based EDI files being dropped in an FTP folder.  We also have some FHIR JSON coming in through a REST-ful web service.

In each case, we’ll be extracting some basic patient information and we’ll use it to populate a relational database and also to execute a simple REST-based JSON service. To accomplish this the user simply configures the eiConsole Interface Assembly Line, a consistent set of stages that are flexible enough to connect any system to any other regardless of data format, or connectivity protocol. Moving from left to right, we start with the source system which maintains descriptive information about the sending system.

Retrieving Your Healthcare Data

Next is the Listener – the entry point of the interface to configure a Listener. We select the appropriate listener type from the drop-down and fill out the related property panels. The eiConsole supports real-time periodic scheduled and triggered listeners. Some commonly implemented listeners include HL7 over LLP, FTP including SFTP and FTPS, SOAP and RESTful based web services, Messaging Middleware such as JMS, MSMQ, and RabbitMQ.

After the listeners, configured processors offer a library of widgets for manipulation and validation prior to data transformation. Common examples include decryption, decompression or validation.

Processors, as is true of other stages in the assembly line, are extensible through an open Java-based API. After pre-processing, we move on to the Source Transformation. Here the data is converted from the inbound format into a canonical XML representation through a two-step process.

Convert Your Data into a Common XML Format

First, Parsing to Convert Data into XML. Transformation modules exist to convert a variety of non-XML formats including CSVs, delimited and fixed-width files, EDI including ANSI X12 EDI and other HIPAA formats, HL7 of course, JSON, Microsoft Excel and others.

Note that the HL7 transformation module has been specifically designed to be lenient. Given all the various versions and non-standard flavors of HL7 messages, this ensures that virtually anything that even looks like HL7 can be parsed for future processing. The second step of transformation is logical mapping, which is always an XML to XML transformation represented as XSLT.

Mapping Your HealthCare Transactions with the Data Mapper

This is the Data Mapper which allows a technical business analyst to drag & drop to build transformations between any two data formats. The 3-panel mapping paradigm is unique in its ability to represent even the most complex mappings graphically.

To use the mapper, the Source and Target data formats are built right in through the format builder dialogue. Various types of metadata such as the HL7 data dictionary can be directly imported. Sample files can also be used to provide an accurate picture of what a system may actually produce or consume.

Note also when using HL7, “Friendly Names” can be applied to the elements to make it easier to determine what fields. Here HL7 was loaded as the Source, and a simple XML sample file was used as the Target. Dragging and dropping these fields together in the center panel creates a one-to-one map.

The palette of functions above the mapping allows for conditional logic, looping logic, data manipulation and much more. All of this can be accomplished without custom or proprietary scripting; however, the mapping is directly based on XSLT, a standard transformation language with a strong developer community.

Routing Your Data to the Target Destination

Returning to the main grid, the next step is Routing. Here XPath or other routing rules can be used to inspect the data and direct the inbound message towards the appropriate target system or systems.

Proactive error notification, via the transaction monitor metaphor, is also configured at this point. We then move to the target side of the interface. Each target system has an accompanying Transformation and Transport.

The Target Transform is the mirror image of the Source Transforming the canonical XML Format to the Format required by the Target system. Note that on the Target side, the Transformation Module executes second, converting the output of any logical mapping into the physical format required by the Target.

Finally, we move on to the Transport stage, and in this case, we’re updating a relational database and also calling a RESTful web service. However, as is true with a Listener, a wide variety of transmission mechanisms are supported. Both synchronous and asynchronous processing is also possible.

Pre and post processes are also available in the Transport step for encrypting, compressing or otherwise manipulating the data prior to transmission. Post processes could be used for cleanup tasks and executing subsequent steps in a complex data flow. Once an interface has been configured in the eiConsole, it can then be switched into testing mode.

Testing Your Interface Routes

In Test mode, we can test a route from end-to-end or choose a subset of stages we wish to test. When we start in the middle of the process, we’re asked to supply a sample file manually. As the file is processed, successful stages are marked with a green check and failures are indicated with a red X.

You can then look at how the data appeared at each stage in our process. Here we supplied an HL7 ADT message for input. The message was converted into XML with “Friendly Names” by the HL7 transformation module. The source mapping extracted key pieces of patient demographic information into a simple XML canonical.

The data was routed to both defined targets. The canonical was mapped first onto SQLXML. Here we indicate an insert into a table called patient and second, we simply rendered some JSON. We then will update the database and post to that RESTful web service.

Moving Interfaces into Production

Now that an interface has been successfully tested from end-to-end, it’s ready for deployment to the eiPlatform runtime environment. Deployment can be handled via drag & drop using the server view, also the PilotFish Web-based eiDashboard, or through your preferred file-based change management process.

Once deployed, the eiPlatform, provides a myriad of reporting, alerting and monitoring utilities. The eiPlatform is used for Production Management Interfaces. And that’s all there is to it!

We Are Here to Answer Your Questions

Please feel free to contact us with any additional technical questions or for a more customized demonstration of the software.  We offer a Free 90-Day Trial of the eiConsole. Try it for yourself!

HL7 is the registered trademark of Health Level Seven International. X12, chartered by the American National Standards Institute for more than 35 years, develops and maintains EDI standards and XML schemas.