RSS

Monthly Archives: August 2015

Healthcare I.T in the big data space: Article 2 – Getting setup, touching some code

This is the 2nd article in this series, you can start with the 1st article if you need some context.

For the infrastructure, I began small.

  1. I launched an Ubuntu ec2 instance
  2. I installed a vnc  server on it so i could access the desktop from a vnc client on my Mac  – I plan to use this ec2 instance as my development machine
  3. I installed mirth – I’ll need this to generate valid hl7 messages that I will feed into spring xd’s big data pipeline
  4. I installed eclipse luna- to develop the spring integration modules that will be uploaded into spring xd.

My 1st use case is to create a hl7 feed and ingest it using spring xd, storing the messages in hdfs.

For the 1st development effort, I plan to create a mirth channel that reads hl7 from a database, sends it over tcp to a spring xd custom source module.
The mirth configuration is shown below

Source tab of the channel

mirth_source_channel
Destination tab of the channel

mirth_destination
For the spring xd to ingest the hl7, I have to create a custom source module as none of the out of the box modules will handle hl7 mllp protocol. This module will use a  camel hl7 endpoint to accept messages from mirth and then place these messages on a spring integration channel.

The plan is to eventually upload this spring integration module into spring xd. 1st things 1st. In eclipse I created a spring boot project. The configuration class / application class looks as follows:

This being the 1st iteration , Ian writing to file just to make sure the data is coming in, ad sure enough after I run the code in eclipse and start my mirth channel, I have hl7 messages dropping into my folder . Neat!!

Spring boot/integration /camel source code iteration 1 of the inbound hl7 to a file destination:

package com.wchr.xd;

import java.io.IOException;

import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import static org.apache.camel.component.hl7.HL7.ack;
import org.apache.camel.component.hl7.HL7MLLPNettyDecoderFactory;
import org.apache.camel.component.hl7.HL7MLLPNettyEncoderFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@SpringBootApplication
@Configuration
public class Application extends RouteBuilder {

	@Autowired
	CamelContext camelContext; 

	//private static CamelContext cmContext;

    public static void main(String[] args) throws IOException, InterruptedException {
    	 SpringApplication.run(Application.class, args);
    	// cmContext = context.getBean(CamelContext.class);
    	}

    @Bean
    public HL7MLLPNettyDecoderFactory hl7decoder(){
    	return new HL7MLLPNettyDecoderFactory();
    }

    @Bean
    public HL7MLLPNettyEncoderFactory hl7encoder(){
    	return new HL7MLLPNettyEncoderFactory();
    }

	@Override
	public void configure() throws Exception {
		// TODO Auto-generated method stub
		from("netty4:tcp://localhost:9000?sync=true&encoder=#hl7encoder&decoder=#hl7decoder")
		.to("file://inbound?autoCreate=true&fileName=${date:now:yyyyMMddHHmmss}.hl7")
		.transform(ack());
	}

}

snapshot of the inbound messages from my eclipse project
inbound hl7

 
Leave a comment

Posted by on August 2, 2015 in java, software, spring

 

Tags: , , , , , , ,

Healthcare I.T in the big data space: Article 1 – The idea

So I’ve had this idea on my mind for a surprisingly long time, maybe even a couple of years. Finally I’m acting on it.

I have over the years seen a lot of EMR SYSTEMS, healthcare integration platforms tools applications both off the shelf vendor products and in house solutions. Most fall in the category I would call legacy (older technologies) or at best current proven technologies. Not much out there that is more bolder in using some of the newer specific big data solutions. That’s understandable since in some cases if the existing solution is not broke, why fix it.

Still this has left me with an itch that I’m going to try address here. I want to see a data integration project that aggregates healthcare data (hl7, custom flat files, x12,….), uses a big data platform to ingest, monitor, transform, store and run some analytics – the whole 9 yards.

If I’m successful in my next few posts I will set up a simple project to do so, using simple use cases at each of the integration steps I have mentioned above.

Down to some specifics:

I plan to use the following technologies: (if I could getaway with it I’d use spring xd solely)

  1. Source of some test data – mirth for hl7, spring xd for other generic data sources, others as my use cases advance
  2. Ingestion of data  –  camel for hl7, spring xd
  3. Monitoring – spring xd counters and gauges
  4. Transformation – spring xd processors, sources and sinks
  5. Persistence – hdfs using spring xd
  6. Analytics -not sure yet; one of the solutions supported by spring xd

My infrastructure consists of Amazon web services ec2 instances (Ubuntu) that I will spin up as required.

In the next post, I will have a short write up about how I got set up with aws ec2 and installed mirth and got started with eclipse

 
Leave a comment

Posted by on August 2, 2015 in programming, software

 

Tags: , , , ,