Hadoop Saves Children’s Hospital from Information Chaos

In the recent times, like the other industry sectors, the healthcare sector has also recognized the uniqueness of Hadoop in re-engineering any outdated database design for facilitating volume, velocity, variety, and veracity of data—normally associated with big data. Quickly following the footsteps of its peers, many prominent hospitals worldwide have embraced the Hadoop technology, thus enabling their staff to work efficiently with big data. In fact, many patient-care systems could not even think of working with very high-volume data or unstructured data before big data and Hadoop technologies were introduced in their work environments.

This post showcases one such patient health-care system, used at Los Angeles Children’s Hospital, which reinvented its data-management architecture with Hadoop. Alina Popescu reports that while attending the 2013 Hadoop Summit, the co-hosts of the CUBE magazine met Mohamed Elmallah, Manager of Enterprise Applications and Architecture at the Children’s Hospital in Los Angeles. In their conversation, Mr. Elmallah seemed very excited about recounting his experience with Hadoop implementation at his hospital —to facilitate fast access to Electronic Medical Records (EMR) data that had been online since 2004.

When Hadoop was considered for th Children’s Hospital, the budget-frendliness of the entire project was a major criterion for getting approval. To this effect, Mr. Elmallah mentioned that there was a match between the needs of research on data and the budget we had.

In a hospital environment, specific staff members like the researchers, physicians, and other clinical trials personnel frequently require to review and compare high volumes of  data from widely disparate sources. Thus storing such high-volume, wide-variety data is in itself a challenge; moreover extracting highly specific data from literally mountains of data and processing such data brings a fresh set of challenges. Hadoop is a highly sought-after solution to meet such a formidable challenge—Hadoop used in combination with big data provides an even better solution.

For the last several years, the researchers or other hospital staff at Los Angeles Children’s Hospital—who required instant access to mountains of their patient data and also other medical research data for their work, were finally able to gain access to essential data at their fingertips because of Hadoop. Hadoop—for those that do not know—is a data-storage technology that utilizes existing hardware and network components to rebuild a Hadoop-enabled network architecture to facilitate speedy storage and retrieval of high volume data from back-end databases.

Elmallah mentioned that the researchers at the hospital frequently needed to monitor the vital signs of patients during their treatment. Traditionally, a nurse or another health-care provider took snapshots of the vital signs and manually inserted them into the EMR system. The researchers were not happy with the amount of data; they needed more frequent data—a whole history, a continuous curve of the vital signs.  For this project, the need for continuous storage of high frequency data was converted into reality through the use of the SQL server before storing the data on HDFS.

As part of their future plan, Elmallah said that the electronic medical records systems are relatively open today…We’ll be able to get the data out of the Hadoop and tier it into the EMR for our end users.

Hadoop has proved its mettle in the big data world, and this case study is just another example of a successful implementation of Hadoop at a children’s hospital. In fact, the back-end of the EMR system has been designed such that the hospital users simply have to know the front end UI to quickly extract the needed patient data, process or update it, and store the data back on the system.

Why is Hadoop a preferred choice among large data systems?

As in the case of Children’s Hospital in Los Angeles, many organizations, handling very high-volume data on a daily basis, have transitioned to Hadoop-enabled database systems. Hadoop has the following characteristics, which make it a favored data platform for data scientists and end users alike:

1. The provision for utilizing ordinary, low-end hardware as components of a Hadoop network.
2. Fast data retrieval through a distributed, parallel processing architecture
3. Data access from any location such as networked mobile devices
4. Fast storage of data without any data loss
5. Low maintenance as distributed servers are free from data clogging
6. Decentralized network: One broken node does not bring down entire system

Hadoop sits silently at the back end—managing all the data storage, data authentication, data retrieval, and data processing needs, while the average user just worries about interfacing through a simple-to-use, friendly UI.

How has Hadoop reshaped the EMR system at the Children’s Hospital?

At Children’s Hospital in Los Angeles, the data scientists and IT folks have reinvented the database design for the EMR system; and now the hospital administrators, hospital staff, research staff and even the IT center are all happy to see such a transparent system helping users (mainly researchers) interact with such powerful data, that was hitherto unknown to them.

Hadoop architecture has also provisioned working with very large data sets and high volume data, usually of variable data types, which may be extracted at high velocity, preserving the veracity of the data. Before Hadoop came into picture, the hospital staff could not dream of handling structured and unstructured data at the same time, and with such high degree of accuracy.

Hadoop enabled data-management expands frontiers of research

This hospital is now rolling out their revived EMR system to outpatient clinics, which can certainly expand the scope of patient data for future research projects. The major considerations behind the hospital’s technology decision— the need to enable research  scientists access large amounts of data, the hospital’s patient data, and worldwide patient or medical research data—have all been fulfilled now.