Collaboration to Optimize the Hadoop Stack and Advance Big Data Technologies in Genomics

As a part of this collaboration, the NextBio and Intel engineers will apply experience they have gained from NextBio’s use of Big Data technologies to the improvement of HDFS (Hadoop distributed file system), Hadoop, and HBase. Any enhancements that NextBio engineers make to the Hadoop stack will be contributed to the open-source community. Intel will also showcase NextBio’s use of Big Data. Hadoop is a distributed work manager for huge amounts of data on a large number of systems. The system is also involved in monitoring, failover, and scheduling.

“NextBio is positioned at the intersection of genomics and Big Data. Every day we deal with the three Vs [volume, variety, and velocity] associated with big Data--we, our collaborators, and our users are adding large volumes of a variety of molecular data to NextBio at an increasing velocity,” said Dr. Satnam Alag, chief technology officer and vice president of engineering at NextBio. “Without the implementation of our algorithms in the MapReduce framework, operational expertise in HDFS, Hadoop, and HBase, and investments in building our secure cloud-based infrastructure, it would have been impossible for us to scale cost-effectively to handle this large-scale data.”

Hadoop is designed as a distributed work manager for huge amounts of data on a large number of systems. However, Hadoop is more than that in that it is also about monitoring, failover, and scheduling. “Intel is firmly committed to the wide adoption and use of big data technologies such as HDFS, Hadoop, and HBase across all industries that need to analyze large amounts of data,” said Girish Juneja, CTO and general manager, Big Data software and services, Intel. “Complex data requiring compute-intensive analysis needs not only big data open source, but a combination of hardware and software management optimizations to help deliver needed scale with a high return on investment. Intel is working closely with NextBio to deliver this showcase reference to the Big Data community and life science industry.”

“The use of big data technologies at NextBio enables researchers and clinicians to mine billions of data points in real-time to discover new biomarkers, clinically assess targets and drug profiles, optimally design clinical trials, and interpret patient molecular data,” Dr. Alag continued. “NextBio has invested significantly in the use of Big Data technologies to handle the tsunami of genomic data being generated and its expected exponential growth. As we further scale our infrastructure to handle this growing data resource, we are excited to work with Intel to make the Hadoop stack better and give back to the open-source community.”

NextBio provides a cutting-edge scientific platform to aggregate and interpret large quantities of genomic and other life sciences data for research and clinical applications. NextBio’s platform integrates data from multiple repositories and diverse technologies by means of a unique correlation engine, which precomputes billions of vital connections between disparate public and proprietary sources of clinical and experimental data. In doing so, the platform enables interoperability from instrument readouts to data interpretation for translational research and discovery.

Backed by highly scalable, big data technology capable of analyzing petabytes of data in real-time, NextBio’s platform is delivered as a SaaS (software as a service) solution resulting in quick deployment and rapid return on investment.

Related Links:
NextBio
Intel