Tuesday, 30 October 2012

Hadoop Analytics is not real time - A Reality or Myth?

Big Data is the talk of the street and Hadoop is emerging as the platform of choice for running analysis on both structured and unstructured Big Data. 

One of the main strengths of Hadoop is ad-hoc massive scale analysis against the data stored in Hadoop. In a typical Hadoop usage, enterprises will dump majorityof their unstructured in HDFS and periodically run Map-Reduce analysis to gain insights into new data and optionally structure it for storage in other external data sources for reuse by other applications.

Following diagram (though overly simplified) reflects this usage of Map-Reduce.

While this model works quite well for offline-batched analytics, its serious limitation is that it cannot be used for real time decision-making. Business use cases that demand quick action on their data (e.g. security markets, fraud detection, fault detection, location-based services, Facebook Insights, Twitter trends etc.), cannot leverage Hadoop Map-Reduce for immediate real time analytics on their new data and leverage alternate technologies to meet the needs.

There is a popular belief that Hadoop Map-Reduce cannot be real time, which is true so far. HFlame (www.hflame.com, a product from Data Advent) breaks the shackle without reinventing Hadoop or any of its components. HFlame transforms customer’s existing Hadoop infrastructure  (e.g. Apache Hadoop, CDH, HDP) with real time data analysis infrastructure.

Following diagram explains the change in Map-Reduce processing with HFlame –

HFlame Map-Reduce jobs are continuously running (i.e. job is still active even when no data is available in HDFS to process). As soon as new data is written to HDFS, it is immediately passed to the appropriate real time Map-Reduce jobs. Real time Map-Reduce will either

  • Produce the immediate insights on the new data or
  • Collect the new data for specific amount of time and produce analytics results on the collective data.

HFlame continuous analysis places Hadoop right in the center of real time business solutions. Businesses can analyze the data stream instantaneously and leverage patterns like continuous query, complex event processing without introducing any further complexity to their infrastructure.
HFlame is completely transparent to the Hadoop users and works with their own Hadoop distribution and installation. HFlame leverages the core of Hadoop HDFS and Map-Reduce data processing framework.

Check out http://www.hflame.com or http://www.dataadvent.com for more details.


  1. This Hadoop trainings will help you comprehend the procedures followed in the Hadoop structure and how it can be used for the company requirements.

  2. Ecorptrainings.com provides ccna in hyderabad with best faculties on real time projects. We give the best online trainingamong the ccna in Hyderabad.
    Classroom Training in Hyderabad India

  3. This comment has been removed by the author.

  4. The Information which you provided is very much useful for Hadoop Online Training Learners Thank You for Sharing Valuable Information

  5. The information which you provides is very much useful for the Hadoop Learners. Thank you for your valuable information. I found hadooponlinetrainings is the best Hadoop Online Traininginstitute in Hyderabad, India .

  6. Nice article very happy to see this Hadoop Online Training Article.. I came to know hadooponlinetrainings.com at hyderabad is also providing excellent hadoop online training.. keep Posting more articles..

  7. I was really impresed by reading this article about
    Hadoop online training
    , It will be useful for Hadoop online training learners

  8. Thanks for your support,I am very interested in learning HADOOP Right now i am learning HADOOP in 123 trainings. They will provide the Best
    HADOOP Online Training at hyderabad.

  9. It was nice article it was very useful for me as well as useful for HADOOP online training learners thanks for providing this valuable information.

  10. The Information you provided is very much useful for Hadoop Learners. This Information was very Intersting, We also provide Hadoop Online training in India.
    Skypeid: rsonlinehyd
    Please contact us India:+91 9052699906,USA :+1 909-666-5386

  11. Thanks for sharing this valuble information and it is useful for me and also Hadoop learners.we also provideHadoop Bigdata Online Training Classes In India

  12. Appreciation for nice Update, I found something new and folks can get useful information about Hadoop Online Training


  13. Thanks for sharing valuable information and it is useful for hadooponlinetrainings provides the best Hadoop Online Training classes.

  14. It is really nice article and i got some info from this article,thanks for sharing it.
    We also provide Hadoop online training in hyderabad,India and for more information contact 91-8897755222 or drop mail to ads@mentorsinn.com.

  15. Best Big Data Hadoop Training in Hyderabad @ Kalyan Orienit

    Follow the below links to know more knowledge on Hadoop










    Best Big Data Hadoop Training in Hyderabad @ Kalyan Orienit

  16. Nice blog and thanks for sharing your information. Hamsini Technologies provide the high-quality online courses training to the students like android, HADOOP, java, SAP, Tableau, Hibernate, Struts, Spring, Salesforce etc. and also provide record sessions

    HADOOP Online Training Hyderabad.
    Hadoop Online training


  17. Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write
    Packers and Movers Thane
    Packers and Movers Navi Mumbai
    Packers and Movers Ghaziabad
    Packers and Movers Faridabad

  18. Nice one.

    Thank you for postining good information about SEO.
    SaiSantoshTechnologies Offers search engine optimization and marketing services, text link brokering, directory submission service, and list of free
    internet based SEO tools.

    seo training in hyderabad

    seo training in hyderabad

  19. Beautiful - I was just about to write some ugly cursor code when I found this.

    Best Software Testing Training Institute in Chennai with Placement

  20. Usually the blog you posted is very useful to us thanks for posting this blog..............................Please contact us to know more about Oracle Fusion Financials Training