Tuesday 30 October 2012

Hadoop Analytics is not real time - A Reality or Myth?

Big Data is the talk of the street and Hadoop is emerging as the platform of choice for running analysis on both structured and unstructured Big Data. 

One of the main strengths of Hadoop is ad-hoc massive scale analysis against the data stored in Hadoop. In a typical Hadoop usage, enterprises will dump majorityof their unstructured in HDFS and periodically run Map-Reduce analysis to gain insights into new data and optionally structure it for storage in other external data sources for reuse by other applications.

Following diagram (though overly simplified) reflects this usage of Map-Reduce.


While this model works quite well for offline-batched analytics, its serious limitation is that it cannot be used for real time decision-making. Business use cases that demand quick action on their data (e.g. security markets, fraud detection, fault detection, location-based services, Facebook Insights, Twitter trends etc.), cannot leverage Hadoop Map-Reduce for immediate real time analytics on their new data and leverage alternate technologies to meet the needs.

There is a popular belief that Hadoop Map-Reduce cannot be real time, which is true so far. HFlame (www.hflame.com, a product from Data Advent) breaks the shackle without reinventing Hadoop or any of its components. HFlame transforms customer’s existing Hadoop infrastructure  (e.g. Apache Hadoop, CDH, HDP) with real time data analysis infrastructure.

Following diagram explains the change in Map-Reduce processing with HFlame –


HFlame Map-Reduce jobs are continuously running (i.e. job is still active even when no data is available in HDFS to process). As soon as new data is written to HDFS, it is immediately passed to the appropriate real time Map-Reduce jobs. Real time Map-Reduce will either

  • Produce the immediate insights on the new data or
  • Collect the new data for specific amount of time and produce analytics results on the collective data.

HFlame continuous analysis places Hadoop right in the center of real time business solutions. Businesses can analyze the data stream instantaneously and leverage patterns like continuous query, complex event processing without introducing any further complexity to their infrastructure.
HFlame is completely transparent to the Hadoop users and works with their own Hadoop distribution and installation. HFlame leverages the core of Hadoop HDFS and Map-Reduce data processing framework.

Check out http://www.hflame.com or http://www.dataadvent.com for more details.
 

18 comments:

  1. Ecorptrainings.com provides ccna in hyderabad with best faculties on real time projects. We give the best online trainingamong the ccna in Hyderabad.
    Classroom Training in Hyderabad India

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. The information which you provides is very much useful for the Hadoop Learners. Thank you for your valuable information. I found hadooponlinetrainings is the best Hadoop Online Traininginstitute in Hyderabad, India .

    ReplyDelete
  4. Nice article very happy to see this Hadoop Online Training Article.. I came to know hadooponlinetrainings.com at hyderabad is also providing excellent hadoop online training.. keep Posting more articles..

    ReplyDelete
  5. I was really impresed by reading this article about
    Hadoop online training
    , It will be useful for Hadoop online training learners

    ReplyDelete
  6. Thanks for your support,I am very interested in learning HADOOP Right now i am learning HADOOP in 123 trainings. They will provide the Best
    HADOOP Online Training at hyderabad.


    ReplyDelete
  7. It was nice article it was very useful for me as well as useful for HADOOP online training learners thanks for providing this valuable information.

    ReplyDelete
  8. Thanks for sharing the valuable information

    Hadoop Online Training

    ReplyDelete
  9. Thanks for sharing this valuble information and it is useful for me and also Hadoop learners.we also provideHadoop Bigdata Online Training Classes In India

    ReplyDelete

  10. Thanks for sharing valuable information and it is useful for hadooponlinetrainings provides the best Hadoop Online Training classes.

    ReplyDelete
  11. It is really nice article and i got some info from this article,thanks for sharing it.
    We also provide Hadoop online training in hyderabad,India and for more information contact 91-8897755222 or drop mail to ads@mentorsinn.com.

    ReplyDelete
  12. Best Big Data Hadoop Training in Hyderabad @ Kalyan Orienit

    Follow the below links to know more knowledge on Hadoop

    WebSites:
    ================
    http://www.kalyanhadooptraining.com/

    http://www.hyderabadhadooptraining.com/

    http://www.bigdatatraininghyderabad.com/

    Videos:
    ===============
    https://www.youtube.com/watch?v=-_fTzrgzVQc

    https://www.youtube.com/watch?v=Df2Odze87dE

    https://www.youtube.com/watch?v=AOfX-tNkYyo

    https://www.youtube.com/watch?v=Cyo3y0vlZ3c

    https://www.youtube.com/watch?v=jOLSXx6koO4

    https://www.youtube.com/watch?v=09mpbNBAmCo


    Best Big Data Hadoop Training in Hyderabad @ Kalyan Orienit

    ReplyDelete
  13. Nice blog and thanks for sharing your information. Hamsini Technologies provide the high-quality online courses training to the students like android, HADOOP, java, SAP, Tableau, Hibernate, Struts, Spring, Salesforce etc. and also provide record sessions

    HADOOP Online Training Hyderabad.
    Hadoop Online training

    ReplyDelete
  14. Nice one.

    Thank you for postining good information about SEO.
    SaiSantoshTechnologies Offers search engine optimization and marketing services, text link brokering, directory submission service, and list of free
    internet based SEO tools.

    seo training in hyderabad

    seo training in hyderabad

    ReplyDelete
  15. Beautiful - I was just about to write some ugly cursor code when I found this.

    Best Software Testing Training Institute in Chennai with Placement

    ReplyDelete
  16. Usually the blog you posted is very useful to us thanks for posting this blog..............................Please contact us to know more about Oracle Fusion Financials Training

    ReplyDelete
  17. Thanks for sharing it. I always enjoy reading such superb content with valuable information. The ideas presented are excellent and really cool, making the post truly enjoyable. Keep up the fantastic work.
    Data Analytics Certifications: Paving the Way to Career Advancement

    ReplyDelete