OREANDA-NEWS. August 22, 2012. Fujitsu Laboratories Limited today announced the development of the industry's first integrated big data development platform for processing large volumes of diverse time-series data.

In recent years, massive amounts of diverse data—as represented by sensor data, human location data, and other kinds of time-series data—continue to grow at an explosive pace. This has prompted the development of parallel batch processing technologies such as Hadoop(1), as well as complex event processing technologies(2) for processing data in real time. However, because each processing technology has employed different types of development and execution environments, it has been difficult to quickly apply insights gained from data analysis results to real-time processing applications. Moreover, maximizing the performance of Fujitsu's event processing engine(3) has required considerable knowledge about parallel application design, such as how to estimate network traffic.

By developing an integrated development platform able to handle description languages for both stored data analysis and complex event processing, Fujitsu was able to reduce development time for both batch and event processing by roughly 80% (from 8 weeks to 1.5 weeks) in a case study involving POS analysis-based coupon issuing. The platform is also equipped with a newly developed parallelism extraction function that automatically improves the processing efficiency of complex event processing. This function automatically extracts parallelism opportunities from event processing applications and recommends how to combine each analysis step in a way that optimizes execution plans without any extra effort.

This is one of the technologies that will be put to use to support human-centric computing, which will provide precisely targeted services anywhere.

Details of the new technology will be published at the IPSJ/SIGSE Software Engineering Symposium 2012 (SES2012), to be held from August 27 - 29, 2012.