An efficient industrial big-data engine Articles uri icon

authors

  • BASANTA VAL, PABLO

publication date

  • April 2018

start page

  • 1

end page

  • 9

International Standard Serial Number (ISSN)

  • 1551-3203

Electronic International Standard Serial Number (EISSN)

  • 1941-0050

abstract

  • Current trends in industrial systems opt for the use of different big-data engines as a mean to process huge amounts of data that cannot be processed with an ordinary infrastructure. The number of issues an industrial infrastructure has to face is large and includes challenges such as the definition of different efficient architecture setups for different applications, and the definition of specific models for industrial analytics. In this context, the article explores the development of a medium size big-data engine (i.e. implementation) able to improve performance in map-reduce computing by splitting the analytic into different segments that may be processed by the engine in parallel using a hierarchical model. This type of facility reduces end-to-end computation time for all segments with their results then merged with other information from other segments after their processing in parallel. This type of setup increases performance of current clusters improving I/O operations remarkably as empirical results revealed.

keywords

  • industrial big-data; efficient infrastructure; map-reduce; industrial infrastructure; big-data engine.