Patterns for distributed real-time stream processing Articles uri icon

publication date

  • November 2017

start page

  • 3243

end page

  • 3257

issue

  • 11

volume

  • 28

International Standard Serial Number (ISSN)

  • 1045-9219

Electronic International Standard Serial Number (EISSN)

  • 1558-2183

abstract

  • In recent years, big data systems have become an active area of research and development. Stream processing is one of the potential application scenarios of big data systems where the goal is to process a continuous, high velocity flow of information items. High frequency trading (HFT) in stock markets or trending topic detection in Twitter are some examples of stream processing applications. In some cases (like, for instance, in HFT), these applications have end-to-end quality-of-service requirements and may benefit from the usage of real-time techniques. Taking this into account, the present article analyzes, from the point of view of real-time systems, a set of patterns that can be used when implementing a stream processing application. For each pattern, we discuss its advantages and disadvantages, as well as its impact in application performance, measured as response time, maximum input frequency and changes in utilization demands due to the pattern.

subjects

  • Telecommunications

keywords

  • real-time patterns; stream processing; big data; real-time systems; computational modeling; storms; big data applications; twitter; time factors