A First Look at Big Data Apache Flink!
There is abundance of interest in learning how to analyze streaming data in large-scale systems, partly because there are situations in which the time-value of data makes real-time analytics so eye-catching. But gathering in-the-moment insights made possible by very low latency applications is just one of the benefits of high-performance stream processing. In this blog we will talk about First Look at Apache Flink, and Apache Flink is a highly innovative open source stream processor with a surprising range of capabilities that help you take advantage of stream-based approaches.
So what is Apache Flink?
The Apache Flink home project has the tagline – Apache Flink is an open source platform for distributed stream and batch data processing. For many of us it’s a real surprise to realize that Flink not only provides real-time streaming with high throughput and exactly-once guaranteed and it’s best suits for batch data processing engine. We could have to do choose between these approaches, but Flink lets us to do both with one tool; that’s Apache Flink. Flink has its origins in the Stratosphere project, a research project conducted by three Berlin-based Universities as well as other European Universities between 2010 and 2014.
A fork of the Stratosphere code was donated in April 2014 to the Apache Software Foundation as an incubating project, with an initial set of committers consisting of the core developers of the system. Shortly thereafter, many of the founding committers left university to start a company to commercialize Flink: data Artisans.
During incubation, the project name had to be changed from Stratosphere because of potential confusion with an unrelated project. The name Flink was selected to honor the style of this stream and batch processor: in German, the word “flink” means fast or agile.
A logo showing a colorful squirrel was chosen because squirrels are fast, agile and in the case of squirrels in Berlin is n amazing shade of reddish-brown.
The project completed incubation quickly, and in December 2014, Flink graduated to become a top-level project of the Apache Software Foundation. Flink is one of the 5 largest big data projects of the Apache Software Foundation, with a community of more than 200developers across the globe and several production installations with some of them in Fortune Global 500 companies In October 2015, the Flink project held its first annual conference in Berlin: Flink Forward. And 2nd Flink Forward is due by 12-14 Sep 2016. Berlin.
Flink not only enables fault-tolerant, truly real-time analytics, it can also analyze historical data and greatly simplify our data pipeline. Perhaps most surprising is that Flink lets us do streaming analytics as well as batch jobs, both with one technology. Flink’s expressivity and robust performance make it easy to develop applications, and Flink’s architecture makes those easy to maintain in production.
Reference – Introduction to Apache Flink, Ellen Friedman, Kostas Tzoumas
Interesting? Please subscribe to our blogs at www.dataottam.com to keep yourself trendy on Big Data, IoT, and Analytics.
And as always please feel free to comment coffee@dataottam.com.
↧
A First Look at Big Data Apache Flink!
↧