Stream Processing Framework “Apache Flink 1.11” Released

July 6, development team behind the open-source distributed stream processing “Apache Flink” released the latest stable version “Apache Flink 1.11.0”.

Apache Flink is an open-source stream processing framework. With distributed stream data flow engine written in Java and Scala, it can be integrated with resources like Kubernetes, Yarn, and Apache Mesos, and use storages like HDFS and Amazon S3.

This release brings Unaligned Checkpoints, a new checkpoint mechanism, to the core data engine. It improves checkpointing performance under heavy backpressure. It is disabled by default, so it needs to be enabled from checkpoint settings.

It also introduces Source API, which simplifies the implementation of sources by unifying batch and streaming.

It used to be that watermark generations relied on two different interfaces (AssignerWithPunctuatedWatermarks and AssignerWithPeriodicWatermarks), but now they are unified into a single interface (WatermarkGenerator). This improves the implementation of new connectors and watermark emission.

Flink SQL introduces support for Change Data Capture (CDC) and support for Python UDF. It has also added improvements to PyFlink.

Apache Flink
https://flink.apache.org/