“Apache Hadoop 3.3” Released, with ARM support

July 28, Apache Hadoop community announced the release of the latest stable version “Apache Hadoop 3.3.0”. It brings many improvements including support for ARM and Java 11 runtime.

Apache Hadoop is an open-source distributed processing framework. It is made up of Hadoop Common, Hadoop Distributed File System, Hadoop YARN (job scheduling and cluster resource manager), Hadoop MapReduce (YARN-based parallel processing system), and Hadoop Ozone.

Apache Hadoop 3.3 is the latest release of the v3 series after v3.2, released in January 2019. This release includes over 2,100 bug fixes since v3.2.

It now offers support for ARM architecture. Support for Java11 runtime is now stable.

Protocol Buffers is upgraded to v3.7.1 from v2.5 as v2.5 reached EOL. Protocol Buffers are language-neutral mechanism, created by Google, for serializing structure data.

Now, it exports Hadoop metrics to Prometheus. As the default value of the property is false, “hadoop.prometheus.endpoint.enabled” needs to be set to true to obtain Prometheus-friendly formatted metrics from ‘/prom’ endpoints of Hadoop daemons.

It supports AuthenticationFilter authentication mechanism. HDFS Router-based Federation (RBF) supports Kerberos authentication and also contains many bug fixes.

This release includes many improvements regarding Amazon S3, adding better support for Delegation Token, better handling of 404 caching, and improved S3guard performance.

This release introduces application catalog system, which provides an editorial and search interface for YARN applications. YARN is added with OPPORTUNISTIC containers that can be queued and executed at NodeManager (NM). Their priority is lower than the existing GUARANTEED containers. Scheduling of opportunistic containers can be made through the central YARN RM and distributed scheduling.

There are many other improvements added to this release.

Apache Hadoop
https://hadoop.apache.org/