Spark 2.4.1 is a maintenance release containing stability fixes. This release is based on the branch-2.4 maintenance branch of Spark. We strongly recommend all 2.4.0 users to upgrade to this stable release.
In Apache Spark 2.4.1, Scala 2.12 support is GA, and it’s no longer experimental. We will drop Scala 2.11 support in Spark 3.0, so please provide us feedback.
You can consult JIRA for the detailed changes.
spark.executor.heartbeatInterval is less than one second, it will always be set to zero resulting timeout.
We would like to acknowledge all community members for contributing patches to this release.