Name of the Vulnerable Software and Affected Versions Apache Spark versions prior to 2.3.3

Description The issue arises when Apache Spark writes user data to local disk unencrypted, despite having spark.io.encryption.enabled set to true. This occurs in specific situations, including when cached blocks are fetched to disk, controlled by spark.maxRemoteBlockSizeFetchToMem, and when using certain functions in SparkR and Pyspark, such as parallelize, broadcast, and python udfs.

Recommendations For versions prior to 2.3.3, update to version 2.3.3 or later to resolve the issue. As a temporary workaround, consider disabling the use of spark.maxRemoteBlockSizeFetchToMem, parallelize, broadcast, and python udfs until a patch is available. Restrict access to sensitive data and minimize the use of affected functions to minimize the risk of exploitation.