java.lang.IllegalAccessError: ai.h2o.sparkling.backend.utils.RestCommunication When Running H2O Job

General
1

Issue:

When running a H2O job on Livy, the following exception is encountered:

java.lang.IllegalAccessError: class ai.h2o.sparkling.backend.utils.RestCommunication
(in unnamed module @0x480846cb) cannot access class
sun.net.www.protocol.http.Handler (in module java.base)
because module java.base does not export
sun.net.www.protocol.http to unnamed module @0x480846cb

Cause:

This error occurs because the H2O library attempts to access the internal JDK class sun.net.www.protocol.http.Handler. Starting with Java 9, the Java Module System restricts access to internal JDK APIs unless they are explicitly exported.

Resolution:

To allow access to the required internal JDK package, add the following Java options to Spark Conf in Livy Connection:

--conf spark.driver.extraJavaOptions="--add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED" \
--conf spark.executor.extraJavaOptions="--add-exports=java.base/sun.net.www.protocol.http=ALL-UNNAMED"

This exports the sun.net.www.protocol.http package from the java.base module to all unnamed modules, allowing the H2O library to access the required class and preventing the IllegalAccessError.