Add the following line to
Refer to Logging.
runAsSparkUser(func: () => Unit)
func function with Hadoop’s
UserGroupInformation of the current user as a thread local variable (and distributed to child threads). It is later used for authenticating HDFS and YARN calls.
runAsSparkUser reads the current username (as SPARK_USER environment variable or the short user name from Hadoop’s
FIXME How to use
You should see the current username printed out in the following DEBUG message in the logs:
DEBUG YarnSparkHadoopUtil: running as user: [user]
It then creates a remote user for the current user (using
UserGroupInformation.createRemoteUser), transfers credential tokens and runs the input
func function as the privileged user.