Ankit Sharma created FLINK-11324:
------------------------------------
Summary: Custom Log4j properties for each flink job
Key: FLINK-11324
URL:
https://issues.apache.org/jira/browse/FLINK-11324 Project: Flink
Issue Type: Improvement
Components: Flink on Tez
Affects Versions: 1.6.2
Reporter: Ankit Sharma
It would be a good idea if we can have a custom log4j files/location for each job we run via flink.
Currently there is no option for that in flink command.
I tried providing log4j properties by using below command but it didn't worked for me:
flink run /usr/lib/flink/examples/batch/WordCount.jar -Dlog4j.configuration=~/log4j_ABC.properties
So, i checked the source code of flink and found that the path of log4j proerpties are hard-coded in flink command script, so did few twik to make it work. It worked for me but i am really not sure about what other components it might affect.
Below are the steps/changes done:
1. Add below code in $FLINK_HOME/bin/flink file just after the config.sh being called, this code will add a new environment variable FLINK_LOG_SETTING in which we will be setting the log4j-cli.properties absolute path:
if [ -z "$FLINK_LOG_SETTING" ]; then FLINK_LOG_SETTING=$FLINK_CONF_DIR; fi
2. In log_setting variable mentioned in flink, change the -Dlog4j.configuration as below:
log_setting=(-Dlog.file="$log" -Dlog4j.configuration=file:"$FLINK_LOG_SETTING"/log4j-cli.properties -Dlogback.configurationFile=file:"$FLINK_CONF_DIR"/logback.xml)
3. save the changes.
4. Now before executing any flink job in cli, export the FLINK_LOG_SETTING to any custom path where you have log4j-cli.properties present for the respective job.
$ export FLINK_LOG_SETTING=/home/hadoop/test
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
