[jira] [Created] (FLINK-22407) upgrade log4j to 2.14

[Shang Yuanchun (Jira)]

Svend Vanderveken created FLINK-22407:
-----------------------------------------

             Summary: upgrade log4j to 2.14
                 Key: FLINK-22407
                 URL: https://issues.apache.org/jira/browse/FLINK-22407
             Project: Flink
          Issue Type: Improvement
            Reporter: Svend Vanderveken


Flink is currently [relying on log4j 2.12.1|https://github.com/apache/flink/blob/master/pom.xml#L110] .Unfortunately, this vesrion has a bug related to json layout that prevents a user from adding additional log fields, as reported here https://issues.apache.org/jira/browse/LOG4J2-2652 as well as here: [https://stackoverflow.com/questions/57003440/why-is-log4j2-jsonlayout-keyvaluepair-printing-empty-logevent-messages]

The problem is fixed in Log4j 2.13.1.

Is there a good reason to keep Log4j 2.12.1, or can we upgrade?

As an illustration, the presence of  {{additional1}} in the snippet below:
{code:java}
rootLogger.level = INFO
rootLogger.appenderRef.console.ref = LogConsole      
appender.console.name = LogConsole
appender.console.type = CONSOLE
appender.console.layout.type = JsonLayout
appender.console.layout.complete = false
appender.console.layout.compact = true
appender.console.layout.eventEol = true
appender.console.layout.properties = true
appender.console.layout.includeStacktrace=true
appender.console.layout.stacktraceAsString=true      
appender.console.layout.additional1.type=KeyValuePair
appender.console.layout.additional1.key=timestamp
appender.console.layout.additional1.value=$${date:yyyy-MM-dd'T'HH:mm:ss.SSSZ}

{code}
 
 leads to missing fields in the resulting logs, e.g.:
{code:java}
{"logEvent":"Recover all persisted job graphs.","timestamp":"2021-04-21T16:50:31.722+0000"}
{"logEvent":"Successfully recovered 0 persisted job graphs.","timestamp":"2021-04-21T16:50:31.723+0000"}
{"logEvent":"Starting the SlotManager.","timestamp":"2021-04-21T16:50:31.732+0000"}
{"logEvent":"Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/rpc/dispatcher_1 .","timestamp":"2021-04-21T16:50:31.822+0000"}
{code}
 
 Removing the {{additional1}} resolves the issue and yield json logs containing all expected fields:
{code:java}
{"thread":"cluster-io-thread-1","level":"INFO","loggerName":"org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess","message":"Successfully recovered 0 persisted job graphs.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":216868000},"threadId":48,"contextMap":{},"threadPriority":5}
{"thread":"flink-akka.actor.default-dispatcher-3","level":"INFO","loggerName":"org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl","message":"Starting the SlotManager.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":313130000},"threadId":19,"contextMap":{},"threadPriority":5}
{"thread":"cluster-io-thread-1","level":"INFO","loggerName":"org.apache.flink.runtime.rpc.akka.AkkaRpcService","message":"Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/rpc/dispatcher_1 .","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1619080838,"nanoOfSecond":408714000},"threadId":48,"contextMap":{},"threadPriority":5}
{code}
 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)