About JobMananger metrics scope

[Lu Niu]

Hi, Flink Dev

First of all, Happy New Year! I have a question about JM monitoring.

According to
https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html,
metrics.scope.jm only have <hostname> one variable, which seems to be not
enough for YARN deployment mode:

1. The metric doesn't contain identifiers like application id or job id.
2. Since multiple flink jobs can run in a single yarn cluster, it is
possible that multiple JM can run in a single host, but inside different
containers. Under current scope, metrics belonging to two flink jobs will
share identical metric name. e,g,  <host>.jobmanager.
Status.JVM.Memory.Heap.Committed

Is there any walkaround or fix in roadmap? Thanks!

Best
Lu

[Chesnay Schepler-3]

The only workaround is to add some constant to this option before
deploying a Flink cluster.

Relevant JIRA: https://issues.apache.org/jira/browse/FLINK-9543

On 1/7/2021 11:58 PM, Lu Niu wrote:

> Hi, Flink Dev
>
> First of all, Happy New Year! I have a question about JM monitoring.
>
> According to
> https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html,
> metrics.scope.jm only have <hostname> one variable, which seems to be not
> enough for YARN deployment mode:
>
> 1. The metric doesn't contain identifiers like application id or job id.
> 2. Since multiple flink jobs can run in a single yarn cluster, it is
> possible that multiple JM can run in a single host, but inside different
> containers. Under current scope, metrics belonging to two flink jobs will
> share identical metric name. e,g,  <host>.jobmanager.
> Status.JVM.Memory.Heap.Committed
>
> Is there any walkaround or fix in roadmap? Thanks!
>
> Best
> Lu
>