[jira] [Created] (FLINK-21024) Dynamic properties get exposed to job's main method if user parameters are passed

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-21024) Dynamic properties get exposed to job's main method if user parameters are passed

Shang Yuanchun (Jira)
Matthias created FLINK-21024:
--------------------------------

             Summary: Dynamic properties get exposed to job's main method if user parameters are passed
                 Key: FLINK-21024
                 URL: https://issues.apache.org/jira/browse/FLINK-21024
             Project: Flink
          Issue Type: Bug
            Reporter: Matthias


A bug was identified in the [user ML|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Application-cluster-standalone-job-some-JVM-Options-added-to-Program-Arguments-td40719.html] by Alexey exposing dynamic properties into the job user code.

I was able to reproduce this issue by slightly adapting the WordCount example (see attached WordCount2.jar).

Initiating a standalone job without using the `--input` parameter will result in printing an empty array:
```
./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2
```
The corresponding `*.out` file looks like this:
```
[]
Executing WordCount2 example with default input data set.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
```

In contrast, initiating the standalone job using the `--input` parameter will expose the dynamic properties:
```
./bin/standalone-job.sh start --job-classname org.apache.flink.streaming.examples.wordcount.WordCount2 --input /opt/flink/config/flink-conf.yaml
```
Resulting in the following output:
```
[--input, /opt/flink/config/flink-conf.yaml, -D, jobmanager.memory.off-heap.size=134217728b, -D, jobmanager.memory.jvm-overhead.min=201326592b, -D, jobmanager.memory.jvm-metaspace.size=268435456b, -D, jobmanager.memory.heap.size=1073741824b, -D, jobmanager.memory.jvm-overhead.max=201326592b]
Printing result to stdout. Use --output to specify output path.
```

Interestingly, this cannot be reproduced on a local standalone session cluster.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)