[jira] [Created] (FLINK-16304) Remove python packages bundled in the flink-python jar

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-16304) Remove python packages bundled in the flink-python jar

Shang Yuanchun (Jira)
Wei Zhong created FLINK-16304:
---------------------------------

             Summary: Remove python packages bundled in the flink-python jar
                 Key: FLINK-16304
                 URL: https://issues.apache.org/jira/browse/FLINK-16304
             Project: Flink
          Issue Type: Improvement
          Components: API / Python
    Affects Versions: 1.11.0
            Reporter: Wei Zhong
             Fix For: 1.11.0


As the Python Dependency Management API is sufficient to control the python environment, and current design that bundles python packages to the flink-python jar and extracts when running Python UDF has been a blocker of the Python UDF performance optimization. The python packages bundled in the flink-python jar need to be removed now.

It means that users need to ensure that the PyFlink and its dependencies are pre-installed in the client and cluster environment when submitting jobs. Of course the cluster environment can be configured via Python Dependency Management API.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)