[jira] [Created] (FLINK-18087) Uploading user artifact for Yarn job cluster could not work

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-18087) Uploading user artifact for Yarn job cluster could not work

Shang Yuanchun (Jira)
Yang Wang created FLINK-18087:
---------------------------------

             Summary: Uploading user artifact for Yarn job cluster could not work
                 Key: FLINK-18087
                 URL: https://issues.apache.org/jira/browse/FLINK-18087
             Project: Flink
          Issue Type: Bug
    Affects Versions: 1.11.0, 1.12.0
            Reporter: Yang Wang
             Fix For: 1.11.0, 1.12.0


In FLINK-17632, we add the support remote user jar. However, uploading user artifact for Yarn job cluster is broken exceptionally. In the following code, we should only upload local files. Now it has the contrary behavior.
{code:java}
// only upload local files
if (Utils.isRemotePath(entry.getValue().filePath)) {
   Path localPath = new Path(entry.getValue().filePath);
   Tuple2<Path, Long> remoteFileInfo =
         fileUploader.uploadLocalFileToRemote(localPath, entry.getKey());
   jobGraph.setUserArtifactRemotePath(entry.getKey(), remoteFileInfo.f0.toString());
}
{code}
 

Another problem is the related tests {{testPerJobModeWithDistributedCache}} does not fail because we do not fetch the artifact from blob server. We directly get it from local file. It also needs to be enhanced.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)