[jira] [Created] (FLINK-16376) Using consistent method to get Yarn application directory

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-16376) Using consistent method to get Yarn application directory

Shang Yuanchun (Jira)
Victor Wong created FLINK-16376:
-----------------------------------

             Summary: Using consistent method to get Yarn application directory
                 Key: FLINK-16376
                 URL: https://issues.apache.org/jira/browse/FLINK-16376
             Project: Flink
          Issue Type: Improvement
          Components: Deployment / YARN
    Affects Versions: 1.10.0
            Reporter: Victor Wong


The Yarn application directory of Flink is "/user/{user.name}/.flink", but this logic is separated in different places.

1. org.apache.flink.yarn.YarnClusterDescriptor#getYarnFilesDir

{code:java}
        private Path getYarnFilesDir(final ApplicationId appId) throws IOException {
                final FileSystem fileSystem = FileSystem.get(yarnConfiguration);
                final Path homeDir = fileSystem.getHomeDirectory();
                return *new Path(homeDir, ".flink/" + appId + '/');*
        }
{code}

2. org.apache.flink.yarn.Utils#uploadLocalFileToRemote


{code:java}
                // copy resource to HDFS
                String suffix =
                        *".flink/"*
                                *+ appId*
                                + (relativeTargetPath.isEmpty() ? "" : "/" + relativeTargetPath)
                                + "/" + localSrcPath.getName();

                Path dst = new Path(homedir, suffix);
{code}

We can extract `getYarnFilesDir` method to `org.apache.flink.yarn.Utils`, and use this method to get Yarn application directory in all the other places.






--
This message was sent by Atlassian Jira
(v8.3.4#803005)