Adrian Zhong created FLINK-22769:
------------------------------------
Summary: yarnship do not support symbolic directory
Key: FLINK-22769
URL:
https://issues.apache.org/jira/browse/FLINK-22769 Project: Flink
Issue Type: Improvement
Components: Deployment / YARN
Affects Versions: 1.13.0, 1.12.2, 1.13.1
Reporter: Adrian Zhong
If we pass `-yt ` a symbolic directory, we will get an exception:
{code:java}
Caused by: java.lang.IllegalArgumentException: File to copy cannot be a directory: file:/path/to/sql/connectors/Caused by: java.lang.IllegalArgumentException: File to copy cannot be a directory: file:/path/to/sql/connectors/ at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138) at org.apache.flink.yarn.YarnApplicationFileUploader.uploadLocalFileToRemote(YarnApplicationFileUploader.java:197) at org.apache.flink.yarn.YarnApplicationFileUploader.registerSingleLocalResource(YarnApplicationFileUploader.java:179) at org.apache.flink.yarn.YarnApplicationFileUploader.registerMultipleLocalResources(YarnApplicationFileUploader.java:289) at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:870)
{code}
The root cause is not here but in :
{code:java}
final java.nio.file.Path shipPath = file.toPath();
final java.nio.file.Path parentPath = shipPath.getParent();
Files.walkFileTree(
shipPath.toRealPath(),
new SimpleFileVisitor<java.nio.file.Path>() {
@Override
public FileVisitResult visitFile(
java.nio.file.Path file, BasicFileAttributes attrs) {
localPaths.add(new Path(file.toUri())); //root cause: localPath is an symbolic file, without any file under real path.
relativePaths.add(
new Path(
localResourcesDirectory,
parentPath.relativize(file).toString()));
return FileVisitResult.CONTINUE;
}
});
{code}
Please assign to me, I'd like to be a contributor.
Any other please *do not implement* or create pr before this issue is assigned.
Note: this bug is not os platform specific.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
