Wei Zhong created FLINK-15248:
---------------------------------
Summary: FileUtils#compressDirectory behaves buggy when processing relative directory path
Key: FLINK-15248
URL:
https://issues.apache.org/jira/browse/FLINK-15248 Project: Flink
Issue Type: Bug
Components: FileSystems
Affects Versions: 1.10.0
Reporter: Wei Zhong
Fix For: 1.10.0
_FileUtils#compressDirectory_ behaves buggy when processing relative directory path. If the path of target directory is a relative path, the relative path inside the target zip file can not be constructed correctly:
{code:java}
public static Path compressDirectory(Path directory, Path target) throws IOException {
FileSystem sourceFs = directory.getFileSystem();
FileSystem targetFs = target.getFileSystem();
try (ZipOutputStream out = new ZipOutputStream(targetFs.create(target, FileSystem.WriteMode.NO_OVERWRITE))) {
addToZip(directory, sourceFs, directory.getParent(), out);
}
return target;
}
private static void addToZip(Path fileOrDirectory, FileSystem fs, Path rootDir, ZipOutputStream out) throws IOException {
String relativePath = fileOrDirectory.getPath().replace(rootDir.getPath() + '/', "");
if (fs.getFileStatus(fileOrDirectory).isDir()) {
out.putNextEntry(new ZipEntry(relativePath + '/'));
// The containedFile.getPath() returns an absolute path but the rootDir
// could be a relative path or an empty string (if user only specify the
// directory name as the relative path). In this case when calling this
// method recursively the replacement at the beginning of this method will
// return a wrong result.
for (FileStatus containedFile : fs.listStatus(fileOrDirectory)) {
addToZip(containedFile.getPath(), fs, rootDir, out);
}
} else {
ZipEntry entry = new ZipEntry(relativePath);
out.putNextEntry(entry);
try (FSDataInputStream in = fs.open(fileOrDirectory)) {
IOUtils.copyBytes(in, out, false);
}
out.closeEntry();
}
}{code}
Currently PyFlink allows users to upload python library directories and requirements cached directory, which will be compressed by _FileUtils#compressDirectory_ eventually. If users specify them via relative paths, this bug will be triggered and causes those features unavailable.
we can fix this bug by converting the directory path to absolute path in _FileUtils#compressDirectory_ before calling _addToZip_ method_._
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
