Scott Kidder created FLINK-6176:
-----------------------------------
Summary: Add JARs to CLASSPATH deterministically
Key: FLINK-6176
URL:
https://issues.apache.org/jira/browse/FLINK-6176 Project: Flink
Issue Type: Bug
Components: Core
Affects Versions: 1.2.0
Reporter: Scott Kidder
The Flink 1.2.0 {{config.sh}} script uses the following shell-script function to build the CLASSPATH variable from a listing of JAR files in the {{$FLINK_HOME/lib}} directory:
{code}
constructFlinkClassPath() {
while read -d '' -r jarfile ; do
if [[ $FLINK_CLASSPATH = "" ]]; then
FLINK_CLASSPATH="$jarfile";
else
FLINK_CLASSPATH="$FLINK_CLASSPATH":"$jarfile"
fi
done < <(find "$FLINK_LIB_DIR" ! -type d -name '*.jar' -print0)
echo $FLINK_CLASSPATH
}
{code}
The {{find}} command as it is specified will return files in directory-order, which is varies by OS and filesystem.
The inconsistent ordering caused problems for me when installing a Flink Docker image I built on a new machine with a newer version of Docker. The differences in the Docker filesystem implementation led to different ordering of the directory contents, which led to a different order of the CLASSPATH element and very puzzling {{ClassNotFoundException}} errors when running my application.
This should be addressed by adding some explicit ordering to the JAR files added to the CLASSPATH used by Flink.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Locked
