liu created FLINK-20230:
---------------------------
Summary: PARTITIONED TABLE, INSERT INTO EMPTY VALUES, THROW FileNotFoundException
Key: FLINK-20230
URL:
https://issues.apache.org/jira/browse/FLINK-20230 Project: Flink
Issue Type: Bug
Components: API / DataSet, API / DataStream
Affects Versions: 1.11.2, 1.11.1, 1.11.0
Environment: TABLE DDL:
CREATE TABLE IF NOT EXISTS test1 (
uid string,
truename string,
dt string
)
COMMENT '用户表1'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ''
STORED AS TEXTFILE;
CREATE TABLE IF NOT EXISTS test2(
uid string COMMENT 'ID',
truename string COMMENT '实际名称'
)
COMMENT '用户表2'
PARTITIONED BY ( `dt` string )
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ''
STORED AS TEXTFILE;
# CODE IS:
EnvironmentSettings settings = EnvironmentSettings.newInstance()
.useBlinkPlanner()
.inBatchMode()
.build();
TableEnvironment tableEnv = TableEnvironment.create(settings);
String nowDt = "202011111";
Table table = tableEnv.sqlQuery(String.format("select \n" +
"uid,\n" +
"truename,\n"
"'%s' as dt\n" +
"from test1 " +
"where dt = '%s'", nowDt, nowDt
));
table.executeInsert("test2");
Reporter: liu
if table has PARTITIONED, USE SQL "INSERT INTO VALUES", WHEN THE VALUES IS NULL, AND IT WILL THROW FileNotFoundException;
*Exception:*
Caused by: java.lang.Exception: Failed to finalize execution on masterCaused by: java.lang.Exception: Failed to finalize execution on master ... 34 moreCaused by: org.apache.flink.table.api.TableException: Exception in finalizeGlobal at org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:97) at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.finalizeOnMaster(InputOutputFormatVertex.java:132) at org.apache.flink.runtime.executiongraph.ExecutionGraph.vertexFinished(ExecutionGraph.java:1286) ... 33 moreCaused by: java.io.FileNotFoundException: File hdfs://nameservice1/user/hive/warehouse/test.db/ad_wide/.staging_1605749732741 does not exist. at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:901) at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:112) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:958) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:958) at org.apache.flink.hive.shaded.fs.hdfs.HadoopFileSystem.listStatus(HadoopFileSystem.java:157) at org.apache.flink.table.filesystem.PartitionTempFileManager.headCheckpoints(PartitionTempFileManager.java:140) at org.apache.flink.table.filesystem.FileSystemCommitter.commitUpToCheckpoint(FileSystemCommitter.java:98) at org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:95)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
