Robert Metzger created FLINK-21475:
-------------------------------------- Summary: StateWithExecutionGraph.suspend fails with IllegalStateException Key: FLINK-21475 URL: https://issues.apache.org/jira/browse/FLINK-21475 Project: Flink Issue Type: New Feature Components: Runtime / Coordination Affects Versions: 1.13.0 Reporter: Robert Metzger Fix For: 1.13.0 https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=13665&view=logs&j=a5ef94ef-68c2-57fd-3794-dc108ed1c495&t=9c1ddabe-d186-5a2c-5fcc-f3cafb3ec699 For discoverability, the Azure output is: {code} 2021-02-23T23:48:50.6572167Z [INFO] BUILD FAILURE 2021-02-23T23:48:50.6573151Z [INFO] ------------------------------------------------------------------------ 2021-02-23T23:48:50.6573684Z [INFO] Total time: 01:59 min 2021-02-23T23:48:50.6574520Z [INFO] Finished at: 2021-02-23T23:48:50+00:00 2021-02-23T23:48:51.4672056Z [INFO] Final Memory: 183M/3491M 2021-02-23T23:48:51.4673656Z [INFO] ------------------------------------------------------------------------ 2021-02-23T23:48:51.4674310Z [WARNING] The requested profile "skip-webui-build" could not be activated because it does not exist. 2021-02-23T23:48:51.4675176Z [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.1:test (integration-tests) on project flink-connector-files: There are test failures. 2021-02-23T23:48:51.4675685Z [ERROR] 2021-02-23T23:48:51.4676248Z [ERROR] Please refer to /__w/2/s/flink-connectors/flink-connector-files/target/surefire-reports for the individual test results. 2021-02-23T23:48:51.4677378Z [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. 2021-02-23T23:48:51.4677963Z [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-02-23T23:48:51.4679891Z [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-connectors/flink-connector-files/target && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire/surefirebooter72596965996179907.jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire 2021-02-23T23-47-06_882-jvmRun1 surefire3083231786399295313tmp surefire_104007394662374862924tmp 2021-02-23T23:48:51.4680846Z [ERROR] Error occurred in starting fork, check output in log 2021-02-23T23:48:51.4681156Z [ERROR] Process Exit Code: 239 2021-02-23T23:48:51.4681413Z [ERROR] Crashed tests: 2021-02-23T23:48:51.4681737Z [ERROR] org.apache.flink.connector.file.sink.writer.FileSinkMigrationITCase 2021-02-23T23:48:51.4682470Z [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-02-23T23:48:51.4684122Z [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-connectors/flink-connector-files/target && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire/surefirebooter72596965996179907.jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire 2021-02-23T23-47-06_882-jvmRun1 surefire3083231786399295313tmp surefire_104007394662374862924tmp 2021-02-23T23:48:51.4685319Z [ERROR] Error occurred in starting fork, check output in log 2021-02-23T23:48:51.4685796Z [ERROR] Process Exit Code: 239 2021-02-23T23:48:51.4686229Z [ERROR] Crashed tests: 2021-02-23T23:48:51.4686926Z [ERROR] org.apache.flink.connector.file.sink.writer.FileSinkMigrationITCase 2021-02-23T23:48:51.4687709Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510) 2021-02-23T23:48:51.4688603Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:457) 2021-02-23T23:48:51.4689451Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:298) 2021-02-23T23:48:51.4690207Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246) 2021-02-23T23:48:51.4691076Z [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183) 2021-02-23T23:48:51.4692029Z [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011) 2021-02-23T23:48:51.4693058Z [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857) 2021-02-23T23:48:51.4693946Z [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132) 2021-02-23T23:48:51.4694768Z [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) 2021-02-23T23:48:51.4695612Z [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) 2021-02-23T23:48:51.4696345Z [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) 2021-02-23T23:48:51.4697299Z [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) 2021-02-23T23:48:51.4698243Z [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) 2021-02-23T23:48:51.4699215Z [ERROR] at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) 2021-02-23T23:48:51.4700380Z [ERROR] at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120) 2021-02-23T23:48:51.4701116Z [ERROR] at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355) 2021-02-23T23:48:51.4701775Z [ERROR] at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) 2021-02-23T23:48:51.4702545Z [ERROR] at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) 2021-02-23T23:48:51.4703179Z [ERROR] at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216) 2021-02-23T23:48:51.4703787Z [ERROR] at org.apache.maven.cli.MavenCli.main(MavenCli.java:160) 2021-02-23T23:48:51.4704402Z [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-02-23T23:48:51.4705114Z [ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-02-23T23:48:51.4705918Z [ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-02-23T23:48:51.4706668Z [ERROR] at java.lang.reflect.Method.invoke(Method.java:498) 2021-02-23T23:48:51.4707481Z [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) 2021-02-23T23:48:51.4708256Z [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) 2021-02-23T23:48:51.4709066Z [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) 2021-02-23T23:48:51.4709880Z [ERROR] at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) 2021-02-23T23:48:51.4710740Z [ERROR] Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-02-23T23:48:51.4714026Z [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-connectors/flink-connector-files/target && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire/surefirebooter72596965996179907.jar /__w/2/s/flink-connectors/flink-connector-files/target/surefire 2021-02-23T23-47-06_882-jvmRun1 surefire3083231786399295313tmp surefire_104007394662374862924tmp 2021-02-23T23:48:51.4715805Z [ERROR] Error occurred in starting fork, check output in log 2021-02-23T23:48:51.4716291Z [ERROR] Process Exit Code: 239 2021-02-23T23:48:51.4716701Z [ERROR] Crashed tests: 2021-02-23T23:48:51.4717323Z [ERROR] org.apache.flink.connector.file.sink.writer.FileSinkMigrationITCase 2021-02-23T23:48:51.4718074Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:669) 2021-02-23T23:48:51.4718917Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115) 2021-02-23T23:48:51.4719729Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444) 2021-02-23T23:48:51.4720466Z [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420) 2021-02-23T23:48:51.4721147Z [ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2021-02-23T23:48:51.4721808Z [ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2021-02-23T23:48:51.4722700Z [ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2021-02-23T23:48:51.4723348Z [ERROR] at java.lang.Thread.run(Thread.java:748) 2021-02-23T23:48:51.4724269Z [ERROR] -> [Help 1] 2021-02-23T23:48:51.4724583Z [ERROR] 2021-02-23T23:48:51.4725296Z [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. 2021-02-23T23:48:51.4726133Z [ERROR] Re-run Maven using the -X switch to enable full debug logging. 2021-02-23T23:48:51.4726565Z [ERROR] 2021-02-23T23:48:51.4727239Z [ERROR] For more information about the errors and possible solutions, please read the following articles: 2021-02-23T23:48:51.4728131Z [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException 2021-02-23T23:48:51.4728631Z [ERROR] 2021-02-23T23:48:51.4729322Z [ERROR] After correcting the problems, you can resume the build with the command 2021-02-23T23:48:51.4730277Z [ERROR] mvn <goals> -rf :flink-connector-files 2021-02-23T23:48:51.9715714Z Process exited with EXIT CODE: 1. 2021-02-23T23:48:51.9723840Z Trying to KILL watchdog (4117). {code} From the logs, the real issue seems to be {code} 23:48:38,279 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager [] - Shutting down TaskExecutorLocalStateStoresManager. 23:48:38,266 [flink-akka.actor.default-dispatcher-3] INFO org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor [] - The RpcEndpoint jobmanager_3 failed. org.apache.flink.runtime.jobmaster.JobMasterException: Could not properly stop the JobMaster. at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:405) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:563) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:186) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[scala-library-2.11.12.jar:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) ~[scala-library-2.11.12.jar:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) ~[scala-library-2.11.12.jar:?] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.invoke(ActorCell.scala:561) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [akka-actor_2.11-2.5.21.jar:2.5.21] Caused by: java.lang.IllegalStateException at org.apache.flink.util.Preconditions.checkState(Preconditions.java:177) ~[flink-core-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.scheduler.adaptive.StateWithExecutionGraph.suspend(StateWithExecutionGraph.java:139) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.suspend(AdaptiveScheduler.java:307) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.stopScheduling(JobMaster.java:945) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.stopJobExecution(JobMaster.java:907) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:399) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] ... 20 more 23:48:38,284 [flink-akka.actor.default-dispatcher-3] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Job 0fbf949ce33fe9af6c47b0cf3ceac0a0 was not finished by JobManager. 23:48:38,306 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl [] - FileChannelManager removed spill file directory /tmp/flink-io-1b0f58d4-884c-4cac-bc87-f9028fb573a5 23:48:38,306 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.io.network.NettyShuffleEnvironment [] - Shutting down the network environment and its components. 23:48:38,285 [flink-akka.actor.default-dispatcher-3] ERROR org.apache.flink.runtime.util.FatalExitExceptionHandler [] - FATAL: Thread 'flink-akka.actor.default-dispatcher-3' produced an uncaught exception. Stopping the process... java.util.concurrent.CompletionException: org.apache.flink.util.FlinkException: Could not properly shut down the JobManagerRunner at java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:338) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.uniRelay(CompletableFuture.java:925) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:913) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_282] at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.lambda$closeAsync$0(JobManagerRunnerImpl.java:224) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_282] at org.apache.flink.runtime.concurrent.FutureUtils.lambda$forwardTo$22(FutureUtils.java:1341) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456) ~[?:1.8.0_282] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_282] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_282] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_282] Caused by: org.apache.flink.util.FlinkException: Could not properly shut down the JobManagerRunner ... 12 more Caused by: org.apache.flink.runtime.jobmaster.JobMasterException: Could not properly stop the JobMaster. at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:405) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:563) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:186) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) ~[scala-library-2.11.12.jar:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) [scala-library-2.11.12.jar:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [scala-library-2.11.12.jar:?] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.invoke(ActorCell.scala:561) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.run(Mailbox.scala:225) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [akka-actor_2.11-2.5.21.jar:2.5.21] Caused by: java.lang.IllegalStateException at org.apache.flink.util.Preconditions.checkState(Preconditions.java:177) ~[flink-core-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.scheduler.adaptive.StateWithExecutionGraph.suspend(StateWithExecutionGraph.java:139) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.suspend(AdaptiveScheduler.java:307) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.stopScheduling(JobMaster.java:945) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.stopJobExecution(JobMaster.java:907) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.jobmaster.JobMaster.onStop(JobMaster.java:399) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStop(RpcEndpoint.java:214) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StartedState.terminate(AkkaRpcActor.java:563) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:186) ~[flink-runtime_2.11-1.13-SNAPSHOT.jar:1.13-SNAPSHOT] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [akka-actor_2.11-2.5.21.jar:2.5.21] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) [scala-library-2.11.12.jar:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [akka-actor_2.11-2.5.21.jar:2.5.21] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) ~[scala-library-2.11.12.jar:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [scala-library-2.11.12.jar:?] at akka.actor.Actor$class.aroundReceive(Actor.scala:517) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.actor.ActorCell.invoke(ActorCell.scala:561) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.run(Mailbox.scala:225) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.Mailbox.exec(Mailbox.scala:235) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) ~[akka-actor_2.11-2.5.21.jar:2.5.21] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) ~[akka-actor_2.11-2.5.21.jar:2.5.21] 23:48:38,307 [flink-akka.actor.default-dispatcher-4] INFO org.apache.flink.runtime.io.disk.FileChannelManagerImpl [] - FileChannelManager removed spill file directory /tmp/flink-netty-shuffle-ac49e716-c423-4dcc-9e42-34303a9e200d {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) |
Free forum by Nabble | Edit this page |