[jira] [Created] (FLINK-17724) PyFlink end-to-end test fails with Cannot run program "venv.zip/.conda/bin/python": error=2, No such file or directory

[Shang Yuanchun (Jira)]

Robert Metzger created FLINK-17724:
--------------------------------------

             Summary: PyFlink end-to-end test fails with Cannot run program "venv.zip/.conda/bin/python": error=2, No such file or directory
                 Key: FLINK-17724
                 URL: https://issues.apache.org/jira/browse/FLINK-17724
             Project: Flink
          Issue Type: Bug
          Components: API / Python, Tests
            Reporter: Robert Metzger
             Fix For: 1.11.0


CI: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8001&view=logs&j=1f3ed471-1849-5d3c-a34c-19792af4ad16&t=2f5b54d0-1d28-5b01-d344-aa50ffe0cdf8

{code}
2020-05-15T09:21:20.5198725Z Verifying transaction: ...working... done
2020-05-15T09:21:20.5918557Z Executing transaction: ...working... done
2020-05-15T09:22:18.8776831Z DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
2020-05-15T09:22:19.0555591Z Starting cluster.
2020-05-15T09:22:21.9516388Z Starting standalonesession daemon on host fv-az678.
2020-05-15T09:22:23.5833254Z Starting taskexecutor daemon on host fv-az678.
2020-05-15T09:22:23.6192099Z Waiting for Dispatcher REST endpoint to come up...
2020-05-15T09:22:24.6699447Z Waiting for Dispatcher REST endpoint to come up...
2020-05-15T09:22:26.0376695Z Waiting for Dispatcher REST endpoint to come up...
2020-05-15T09:22:27.1345574Z Waiting for Dispatcher REST endpoint to come up...
2020-05-15T09:22:28.1809673Z Dispatcher REST endpoint is up.
2020-05-15T09:22:28.1842051Z Test submitting python job:\n
2020-05-15T09:22:29.6520483Z Results directory: /tmp/result
2020-05-15T09:23:23.2171819Z Traceback (most recent call last):
2020-05-15T09:23:23.2174222Z   File "/home/vsts/work/1/s/flink-end-to-end-tests/flink-python-test/python/python_job.py", line 82, in <module>
2020-05-15T09:23:23.2174857Z     word_count()
2020-05-15T09:23:23.2175685Z   File "/home/vsts/work/1/s/flink-end-to-end-tests/flink-python-test/python/python_job.py", line 76, in word_count
2020-05-15T09:23:23.2176310Z     t_env.execute("word_count")
2020-05-15T09:23:23.2177228Z   File "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/pyflink.zip/pyflink/table/table_environment.py", line 1049, in execute
2020-05-15T09:23:23.2179484Z   File "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1286, in __call__
2020-05-15T09:23:23.2181045Z   File "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/pyflink.zip/pyflink/util/exceptions.py", line 147, in deco
2020-05-15T09:23:23.2182205Z   File "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 328, in get_return_value
2020-05-15T09:23:23.2182889Z py4j.protocol.Py4JJavaError: An error occurred while calling o2.execute.
2020-05-15T09:23:23.2183634Z : java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
2020-05-15T09:23:23.2184387Z at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:290)
2020-05-15T09:23:23.2184960Z at org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:325)
2020-05-15T09:23:23.2185508Z at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-05-15T09:23:23.2186320Z at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-05-15T09:23:23.2186930Z at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-05-15T09:23:23.2187472Z at java.lang.reflect.Method.invoke(Method.java:498)
2020-05-15T09:23:23.2188027Z at org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
2020-05-15T09:23:23.2188672Z at org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
2020-05-15T09:23:23.2189287Z at org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
2020-05-15T09:23:23.2189911Z at org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
2020-05-15T09:23:23.2190546Z at org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
2020-05-15T09:23:23.2191178Z at org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
2020-05-15T09:23:23.2191702Z at java.lang.Thread.run(Thread.java:748)
2020-05-15T09:23:23.2192357Z Caused by: java.util.concurrent.ExecutionException: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
2020-05-15T09:23:23.2193084Z at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
2020-05-15T09:23:23.2193645Z at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
2020-05-15T09:23:23.2194213Z at org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:319)
2020-05-15T09:23:23.2194827Z ... 11 more
2020-05-15T09:23:23.2196472Z Caused by: org.apache.flink.client.program.ProgramInvocationException: Job failed (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
2020-05-15T09:23:23.2198222Z at org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:116)
2020-05-15T09:23:23.2198778Z at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
2020-05-15T09:23:23.2199403Z at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
2020-05-15T09:23:23.2199858Z at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
2020-05-15T09:23:23.2200313Z at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
2020-05-15T09:23:23.2200826Z at org.apache.flink.client.program.rest.RestClusterClient.lambda$pollResourceAsync$22(RestClusterClient.java:602)
2020-05-15T09:23:23.2201336Z at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
2020-05-15T09:23:23.2201822Z at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
2020-05-15T09:23:23.2202290Z at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
2020-05-15T09:23:23.2224141Z at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
2020-05-15T09:23:23.2224660Z at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:307)
2020-05-15T09:23:23.2225150Z at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
2020-05-15T09:23:23.2225628Z at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
2020-05-15T09:23:23.2226080Z at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
2020-05-15T09:23:23.2226519Z at java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
2020-05-15T09:23:23.2226971Z at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:943)
2020-05-15T09:23:23.2227416Z at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
2020-05-15T09:23:23.2227864Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2020-05-15T09:23:23.2228293Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2020-05-15T09:23:23.2228601Z ... 1 more
2020-05-15T09:23:23.2228905Z Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
2020-05-15T09:23:23.2229343Z at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:147)
2020-05-15T09:23:23.2229861Z at org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:114)
2020-05-15T09:23:23.2230242Z ... 19 more
2020-05-15T09:23:23.2230569Z Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
2020-05-15T09:23:23.2231404Z at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:112)
2020-05-15T09:23:23.2232059Z at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:78)
2020-05-15T09:23:23.2232647Z at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:189)
2020-05-15T09:23:23.2233188Z at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:183)
2020-05-15T09:23:23.2233734Z at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:177)
2020-05-15T09:23:23.2234253Z at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:505)
2020-05-15T09:23:23.2234748Z at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:386)
2020-05-15T09:23:23.2235150Z at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-05-15T09:23:23.2235515Z at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-05-15T09:23:23.2235967Z at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-05-15T09:23:23.2236466Z at java.lang.reflect.Method.invoke(Method.java:498)
2020-05-15T09:23:23.2236860Z at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:284)
2020-05-15T09:23:23.2237478Z at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:199)
2020-05-15T09:23:23.2238115Z at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
2020-05-15T09:23:23.2238594Z at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
2020-05-15T09:23:23.2238985Z at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
2020-05-15T09:23:23.2239352Z at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
2020-05-15T09:23:23.2239726Z at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
2020-05-15T09:23:23.2240095Z at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
2020-05-15T09:23:23.2240569Z at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
2020-05-15T09:23:23.2240938Z at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
2020-05-15T09:23:23.2241326Z at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
2020-05-15T09:23:23.2241685Z at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
2020-05-15T09:23:23.2242024Z at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
2020-05-15T09:23:23.2242385Z at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
2020-05-15T09:23:23.2242702Z at akka.actor.ActorCell.invoke(ActorCell.scala:561)
2020-05-15T09:23:23.2243033Z at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
2020-05-15T09:23:23.2243357Z at akka.dispatch.Mailbox.run(Mailbox.scala:225)
2020-05-15T09:23:23.2243644Z at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
2020-05-15T09:23:23.2243987Z at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
2020-05-15T09:23:23.2244373Z at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
2020-05-15T09:23:23.2244784Z at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2020-05-15T09:23:23.2245181Z at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2020-05-15T09:23:23.2246306Z Caused by: java.lang.Exception: The user defined 'open(Configuration)' method in class org.apache.flink.table.runtime.functions.python.PythonScalarFunctionFlatMap caused an exception: Failed to create stage bundle factory!
2020-05-15T09:23:23.2246989Z at org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1352)
2020-05-15T09:23:23.2247464Z at org.apache.flink.runtime.operators.chaining.ChainedFlatMapDriver.openTask(ChainedFlatMapDriver.java:47)
2020-05-15T09:23:23.2247930Z at org.apache.flink.runtime.operators.BatchTask.openChainedTasks(BatchTask.java:1392)
2020-05-15T09:23:23.2248370Z at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:157)
2020-05-15T09:23:23.2248765Z at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:713)
2020-05-15T09:23:23.2249133Z at org.apache.flink.runtime.taskmanager.Task.run(Task.java:539)
2020-05-15T09:23:23.2249456Z at java.lang.Thread.run(Thread.java:748)
2020-05-15T09:23:23.2249774Z Caused by: java.lang.RuntimeException: Failed to create stage bundle factory!
2020-05-15T09:23:23.2250246Z at org.apache.flink.python.AbstractPythonFunctionRunner.createStageBundleFactory(AbstractPythonFunctionRunner.java:197)
2020-05-15T09:23:23.2250755Z at org.apache.flink.python.AbstractPythonFunctionRunner.open(AbstractPythonFunctionRunner.java:164)
2020-05-15T09:23:23.2251342Z at org.apache.flink.table.runtime.runners.python.scalar.AbstractGeneralPythonScalarFunctionRunner.open(AbstractGeneralPythonScalarFunctionRunner.java:65)
2020-05-15T09:23:23.2252007Z at org.apache.flink.table.runtime.functions.python.AbstractPythonStatelessFunctionFlatMap.open(AbstractPythonStatelessFunctionFlatMap.java:211)
2020-05-15T09:23:23.2252624Z at org.apache.flink.table.runtime.functions.python.AbstractPythonScalarFunctionFlatMap.open(AbstractPythonScalarFunctionFlatMap.java:69)
2020-05-15T09:23:23.2253173Z at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
2020-05-15T09:23:23.2253718Z at org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1350)
2020-05-15T09:23:23.2253997Z ... 6 more
2020-05-15T09:23:23.2254328Z Caused by: java.io.IOException: Cannot run program "venv.zip/.conda/bin/python": error=2, No such file or directory
2020-05-15T09:23:23.2254737Z at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
2020-05-15T09:23:23.2255178Z at org.apache.flink.python.util.PythonEnvironmentManagerUtils.execute(PythonEnvironmentManagerUtils.java:176)
2020-05-15T09:23:23.2255723Z at org.apache.flink.python.util.PythonEnvironmentManagerUtils.getSitePackagesPath(PythonEnvironmentManagerUtils.java:156)
2020-05-15T09:23:23.2256434Z at org.apache.flink.python.util.PythonEnvironmentManagerUtils.pipInstallRequirements(PythonEnvironmentManagerUtils.java:98)
2020-05-15T09:23:23.2257046Z at org.apache.flink.python.env.ProcessPythonEnvironmentManager.createEnvironment(ProcessPythonEnvironmentManager.java:172)
2020-05-15T09:23:23.2257609Z at org.apache.flink.python.AbstractPythonFunctionRunner.createPythonExecutionEnvironment(AbstractPythonFunctionRunner.java:249)
2020-05-15T09:23:23.2258213Z at org.apache.flink.table.runtime.runners.python.AbstractPythonStatelessFunctionRunner.createExecutableStage(AbstractPythonStatelessFunctionRunner.java:158)
2020-05-15T09:23:23.2258821Z at org.apache.flink.python.AbstractPythonFunctionRunner.createStageBundleFactory(AbstractPythonFunctionRunner.java:195)
2020-05-15T09:23:23.2259172Z ... 12 more
2020-05-15T09:23:23.2259405Z Caused by: java.io.IOException: error=2, No such file or directory
2020-05-15T09:23:23.2259710Z at java.lang.UNIXProcess.forkAndExec(Native Method)
2020-05-15T09:23:23.2259991Z at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
2020-05-15T09:23:23.2260313Z at java.lang.ProcessImpl.start(ProcessImpl.java:134)
2020-05-15T09:23:23.2260623Z at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
2020-05-15T09:23:23.2260875Z ... 19 more
2020-05-15T09:23:23.2260980Z
2020-05-15T09:23:23.2406372Z org.apache.flink.client.program.ProgramAbortException
2020-05-15T09:23:23.2406771Z at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:95)
2020-05-15T09:23:23.2407331Z at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2020-05-15T09:23:23.2407892Z at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2020-05-15T09:23:23.2408342Z at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2020-05-15T09:23:23.2408748Z at java.lang.reflect.Method.invoke(Method.java:498)
2020-05-15T09:23:23.2409138Z at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
2020-05-15T09:23:23.2409654Z at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
2020-05-15T09:23:23.2410157Z at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:148)
2020-05-15T09:23:23.2410664Z at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:707)
2020-05-15T09:23:23.2411172Z at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
2020-05-15T09:23:23.2411577Z at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:924)
2020-05-15T09:23:23.2412014Z at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1000)
2020-05-15T09:23:23.2412499Z at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
2020-05-15T09:23:23.2413178Z at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1000)
2020-05-15T09:23:23.8192063Z Stopping taskexecutor daemon (pid: 89153) on host fv-az678.
2020-05-15T09:23:23.9113567Z Stopping standalonesession daemon (pid: 88857) on host fv-az678.
2020-05-15T09:23:24.0749389Z [FAIL] Test script contains errors.
2020-05-15T09:23:24.0757935Z Checking of logs skipped.
2020-05-15T09:23:24.0758481Z
2020-05-15T09:23:24.0759921Z [FAIL] 'PyFlink end-to-end test' failed after 3 minutes and 24 seconds! Test exited with exit code 1
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)