Karrtik Iyer created FLINK-8615:
-----------------------------------
Summary: Configuring Apache Flink Local Set up with Pseudo distributed Yarn
Key: FLINK-8615
URL:
https://issues.apache.org/jira/browse/FLINK-8615 Project: Flink
Issue Type: Bug
Affects Versions: 1.4.0
Environment: Mac Os, Hadoop 2.8.3 and Flink 1.4.0
Reporter: Karrtik Iyer
I have set up HADOOP 2.8.3 and YARN on my Mac machine in Pseudo distributed mode, following this blog: [Hadoop In Pseudo Distributed Mode|
http://zhongyaonan.com/hadoop-tutorial/setting-up-hadoop-2-6-on-mac-osx-yosemite.html] I have been able to successfully start hdfs and yarn. And also able to submit Map reduce jobs.
After that I have download Apache Flink 1.4 from [here|
http://redrockdigimark.com/apachemirror/flink/flink-1.4.0/flink-1.4.0-bin-hadoop28-scala_2.11.tgz]. Now I am trying to set up Flink on the above Yarn cluster by following the steps [here|
https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/yarn_setup.html]. When I try to run /bin/yarn-session.sh -n 4 -jm 1024 -tm 4096, I am getting below error which I am unable to resolve. Can someone please advise and help me with the same?
{{Error while deploying YARN cluster: Couldn't deploy Yarn session cluster java.lang.RuntimeException: Couldn't deploy Yarn session cluster at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:372) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:679) at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:514) at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:511) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:511) Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of virtual cores per node were configured with 1 but Yarn only has -1 virtual cores available. Please note that the number of virtual cores is set to the number of task slots by default unless configured in the Flink config with 'yarn.containers.vcores.' at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:265) at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:415) at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:367)}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
