[
https://issues.apache.org/jira/browse/FLINK-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038671#comment-14038671 ]
Tobias commented on FLINK-959:
------------------------------
Bug: When security groups are created access from slave to slave group is not granted. Even if requested with boto.
Solution: Allow all UDP and TCP traffic from slave group to slave group manually via the web interface.
> Automated bare-metal deployment of FLINK on Amazon EC2 and OpenStack instances
> ------------------------------------------------------------------------------
>
> Key: FLINK-959
> URL:
https://issues.apache.org/jira/browse/FLINK-959> Project: Flink
> Issue Type: New Feature
> Affects Versions: pre-apache-0.5
> Reporter: Tobias
> Assignee: Tobias
> Fix For: pre-apache-0.5
>
>
> This python script does start Amazon ec2|OpenStack instances to install java+hadoop and configure hdfs/yarn via puppet. In order to run FLINK on top of hadoop YARN.
> In order to install java and hadoop binaries are downloaded by the script and handed over to puppet for automated provisioning.
> User-data scripts are used to install puppet (only debian) on the master and slave instances. Accordingly security groups are created and configured.
> The master instance then starts a self configuration process, so that the puppet modules are set up according to the cluster structure.
> The master detects if hadoop YARN web interface is accessible and waits for all expected nodes to be up and running. Then a stratosphere yarn session is started. Taskmanager and Jobmanager memory allocations are set up in the instances.cfg.
> Notes:
> - Configuration reserves 600mb for the operating system and allocates the rest for the YARN node.
> - The Flink web interface is not accessible because the yarn.web.proxy throws a NullpointerException
> - Only runs on Debian derivatives because it uses apt-get
> - Tested with ubuntu-13.08
> - FLINK is still named Stratosphere
> Code at:
https://github.com/tobwiens/StratopshereBareMetalProvPuppet--
This message was sent by Atlassian JIRA
(v6.2#6252)
Locked
