Problem to deploy of flink program in cluster

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem to deploy of flink program in cluster

simone sguazza

Hi, I'm a student and I using flink v1.2.1 to develop a workflow to analyze data coming from kafka v0.10.


At the moment everything work fine, from eclipse I can export the runnable jar with dependencies and with "sudo java -jar flinkApp.jar" I can run the flink application on ubuntu server.

Until here all ok.

I configured flink to be in High Avaliabity mode, it's start correcly.

When I try to export with maven, it create a fat jar, i launch that with "sudo bin/flink run flinkApp.jar". It's work and on the dashboard I can the see the job is running. The problem is that is not receive a single data from kafka. The cuorious thing is that only when I export the jar in order to run in cluster mode I've this problem, whereas I export a runnable jar everything work fine.


I put in attachment three files:

- webUI_producer: it shows the flink dashboard with my kafka producer

- problemFlinkCluster: it shows the ubuntu server where my flink application run. The top left square is zookeeper (from kafka), the bottom left is the flink program running with "sudo java -jar flinkapp.jar" and the right most column is the execution of flink app with "sudo bin/flink run appFlink.jar". You can see The flink application start and than do nothing more.

- the pom.xml


I think the problem is in the pom.xml but I don't know where.


Best regards, simone


pom.xml (19K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problem to deploy of flink program in cluster

郭健
Hi Simone,

the output of your app is on the task manager in cluster mode, not in client side.

发自我的 iPad

在 2017年7月3日,23:12,simone sguazza <[hidden email]<mailto:[hidden email]>> 写道:


Hi, I'm a student and I using flink v1.2.1 to develop a workflow to analyze data coming from kafka v0.10.


At the moment everything work fine, from eclipse I can export the runnable jar with dependencies and with "sudo java -jar flinkApp.jar" I can run the flink application on ubuntu server.

Until here all ok.

I configured flink to be in High Avaliabity mode, it's start correcly.

When I try to export with maven, it create a fat jar, i launch that with "sudo bin/flink run flinkApp.jar". It's work and on the dashboard I can the see the job is running. The problem is that is not receive a single data from kafka. The cuorious thing is that only when I export the jar in order to run in cluster mode I've this problem, whereas I export a runnable jar everything work fine.


I put in attachment three files:

- webUI_producer: it shows the flink dashboard with my kafka producer

- problemFlinkCluster: it shows the ubuntu server where my flink application run. The top left square is zookeeper (from kafka), the bottom left is the flink program running with "sudo java -jar flinkapp.jar" and the right most column is the execution of flink app with "sudo bin/flink run appFlink.jar". You can see The flink application start and than do nothing more.

- the pom.xml


I think the problem is in the pom.xml but I don't know where.


Best regards, simone

<pom.xml>
Reply | Threaded
Open this post in threaded view
|

Problem to deploy of flink program in cluster

simone sguazza
In reply to this post by simone sguazza

Hi, below you can find my mail and my problem before the my subscribe to mailing list.


simone


Da: simone sguazza <[hidden email]>
Inviato: lunedì 3 luglio 2017 11:57
A: [hidden email]
Oggetto: Problem to deploy of flink program in cluster
 

Hi, I'm a student and I using flink v1.2.1 to develop a workflow to analyze data coming from kafka v0.10.


At the moment everything work fine, from eclipse I can export the runnable jar with dependencies and with "sudo java -jar flinkApp.jar" I can run the flink application on ubuntu server.

Until here all ok.

I configured flink to be in High Avaliabity mode, it's start correcly.

When I try to export with maven, it create a fat jar, i launch that with "sudo bin/flink run flinkApp.jar". It's work and on the dashboard I can the see the job is running. The problem is that is not receive a single data from kafka. The cuorious thing is that only when I export the jar in order to run in cluster mode I've this problem, whereas I export a runnable jar everything work fine.


I put in attachment three files:

- webUI_producer: it shows the flink dashboard with my kafka producer

- problemFlinkCluster: it shows the ubuntu server where my flink application run. The top left square is zookeeper (from kafka), the bottom left is the flink program running with "sudo java -jar flinkapp.jar" and the right most column is the execution of flink app with "sudo bin/flink run appFlink.jar". You can see The flink application start and than do nothing more.

- the pom.xml


I think the problem is in the pom.xml but I don't know where.


Best regards, simone


pom.xml (19K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problem to deploy of flink program in cluster

Tzu-Li (Gordon) Tai
Hi Simone,

Is the job actually running? Or did it fail with some exception? If it failed, there should be some error message in the JM logs.
Could you take a look at that and see if it hints anything?

Cheers,
Gordon


On 4 July 2017 at 3:35:07 PM, simone sguazza ([hidden email]) wrote:

Hi, below you can find my mail and my problem before the my subscribe to mailing list.


simone

Da: simone sguazza <[hidden email]>
Inviato: lunedì 3 luglio 2017 11:57
A: [hidden email]
Oggetto: Problem to deploy of flink program in cluster
 
Hi, I'm a student and I using flink v1.2.1 to develop a workflow to analyze data coming from kafka v0.10.



At the moment everything work fine, from eclipse I can export the runnable jar with dependencies and with "sudo java -jar flinkApp.jar" I can run the flink application on ubuntu server.

Until here all ok.

I configured flink to be in High Avaliabity mode, it's start correcly.

When I try to export with maven, it create a fat jar, i launch that with "sudo bin/flink run flinkApp.jar". It's work and on the dashboard I can the see the job is running. The problem is that is not receive a single data from kafka. The cuorious thing is that only when I export the jar in order to run in cluster mode I've this problem, whereas I export a runnable jar everything work fine.



I put in attachment three files:

- webUI_producer: it shows the flink dashboard with my kafka producer

- problemFlinkCluster: it shows the ubuntu server where my flink application run. The top left square is zookeeper (from kafka), the bottom left is the flink program running with "sudo java -jar flinkapp.jar" and the right most column is the execution of flink app with "sudo bin/flink run appFlink.jar". You can see The flink application start and than do nothing more.

- the pom.xml



I think the problem is in the pom.xml but I don't know where.



Best regards, simone
Reply | Threaded
Open this post in threaded view
|

Re: Problem to deploy of flink program in cluster

Ted Yu
In reply to this post by simone sguazza
I only see pom.xml in the attachment. 
Consider using pastebin for the other files. 
-------- Original message --------From: simone sguazza <[hidden email]> Date: 7/4/17  12:34 AM  (GMT-08:00) To: [hidden email] Subject: Problem to deploy of flink program in cluster


Hi, below you can find my mail and my problem before the my subscribe to mailing list.





simone




Da: simone sguazza <[hidden email]>

Inviato: lunedì 3 luglio 2017 11:57

A: [hidden email]

Oggetto: Problem to deploy of flink program in cluster
 



Hi,
 I'm a student and I using flink v1.2.1 to develop a workflow to analyze data coming from kafka v0.10.




















At the moment everything work fine, from eclipse I can export the runnable jar with dependencies and with "sudo java -jar flinkApp.jar" I can run the flink application on ubuntu server.
Until here all ok.
I configured flink to be in High Avaliabity mode, it's start correcly.
When I try to export with maven, it create a fat jar, i launch that with "sudo bin/flink run flinkApp.jar". It's work and on the dashboard I can the see the job is running. The problem is that is not receive a single data from kafka. The cuorious thing is
 that only when I export the jar in order to run in cluster mode I've this problem, whereas I export a runnable jar everything work fine.



I put in attachment three files:
- webUI_producer: it shows the flink dashboard with my kafka producer
- problemFlinkCluster: it shows the ubuntu server where my flink application run. The top left square is zookeeper (from kafka), the bottom left is the flink program running with "sudo java -jar flinkapp.jar" and the right most column is the execution of
 flink app with "sudo bin/flink run appFlink.jar". You can see The flink application start and than do nothing more.
- the pom.xml



I think the problem is in the pom.xml but I don't know where.



Best regards, simone