Kafka doesn't work in very simple test

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Kafka doesn't work in very simple test

Adrian Portabales
Hi everyone,
I wrote this post in stackoverflow
http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work

What's happening?

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Aljoscha Krettek-2
Hi,
could you please provide a complete minimal example and the commands that
you use to start the cluster and run the job?

Cheers,
Aljoscha

On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <[hidden email]>
wrote:

> Hi everyone,
> I wrote this post in stackoverflow
>
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
>
> What's happening?
>
> Thanks
>
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Adrian Portabales
Other jobs works fine in Flink (like batch wordcount) but this doesn't.
I use "bin/flink run  -m analytics1:6123 -c KafkaWordCountFlink
../benchmark_Spark-Flink-assembly-1.0.jar wordCount2"

Thanks
Adrian

2016-06-17 11:10 GMT+02:00 Aljoscha Krettek <[hidden email]>:

> Hi,
> could you please provide a complete minimal example and the commands that
> you use to start the cluster and run the job?
>
> Cheers,
> Aljoscha
>
> On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <[hidden email]
> >
> wrote:
>
> > Hi everyone,
> > I wrote this post in stackoverflow
> >
> >
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
> >
> > What's happening?
> >
> > Thanks
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Robert Metzger
Hi Adrian,

you need to provide a bit more information than this to understand the
problem.
What are the taskmanager logs saying?
How do you test if data is being read from Kafka or not?

On Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales <
[hidden email]> wrote:

> Other jobs works fine in Flink (like batch wordcount) but this doesn't.
> I use "bin/flink run  -m analytics1:6123 -c KafkaWordCountFlink
> ../benchmark_Spark-Flink-assembly-1.0.jar wordCount2"
>
> Thanks
> Adrian
>
> 2016-06-17 11:10 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>
> > Hi,
> > could you please provide a complete minimal example and the commands that
> > you use to start the cluster and run the job?
> >
> > Cheers,
> > Aljoscha
> >
> > On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <
> [hidden email]
> > >
> > wrote:
> >
> > > Hi everyone,
> > > I wrote this post in stackoverflow
> > >
> > >
> >
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
> > >
> > > What's happening?
> > >
> > > Thanks
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Adrian Portabales
Hi Robert, thanks for the answer.

How do you test if data is being read from Kafka or not?
-Tested with kafka consumer script

What are the taskmanager logs saying?
-http://pastebin.com/AA2MwW5W

I think the problem is with the line 89.

Thanks!

2016-06-21 18:17 GMT+02:00 Robert Metzger <[hidden email]>:

> Hi Adrian,
>
> you need to provide a bit more information than this to understand the
> problem.
> What are the taskmanager logs saying?
> How do you test if data is being read from Kafka or not?
>
> On Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales <
> [hidden email]> wrote:
>
> > Other jobs works fine in Flink (like batch wordcount) but this doesn't.
> > I use "bin/flink run  -m analytics1:6123 -c KafkaWordCountFlink
> > ../benchmark_Spark-Flink-assembly-1.0.jar wordCount2"
> >
> > Thanks
> > Adrian
> >
> > 2016-06-17 11:10 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> >
> > > Hi,
> > > could you please provide a complete minimal example and the commands
> that
> > > you use to start the cluster and run the job?
> > >
> > > Cheers,
> > > Aljoscha
> > >
> > > On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Hi everyone,
> > > > I wrote this post in stackoverflow
> > > >
> > > >
> > >
> >
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
> > > >
> > > > What's happening?
> > > >
> > > > Thanks
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Aljoscha Krettek-2
Once again, if you could please provide some example code that would allow
us to figure out what the problem might be. From just looking at the logs
it is not obvious, at the end the job seems to be running.

On Wed, 22 Jun 2016 at 11:06 Adrian Portabales <[hidden email]>
wrote:

> Hi Robert, thanks for the answer.
>
> How do you test if data is being read from Kafka or not?
> -Tested with kafka consumer script
>
> What are the taskmanager logs saying?
> -http://pastebin.com/AA2MwW5W
>
> I think the problem is with the line 89.
>
> Thanks!
>
> 2016-06-21 18:17 GMT+02:00 Robert Metzger <[hidden email]>:
>
> > Hi Adrian,
> >
> > you need to provide a bit more information than this to understand the
> > problem.
> > What are the taskmanager logs saying?
> > How do you test if data is being read from Kafka or not?
> >
> > On Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales <
> > [hidden email]> wrote:
> >
> > > Other jobs works fine in Flink (like batch wordcount) but this doesn't.
> > > I use "bin/flink run  -m analytics1:6123 -c KafkaWordCountFlink
> > > ../benchmark_Spark-Flink-assembly-1.0.jar wordCount2"
> > >
> > > Thanks
> > > Adrian
> > >
> > > 2016-06-17 11:10 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> > >
> > > > Hi,
> > > > could you please provide a complete minimal example and the commands
> > that
> > > > you use to start the cluster and run the job?
> > > >
> > > > Cheers,
> > > > Aljoscha
> > > >
> > > > On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > > I wrote this post in stackoverflow
> > > > >
> > > > >
> > > >
> > >
> >
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
> > > > >
> > > > > What's happening?
> > > > >
> > > > > Thanks
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Kafka doesn't work in very simple test

Adrian Portabales
The code was in the link of stackoverflow:
http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work

2016-06-22 11:28 GMT+02:00 Aljoscha Krettek <[hidden email]>:

> Once again, if you could please provide some example code that would allow
> us to figure out what the problem might be. From just looking at the logs
> it is not obvious, at the end the job seems to be running.
>
> On Wed, 22 Jun 2016 at 11:06 Adrian Portabales <[hidden email]
> >
> wrote:
>
> > Hi Robert, thanks for the answer.
> >
> > How do you test if data is being read from Kafka or not?
> > -Tested with kafka consumer script
> >
> > What are the taskmanager logs saying?
> > -http://pastebin.com/AA2MwW5W
> >
> > I think the problem is with the line 89.
> >
> > Thanks!
> >
> > 2016-06-21 18:17 GMT+02:00 Robert Metzger <[hidden email]>:
> >
> > > Hi Adrian,
> > >
> > > you need to provide a bit more information than this to understand the
> > > problem.
> > > What are the taskmanager logs saying?
> > > How do you test if data is being read from Kafka or not?
> > >
> > > On Fri, Jun 17, 2016 at 11:36 AM, Adrian Portabales <
> > > [hidden email]> wrote:
> > >
> > > > Other jobs works fine in Flink (like batch wordcount) but this
> doesn't.
> > > > I use "bin/flink run  -m analytics1:6123 -c KafkaWordCountFlink
> > > > ../benchmark_Spark-Flink-assembly-1.0.jar wordCount2"
> > > >
> > > > Thanks
> > > > Adrian
> > > >
> > > > 2016-06-17 11:10 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> > > >
> > > > > Hi,
> > > > > could you please provide a complete minimal example and the
> commands
> > > that
> > > > > you use to start the cluster and run the job?
> > > > >
> > > > > Cheers,
> > > > > Aljoscha
> > > > >
> > > > > On Fri, 17 Jun 2016 at 10:53 Adrian Portabales <
> > > > [hidden email]
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi everyone,
> > > > > > I wrote this post in stackoverflow
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://stackoverflow.com/questions/37841342/flink-with-kafka-consumer-doesnt-work
> > > > > >
> > > > > > What's happening?
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > >
> > > >
> > >
> >
>