Re: Kafka source stuck while canceling

Posted by Stephan Ewen on Oct 21, 2015; 5:04pm
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/Kafka-source-stuck-while-canceling-tp8494p8566.html

The Kafka consumer API has issue with being interruptible that will
hopefully get better in Kafka 0.8.3.

There must be another issue here in addition, though. Does the stacktrace
go any deeper than that?
I would assume that the main invokable thread is stuck in some blocking
method, or in a loop that does not terminate. It might also be stuck on a
lock, in which case it would be waiting for the lock holder to terminate.

Do you have the traces from other threads as well, so we could look which
one actually is stuck while holding the lock?

Greetings,
Stephan


On Mon, Oct 19, 2015 at 12:06 PM, Gyula Fóra <[hidden email]> wrote:

> Hey guys,
>
> Has anyone ever got something similar working with the kafka sources?
>
> 11:52:48,838 WARN  org.apache.flink.runtime.taskmanager.Task
>       - Task 'Source: Kafka[***] (3/4)' did not react to cancelling signal,
> but is stuck in method:
>
>  org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:208)
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:584)
> java.lang.Thread.run(Thread.java:745)
>
> The failure was caused by a different operator in the pipeline, but the job
> could never be fully cancelled and restarted due to this error.
>
> Any idea is appreciated :)
>
> Cheers,
> Gyula
>