[jira] [Created] (FLINK-18398) ElasticSearch unavailibility causes TM shutdown

[Shang Yuanchun (Jira)]

Alexander Fedulov created FLINK-18398:
-----------------------------------------

             Summary: ElasticSearch unavailibility causes TM shutdown
                 Key: FLINK-18398
                 URL: https://issues.apache.org/jira/browse/FLINK-18398
             Project: Flink
          Issue Type: Bug
          Components: Connectors / ElasticSearch
    Affects Versions: 1.10.0
            Reporter: Alexander Fedulov
         Attachments: elastic_jm_log.txt, elastic_tm_log.txt

Similarly to [FLINK-17327|https://issues.apache.org/jira/browse/FLINK-17327], unavailibility of ElasticSearch cluster causes Tasks cancellation to timeout and Task Manager to be killed. The following exceptions can be found in the logs:

 
{code:java}
2020-06-15 19:52:03.664Z ERROR [  I/O dispatcher 229] .f.s.c.e.ElasticsearchSinkBase : Failed Elasticsearch bulk request: request retries exceeded max retry timeout [30000]java.io.IOException: request retries exceeded max retry timeout [30000]
...
2020-06-15 19:55:03.861Z  WARN [43df85ee0f907ae9d0).] o.a.f.r.taskmanager.Task       : Task 'graph53 (1/1)' did not react to cancelling signal for 30 seconds, but is stuck in method:
 org.elasticsearch.action.bulk.BulkProcessor.flush(BulkProcessor.java:356)
...
2020-06-15 19:55:04.120Z ERROR [663038f87ef09c4da6).] o.a.f.r.taskmanager.Task       : Task did not exit gracefully within 180 + seconds.
2020-06-15 19:55:04.121Z ERROR [663038f87ef09c4da6).] o.a.f.r.t.TaskExecutor         : Task did not exit gracefully within 180 + seconds.
2020-06-15 19:55:04.121Z ERROR [663038f87ef09c4da6).] o.a.f.r.t.TaskManagerRunner    : Fatal error occurred while executing the TaskManager. Shutting it down...
{code}

Detailed logs  are attached.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)