[jira] [Created] (FLINK-21379) Elasticsearch (v6.3.1) sink end-to-end test fails on azure

[Shang Yuanchun (Jira)]

Dawid Wysakowicz created FLINK-21379:
----------------------------------------

             Summary: Elasticsearch (v6.3.1) sink end-to-end test fails on azure
                 Key: FLINK-21379
                 URL: https://issues.apache.org/jira/browse/FLINK-21379
             Project: Flink
          Issue Type: Bug
          Components: Connectors / ElasticSearch, Runtime / Checkpointing, Tests
    Affects Versions: 1.11.3
            Reporter: Dawid Wysakowicz


https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=13356&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=2b7514ee-e706-5046-657b-3430666e7bd9

{code}
2021-02-15 23:15:39,939 ERROR org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase [] - Failed Elasticsearch item request: [:intentional invalid index:] ElasticsearchException[Elasticsearch exception [type=invalid_index_name_exception, reason=Invalid index name [:intentional invalid index:], must not contain the following characters [ , ", *, \, <, |, ,, >, /, ?]]]
org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=invalid_index_name_exception, reason=Invalid index name [:intentional invalid index:], must not contain the following characters [ , ", *, \, <, |, ,, >, /, ?]]
        at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:510) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:421) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:135) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:198) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:653) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$3(RestHighLevelClient.java:549) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:580) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:621) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:375) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:366) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
2021-02-15 23:15:39,979 ERROR org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase [] - Failed Elasticsearch item request: [:intentional invalid index:] ElasticsearchException[Elasticsearch exception [type=invalid_index_name_exception, reason=Invalid index name [:intentional invalid index:], must not contain the following characters [ , ", *, \, <, |, ,, >, /, ?]]]
org.elasticsearch.ElasticsearchException: Elasticsearch exception [type=invalid_index_name_exception, reason=Invalid index name [:intentional invalid index:], must not contain the following characters [ , ", *, \, <, |, ,, >, /, ?]]
        at org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:510) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:421) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.action.bulk.BulkItemResponse.fromXContent(BulkItemResponse.java:135) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.action.bulk.BulkResponse.fromXContent(BulkResponse.java:198) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:653) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAsyncAndParseEntity$3(RestHighLevelClient.java:549) ~[blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestHighLevelClient$1.onSuccess(RestHighLevelClient.java:580) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:621) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:375) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.elasticsearch.client.RestClient$1.completed(RestClient.java:366) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) [blob_p-1cb3213b045f842a05dea3132500fab00b6335e6-dcafee2d0ae53e4014367dc963a9c5ba:1.11-SNAPSHOT]
        at org.apache.flink.runtime.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:501) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:60) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        at org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:113) ~[flink-dist_2.11-1.11-SNAPSHOT.jar:1.11-SNAPSHOT]
        ... 3 more
{code}

I am not entirely sure what is the root cause. It might be something in the elasticsearch connector, but the first exception fails for index that's called: "intentionally invalid".

The second exception is about checkpointing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)