Hi Satish,
usually you would side-outputs [1] for that but afaik asyncIO doesn't
support that (yet).
So your option works well to use some union type. You can then chain a map
function that uses side-outputs.
[1]
https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/side_output/On Fri, Jun 11, 2021 at 7:49 PM Satish Saley <
[hidden email]> wrote:
> One way I thought to achieve this is -
> 1. For failures, add a special record to collection in RichAsyncFunction
> 2. Filter out those special records from the DataStream and push to
> another Kafka
> Let me know if it makes sense.
>
>
> On Fri, Jun 11, 2021 at 10:40 AM Satish Saley <
[hidden email]>
> wrote:
>
>> Hi,
>> - I have a kafka consumer to read events.
>> - Then, I have RichAsyncFunction to call a remote service to get
>> more information about that event.
>>
>> If the remote call fails after X number of retries, I don't want flink to
>> fail the job and start processing from the beginning. Instead I would like
>> to push info about failed call to another Kafka topic. Is there a way to
>> achieve this?
>>
>
Locked
