Prasanna,
The Flink project does not have an SQS connector, and a quick google search
hasn't found one. Nor does Flink have an HTTP sink, but with a bit of
googling you can find that various folks have implemented this themselves.
As for implementing SQS as a custom sink, if you need exactly once
guarantees I believe you will need to limit yourself to using SQS FIFO
queues, and you'll need to attach a MessageDeduplicationId to each record.
SQS offers to do deduplication for you, but only supports deduplication
within a 5 minute window -- which I would find too short to rely on during
recovery or redeployment.
See
https://stackoverflow.com/a/28111986/2000823 for more on exactly once
with SQS.
Regards,
David
On Fri, Jul 17, 2020 at 9:55 PM Prasanna kumar <
[hidden email]> wrote:
> Hi ,
>
> I did not find out of box flink sink connector for http and SQS
> mechanism.
>
> Has anyone implemented it?
> Wanted to know if we are writing a custom sink function , whether it
> would affect semantic exactly one guarantees ?
>
>
> Thanks ,
> Prasanna
>
Locked
