How to use or configure flink checkpointing with siddhi internal state

[Dipanjan Mazumder]

Hi ,
   I was trying to do checkpointing while using siddhi as the CEP engine running on flink. While using siddhi windowing , it uses an internal state to aggregated or perform operation on a bucket of events pertaining to a specific time window. But what i am not sure is how can that state be mapped to Flinks internal state so that i can use flink checkpointing to safeguard the internal state of the siddhi operators in the face of failure.
Any help or pointer for this will be of great help to me.Thanks in advance.
Dipanjan -

[Till Rohrmann]

Hi Dipanjan,

I am assuming that you are using the flink-siddhi library [1]. I am not an
expert but it looks as if the AbstractSiddhiOperator overrides the
snapshotState [2] method to store the Siddhi state in Flink.

[1] https://github.com/haoch/flink-siddhi
[2]
https://github.com/haoch/flink-siddhi/blob/master/core/src/main/java/org/apache/flink/streaming/siddhi/operator/AbstractSiddhiOperator.java#L331

Cheers,
Till

On Mon, May 31, 2021 at 7:27 PM Dipanjan Mazumder <[hidden email]>
wrote:

> Hi ,
>    I was trying to do checkpointing while using siddhi as the CEP engine
> running on flink. While using siddhi windowing , it uses an internal state
> to aggregated or perform operation on a bucket of events pertaining to a
> specific time window. But what i am not sure is how can that state be
> mapped to Flinks internal state so that i can use flink checkpointing to
> safeguard the internal state of the siddhi operators in the face of failure.
> Any help or pointer for this will be of great help to me.Thanks in advance.
> Dipanjan -

[Dipanjan Mazumder]

 Hi Till,
    Thanks so that means it should work will try and see ..
RegardsDipanjan
    On Tuesday, June 1, 2021, 01:48:19 PM GMT+5:30, Till Rohrmann <[hidden email]> wrote:  
 
 Hi Dipanjan,
I am assuming that you are using the flink-siddhi library [1]. I am not an expert but it looks as if the AbstractSiddhiOperator overrides the snapshotState [2] method to store the Siddhi state in Flink.
[1] https://github.com/haoch/flink-siddhi[2] https://github.com/haoch/flink-siddhi/blob/master/core/src/main/java/org/apache/flink/streaming/siddhi/operator/AbstractSiddhiOperator.java#L331
Cheers,Till
On Mon, May 31, 2021 at 7:27 PM Dipanjan Mazumder <[hidden email]> wrote:

Hi ,
   I was trying to do checkpointing while using siddhi as the CEP engine running on flink. While using siddhi windowing , it uses an internal state to aggregated or perform operation on a bucket of events pertaining to a specific time window. But what i am not sure is how can that state be mapped to Flinks internal state so that i can use flink checkpointing to safeguard the internal state of the siddhi operators in the face of failure.
Any help or pointer for this will be of great help to me.Thanks in advance.
Dipanjan -