Re: [jira] [Commented] (FLINK-7606) CEP operator leaks state

[Kostas Kloudas]

Yes. In general, and for every keyed state, the amount of state you store
is proportional to the number of active keys that you expect to have.

On Sep 15, 2017 5:33 PM, "Paolo Rendano (JIRA)" <[hidden email]> wrote:

>
>     [ https://issues.apache.org/jira/browse/FLINK-7606?page=
> com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=16168025#comment-16168025 ]
>
> Paolo Rendano commented on FLINK-7606:
> --------------------------------------
>
> Hi [~kkl0u],
> from your comment I understand that the basic memory (so with minimal load
> of messages) I have to setup for a single instance is directly related with
> the number of keys I want to be able to manage. I correctly understood, do
> I?
>
> > CEP operator leaks state
> > ------------------------
> >
> >                 Key: FLINK-7606
> >                 URL: https://issues.apache.org/jira/browse/FLINK-7606
> >             Project: Flink
> >          Issue Type: Bug
> >          Components: CEP
> >    Affects Versions: 1.3.1
> >            Reporter: Matteo Ferrario
> >         Attachments: heap-dump1.png, heap-dump2.png, heap-dump3.png
> >
> >
> > The NestedMapsStateTable grows up continuously without free the heap
> memory.
> > We created a simple job that processes a stream of messages and uses CEP
> to generate an outcome message when a specific pattern is identified.
> > The messages coming from the stream are grouped by a key defined in a
> specific field of the message.
> > We've also added the "within" clause (set as 5 minutes), indicating that
> two incoming messages match the pattern only if they come in a certain time
> window.
> > What we've seen is that for every key present in the message, an NFA
> object is instantiated in the NestedMapsStateTable and it is never
> deallocated.
> > Also the "within" clause didn't help: we've seen that if we send
> messages that don't match the pattern, the memory grows up (I suppose that
> the state of NFA is updated) but it is not cleaned also after the 5 minutes
> of time window defined in "within" clause.
> > If you need, I can provide more details about the job we've implemented
> and also the screenshots about the memory leak.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.4.14#64029)
>