[DISCUSS] Support notFollowedBy with interval as the last part of a Pattern

[Shuai Xu]

Hi all,
CEP is broadly used in more and more applications now. In may cases, users
need to use the pattern CEP.begin().notFollowedBy(). For example, they may
want to get the uses who created an oder but didn't pay in 10 minutes and
so on.

However, CEP doesn't support notFollowBy() as the last part of a pattern
now. So I propose to enable it as following:

If the pattern is ended with notFollowBy() with a time interval within(t),
we take it as a valid pattern. This pattern will be triggered after time t
from the begin stage if the previous pattern is matched and the
notFollowBy() pattern doesn't appear during the interval.

For example, Pattern.begin<Event>("start").where(event.getId() ==
1).notFollowBy("not").where(event.getId() == 2).within(Time.minutes(10)) is
a valid pattern now.
If the ids of the input events are 1, 3, 3..., then after 10 minutes from
getting event with id 1, it will get a match with 1.

This change will not add any new public interface, it only makes some
patterns not to be invalid any more.

The detail implement design is in:
https://docs.google.com/document/d/1swUSHcVxbkWm7EPdOfOQXWj-A4gGDA8Y8R1DOUjokds/edit#

Similar requirements from users can be found in:
https://issues.apache.org/jira/browse/FLINK-9431?filter=12347662

Please let me know if you have any questions or suggestions to improve this
proposal.

[dwysakowicz]

Hi Shuai,

First of all let me apologize for a late reply. Unfortunately I don't
have enough capacity to properly review and help with the proposal at
this time. If there is another committer in the community willing to
shepherd the proposal feel free to proceed with it.

Another suggestion I may have is you could try with hosting a fork of
the library in flink-packages.org to see if it attracts the interest of
users.

Best,

Dawid

On 11/02/2020 10:10, Shuai Xu wrote:

> Hi all,
> CEP is broadly used in more and more applications now. In may cases, users
> need to use the pattern CEP.begin().notFollowedBy(). For example, they may
> want to get the uses who created an oder but didn't pay in 10 minutes and
> so on.
>
> However, CEP doesn't support notFollowBy() as the last part of a pattern
> now. So I propose to enable it as following:
>
> If the pattern is ended with notFollowBy() with a time interval within(t),
> we take it as a valid pattern. This pattern will be triggered after time t
> from the begin stage if the previous pattern is matched and the
> notFollowBy() pattern doesn't appear during the interval.
>
> For example, Pattern.begin<Event>("start").where(event.getId() ==
> 1).notFollowBy("not").where(event.getId() == 2).within(Time.minutes(10)) is
> a valid pattern now.
> If the ids of the input events are 1, 3, 3..., then after 10 minutes from
> getting event with id 1, it will get a match with 1.
>
> This change will not add any new public interface, it only makes some
> patterns not to be invalid any more.
>
> The detail implement design is in:
> https://docs.google.com/document/d/1swUSHcVxbkWm7EPdOfOQXWj-A4gGDA8Y8R1DOUjokds/edit#
>
> Similar requirements from users can be found in:
> https://issues.apache.org/jira/browse/FLINK-9431?filter=12347662
>
> Please let me know if you have any questions or suggestions to improve this
> proposal.
>