Hey All!
Is there a proper way of using a Flink Streaming source with event timestamps and watermarks? What I mean here is instead of implementing a custom SourceFunction, use an existing one and provide some Timestamp extractor (like the one currently used for Time windows), which will also automatically generate the watermarks based on event time (assuming that timestamps are partially ordered). Maybe I missed something, or there might be somebody working on this already :) Cheers, Gyula |
Hey Gyula,
I have been recently looking at the streaming UdfOperators and can not recall a utility for the sources that you are looking for, but maybe I am also missing it. :) It would be a convenient addition though. Best, Marton On Sun, Sep 13, 2015 at 8:59 PM, Gyula Fóra <[hidden email]> wrote: > Hey All! > > Is there a proper way of using a Flink Streaming source with event > timestamps and watermarks? What I mean here is instead of implementing a > custom SourceFunction, use an existing one and provide some Timestamp > extractor (like the one currently used for Time windows), which will also > automatically generate the watermarks based on event time (assuming that > timestamps are partially ordered). > > Maybe I missed something, or there might be somebody working on this > already :) > > Cheers, > Gyula > |
Hi,
there is nothing like that in there yet but it would be very nice to have it. :D Also, as far as I know no-one is working on this right now. Cheers, Aljoscha On Mon, 14 Sep 2015 at 06:27 Márton Balassi <[hidden email]> wrote: > Hey Gyula, > > I have been recently looking at the streaming UdfOperators and can not > recall a utility for the sources that you are looking for, but maybe I am > also missing it. :) > It would be a convenient addition though. > > Best, > > Marton > > On Sun, Sep 13, 2015 at 8:59 PM, Gyula Fóra <[hidden email]> wrote: > > > Hey All! > > > > Is there a proper way of using a Flink Streaming source with event > > timestamps and watermarks? What I mean here is instead of implementing a > > custom SourceFunction, use an existing one and provide some Timestamp > > extractor (like the one currently used for Time windows), which will also > > automatically generate the watermarks based on event time (assuming that > > timestamps are partially ordered). > > > > Maybe I missed something, or there might be somebody working on this > > already :) > > > > Cheers, > > Gyula > > > |
Then as a first step, I will open a Jira for it :)
Aljoscha Krettek <[hidden email]> ezt írta (időpont: 2015. szept. 14., H, 10:18): > Hi, > there is nothing like that in there yet but it would be very nice to have > it. :D Also, as far as I know no-one is working on this right now. > > Cheers, > Aljoscha > > On Mon, 14 Sep 2015 at 06:27 Márton Balassi <[hidden email]> > wrote: > > > Hey Gyula, > > > > I have been recently looking at the streaming UdfOperators and can not > > recall a utility for the sources that you are looking for, but maybe I am > > also missing it. :) > > It would be a convenient addition though. > > > > Best, > > > > Marton > > > > On Sun, Sep 13, 2015 at 8:59 PM, Gyula Fóra <[hidden email]> wrote: > > > > > Hey All! > > > > > > Is there a proper way of using a Flink Streaming source with event > > > timestamps and watermarks? What I mean here is instead of implementing > a > > > custom SourceFunction, use an existing one and provide some Timestamp > > > extractor (like the one currently used for Time windows), which will > also > > > automatically generate the watermarks based on event time (assuming > that > > > timestamps are partially ordered). > > > > > > Maybe I missed something, or there might be somebody working on this > > > already :) > > > > > > Cheers, > > > Gyula > > > > > > |
Free forum by Nabble | Edit this page |