Error with window fold

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Error with window fold

Gyula Fóra-2
Hey,

Running the following simple application gives me an error:

//just counting by key, the
streamOfIntegers.keyBy(x -> x).timeWindow(Time.milliseconds(3000)).fold(0, (
c, next) -> c + 1).print();

Executing this gives the following error:

"No initial value was serialized for the fold window function. Probably the
setOutputType method was not called."

Any idea?

Cheers,

Gyula
mxm
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

mxm
Hi Gyula,

Trying to reproduce this error now. I'm assuming this is 0.10-SNAPSHOT?

Cheers,
Max

On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]> wrote:

> Hey,
>
> Running the following simple application gives me an error:
>
> //just counting by key, the
> streamOfIntegers.keyBy(x -> x).timeWindow(Time.milliseconds(3000)).fold(0, (
> c, next) -> c + 1).print();
>
> Executing this gives the following error:
>
> "No initial value was serialized for the fold window function. Probably the
> setOutputType method was not called."
>
> Any idea?
>
> Cheers,
>
> Gyula
mxm
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

mxm
It's a bug. It also occurs in the Java API. Perhaps we can find a fix
for the release..

On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]> wrote:

> Hi Gyula,
>
> Trying to reproduce this error now. I'm assuming this is 0.10-SNAPSHOT?
>
> Cheers,
> Max
>
> On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]> wrote:
>> Hey,
>>
>> Running the following simple application gives me an error:
>>
>> //just counting by key, the
>> streamOfIntegers.keyBy(x -> x).timeWindow(Time.milliseconds(3000)).fold(0, (
>> c, next) -> c + 1).print();
>>
>> Executing this gives the following error:
>>
>> "No initial value was serialized for the fold window function. Probably the
>> setOutputType method was not called."
>>
>> Any idea?
>>
>> Cheers,
>>
>> Gyula
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

Gyula Fóra
This was java 8, snapshot 1.0 :)

Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. nov. 4., Sze,
14:47):

> It's a bug. It also occurs in the Java API. Perhaps we can find a fix
> for the release..
>
> On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]> wrote:
> > Hi Gyula,
> >
> > Trying to reproduce this error now. I'm assuming this is 0.10-SNAPSHOT?
> >
> > Cheers,
> > Max
> >
> > On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]> wrote:
> >> Hey,
> >>
> >> Running the following simple application gives me an error:
> >>
> >> //just counting by key, the
> >> streamOfIntegers.keyBy(x ->
> x).timeWindow(Time.milliseconds(3000)).fold(0, (
> >> c, next) -> c + 1).print();
> >>
> >> Executing this gives the following error:
> >>
> >> "No initial value was serialized for the fold window function. Probably
> the
> >> setOutputType method was not called."
> >>
> >> Any idea?
> >>
> >> Cheers,
> >>
> >> Gyula
>
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

Till Rohrmann
The error should also be in 0.10. The problem is that due to the latest
stream refactorings we not only have fold operations which are
StreamOperators but also WindowFunctions (if you apply them on windows).
For the StreamOperators the output type is correctly set in the StreamGraph
and thus the initial value can be serialized. For the latter case the
function is part of the AccumulatingProcessingTimeWindowOperator which does
not implement the OutputTypeConfigurable interface. However, the
FoldWindowFunction does, which does not make sense at the moment, because
the WindowFunctions are not checked whether they implement the
OutputTypeConfigurable interface or not. I think this has to be added.

Cheers,

Till


On Wed, Nov 4, 2015 at 2:54 PM, Gyula Fóra <[hidden email]> wrote:

> This was java 8, snapshot 1.0 :)
>
> Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. nov. 4., Sze,
> 14:47):
>
> > It's a bug. It also occurs in the Java API. Perhaps we can find a fix
> > for the release..
> >
> > On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]>
> wrote:
> > > Hi Gyula,
> > >
> > > Trying to reproduce this error now. I'm assuming this is 0.10-SNAPSHOT?
> > >
> > > Cheers,
> > > Max
> > >
> > > On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]> wrote:
> > >> Hey,
> > >>
> > >> Running the following simple application gives me an error:
> > >>
> > >> //just counting by key, the
> > >> streamOfIntegers.keyBy(x ->
> > x).timeWindow(Time.milliseconds(3000)).fold(0, (
> > >> c, next) -> c + 1).print();
> > >>
> > >> Executing this gives the following error:
> > >>
> > >> "No initial value was serialized for the fold window function.
> Probably
> > the
> > >> setOutputType method was not called."
> > >>
> > >> Any idea?
> > >>
> > >> Cheers,
> > >>
> > >> Gyula
> >
>
mxm
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

mxm
That's basically what I also found out too so far. If you want to fix
it please go ahead.

On Wed, Nov 4, 2015 at 3:11 PM, Till Rohrmann <[hidden email]> wrote:

> The error should also be in 0.10. The problem is that due to the latest
> stream refactorings we not only have fold operations which are
> StreamOperators but also WindowFunctions (if you apply them on windows).
> For the StreamOperators the output type is correctly set in the StreamGraph
> and thus the initial value can be serialized. For the latter case the
> function is part of the AccumulatingProcessingTimeWindowOperator which does
> not implement the OutputTypeConfigurable interface. However, the
> FoldWindowFunction does, which does not make sense at the moment, because
> the WindowFunctions are not checked whether they implement the
> OutputTypeConfigurable interface or not. I think this has to be added.
>
> Cheers,
>
> Till
>
>
> On Wed, Nov 4, 2015 at 2:54 PM, Gyula Fóra <[hidden email]> wrote:
>
>> This was java 8, snapshot 1.0 :)
>>
>> Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. nov. 4., Sze,
>> 14:47):
>>
>> > It's a bug. It also occurs in the Java API. Perhaps we can find a fix
>> > for the release..
>> >
>> > On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]>
>> wrote:
>> > > Hi Gyula,
>> > >
>> > > Trying to reproduce this error now. I'm assuming this is 0.10-SNAPSHOT?
>> > >
>> > > Cheers,
>> > > Max
>> > >
>> > > On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]> wrote:
>> > >> Hey,
>> > >>
>> > >> Running the following simple application gives me an error:
>> > >>
>> > >> //just counting by key, the
>> > >> streamOfIntegers.keyBy(x ->
>> > x).timeWindow(Time.milliseconds(3000)).fold(0, (
>> > >> c, next) -> c + 1).print();
>> > >>
>> > >> Executing this gives the following error:
>> > >>
>> > >> "No initial value was serialized for the fold window function.
>> Probably
>> > the
>> > >> setOutputType method was not called."
>> > >>
>> > >> Any idea?
>> > >>
>> > >> Cheers,
>> > >>
>> > >> Gyula
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

Aljoscha Krettek-2
Ahh, there are no fold tests for the aligned time window operators.

If you use Ingestion time or event time as stream characteristic it works
correctly.

On Wed, Nov 4, 2015, 15:23 Maximilian Michels <[hidden email]> wrote:

> That's basically what I also found out too so far. If you want to fix
> it please go ahead.
>
> On Wed, Nov 4, 2015 at 3:11 PM, Till Rohrmann <[hidden email]>
> wrote:
> > The error should also be in 0.10. The problem is that due to the latest
> > stream refactorings we not only have fold operations which are
> > StreamOperators but also WindowFunctions (if you apply them on windows).
> > For the StreamOperators the output type is correctly set in the
> StreamGraph
> > and thus the initial value can be serialized. For the latter case the
> > function is part of the AccumulatingProcessingTimeWindowOperator which
> does
> > not implement the OutputTypeConfigurable interface. However, the
> > FoldWindowFunction does, which does not make sense at the moment, because
> > the WindowFunctions are not checked whether they implement the
> > OutputTypeConfigurable interface or not. I think this has to be added.
> >
> > Cheers,
> >
> > Till
> >
> >
> > On Wed, Nov 4, 2015 at 2:54 PM, Gyula Fóra <[hidden email]> wrote:
> >
> >> This was java 8, snapshot 1.0 :)
> >>
> >> Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. nov. 4.,
> Sze,
> >> 14:47):
> >>
> >> > It's a bug. It also occurs in the Java API. Perhaps we can find a fix
> >> > for the release..
> >> >
> >> > On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]>
> >> wrote:
> >> > > Hi Gyula,
> >> > >
> >> > > Trying to reproduce this error now. I'm assuming this is
> 0.10-SNAPSHOT?
> >> > >
> >> > > Cheers,
> >> > > Max
> >> > >
> >> > > On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]>
> wrote:
> >> > >> Hey,
> >> > >>
> >> > >> Running the following simple application gives me an error:
> >> > >>
> >> > >> //just counting by key, the
> >> > >> streamOfIntegers.keyBy(x ->
> >> > x).timeWindow(Time.milliseconds(3000)).fold(0, (
> >> > >> c, next) -> c + 1).print();
> >> > >>
> >> > >> Executing this gives the following error:
> >> > >>
> >> > >> "No initial value was serialized for the fold window function.
> >> Probably
> >> > the
> >> > >> setOutputType method was not called."
> >> > >>
> >> > >> Any idea?
> >> > >>
> >> > >> Cheers,
> >> > >>
> >> > >> Gyula
> >> >
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: Error with window fold

Till Rohrmann
I have a fix for it. Will open a PR.

On Wed, Nov 4, 2015 at 3:31 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Ahh, there are no fold tests for the aligned time window operators.
>
> If you use Ingestion time or event time as stream characteristic it works
> correctly.
>
> On Wed, Nov 4, 2015, 15:23 Maximilian Michels <[hidden email]> wrote:
>
> > That's basically what I also found out too so far. If you want to fix
> > it please go ahead.
> >
> > On Wed, Nov 4, 2015 at 3:11 PM, Till Rohrmann <[hidden email]>
> > wrote:
> > > The error should also be in 0.10. The problem is that due to the latest
> > > stream refactorings we not only have fold operations which are
> > > StreamOperators but also WindowFunctions (if you apply them on
> windows).
> > > For the StreamOperators the output type is correctly set in the
> > StreamGraph
> > > and thus the initial value can be serialized. For the latter case the
> > > function is part of the AccumulatingProcessingTimeWindowOperator which
> > does
> > > not implement the OutputTypeConfigurable interface. However, the
> > > FoldWindowFunction does, which does not make sense at the moment,
> because
> > > the WindowFunctions are not checked whether they implement the
> > > OutputTypeConfigurable interface or not. I think this has to be added.
> > >
> > > Cheers,
> > >
> > > Till
> > >
> > >
> > > On Wed, Nov 4, 2015 at 2:54 PM, Gyula Fóra <[hidden email]>
> wrote:
> > >
> > >> This was java 8, snapshot 1.0 :)
> > >>
> > >> Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. nov. 4.,
> > Sze,
> > >> 14:47):
> > >>
> > >> > It's a bug. It also occurs in the Java API. Perhaps we can find a
> fix
> > >> > for the release..
> > >> >
> > >> > On Wed, Nov 4, 2015 at 2:40 PM, Maximilian Michels <[hidden email]>
> > >> wrote:
> > >> > > Hi Gyula,
> > >> > >
> > >> > > Trying to reproduce this error now. I'm assuming this is
> > 0.10-SNAPSHOT?
> > >> > >
> > >> > > Cheers,
> > >> > > Max
> > >> > >
> > >> > > On Wed, Nov 4, 2015 at 1:49 PM, Gyula Fóra <[hidden email]>
> > wrote:
> > >> > >> Hey,
> > >> > >>
> > >> > >> Running the following simple application gives me an error:
> > >> > >>
> > >> > >> //just counting by key, the
> > >> > >> streamOfIntegers.keyBy(x ->
> > >> > x).timeWindow(Time.milliseconds(3000)).fold(0, (
> > >> > >> c, next) -> c + 1).print();
> > >> > >>
> > >> > >> Executing this gives the following error:
> > >> > >>
> > >> > >> "No initial value was serialized for the fold window function.
> > >> Probably
> > >> > the
> > >> > >> setOutputType method was not called."
> > >> > >>
> > >> > >> Any idea?
> > >> > >>
> > >> > >> Cheers,
> > >> > >>
> > >> > >> Gyula
> > >> >
> > >>
> >
>