Hi all,
Recently, I was tracing the source code in streaming api and I was confused about some implementations. When using reduce function with evictor, the *WindowStream* will wrap the *ReduceFunction* and *ProcessWindowFunction* into *ReduceApplyProcessWindonwFunction* and put it in *InternalIterableProcessWindowFunction*. So does fold function. However, when using aggregate, the *InternalIterableProcessWindowFunction* was changed to *InternalAggregateProcessWindowFunction* which was applied aggregation in the process() method. My question is why not implement an *AggregateApplyProcessWindowFunction* and use *InternalIterableProcessWindowFunction* instead just like reduce, fold function did. Is there any concern? Many thanks, Tony Wei |
Hi Tony,
The reason for this is that the aggregate code is newer. The new code has less layers, compared to the reduce/fold implementation where it is InternalFunction(ReduceApplyFunction(Reduce)) instead of InteralAggregateFunction(Aggregate). Best, Aljoscha > On 26. Apr 2017, at 06:39, 魏偉哲 <[hidden email]> wrote: > > Hi all, > > Recently, I was tracing the source code in streaming api and I was confused > about some implementations. > > When using reduce function with evictor, the *WindowStream* will wrap the > *ReduceFunction* and *ProcessWindowFunction* into > *ReduceApplyProcessWindonwFunction* and put it in > *InternalIterableProcessWindowFunction*. So does fold function. > > However, when using aggregate, the *InternalIterableProcessWindowFunction* > was changed to *InternalAggregateProcessWindowFunction* which was applied > aggregation in the process() method. > > My question is why not implement an *AggregateApplyProcessWindowFunction* > and use *InternalIterableProcessWindowFunction* instead just like reduce, > fold function did. Is there any concern? > > Many thanks, > Tony Wei |
Hi Aljoscha,
I know the aggregate code is newer. I am confused because the implementations are not consistent. Does it mean that the reduce/fold implementation would need to be refactored for the purpose of having less layers ? Or is it better to remain the current implementations for some reasons? Many thanks, Tony Wei 2017-04-26 20:24 GMT+08:00 Aljoscha Krettek <[hidden email]>: > Hi Tony, > The reason for this is that the aggregate code is newer. The new code has > less layers, compared to the reduce/fold implementation where it is > InternalFunction(ReduceApplyFunction(Reduce)) instead of > InteralAggregateFunction(Aggregate). > > Best, > Aljoscha > > On 26. Apr 2017, at 06:39, 魏偉哲 <[hidden email]> wrote: > > > > Hi all, > > > > Recently, I was tracing the source code in streaming api and I was > confused > > about some implementations. > > > > When using reduce function with evictor, the *WindowStream* will wrap the > > *ReduceFunction* and *ProcessWindowFunction* into > > *ReduceApplyProcessWindonwFunction* and put it in > > *InternalIterableProcessWindowFunction*. So does fold function. > > > > However, when using aggregate, the *InternalIterableProcessWindowF > unction* > > was changed to *InternalAggregateProcessWindowFunction* which was > applied > > aggregation in the process() method. > > > > My question is why not implement an *AggregateApplyProcessWindowFun > ction* > > and use *InternalIterableProcessWindowFunction* instead just like > reduce, > > fold function did. Is there any concern? > > > > Many thanks, > > Tony Wei > > |
Hi,
Both implementations work so no one bothered to change the older implementations yet. I don’t think it’s a problem but if you want you can adapt reduce/fold to the newer implementation. Best, Aljoscha > On 26. Apr 2017, at 14:51, 魏偉哲 <[hidden email]> wrote: > > Hi Aljoscha, > > I know the aggregate code is newer. I am confused because the > implementations are not consistent. > Does it mean that the reduce/fold implementation would need to be > refactored for the purpose of having less layers ? > Or is it better to remain the current implementations for some reasons? > > Many thanks, > Tony Wei > > 2017-04-26 20:24 GMT+08:00 Aljoscha Krettek <[hidden email]>: > >> Hi Tony, >> The reason for this is that the aggregate code is newer. The new code has >> less layers, compared to the reduce/fold implementation where it is >> InternalFunction(ReduceApplyFunction(Reduce)) instead of >> InteralAggregateFunction(Aggregate). >> >> Best, >> Aljoscha >>> On 26. Apr 2017, at 06:39, 魏偉哲 <[hidden email]> wrote: >>> >>> Hi all, >>> >>> Recently, I was tracing the source code in streaming api and I was >> confused >>> about some implementations. >>> >>> When using reduce function with evictor, the *WindowStream* will wrap the >>> *ReduceFunction* and *ProcessWindowFunction* into >>> *ReduceApplyProcessWindonwFunction* and put it in >>> *InternalIterableProcessWindowFunction*. So does fold function. >>> >>> However, when using aggregate, the *InternalIterableProcessWindowF >> unction* >>> was changed to *InternalAggregateProcessWindowFunction* which was >> applied >>> aggregation in the process() method. >>> >>> My question is why not implement an *AggregateApplyProcessWindowFun >> ction* >>> and use *InternalIterableProcessWindowFunction* instead just like >> reduce, >>> fold function did. Is there any concern? >>> >>> Many thanks, >>> Tony Wei >> >> |
Hi Aljoscha,
I see. Thanks for your reply. Best, Tony Wei Aljoscha Krettek <[hidden email]>於 2017年4月26日 週三,下午10:29寫道: > Hi, > Both implementations work so no one bothered to change the older > implementations yet. I don’t think it’s a problem but if you want you can > adapt reduce/fold to the newer implementation. > > Best, > Aljoscha > > On 26. Apr 2017, at 14:51, 魏偉哲 <[hidden email]> wrote: > > > > Hi Aljoscha, > > > > I know the aggregate code is newer. I am confused because the > > implementations are not consistent. > > Does it mean that the reduce/fold implementation would need to be > > refactored for the purpose of having less layers ? > > Or is it better to remain the current implementations for some reasons? > > > > Many thanks, > > Tony Wei > > > > 2017-04-26 20:24 GMT+08:00 Aljoscha Krettek <[hidden email]>: > > > >> Hi Tony, > >> The reason for this is that the aggregate code is newer. The new code > has > >> less layers, compared to the reduce/fold implementation where it is > >> InternalFunction(ReduceApplyFunction(Reduce)) instead of > >> InteralAggregateFunction(Aggregate). > >> > >> Best, > >> Aljoscha > >>> On 26. Apr 2017, at 06:39, 魏偉哲 <[hidden email]> wrote: > >>> > >>> Hi all, > >>> > >>> Recently, I was tracing the source code in streaming api and I was > >> confused > >>> about some implementations. > >>> > >>> When using reduce function with evictor, the *WindowStream* will wrap > the > >>> *ReduceFunction* and *ProcessWindowFunction* into > >>> *ReduceApplyProcessWindonwFunction* and put it in > >>> *InternalIterableProcessWindowFunction*. So does fold function. > >>> > >>> However, when using aggregate, the *InternalIterableProcessWindowF > >> unction* > >>> was changed to *InternalAggregateProcessWindowFunction* which was > >> applied > >>> aggregation in the process() method. > >>> > >>> My question is why not implement an *AggregateApplyProcessWindowFun > >> ction* > >>> and use *InternalIterableProcessWindowFunction* instead just like > >> reduce, > >>> fold function did. Is there any concern? > >>> > >>> Many thanks, > >>> Tony Wei > >> > >> > > |
Free forum by Nabble | Edit this page |