Hello!
The `CheckpointedFunction` interface was introduced to provide user functions with a context for partitioned state storage. This works for user functions that are provided directly to the operator (assuming it extends from `AbstractUdfStreamOperator`). However, I see that some operators wrap the user function in an internal function (e.g. `WrappingFunction`, `InternalWindowFunction`). This defeats the callbacks on `CheckpointedFunction` because those wrappers don’t implement `CheckpointedFunction`. Is this by design or a bug? For example, the user function passed to `WindowedStream::apply` will get callbacks for `RichFunction` but not for `CheckpointedFunction`. To fix it, the wrapper would assumedly implement `CheckpointedFunction` and contain logic similar to that found in `AbstractUdfStreamOperator` (including the handling of `ListCheckpointed`). Thanks Eron |
Hi,
you are correct, the wrappers are lacking callbacks on CheckpointedFunction. This is not the intended design and should be added. Best, Stefan > Am 05.01.2017 um 04:40 schrieb Eron Wright <[hidden email]>: > > Hello! > > The `CheckpointedFunction` interface was introduced to provide user functions with a context for partitioned state storage. This works for user functions that are provided directly to the operator (assuming it extends from `AbstractUdfStreamOperator`). However, I see that some operators wrap the user function in an internal function (e.g. `WrappingFunction`, `InternalWindowFunction`). This defeats the callbacks on `CheckpointedFunction` because those wrappers don’t implement `CheckpointedFunction`. Is this by design or a bug? > > For example, the user function passed to `WindowedStream::apply` will get callbacks for `RichFunction` but not for `CheckpointedFunction`. > > To fix it, the wrapper would assumedly implement `CheckpointedFunction` and contain logic similar to that found in `AbstractUdfStreamOperator` (including the handling of `ListCheckpointed`). > > Thanks > Eron |
Opened FLINK-5416 (unassigned).
________________________________ From: Stefan Richter <[hidden email]> Sent: Thursday, January 5, 2017 5:54:30 AM To: [hidden email] Subject: Re: CheckpointedFunction applicability? Hi, you are correct, the wrappers are lacking callbacks on CheckpointedFunction. This is not the intended design and should be added. Best, Stefan > Am 05.01.2017 um 04:40 schrieb Eron Wright <[hidden email]>: > > Hello! > > The `CheckpointedFunction` interface was introduced to provide user functions with a context for partitioned state storage. This works for user functions that are provided directly to the operator (assuming it extends from `AbstractUdfStreamOperator`). However, I see that some operators wrap the user function in an internal function (e.g. `WrappingFunction`, `InternalWindowFunction`). This defeats the callbacks on `CheckpointedFunction` because those wrappers don’t implement `CheckpointedFunction`. Is this by design or a bug? > > For example, the user function passed to `WindowedStream::apply` will get callbacks for `RichFunction` but not for `CheckpointedFunction`. > > To fix it, the wrapper would assumedly implement `CheckpointedFunction` and contain logic similar to that found in `AbstractUdfStreamOperator` (including the handling of `ListCheckpointed`). > > Thanks > Eron |
Free forum by Nabble | Edit this page |