Force enabling checkpoints for iterative streaming jobs

classic Classic list List threaded Threaded
22 messages Options
12
mxm
Reply | Threaded
Open this post in threaded view
|

Re: Force enabling checkpoints for iterative streaming jobs

mxm
Let's mark the method of the environment as deprecated like Aljoscha
suggested. Then I think we could merge it.

On Wed, Jun 10, 2015 at 2:50 PM, Gyula Fóra <[hidden email]> wrote:

> Then I suggest we leave it in the environment along with the other
> checkpointing methods.
>
> I updated my PR so it includes hints how to force enable checkpoints (and
> the reduced guarantees) when an error is thrown for iterative jobs.
>
> On Wed, Jun 10, 2015 at 2:46 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > We could add a method on the ExecutionConfig but mark it as deprecated
> > and explain that it will go away once the interplay of iterations,
> > state and so on is properly figured out.
> >
> > On Wed, Jun 10, 2015 at 2:36 PM, Ufuk Celebi <[hidden email]> wrote:
> > > On 10 Jun 2015, at 14:29, Gyula Fóra <[hidden email]> wrote:
> > >
> > >> Max suggested that I add this feature slightly hidden to the execution
> > >> config instance.
> > >>
> > >> The problem then is that I either make a public field in the config or
> > once
> > >> again add a method.
> > >>
> > >> Any ideas?
> > >
> > > I thought about this as well. The only way to add this in a "hidden"
> way
> > would be have access to the Configuration instance – which the user does
> > not. So I think there is no way w/o adding public access.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Force enabling checkpoints for iterative streaming jobs

Gyula Fóra
Done, I will merge it after travis passes.

Maximilian Michels <[hidden email]> ezt írta (időpont: 2015. jún. 10., Sze,
15:25):

> Let's mark the method of the environment as deprecated like Aljoscha
> suggested. Then I think we could merge it.
>
> On Wed, Jun 10, 2015 at 2:50 PM, Gyula Fóra <[hidden email]> wrote:
>
> > Then I suggest we leave it in the environment along with the other
> > checkpointing methods.
> >
> > I updated my PR so it includes hints how to force enable checkpoints (and
> > the reduced guarantees) when an error is thrown for iterative jobs.
> >
> > On Wed, Jun 10, 2015 at 2:46 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> > > We could add a method on the ExecutionConfig but mark it as deprecated
> > > and explain that it will go away once the interplay of iterations,
> > > state and so on is properly figured out.
> > >
> > > On Wed, Jun 10, 2015 at 2:36 PM, Ufuk Celebi <[hidden email]> wrote:
> > > > On 10 Jun 2015, at 14:29, Gyula Fóra <[hidden email]> wrote:
> > > >
> > > >> Max suggested that I add this feature slightly hidden to the
> execution
> > > >> config instance.
> > > >>
> > > >> The problem then is that I either make a public field in the config
> or
> > > once
> > > >> again add a method.
> > > >>
> > > >> Any ideas?
> > > >
> > > > I thought about this as well. The only way to add this in a "hidden"
> > way
> > > would be have access to the Configuration instance – which the user
> does
> > > not. So I think there is no way w/o adding public access.
> > >
> >
>
12