DagConnection tempMode vs breakPipeline property

classic Classic list List threaded Threaded
3 messages Options
CPC
Reply | Threaded
Open this post in threaded view
|

DagConnection tempMode vs breakPipeline property

CPC
Hi everybody,

I am trying to understand flink optimizer in DataSet api. But i dont
understant tempMode and breakPipeline properties. TempMode enum also has a
breakPipeline property but they both accessed and set in different places.
DagConnection.breakPipeline is used to select network io DataExchangeMode.
Though i dont understand DagConnection.TempMode and also tempMode in
Channel. Could somebody explain why there is two different breakPipeline
property and the duty of TempMode. Thank you in advance.
Reply | Threaded
Open this post in threaded view
|

Re: DagConnection tempMode vs breakPipeline property

Ufuk Celebi-2
The data exchange mode has been introduced recently as a replacement
for the pipeline break logic, which was buggy. I'm not too familiar
with the optimizer, but I would expect everything that goes back to
the DataExchangeMode to be correct. The rest should be an artifact of
the old pipeline breaker logic and not be used any more. Does this
help in any way?

On Thu, Jun 16, 2016 at 4:46 PM, CPC <[hidden email]> wrote:
> Hi everybody,
>
> I am trying to understand flink optimizer in DataSet api. But i dont
> understant tempMode and breakPipeline properties. TempMode enum also has a
> breakPipeline property but they both accessed and set in different places.
> DagConnection.breakPipeline is used to select network io DataExchangeMode.
> Though i dont understand DagConnection.TempMode and also tempMode in
> Channel. Could somebody explain why there is two different breakPipeline
> property and the duty of TempMode. Thank you in advance.
CPC
Reply | Threaded
Open this post in threaded view
|

Re: DagConnection tempMode vs breakPipeline property

CPC
Thank you ufuk. Yes it helps alot. Yardımın için teşekkürler.
On Jul 4, 2016 2:02 PM, "Ufuk Celebi" <[hidden email]> wrote:

> The data exchange mode has been introduced recently as a replacement
> for the pipeline break logic, which was buggy. I'm not too familiar
> with the optimizer, but I would expect everything that goes back to
> the DataExchangeMode to be correct. The rest should be an artifact of
> the old pipeline breaker logic and not be used any more. Does this
> help in any way?
>
> On Thu, Jun 16, 2016 at 4:46 PM, CPC <[hidden email]> wrote:
> > Hi everybody,
> >
> > I am trying to understand flink optimizer in DataSet api. But i dont
> > understant tempMode and breakPipeline properties. TempMode enum also has
> a
> > breakPipeline property but they both accessed and set in different
> places.
> > DagConnection.breakPipeline is used to select network io
> DataExchangeMode.
> > Though i dont understand DagConnection.TempMode and also tempMode in
> > Channel. Could somebody explain why there is two different breakPipeline
> > property and the duty of TempMode. Thank you in advance.
>