Hi,
I have different plans for the DOPChangeTest in java 7 and 8. Here are the two plans that are generated: Java 7: http://pastebin.com/krzF0gmJ Java 8: http://pastebin.com/a6TuBjzB The plans are equivalent and both okay. The question is how to deal with this in a testcase when we have to expect a specific plan. The problem arises in this pull request: https://github.com/apache/incubator-flink/pull/83 Sebastian |
Hi!
The optimizer pushes certain strategies down before the map operators. That is correct, but it would be good to have a bias to not do that unless it yields a clear advantage. For that case, the simplest fix would be to give the mapper a selectivity smaller than one. That should do it. Stephan On Mon, Sep 1, 2014 at 1:47 PM, Sebastian Kunert <[hidden email]> wrote: > Hi, > > I have different plans for the DOPChangeTest in java 7 and 8. Here are the > two plans that are generated: > > Java 7: http://pastebin.com/krzF0gmJ > Java 8: http://pastebin.com/a6TuBjzB > > The plans are equivalent and both okay. The question is how to deal with > this in a testcase when we have to expect a specific plan. The problem > arises in this pull request: > https://github.com/apache/incubator-flink/pull/83 > > Sebastian > |
Hi Stephan,
I set the "filterFactor" in the CompilerHints of the mapper to 0.5, I hope that was what you meant. Nevertheless, it seems to have no effect on the output plan. Sebastian 2014-09-01 13:56 GMT+02:00 Stephan Ewen <[hidden email]>: > Hi! > > The optimizer pushes certain strategies down before the map operators. That > is correct, but it would be good to have a bias to not do that unless it > yields a clear advantage. > > For that case, the simplest fix would be to give the mapper a selectivity > smaller than one. That should do it. > > Stephan > > > > On Mon, Sep 1, 2014 at 1:47 PM, Sebastian Kunert <[hidden email]> > wrote: > > > Hi, > > > > I have different plans for the DOPChangeTest in java 7 and 8. Here are > the > > two plans that are generated: > > > > Java 7: http://pastebin.com/krzF0gmJ > > Java 8: http://pastebin.com/a6TuBjzB > > > > The plans are equivalent and both okay. The question is how to deal with > > this in a testcase when we have to expect a specific plan. The problem > > arises in this pull request: > > https://github.com/apache/incubator-flink/pull/83 > > > > Sebastian > > > |
Which mapper did you attach the filter factor to?
On Mon, Sep 1, 2014 at 2:20 PM, Sebastian Kunert <[hidden email]> wrote: > Hi Stephan, > > I set the "filterFactor" in the CompilerHints of the mapper to 0.5, I hope > that was what you meant. Nevertheless, it seems to have no effect on the > output plan. > > Sebastian > > > 2014-09-01 13:56 GMT+02:00 Stephan Ewen <[hidden email]>: > > > Hi! > > > > The optimizer pushes certain strategies down before the map operators. > That > > is correct, but it would be good to have a bias to not do that unless it > > yields a clear advantage. > > > > For that case, the simplest fix would be to give the mapper a selectivity > > smaller than one. That should do it. > > > > Stephan > > > > > > > > On Mon, Sep 1, 2014 at 1:47 PM, Sebastian Kunert <[hidden email]> > > wrote: > > > > > Hi, > > > > > > I have different plans for the DOPChangeTest in java 7 and 8. Here are > > the > > > two plans that are generated: > > > > > > Java 7: http://pastebin.com/krzF0gmJ > > > Java 8: http://pastebin.com/a6TuBjzB > > > > > > The plans are equivalent and both okay. The question is how to deal > with > > > this in a testcase when we have to expect a specific plan. The problem > > > arises in this pull request: > > > https://github.com/apache/incubator-flink/pull/83 > > > > > > Sebastian > > > > > > |
I applied it to the second mapper initially. I also tried applying the
filter to the first mapper and for both. 2014-09-01 15:00 GMT+02:00 Stephan Ewen <[hidden email]>: > Which mapper did you attach the filter factor to? > > > On Mon, Sep 1, 2014 at 2:20 PM, Sebastian Kunert <[hidden email]> > wrote: > > > Hi Stephan, > > > > I set the "filterFactor" in the CompilerHints of the mapper to 0.5, I > hope > > that was what you meant. Nevertheless, it seems to have no effect on the > > output plan. > > > > Sebastian > > > > > > 2014-09-01 13:56 GMT+02:00 Stephan Ewen <[hidden email]>: > > > > > Hi! > > > > > > The optimizer pushes certain strategies down before the map operators. > > That > > > is correct, but it would be good to have a bias to not do that unless > it > > > yields a clear advantage. > > > > > > For that case, the simplest fix would be to give the mapper a > selectivity > > > smaller than one. That should do it. > > > > > > Stephan > > > > > > > > > > > > On Mon, Sep 1, 2014 at 1:47 PM, Sebastian Kunert <[hidden email]> > > > wrote: > > > > > > > Hi, > > > > > > > > I have different plans for the DOPChangeTest in java 7 and 8. Here > are > > > the > > > > two plans that are generated: > > > > > > > > Java 7: http://pastebin.com/krzF0gmJ > > > > Java 8: http://pastebin.com/a6TuBjzB > > > > > > > > The plans are equivalent and both okay. The question is how to deal > > with > > > > this in a testcase when we have to expect a specific plan. The > problem > > > > arises in this pull request: > > > > https://github.com/apache/incubator-flink/pull/83 > > > > > > > > Sebastian > > > > > > > > > > |
Free forum by Nabble | Edit this page |