Hi everyone!
I would suggest to fork a 0.8 release branch very soon, to not stall the development of features that go into 0.9. The new Akka based coordination and the network stack extensions are required for many new features which are in the pipeline. How about we merge the below listed pull requests into the master, and then branch the 0.8 release branch off and bump the master version to 0.9-SNAPSHOT. After we have forked the release branch, Marton (as release manager) would create release candidates and collect status of the tests and patches. We can fix bugs on the release branch and cherry-pick the patches to the 0.9-SNAPSHOT branch. ---------------------------------------------- Important Pull Requests to merge: ---------------------------------------------- [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees of freedom Fixed java quickstart example [FLINK-998] Close TCP connections after destroying logical channels [FLINK-1302] Make JDBCInputFormat implement the NonParallelInput interface [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable support Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of ExecutionEnvironment (this is partially merged, there are a few POM entries missing for Hadoop 1 support) ---------------------------------------------- Optional Pull Requests to merge: ---------------------------------------------- Add support for Subclasses, Interfaces, Abstract Classes as POJOs [FLINK-1245] Introduce TypeHints for Java API operators enable CSV Reader to ignore invalid lines like an empty line at the end and comments - FLINK-1208 Please veto the important pull requests (if they should not go in) and +1 or -1 the optional ones, as you deem fitting. Greetings, Stephan |
+1.
I think adding the TypeHints pull request to the release is a good idea. This allows us to give users a workaround if the type extraction is not working. I'll have a look at the Hbase hadoop1 support. On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote: > Hi everyone! > > I would suggest to fork a 0.8 release branch very soon, to not stall the > development of features that go into 0.9. The new Akka based coordination > and the network stack extensions are required for many new features which > are in the pipeline. > > How about we merge the below listed pull requests into the master, and then > branch the 0.8 release branch off and bump the master version to > 0.9-SNAPSHOT. > > After we have forked the release branch, Marton (as release manager) would > create release candidates and collect status of the tests and patches. We > can fix bugs on the release branch and cherry-pick the patches to the > 0.9-SNAPSHOT branch. > > ---------------------------------------------- > Important Pull Requests to merge: > ---------------------------------------------- > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees of > freedom > > Fixed java quickstart example > > [FLINK-998] Close TCP connections after destroying logical channels > > [FLINK-1302] Make JDBCInputFormat implement the NonParallelInput interface > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > support > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > ExecutionEnvironment > (this is partially merged, there are a few POM entries missing for Hadoop 1 > support) > > > ---------------------------------------------- > Optional Pull Requests to merge: > ---------------------------------------------- > > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > [FLINK-1245] Introduce TypeHints for Java API operators > > enable CSV Reader to ignore invalid lines like an empty line at the end and > comments - FLINK-1208 > > > Please veto the important pull requests (if they should not go in) and +1 > or -1 the optional ones, as you deem fitting. > > > Greetings, > Stephan > |
+1
Looking forward merging the Akka changes into the 0.9-SNAPSHOT master. On Wed, Dec 10, 2014 at 9:06 PM, Robert Metzger <[hidden email]> wrote: > +1. > > I think adding the TypeHints pull request to the release is a good idea. > This allows us to give users a workaround if the type extraction is not > working. > > I'll have a look at the Hbase hadoop1 support. > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote: > > > Hi everyone! > > > > I would suggest to fork a 0.8 release branch very soon, to not stall the > > development of features that go into 0.9. The new Akka based coordination > > and the network stack extensions are required for many new features which > > are in the pipeline. > > > > How about we merge the below listed pull requests into the master, and > then > > branch the 0.8 release branch off and bump the master version to > > 0.9-SNAPSHOT. > > > > After we have forked the release branch, Marton (as release manager) > would > > create release candidates and collect status of the tests and patches. We > > can fix bugs on the release branch and cherry-pick the patches to the > > 0.9-SNAPSHOT branch. > > > > ---------------------------------------------- > > Important Pull Requests to merge: > > ---------------------------------------------- > > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees of > > freedom > > > > Fixed java quickstart example > > > > [FLINK-998] Close TCP connections after destroying logical channels > > > > [FLINK-1302] Make JDBCInputFormat implement the NonParallelInput > interface > > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > > support > > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > > ExecutionEnvironment > > (this is partially merged, there are a few POM entries missing for > Hadoop 1 > > support) > > > > > > ---------------------------------------------- > > Optional Pull Requests to merge: > > ---------------------------------------------- > > > > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > > > [FLINK-1245] Introduce TypeHints for Java API operators > > > > enable CSV Reader to ignore invalid lines like an empty line at the end > and > > comments - FLINK-1208 > > > > > > Please veto the important pull requests (if they should not go in) and +1 > > or -1 the optional ones, as you deem fitting. > > > > > > Greetings, > > Stephan > > > |
+1
2014-12-11 11:00 GMT+01:00 Till Rohrmann <[hidden email]>: > > +1 > > Looking forward merging the Akka changes into the 0.9-SNAPSHOT master. > > On Wed, Dec 10, 2014 at 9:06 PM, Robert Metzger <[hidden email]> > wrote: > > > +1. > > > > I think adding the TypeHints pull request to the release is a good idea. > > This allows us to give users a workaround if the type extraction is not > > working. > > > > I'll have a look at the Hbase hadoop1 support. > > > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote: > > > > > Hi everyone! > > > > > > I would suggest to fork a 0.8 release branch very soon, to not stall > the > > > development of features that go into 0.9. The new Akka based > coordination > > > and the network stack extensions are required for many new features > which > > > are in the pipeline. > > > > > > How about we merge the below listed pull requests into the master, and > > then > > > branch the 0.8 release branch off and bump the master version to > > > 0.9-SNAPSHOT. > > > > > > After we have forked the release branch, Marton (as release manager) > > would > > > create release candidates and collect status of the tests and patches. > We > > > can fix bugs on the release branch and cherry-pick the patches to the > > > 0.9-SNAPSHOT branch. > > > > > > ---------------------------------------------- > > > Important Pull Requests to merge: > > > ---------------------------------------------- > > > > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees > of > > > freedom > > > > > > Fixed java quickstart example > > > > > > [FLINK-998] Close TCP connections after destroying logical channels > > > > > > [FLINK-1302] Make JDBCInputFormat implement the NonParallelInput > > interface > > > > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > > > support > > > > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > > > ExecutionEnvironment > > > (this is partially merged, there are a few POM entries missing for > > Hadoop 1 > > > support) > > > > > > > > > ---------------------------------------------- > > > Optional Pull Requests to merge: > > > ---------------------------------------------- > > > > > > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > > > > > [FLINK-1245] Introduce TypeHints for Java API operators > > > > > > enable CSV Reader to ignore invalid lines like an empty line at the end > > and > > > comments - FLINK-1208 > > > > > > > > > Please veto the important pull requests (if they should not go in) and > +1 > > > or -1 the optional ones, as you deem fitting. > > > > > > > > > Greetings, > > > Stephan > > > > > > |
In reply to this post by Stephan Ewen
On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote:
> > ---------------------------------------------- > Important Pull Requests to merge: > ---------------------------------------------- > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees of > freedom > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > support > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > ExecutionEnvironment > (this is partially merged, there are a few POM entries missing for Hadoop 1 > support) > The first two PRs are still open. I will have a look at them as well (Robert already tested one). @Robert: Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable flink-hbase module for hadoop1 build as well")? It would be nice if had a look at the remaining important ones today. :-) |
Hey,
Checking the PRs listed by Stephan it seems every import one has been merged. As for the optional ones the following two are still pending: Add support for Subclasses, Interfaces, Abstract Classes as POJOs [FLINK-1245] Introduce TypeHints for Java API operators Robert +1'd the TypeHints PR and Timo has just updated it. Should we wait for it? Cheers, Marton On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote: > > > > ---------------------------------------------- > > Important Pull Requests to merge: > > ---------------------------------------------- > > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees of > > freedom > > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > > support > > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > > ExecutionEnvironment > > (this is partially merged, there are a few POM entries missing for > Hadoop 1 > > support) > > > > The first two PRs are still open. I will have a look at them as well > (Robert already tested one). > > @Robert: > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable flink-hbase > module for hadoop1 build as well")? > > > It would be nice if had a look at the remaining important ones today. :-) > |
Hi!
The type hints PR is ready now, Timo made some final adjustments yesterday evening. I'll review and merge it now, then we can fork the branch. Cheers, Stephan On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi <[hidden email]> wrote: > > Hey, > > Checking the PRs listed by Stephan it seems every import one has been > merged. As for the optional ones the following two are still pending: > > > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > [FLINK-1245] Introduce TypeHints for Java API operators > > > Robert +1'd the TypeHints PR and Timo has just updated it. Should we wait > for it? > > Cheers, > > Marton > > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: > > > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> wrote: > > > > > > ---------------------------------------------- > > > Important Pull Requests to merge: > > > ---------------------------------------------- > > > > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less degrees > of > > > freedom > > > > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and NullWritable > > > support > > > > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > > > ExecutionEnvironment > > > (this is partially merged, there are a few POM entries missing for > > Hadoop 1 > > > support) > > > > > > > The first two PRs are still open. I will have a look at them as well > > (Robert already tested one). > > > > @Robert: > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable flink-hbase > > module for hadoop1 build as well")? > > > > > > It would be nice if had a look at the remaining important ones today. :-) > > > |
Cool, thanks.
On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> wrote: > > Hi! > > The type hints PR is ready now, Timo made some final adjustments yesterday > evening. I'll review and merge it now, then we can fork the branch. > > Cheers, > Stephan > > > On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi <[hidden email] > > > wrote: > > > > Hey, > > > > Checking the PRs listed by Stephan it seems every import one has been > > merged. As for the optional ones the following two are still pending: > > > > > > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > > > [FLINK-1245] Introduce TypeHints for Java API operators > > > > > > Robert +1'd the TypeHints PR and Timo has just updated it. Should we wait > > for it? > > > > Cheers, > > > > Marton > > > > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: > > > > > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> > wrote: > > > > > > > > ---------------------------------------------- > > > > Important Pull Requests to merge: > > > > ---------------------------------------------- > > > > > > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less > degrees > > of > > > > freedom > > > > > > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > NullWritable > > > > support > > > > > > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > > > > ExecutionEnvironment > > > > (this is partially merged, there are a few POM entries missing for > > > Hadoop 1 > > > > support) > > > > > > > > > > The first two PRs are still open. I will have a look at them as well > > > (Robert already tested one). > > > > > > @Robert: > > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable flink-hbase > > > module for hadoop1 build as well")? > > > > > > > > > It would be nice if had a look at the remaining important ones today. > :-) > > > > > > |
Do we want the Java Closure Cleaner in 0.8 as well?
On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi <[hidden email]> wrote: > Cool, thanks. > > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> wrote: >> >> Hi! >> >> The type hints PR is ready now, Timo made some final adjustments yesterday >> evening. I'll review and merge it now, then we can fork the branch. >> >> Cheers, >> Stephan >> >> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi <[hidden email] >> > >> wrote: >> > >> > Hey, >> > >> > Checking the PRs listed by Stephan it seems every import one has been >> > merged. As for the optional ones the following two are still pending: >> > >> > >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs >> > >> > [FLINK-1245] Introduce TypeHints for Java API operators >> > >> > >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should we wait >> > for it? >> > >> > Cheers, >> > >> > Marton >> > >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: >> > > >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> >> wrote: >> > > > >> > > > ---------------------------------------------- >> > > > Important Pull Requests to merge: >> > > > ---------------------------------------------- >> > > > >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less >> degrees >> > of >> > > > freedom >> > > > >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and >> NullWritable >> > > > support >> > > > >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of >> > > > ExecutionEnvironment >> > > > (this is partially merged, there are a few POM entries missing for >> > > Hadoop 1 >> > > > support) >> > > > >> > > >> > > The first two PRs are still open. I will have a look at them as well >> > > (Robert already tested one). >> > > >> > > @Robert: >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable flink-hbase >> > > module for hadoop1 build as well")? >> > > >> > > >> > > It would be nice if had a look at the remaining important ones today. >> :-) >> > > >> > >> |
+1 for adding the closure cleaner...
Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email]>: > Do we want the Java Closure Cleaner in 0.8 as well? > > On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > <[hidden email]> wrote: > > Cool, thanks. > > > > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> wrote: > >> > >> Hi! > >> > >> The type hints PR is ready now, Timo made some final adjustments > yesterday > >> evening. I'll review and merge it now, then we can fork the branch. > >> > >> Cheers, > >> Stephan > >> > >> > >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > [hidden email] > >> > > >> wrote: > >> > > >> > Hey, > >> > > >> > Checking the PRs listed by Stephan it seems every import one has been > >> > merged. As for the optional ones the following two are still pending: > >> > > >> > > >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > >> > > >> > [FLINK-1245] Introduce TypeHints for Java API operators > >> > > >> > > >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should we > wait > >> > for it? > >> > > >> > Cheers, > >> > > >> > Marton > >> > > >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: > >> > > > >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> > >> wrote: > >> > > > > >> > > > ---------------------------------------------- > >> > > > Important Pull Requests to merge: > >> > > > ---------------------------------------------- > >> > > > > >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less > >> degrees > >> > of > >> > > > freedom > >> > > > > >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > >> NullWritable > >> > > > support > >> > > > > >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of > >> > > > ExecutionEnvironment > >> > > > (this is partially merged, there are a few POM entries missing for > >> > > Hadoop 1 > >> > > > support) > >> > > > > >> > > > >> > > The first two PRs are still open. I will have a look at them as well > >> > > (Robert already tested one). > >> > > > >> > > @Robert: > >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > flink-hbase > >> > > module for hadoop1 build as well")? > >> > > > >> > > > >> > > It would be nice if had a look at the remaining important ones > today. > >> :-) > >> > > > >> > > >> > |
I am currently working through the type hints pull request...
Right now, I actually think that we should postpone this for the next bugfix release. This pull request changes actually some tricky mechanisms in the Java API (it delays the type information), which may have implications that me and Timo have not fully caught now. I don't feel Merging this last minute - this is prone to introduce errors. Any opinions on that? On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> wrote: > > +1 for adding the closure cleaner... > Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email]>: > > Do we want the Java Closure Cleaner in 0.8 as well? >> >> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi >> <[hidden email]> wrote: >> > Cool, thanks. >> > >> > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> >> wrote: >> >> >> >> Hi! >> >> >> >> The type hints PR is ready now, Timo made some final adjustments >> yesterday >> >> evening. I'll review and merge it now, then we can fork the branch. >> >> >> >> Cheers, >> >> Stephan >> >> >> >> >> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < >> [hidden email] >> >> > >> >> wrote: >> >> > >> >> > Hey, >> >> > >> >> > Checking the PRs listed by Stephan it seems every import one has been >> >> > merged. As for the optional ones the following two are still pending: >> >> > >> >> > >> >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs >> >> > >> >> > [FLINK-1245] Introduce TypeHints for Java API operators >> >> > >> >> > >> >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should we >> wait >> >> > for it? >> >> > >> >> > Cheers, >> >> > >> >> > Marton >> >> > >> >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: >> >> > > >> >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> >> >> wrote: >> >> > > > >> >> > > > ---------------------------------------------- >> >> > > > Important Pull Requests to merge: >> >> > > > ---------------------------------------------- >> >> > > > >> >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less >> >> degrees >> >> > of >> >> > > > freedom >> >> > > > >> >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and >> >> NullWritable >> >> > > > support >> >> > > > >> >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of >> >> > > > ExecutionEnvironment >> >> > > > (this is partially merged, there are a few POM entries missing >> for >> >> > > Hadoop 1 >> >> > > > support) >> >> > > > >> >> > > >> >> > > The first two PRs are still open. I will have a look at them as >> well >> >> > > (Robert already tested one). >> >> > > >> >> > > @Robert: >> >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable >> flink-hbase >> >> > > module for hadoop1 build as well")? >> >> > > >> >> > > >> >> > > It would be nice if had a look at the remaining important ones >> today. >> >> :-) >> >> > > >> >> > >> >> >> > |
+1 on postponing if it introduces behaviour we don't fully understand
On Tue, Dec 16, 2014 at 4:50 PM, Stephan Ewen <[hidden email]> wrote: > I am currently working through the type hints pull request... > > Right now, I actually think that we should postpone this for the next > bugfix release. > > This pull request changes actually some tricky mechanisms in the Java API > (it delays the type information), which may have implications that me and > Timo have not fully caught now. > > I don't feel Merging this last minute - this is prone to introduce errors. > > Any opinions on that? > > > > On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> wrote: >> >> +1 for adding the closure cleaner... >> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email]>: >> >> Do we want the Java Closure Cleaner in 0.8 as well? >>> >>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi >>> <[hidden email]> wrote: >>> > Cool, thanks. >>> > >>> > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> >>> wrote: >>> >> >>> >> Hi! >>> >> >>> >> The type hints PR is ready now, Timo made some final adjustments >>> yesterday >>> >> evening. I'll review and merge it now, then we can fork the branch. >>> >> >>> >> Cheers, >>> >> Stephan >>> >> >>> >> >>> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < >>> [hidden email] >>> >> > >>> >> wrote: >>> >> > >>> >> > Hey, >>> >> > >>> >> > Checking the PRs listed by Stephan it seems every import one has been >>> >> > merged. As for the optional ones the following two are still pending: >>> >> > >>> >> > >>> >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs >>> >> > >>> >> > [FLINK-1245] Introduce TypeHints for Java API operators >>> >> > >>> >> > >>> >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should we >>> wait >>> >> > for it? >>> >> > >>> >> > Cheers, >>> >> > >>> >> > Marton >>> >> > >>> >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> wrote: >>> >> > > >>> >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> >>> >> wrote: >>> >> > > > >>> >> > > > ---------------------------------------------- >>> >> > > > Important Pull Requests to merge: >>> >> > > > ---------------------------------------------- >>> >> > > > >>> >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less >>> >> degrees >>> >> > of >>> >> > > > freedom >>> >> > > > >>> >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and >>> >> NullWritable >>> >> > > > support >>> >> > > > >>> >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix of >>> >> > > > ExecutionEnvironment >>> >> > > > (this is partially merged, there are a few POM entries missing >>> for >>> >> > > Hadoop 1 >>> >> > > > support) >>> >> > > > >>> >> > > >>> >> > > The first two PRs are still open. I will have a look at them as >>> well >>> >> > > (Robert already tested one). >>> >> > > >>> >> > > @Robert: >>> >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable >>> flink-hbase >>> >> > > module for hadoop1 build as well")? >>> >> > > >>> >> > > >>> >> > > It would be nice if had a look at the remaining important ones >>> today. >>> >> :-) >>> >> > > >>> >> > >>> >> >>> >> |
In reply to this post by Stephan Ewen
+1 for stability first
2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > > I am currently working through the type hints pull request... > > Right now, I actually think that we should postpone this for the next > bugfix release. > > This pull request changes actually some tricky mechanisms in the Java API > (it delays the type information), which may have implications that me and > Timo have not fully caught now. > > I don't feel Merging this last minute - this is prone to introduce errors. > > Any opinions on that? > > > > On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> wrote: > > > > +1 for adding the closure cleaner... > > Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email]>: > > > > Do we want the Java Closure Cleaner in 0.8 as well? > >> > >> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > >> <[hidden email]> wrote: > >> > Cool, thanks. > >> > > >> > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> > >> wrote: > >> >> > >> >> Hi! > >> >> > >> >> The type hints PR is ready now, Timo made some final adjustments > >> yesterday > >> >> evening. I'll review and merge it now, then we can fork the branch. > >> >> > >> >> Cheers, > >> >> Stephan > >> >> > >> >> > >> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > >> [hidden email] > >> >> > > >> >> wrote: > >> >> > > >> >> > Hey, > >> >> > > >> >> > Checking the PRs listed by Stephan it seems every import one has > been > >> >> > merged. As for the optional ones the following two are still > pending: > >> >> > > >> >> > > >> >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > >> >> > > >> >> > [FLINK-1245] Introduce TypeHints for Java API operators > >> >> > > >> >> > > >> >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should > we > >> wait > >> >> > for it? > >> >> > > >> >> > Cheers, > >> >> > > >> >> > Marton > >> >> > > >> >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> > wrote: > >> >> > > > >> >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen <[hidden email]> > >> >> wrote: > >> >> > > > > >> >> > > > ---------------------------------------------- > >> >> > > > Important Pull Requests to merge: > >> >> > > > ---------------------------------------------- > >> >> > > > > >> >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with less > >> >> degrees > >> >> > of > >> >> > > > freedom > >> >> > > > > >> >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > >> >> NullWritable > >> >> > > > support > >> >> > > > > >> >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix > of > >> >> > > > ExecutionEnvironment > >> >> > > > (this is partially merged, there are a few POM entries missing > >> for > >> >> > > Hadoop 1 > >> >> > > > support) > >> >> > > > > >> >> > > > >> >> > > The first two PRs are still open. I will have a look at them as > >> well > >> >> > > (Robert already tested one). > >> >> > > > >> >> > > @Robert: > >> >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > >> flink-hbase > >> >> > > module for hadoop1 build as well")? > >> >> > > > >> >> > > > >> >> > > It would be nice if had a look at the remaining important ones > >> today. > >> >> :-) > >> >> > > > >> >> > > >> >> > >> > > > |
Yes, one thing I just realized is that the streaming API is not aware of
MissingTypeInfos and it may cause weird behavior. On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> wrote: > > +1 for stability first > > 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > > > > I am currently working through the type hints pull request... > > > > Right now, I actually think that we should postpone this for the next > > bugfix release. > > > > This pull request changes actually some tricky mechanisms in the Java API > > (it delays the type information), which may have implications that me and > > Timo have not fully caught now. > > > > I don't feel Merging this last minute - this is prone to introduce > errors. > > > > Any opinions on that? > > > > > > > > On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> wrote: > > > > > > +1 for adding the closure cleaner... > > > Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email]>: > > > > > > Do we want the Java Closure Cleaner in 0.8 as well? > > >> > > >> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > > >> <[hidden email]> wrote: > > >> > Cool, thanks. > > >> > > > >> > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> > > >> wrote: > > >> >> > > >> >> Hi! > > >> >> > > >> >> The type hints PR is ready now, Timo made some final adjustments > > >> yesterday > > >> >> evening. I'll review and merge it now, then we can fork the branch. > > >> >> > > >> >> Cheers, > > >> >> Stephan > > >> >> > > >> >> > > >> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > > >> [hidden email] > > >> >> > > > >> >> wrote: > > >> >> > > > >> >> > Hey, > > >> >> > > > >> >> > Checking the PRs listed by Stephan it seems every import one has > > been > > >> >> > merged. As for the optional ones the following two are still > > pending: > > >> >> > > > >> >> > > > >> >> > Add support for Subclasses, Interfaces, Abstract Classes as POJOs > > >> >> > > > >> >> > [FLINK-1245] Introduce TypeHints for Java API operators > > >> >> > > > >> >> > > > >> >> > Robert +1'd the TypeHints PR and Timo has just updated it. Should > > we > > >> wait > > >> >> > for it? > > >> >> > > > >> >> > Cheers, > > >> >> > > > >> >> > Marton > > >> >> > > > >> >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> > > wrote: > > >> >> > > > > >> >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < > [hidden email]> > > >> >> wrote: > > >> >> > > > > > >> >> > > > ---------------------------------------------- > > >> >> > > > Important Pull Requests to merge: > > >> >> > > > ---------------------------------------------- > > >> >> > > > > > >> >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with > less > > >> >> degrees > > >> >> > of > > >> >> > > > freedom > > >> >> > > > > > >> >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > > >> >> NullWritable > > >> >> > > > support > > >> >> > > > > > >> >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + fix > > of > > >> >> > > > ExecutionEnvironment > > >> >> > > > (this is partially merged, there are a few POM entries > missing > > >> for > > >> >> > > Hadoop 1 > > >> >> > > > support) > > >> >> > > > > > >> >> > > > > >> >> > > The first two PRs are still open. I will have a look at them as > > >> well > > >> >> > > (Robert already tested one). > > >> >> > > > > >> >> > > @Robert: > > >> >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > > >> flink-hbase > > >> >> > > module for hadoop1 build as well")? > > >> >> > > > > >> >> > > > > >> >> > > It would be nice if had a look at the remaining important ones > > >> today. > > >> >> :-) > > >> >> > > > > >> >> > > > >> >> > > >> > > > > > > |
I've worked on adding Kryo support to Flink today. I would really like to
include this into the release. Many users were complaining about this ( https://issues.apache.org/jira/browse/FLINK-610). I've based my work on top of Timo's pull request. If we're not going to merge it, I will have to rebase my code to the current master. ... I'll try and see if its easily doable. On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> wrote: > > Yes, one thing I just realized is that the streaming API is not aware of > MissingTypeInfos and it may cause weird behavior. > > On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> wrote: > > > > +1 for stability first > > > > 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > > > > > > I am currently working through the type hints pull request... > > > > > > Right now, I actually think that we should postpone this for the next > > > bugfix release. > > > > > > This pull request changes actually some tricky mechanisms in the Java > API > > > (it delays the type information), which may have implications that me > and > > > Timo have not fully caught now. > > > > > > I don't feel Merging this last minute - this is prone to introduce > > errors. > > > > > > Any opinions on that? > > > > > > > > > > > > On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> > wrote: > > > > > > > > +1 for adding the closure cleaner... > > > > Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email] > >: > > > > > > > > Do we want the Java Closure Cleaner in 0.8 as well? > > > >> > > > >> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > > > >> <[hidden email]> wrote: > > > >> > Cool, thanks. > > > >> > > > > >> > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> > > > >> wrote: > > > >> >> > > > >> >> Hi! > > > >> >> > > > >> >> The type hints PR is ready now, Timo made some final adjustments > > > >> yesterday > > > >> >> evening. I'll review and merge it now, then we can fork the > branch. > > > >> >> > > > >> >> Cheers, > > > >> >> Stephan > > > >> >> > > > >> >> > > > >> >> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > > > >> [hidden email] > > > >> >> > > > > >> >> wrote: > > > >> >> > > > > >> >> > Hey, > > > >> >> > > > > >> >> > Checking the PRs listed by Stephan it seems every import one > has > > > been > > > >> >> > merged. As for the optional ones the following two are still > > > pending: > > > >> >> > > > > >> >> > > > > >> >> > Add support for Subclasses, Interfaces, Abstract Classes as > POJOs > > > >> >> > > > > >> >> > [FLINK-1245] Introduce TypeHints for Java API operators > > > >> >> > > > > >> >> > > > > >> >> > Robert +1'd the TypeHints PR and Timo has just updated it. > Should > > > we > > > >> wait > > > >> >> > for it? > > > >> >> > > > > >> >> > Cheers, > > > >> >> > > > > >> >> > Marton > > > >> >> > > > > >> >> > On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> > > > wrote: > > > >> >> > > > > > >> >> > > On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < > > [hidden email]> > > > >> >> wrote: > > > >> >> > > > > > > >> >> > > > ---------------------------------------------- > > > >> >> > > > Important Pull Requests to merge: > > > >> >> > > > ---------------------------------------------- > > > >> >> > > > > > > >> >> > > > [FLINK-1287] LocalizableSplitAssigner prefers splits with > > less > > > >> >> degrees > > > >> >> > of > > > >> >> > > > freedom > > > >> >> > > > > > > >> >> > > > [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > > > >> >> NullWritable > > > >> >> > > > support > > > >> >> > > > > > > >> >> > > > Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + > fix > > > of > > > >> >> > > > ExecutionEnvironment > > > >> >> > > > (this is partially merged, there are a few POM entries > > missing > > > >> for > > > >> >> > > Hadoop 1 > > > >> >> > > > support) > > > >> >> > > > > > > >> >> > > > > > >> >> > > The first two PRs are still open. I will have a look at them > as > > > >> well > > > >> >> > > (Robert already tested one). > > > >> >> > > > > > >> >> > > @Robert: > > > >> >> > > Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > > > >> flink-hbase > > > >> >> > > module for hadoop1 build as well")? > > > >> >> > > > > > >> >> > > > > > >> >> > > It would be nice if had a look at the remaining important > ones > > > >> today. > > > >> >> :-) > > > >> >> > > > > > >> >> > > > > >> >> > > > >> > > > > > > > > > > |
Sorry I totally forgot to check the streaming API. Do they use the getXXXReturnTypes() methods of the TypeExtractor?
> Am 16.12.2014 um 17:23 schrieb Robert Metzger <[hidden email]>: > > I've worked on adding Kryo support to Flink today. I would really like to > include this into the release. Many users were complaining about this ( > https://issues.apache.org/jira/browse/FLINK-610). > > I've based my work on top of Timo's pull request. If we're not going to > merge it, I will have to rebase my code to the current master. ... I'll try > and see if its easily doable. > >> On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> wrote: >> >> Yes, one thing I just realized is that the streaming API is not aware of >> MissingTypeInfos and it may cause weird behavior. >> >>> On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> wrote: >>> >>> +1 for stability first >>> >>> 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: >>>> >>>> I am currently working through the type hints pull request... >>>> >>>> Right now, I actually think that we should postpone this for the next >>>> bugfix release. >>>> >>>> This pull request changes actually some tricky mechanisms in the Java >> API >>>> (it delays the type information), which may have implications that me >> and >>>> Timo have not fully caught now. >>>> >>>> I don't feel Merging this last minute - this is prone to introduce >>> errors. >>>> >>>> Any opinions on that? >>>> >>>> >>>> >>>> On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> >> wrote: >>>>> >>>>> +1 for adding the closure cleaner... >>>>> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email] >>> : >>>>> >>>>> Do we want the Java Closure Cleaner in 0.8 as well? >>>>>> >>>>>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi >>>>>> <[hidden email]> wrote: >>>>>>> Cool, thanks. >>>>>>> >>>>>>> On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> >>>>>> wrote: >>>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> The type hints PR is ready now, Timo made some final adjustments >>>>>> yesterday >>>>>>>> evening. I'll review and merge it now, then we can fork the >> branch. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Stephan >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < >>>>>> [hidden email] >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hey, >>>>>>>>> >>>>>>>>> Checking the PRs listed by Stephan it seems every import one >> has >>>> been >>>>>>>>> merged. As for the optional ones the following two are still >>>> pending: >>>>>>>>> >>>>>>>>> >>>>>>>>> Add support for Subclasses, Interfaces, Abstract Classes as >> POJOs >>>>>>>>> >>>>>>>>> [FLINK-1245] Introduce TypeHints for Java API operators >>>>>>>>> >>>>>>>>> >>>>>>>>> Robert +1'd the TypeHints PR and Timo has just updated it. >> Should >>>> we >>>>>> wait >>>>>>>>> for it? >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> >>>>>>>>> Marton >>>>>>>>> >>>>>>>>> On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> >>>> wrote: >>>>>>>>>> >>>>>>>>>> On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < >>> [hidden email]> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> ---------------------------------------------- >>>>>>>>>>> Important Pull Requests to merge: >>>>>>>>>>> ---------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> [FLINK-1287] LocalizableSplitAssigner prefers splits with >>> less >>>>>>>> degrees >>>>>>>>> of >>>>>>>>>>> freedom >>>>>>>>>>> >>>>>>>>>>> [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and >>>>>>>> NullWritable >>>>>>>>>>> support >>>>>>>>>>> >>>>>>>>>>> Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + >> fix >>>> of >>>>>>>>>>> ExecutionEnvironment >>>>>>>>>>> (this is partially merged, there are a few POM entries >>> missing >>>>>> for >>>>>>>>>> Hadoop 1 >>>>>>>>>>> support) >>>>>>>>>> >>>>>>>>>> The first two PRs are still open. I will have a look at them >> as >>>>>> well >>>>>>>>>> (Robert already tested one). >>>>>>>>>> >>>>>>>>>> @Robert: >>>>>>>>>> Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable >>>>>> flink-hbase >>>>>>>>>> module for hadoop1 build as well")? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> It would be nice if had a look at the remaining important >> ones >>>>>> today. >>>>>>>> :-) >> |
Yes, the fact that we both did not realize that until now sort of reveals
that there are more implications than anticipated. That is never good for "last minute" patches. That's why I would like to merge it into 0.8.1 and 0.9, rather than 0.8.0 That gives the code a bit more testing/use and us a bit more time to thoroughly think the implications through. On Tue, Dec 16, 2014 at 5:30 PM, Timo Walther <[hidden email]> wrote: > > Sorry I totally forgot to check the streaming API. Do they use the > getXXXReturnTypes() methods of the TypeExtractor? > > > Am 16.12.2014 um 17:23 schrieb Robert Metzger <[hidden email]>: > > > > I've worked on adding Kryo support to Flink today. I would really like to > > include this into the release. Many users were complaining about this ( > > https://issues.apache.org/jira/browse/FLINK-610). > > > > I've based my work on top of Timo's pull request. If we're not going to > > merge it, I will have to rebase my code to the current master. ... I'll > try > > and see if its easily doable. > > > >> On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> wrote: > >> > >> Yes, one thing I just realized is that the streaming API is not aware of > >> MissingTypeInfos and it may cause weird behavior. > >> > >>> On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> > wrote: > >>> > >>> +1 for stability first > >>> > >>> 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > >>>> > >>>> I am currently working through the type hints pull request... > >>>> > >>>> Right now, I actually think that we should postpone this for the next > >>>> bugfix release. > >>>> > >>>> This pull request changes actually some tricky mechanisms in the Java > >> API > >>>> (it delays the type information), which may have implications that me > >> and > >>>> Timo have not fully caught now. > >>>> > >>>> I don't feel Merging this last minute - this is prone to introduce > >>> errors. > >>>> > >>>> Any opinions on that? > >>>> > >>>> > >>>> > >>>> On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> > >> wrote: > >>>>> > >>>>> +1 for adding the closure cleaner... > >>>>> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email] > >>> : > >>>>> > >>>>> Do we want the Java Closure Cleaner in 0.8 as well? > >>>>>> > >>>>>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > >>>>>> <[hidden email]> wrote: > >>>>>>> Cool, thanks. > >>>>>>> > >>>>>>> On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> > >>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi! > >>>>>>>> > >>>>>>>> The type hints PR is ready now, Timo made some final adjustments > >>>>>> yesterday > >>>>>>>> evening. I'll review and merge it now, then we can fork the > >> branch. > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Stephan > >>>>>>>> > >>>>>>>> > >>>>>>>> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > >>>>>> [hidden email] > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> Hey, > >>>>>>>>> > >>>>>>>>> Checking the PRs listed by Stephan it seems every import one > >> has > >>>> been > >>>>>>>>> merged. As for the optional ones the following two are still > >>>> pending: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Add support for Subclasses, Interfaces, Abstract Classes as > >> POJOs > >>>>>>>>> > >>>>>>>>> [FLINK-1245] Introduce TypeHints for Java API operators > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Robert +1'd the TypeHints PR and Timo has just updated it. > >> Should > >>>> we > >>>>>> wait > >>>>>>>>> for it? > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> > >>>>>>>>> Marton > >>>>>>>>> > >>>>>>>>> On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> > >>>> wrote: > >>>>>>>>>> > >>>>>>>>>> On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < > >>> [hidden email]> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> ---------------------------------------------- > >>>>>>>>>>> Important Pull Requests to merge: > >>>>>>>>>>> ---------------------------------------------- > >>>>>>>>>>> > >>>>>>>>>>> [FLINK-1287] LocalizableSplitAssigner prefers splits with > >>> less > >>>>>>>> degrees > >>>>>>>>> of > >>>>>>>>>>> freedom > >>>>>>>>>>> > >>>>>>>>>>> [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > >>>>>>>> NullWritable > >>>>>>>>>>> support > >>>>>>>>>>> > >>>>>>>>>>> Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + > >> fix > >>>> of > >>>>>>>>>>> ExecutionEnvironment > >>>>>>>>>>> (this is partially merged, there are a few POM entries > >>> missing > >>>>>> for > >>>>>>>>>> Hadoop 1 > >>>>>>>>>>> support) > >>>>>>>>>> > >>>>>>>>>> The first two PRs are still open. I will have a look at them > >> as > >>>>>> well > >>>>>>>>>> (Robert already tested one). > >>>>>>>>>> > >>>>>>>>>> @Robert: > >>>>>>>>>> Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > >>>>>> flink-hbase > >>>>>>>>>> module for hadoop1 build as well")? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> It would be nice if had a look at the remaining important > >> ones > >>>>>> today. > >>>>>>>> :-) > >> > |
Yes, I also think so. Stability is more important.
> Am 16.12.2014 um 17:46 schrieb Stephan Ewen <[hidden email]>: > > Yes, the fact that we both did not realize that until now sort of reveals > that there are more implications than anticipated. That is never good for > "last minute" patches. > > That's why I would like to merge it into 0.8.1 and 0.9, rather than 0.8.0 > That gives the code a bit more testing/use and us a bit more time to > thoroughly think the implications through. > >> On Tue, Dec 16, 2014 at 5:30 PM, Timo Walther <[hidden email]> wrote: >> >> Sorry I totally forgot to check the streaming API. Do they use the >> getXXXReturnTypes() methods of the TypeExtractor? >> >>> Am 16.12.2014 um 17:23 schrieb Robert Metzger <[hidden email]>: >>> >>> I've worked on adding Kryo support to Flink today. I would really like to >>> include this into the release. Many users were complaining about this ( >>> https://issues.apache.org/jira/browse/FLINK-610). >>> >>> I've based my work on top of Timo's pull request. If we're not going to >>> merge it, I will have to rebase my code to the current master. ... I'll >> try >>> and see if its easily doable. >>> >>>> On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> wrote: >>>> >>>> Yes, one thing I just realized is that the streaming API is not aware of >>>> MissingTypeInfos and it may cause weird behavior. >>>> >>>>> On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> >> wrote: >>>>> >>>>> +1 for stability first >>>>> >>>>> 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: >>>>>> >>>>>> I am currently working through the type hints pull request... >>>>>> >>>>>> Right now, I actually think that we should postpone this for the next >>>>>> bugfix release. >>>>>> >>>>>> This pull request changes actually some tricky mechanisms in the Java >>>> API >>>>>> (it delays the type information), which may have implications that me >>>> and >>>>>> Timo have not fully caught now. >>>>>> >>>>>> I don't feel Merging this last minute - this is prone to introduce >>>>> errors. >>>>>> >>>>>> Any opinions on that? >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> >>>> wrote: >>>>>>> >>>>>>> +1 for adding the closure cleaner... >>>>>>> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" <[hidden email] >>>>> : >>>>>>> >>>>>>> Do we want the Java Closure Cleaner in 0.8 as well? >>>>>>>> >>>>>>>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi >>>>>>>> <[hidden email]> wrote: >>>>>>>>> Cool, thanks. >>>>>>>>> >>>>>>>>> On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email]> >>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi! >>>>>>>>>> >>>>>>>>>> The type hints PR is ready now, Timo made some final adjustments >>>>>>>> yesterday >>>>>>>>>> evening. I'll review and merge it now, then we can fork the >>>> branch. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Stephan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < >>>>>>>> [hidden email] >>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hey, >>>>>>>>>>> >>>>>>>>>>> Checking the PRs listed by Stephan it seems every import one >>>> has >>>>>> been >>>>>>>>>>> merged. As for the optional ones the following two are still >>>>>> pending: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Add support for Subclasses, Interfaces, Abstract Classes as >>>> POJOs >>>>>>>>>>> >>>>>>>>>>> [FLINK-1245] Introduce TypeHints for Java API operators >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Robert +1'd the TypeHints PR and Timo has just updated it. >>>> Should >>>>>> we >>>>>>>> wait >>>>>>>>>>> for it? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> >>>>>>>>>>> Marton >>>>>>>>>>> >>>>>>>>>>> On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> >>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < >>>>> [hidden email]> >>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> ---------------------------------------------- >>>>>>>>>>>>> Important Pull Requests to merge: >>>>>>>>>>>>> ---------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> [FLINK-1287] LocalizableSplitAssigner prefers splits with >>>>> less >>>>>>>>>> degrees >>>>>>>>>>> of >>>>>>>>>>>>> freedom >>>>>>>>>>>>> >>>>>>>>>>>>> [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and >>>>>>>>>> NullWritable >>>>>>>>>>>>> support >>>>>>>>>>>>> >>>>>>>>>>>>> Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + >>>> fix >>>>>> of >>>>>>>>>>>>> ExecutionEnvironment >>>>>>>>>>>>> (this is partially merged, there are a few POM entries >>>>> missing >>>>>>>> for >>>>>>>>>>>> Hadoop 1 >>>>>>>>>>>>> support) >>>>>>>>>>>> >>>>>>>>>>>> The first two PRs are still open. I will have a look at them >>>> as >>>>>>>> well >>>>>>>>>>>> (Robert already tested one). >>>>>>>>>>>> >>>>>>>>>>>> @Robert: >>>>>>>>>>>> Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable >>>>>>>> flink-hbase >>>>>>>>>>>> module for hadoop1 build as well")? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It would be nice if had a look at the remaining important >>>> ones >>>>>>>> today. >>>>>>>>>> :-) >> |
@Timo&Stephan: The streaming code uses this since last week [1], Timo
starting working on the issue prior way before that - so it also has to do with the streaming team pushing last minute features to be honest... Then are we waiting for the Java Clousure cleaner and the Kyro support, I guess. Aljoscha has already issued a PR for the former, some additional testing is needed from the looks of it. [2] [1] https://git-wip-us.apache.org/repos/asf?p=incubator-flink.git;a=commit;h=51c1f67791307c2b9355171f7398d104befc8de5 [2] https://github.com/apache/incubator-flink/pull/269 On Tue, Dec 16, 2014 at 5:52 PM, Timo Walther <[hidden email]> wrote: > > Yes, I also think so. Stability is more important. > > > Am 16.12.2014 um 17:46 schrieb Stephan Ewen <[hidden email]>: > > > > Yes, the fact that we both did not realize that until now sort of reveals > > that there are more implications than anticipated. That is never good for > > "last minute" patches. > > > > That's why I would like to merge it into 0.8.1 and 0.9, rather than 0.8.0 > > That gives the code a bit more testing/use and us a bit more time to > > thoroughly think the implications through. > > > >> On Tue, Dec 16, 2014 at 5:30 PM, Timo Walther <[hidden email]> > wrote: > >> > >> Sorry I totally forgot to check the streaming API. Do they use the > >> getXXXReturnTypes() methods of the TypeExtractor? > >> > >>> Am 16.12.2014 um 17:23 schrieb Robert Metzger <[hidden email]>: > >>> > >>> I've worked on adding Kryo support to Flink today. I would really like > to > >>> include this into the release. Many users were complaining about this ( > >>> https://issues.apache.org/jira/browse/FLINK-610). > >>> > >>> I've based my work on top of Timo's pull request. If we're not going to > >>> merge it, I will have to rebase my code to the current master. ... I'll > >> try > >>> and see if its easily doable. > >>> > >>>> On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> > wrote: > >>>> > >>>> Yes, one thing I just realized is that the streaming API is not aware > of > >>>> MissingTypeInfos and it may cause weird behavior. > >>>> > >>>>> On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> > >> wrote: > >>>>> > >>>>> +1 for stability first > >>>>> > >>>>> 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > >>>>>> > >>>>>> I am currently working through the type hints pull request... > >>>>>> > >>>>>> Right now, I actually think that we should postpone this for the > next > >>>>>> bugfix release. > >>>>>> > >>>>>> This pull request changes actually some tricky mechanisms in the > Java > >>>> API > >>>>>> (it delays the type information), which may have implications that > me > >>>> and > >>>>>> Timo have not fully caught now. > >>>>>> > >>>>>> I don't feel Merging this last minute - this is prone to introduce > >>>>> errors. > >>>>>> > >>>>>> Any opinions on that? > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> > >>>> wrote: > >>>>>>> > >>>>>>> +1 for adding the closure cleaner... > >>>>>>> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" < > [hidden email] > >>>>> : > >>>>>>> > >>>>>>> Do we want the Java Closure Cleaner in 0.8 as well? > >>>>>>>> > >>>>>>>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > >>>>>>>> <[hidden email]> wrote: > >>>>>>>>> Cool, thanks. > >>>>>>>>> > >>>>>>>>> On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen <[hidden email] > > > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Hi! > >>>>>>>>>> > >>>>>>>>>> The type hints PR is ready now, Timo made some final adjustments > >>>>>>>> yesterday > >>>>>>>>>> evening. I'll review and merge it now, then we can fork the > >>>> branch. > >>>>>>>>>> > >>>>>>>>>> Cheers, > >>>>>>>>>> Stephan > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > >>>>>>>> [hidden email] > >>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hey, > >>>>>>>>>>> > >>>>>>>>>>> Checking the PRs listed by Stephan it seems every import one > >>>> has > >>>>>> been > >>>>>>>>>>> merged. As for the optional ones the following two are still > >>>>>> pending: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Add support for Subclasses, Interfaces, Abstract Classes as > >>>> POJOs > >>>>>>>>>>> > >>>>>>>>>>> [FLINK-1245] Introduce TypeHints for Java API operators > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Robert +1'd the TypeHints PR and Timo has just updated it. > >>>> Should > >>>>>> we > >>>>>>>> wait > >>>>>>>>>>> for it? > >>>>>>>>>>> > >>>>>>>>>>> Cheers, > >>>>>>>>>>> > >>>>>>>>>>> Marton > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email]> > >>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < > >>>>> [hidden email]> > >>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> ---------------------------------------------- > >>>>>>>>>>>>> Important Pull Requests to merge: > >>>>>>>>>>>>> ---------------------------------------------- > >>>>>>>>>>>>> > >>>>>>>>>>>>> [FLINK-1287] LocalizableSplitAssigner prefers splits with > >>>>> less > >>>>>>>>>> degrees > >>>>>>>>>>> of > >>>>>>>>>>>>> freedom > >>>>>>>>>>>>> > >>>>>>>>>>>>> [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > >>>>>>>>>> NullWritable > >>>>>>>>>>>>> support > >>>>>>>>>>>>> > >>>>>>>>>>>>> Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + > >>>> fix > >>>>>> of > >>>>>>>>>>>>> ExecutionEnvironment > >>>>>>>>>>>>> (this is partially merged, there are a few POM entries > >>>>> missing > >>>>>>>> for > >>>>>>>>>>>> Hadoop 1 > >>>>>>>>>>>>> support) > >>>>>>>>>>>> > >>>>>>>>>>>> The first two PRs are still open. I will have a look at them > >>>> as > >>>>>>>> well > >>>>>>>>>>>> (Robert already tested one). > >>>>>>>>>>>> > >>>>>>>>>>>> @Robert: > >>>>>>>>>>>> Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > >>>>>>>> flink-hbase > >>>>>>>>>>>> module for hadoop1 build as well")? > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> It would be nice if had a look at the remaining important > >>>> ones > >>>>>>>> today. > >>>>>>>>>> :-) > >> > |
Hi!
I forked a branch "release-0.8" and bumped the master version to "0.9-incubating-SNAPSHOT". The closure cleaner is merged. Robert has opened a PR for Kryo - I would suggest to merge that to "master", cherry-pick to "release-0.8" and then: Merry releasing! Greetings, Stephan On Tue, Dec 16, 2014 at 6:43 PM, Márton Balassi <[hidden email]> wrote: > > @Timo&Stephan: The streaming code uses this since last week [1], Timo > starting working on the issue prior way before that - so it also has to do > with the streaming team pushing last minute features to be honest... > > Then are we waiting for the Java Clousure cleaner and the Kyro support, I > guess. > Aljoscha has already issued a PR for the former, some additional testing is > needed from the looks of it. [2] > > [1] > > https://git-wip-us.apache.org/repos/asf?p=incubator-flink.git;a=commit;h=51c1f67791307c2b9355171f7398d104befc8de5 > [2] https://github.com/apache/incubator-flink/pull/269 > > On Tue, Dec 16, 2014 at 5:52 PM, Timo Walther <[hidden email]> wrote: > > > > Yes, I also think so. Stability is more important. > > > > > Am 16.12.2014 um 17:46 schrieb Stephan Ewen <[hidden email]>: > > > > > > Yes, the fact that we both did not realize that until now sort of > reveals > > > that there are more implications than anticipated. That is never good > for > > > "last minute" patches. > > > > > > That's why I would like to merge it into 0.8.1 and 0.9, rather than > 0.8.0 > > > That gives the code a bit more testing/use and us a bit more time to > > > thoroughly think the implications through. > > > > > >> On Tue, Dec 16, 2014 at 5:30 PM, Timo Walther <[hidden email]> > > wrote: > > >> > > >> Sorry I totally forgot to check the streaming API. Do they use the > > >> getXXXReturnTypes() methods of the TypeExtractor? > > >> > > >>> Am 16.12.2014 um 17:23 schrieb Robert Metzger <[hidden email]>: > > >>> > > >>> I've worked on adding Kryo support to Flink today. I would really > like > > to > > >>> include this into the release. Many users were complaining about > this ( > > >>> https://issues.apache.org/jira/browse/FLINK-610). > > >>> > > >>> I've based my work on top of Timo's pull request. If we're not going > to > > >>> merge it, I will have to rebase my code to the current master. ... > I'll > > >> try > > >>> and see if its easily doable. > > >>> > > >>>> On Tue, Dec 16, 2014 at 5:02 PM, Stephan Ewen <[hidden email]> > > wrote: > > >>>> > > >>>> Yes, one thing I just realized is that the streaming API is not > aware > > of > > >>>> MissingTypeInfos and it may cause weird behavior. > > >>>> > > >>>>> On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske <[hidden email]> > > >> wrote: > > >>>>> > > >>>>> +1 for stability first > > >>>>> > > >>>>> 2014-12-16 16:50 GMT+01:00 Stephan Ewen <[hidden email]>: > > >>>>>> > > >>>>>> I am currently working through the type hints pull request... > > >>>>>> > > >>>>>> Right now, I actually think that we should postpone this for the > > next > > >>>>>> bugfix release. > > >>>>>> > > >>>>>> This pull request changes actually some tricky mechanisms in the > > Java > > >>>> API > > >>>>>> (it delays the type information), which may have implications that > > me > > >>>> and > > >>>>>> Timo have not fully caught now. > > >>>>>> > > >>>>>> I don't feel Merging this last minute - this is prone to introduce > > >>>>> errors. > > >>>>>> > > >>>>>> Any opinions on that? > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> On Tue, Dec 16, 2014 at 3:36 PM, Stephan Ewen <[hidden email]> > > >>>> wrote: > > >>>>>>> > > >>>>>>> +1 for adding the closure cleaner... > > >>>>>>> Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" < > > [hidden email] > > >>>>> : > > >>>>>>> > > >>>>>>> Do we want the Java Closure Cleaner in 0.8 as well? > > >>>>>>>> > > >>>>>>>> On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > > >>>>>>>> <[hidden email]> wrote: > > >>>>>>>>> Cool, thanks. > > >>>>>>>>> > > >>>>>>>>> On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen < > [hidden email] > > > > > >>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>> Hi! > > >>>>>>>>>> > > >>>>>>>>>> The type hints PR is ready now, Timo made some final > adjustments > > >>>>>>>> yesterday > > >>>>>>>>>> evening. I'll review and merge it now, then we can fork the > > >>>> branch. > > >>>>>>>>>> > > >>>>>>>>>> Cheers, > > >>>>>>>>>> Stephan > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi < > > >>>>>>>> [hidden email] > > >>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> Hey, > > >>>>>>>>>>> > > >>>>>>>>>>> Checking the PRs listed by Stephan it seems every import one > > >>>> has > > >>>>>> been > > >>>>>>>>>>> merged. As for the optional ones the following two are still > > >>>>>> pending: > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> Add support for Subclasses, Interfaces, Abstract Classes as > > >>>> POJOs > > >>>>>>>>>>> > > >>>>>>>>>>> [FLINK-1245] Introduce TypeHints for Java API operators > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> Robert +1'd the TypeHints PR and Timo has just updated it. > > >>>> Should > > >>>>>> we > > >>>>>>>> wait > > >>>>>>>>>>> for it? > > >>>>>>>>>>> > > >>>>>>>>>>> Cheers, > > >>>>>>>>>>> > > >>>>>>>>>>> Marton > > >>>>>>>>>>> > > >>>>>>>>>>> On Fri, Dec 12, 2014 at 2:29 PM, Ufuk Celebi <[hidden email] > > > > >>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Wed, Dec 10, 2014 at 7:35 PM, Stephan Ewen < > > >>>>> [hidden email]> > > >>>>>>>>>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> ---------------------------------------------- > > >>>>>>>>>>>>> Important Pull Requests to merge: > > >>>>>>>>>>>>> ---------------------------------------------- > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> [FLINK-1287] LocalizableSplitAssigner prefers splits with > > >>>>> less > > >>>>>>>>>> degrees > > >>>>>>>>>>> of > > >>>>>>>>>>>>> freedom > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> [FLINK-1305] [FLINK-1304] Test for HadoopInputWrapper and > > >>>>>>>>>> NullWritable > > >>>>>>>>>>>>> support > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Upgraded HBase addon to HBase 0.98.x and new Tuple APIs + > > >>>> fix > > >>>>>> of > > >>>>>>>>>>>>> ExecutionEnvironment > > >>>>>>>>>>>>> (this is partially merged, there are a few POM entries > > >>>>> missing > > >>>>>>>> for > > >>>>>>>>>>>> Hadoop 1 > > >>>>>>>>>>>>> support) > > >>>>>>>>>>>> > > >>>>>>>>>>>> The first two PRs are still open. I will have a look at them > > >>>> as > > >>>>>>>> well > > >>>>>>>>>>>> (Robert already tested one). > > >>>>>>>>>>>> > > >>>>>>>>>>>> @Robert: > > >>>>>>>>>>>> Regarding the Hbase PR: Is it resolved by 1b69749 ("Enable > > >>>>>>>> flink-hbase > > >>>>>>>>>>>> module for hadoop1 build as well")? > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> It would be nice if had a look at the remaining important > > >>>> ones > > >>>>>>>> today. > > >>>>>>>>>> :-) > > >> > > > |
Free forum by Nabble | Edit this page |