Hi Squirrels!
I think it is time we started finalizing the the 0.9 release. The latest milestone is a few weeks old and given the sheer amount of new features and the big interest in Flink these days, we should publish the next release rather soon in my opinion. There are a few issues that we need to address to make this an awesome and high quality release. I started collecting issues in the google doc behind the link below. Everyone should have permission to comment and suggest. Please bear in mind that this is to collect critical issues that are either important bugs or critical features that are unreliable right now. For requests to extend or add features, please file a JIRA rather than adding it for this document. https://docs.google.com/document/d/1VOqEpHFWSHyQ1zIVDtYKNBC-iT3r4jQc8COTYcO2Q30/edit?usp=sharing Greetings, Stephan |
+1 for cutting a release soon.
The planning document looks reasonable .. On Wed, May 13, 2015 at 1:37 PM, Stephan Ewen <[hidden email]> wrote: > Hi Squirrels! > > I think it is time we started finalizing the the 0.9 release. The latest > milestone is a few weeks old and given the sheer amount of new features and > the big interest in Flink these days, we should publish the next release > rather soon in my opinion. > > There are a few issues that we need to address to make this an awesome and > high quality release. > > I started collecting issues in the google doc behind the link below. > Everyone should have permission to comment and suggest. > > Please bear in mind that this is to collect critical issues that are either > important bugs or critical features that are unreliable right now. For > requests to extend or add features, please file a JIRA rather than adding > it for this document. > > > https://docs.google.com/document/d/1VOqEpHFWSHyQ1zIVDtYKNBC-iT3r4jQc8COTYcO2Q30/edit?usp=sharing > > > > Greetings, > Stephan > |
What is the status of the 0.9 release planning.
It seems like many of the open issues from the document have been closed. When do you think are we able to fork off the "release-0.9" branch and create the first RC ? On Wed, May 13, 2015 at 2:17 PM, Robert Metzger <[hidden email]> wrote: > +1 for cutting a release soon. > The planning document looks reasonable .. > > > > > On Wed, May 13, 2015 at 1:37 PM, Stephan Ewen <[hidden email]> wrote: > >> Hi Squirrels! >> >> I think it is time we started finalizing the the 0.9 release. The latest >> milestone is a few weeks old and given the sheer amount of new features >> and >> the big interest in Flink these days, we should publish the next release >> rather soon in my opinion. >> >> There are a few issues that we need to address to make this an awesome and >> high quality release. >> >> I started collecting issues in the google doc behind the link below. >> Everyone should have permission to comment and suggest. >> >> Please bear in mind that this is to collect critical issues that are >> either >> important bugs or critical features that are unreliable right now. For >> requests to extend or add features, please file a JIRA rather than adding >> it for this document. >> >> >> https://docs.google.com/document/d/1VOqEpHFWSHyQ1zIVDtYKNBC-iT3r4jQc8COTYcO2Q30/edit?usp=sharing >> >> >> >> Greetings, >> Stephan >> > > |
On 03 Jun 2015, at 17:00, Robert Metzger <[hidden email]> wrote: > What is the status of the 0.9 release planning. > > It seems like many of the open issues from the document have been closed. > When do you think are we able to fork off the "release-0.9" branch and > create the first RC ? It would be great to do it in the next days. If there are no objections, I would like to act as release manager. :-) Critical issues: - Skipped buffer (http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html) (I'm on it) - Execution graph deadlock (FLINK-2133) Big open issue: - There has been consensus to merge the static code analysis PR for this release, which needs some further testing (I'm on it) - Batch scheduling. I think Till and Stephan are working on this. Can you guys give an estimate whether we will be able to have it in the release? The other big issues have been postponed. – Ufuk |
Ping.
On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > Critical issues: > > - Skipped buffer (http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html) (I'm on it) > - Execution graph deadlock (FLINK-2133) I'm investigating the skipped buffer. I vote to not block the release on this though if I can't reproduce it in the next days. What about FLINK-2133? > Big open issue: > - There has been consensus to merge the static code analysis PR for this release, which needs some further testing (I'm on it) > - Batch scheduling. I think Till and Stephan are working on this. Can you guys give an estimate whether we will be able to have it in the release? @Till, Stephan: Will batch scheduling make it into the release? :) |
Yes, Till, Max, Ufuk, and me have reached consensus that we will postpone
the batch scheduling to 0.10. It is a crucial feature, but we would not like to have it in a highly pre-mature version in there. What came up a lot is that there are a good set of additions that are close to completion. We will probably bring up the suggestion of a 0.10 release very soon after the 0.9 release. On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: > Ping. > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > Critical issues: > > > > - Skipped buffer ( > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html) > (I'm on it) > > - Execution graph deadlock (FLINK-2133) > > I'm investigating the skipped buffer. I vote to not block the release on > this though if I can't reproduce it in the next days. > > What about FLINK-2133? > > > Big open issue: > > - There has been consensus to merge the static code analysis PR for this > release, which needs some further testing (I'm on it) > > - Batch scheduling. I think Till and Stephan are working on this. Can > you guys give an estimate whether we will be able to have it in the release? > > @Till, Stephan: Will batch scheduling make it into the release? :) |
I'll address the remaining documentation issues today.
What about - Sync Streaming Java/Scala API - Consolidate names across batch/streaming (discussion) - Merge static code analysis and the gelly TODOs - FLINK-1522 Add tests for the library methods and examples - FLINK-1943 Add Gelly-GSA compiler and translation tests They seem both unresolved. Other than that it seems we are good to go. Maybe we can manage to get the first release candidate out today? I'm uncertain whether we should fork off the "release-0.9" branch as part of the RC creation or whether we should wait a bit with that. Forking it of would allow us to merge some of the pull requests (storm compat), but we would need to apply many patches to two branches. On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> wrote: > Yes, Till, Max, Ufuk, and me have reached consensus that we will postpone > the batch scheduling to 0.10. It is a crucial feature, but we would not > like to have it in a highly pre-mature version in there. > > > > What came up a lot is that there are a good set of additions that are close > to completion. We will probably bring up the suggestion of a 0.10 release > very soon after the 0.9 release. > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: > > > Ping. > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > Critical issues: > > > > > > - Skipped buffer ( > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > ) > > (I'm on it) > > > - Execution graph deadlock (FLINK-2133) > > > > I'm investigating the skipped buffer. I vote to not block the release on > > this though if I can't reproduce it in the next days. > > > > What about FLINK-2133? > > > > > Big open issue: > > > - There has been consensus to merge the static code analysis PR for > this > > release, which needs some further testing (I'm on it) > > > - Batch scheduling. I think Till and Stephan are working on this. Can > > you guys give an estimate whether we will be able to have it in the > release? > > > > @Till, Stephan: Will batch scheduling make it into the release? :) > |
Hi all,
regarding the 2 gelly issues, I'm sorry but I haven't had time to work on these. And most certainly I won't be able to work on these today :S In any case, I wouldn't consider them blocker issues, so if you agree, please go ahead with the release candidate. -Vasia. On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: > I'll address the remaining documentation issues today. > > What about > > - > > Sync Streaming Java/Scala API > - Consolidate names across batch/streaming (discussion) > - Merge static code analysis > > and the gelly TODOs > > - > > FLINK-1522 Add tests for the library methods and examples > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > They seem both unresolved. > > Other than that it seems we are good to go. > Maybe we can manage to get the first release candidate out today? > > I'm uncertain whether we should fork off the "release-0.9" branch as part > of the RC creation or whether we should wait a bit with that. > > Forking it of would allow us to merge some of the pull requests (storm > compat), but we would need to apply many patches to two branches. > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> wrote: > > > Yes, Till, Max, Ufuk, and me have reached consensus that we will postpone > > the batch scheduling to 0.10. It is a crucial feature, but we would not > > like to have it in a highly pre-mature version in there. > > > > > > > > What came up a lot is that there are a good set of additions that are > close > > to completion. We will probably bring up the suggestion of a 0.10 release > > very soon after the 0.9 release. > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: > > > > > Ping. > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > > > Critical issues: > > > > > > > > - Skipped buffer ( > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > ) > > > (I'm on it) > > > > - Execution graph deadlock (FLINK-2133) > > > > > > I'm investigating the skipped buffer. I vote to not block the release > on > > > this though if I can't reproduce it in the next days. > > > > > > What about FLINK-2133? > > > > > > > Big open issue: > > > > - There has been consensus to merge the static code analysis PR for > > this > > > release, which needs some further testing (I'm on it) > > > > - Batch scheduling. I think Till and Stephan are working on this. Can > > > you guys give an estimate whether we will be able to have it in the > > release? > > > > > > @Till, Stephan: Will batch scheduling make it into the release? :) > > > |
In reply to this post by Robert Metzger
On 05 Jun 2015, at 11:46, Robert Metzger <[hidden email]> wrote: > I'll address the remaining documentation issues today. Thank you so much for doing this. > What about > Sync Streaming Java/Scala API > - Consolidate names across batch/streaming (discussion) > - Merge static code analysis > > They seem both unresolved. As I've said in the previous emails, I'm testing the code analysis. It looks good modulo some minor issues, which I will address and merge. > Other than that it seems we are good to go. > Maybe we can manage to get the first release candidate out today? We need 1) to do some initial testing (with the recent streaming changes), and 2) I think the ExecutionGraph deadlock should be fixed (we know exactly what it is). Regarding 1): it does not make a difference whether we do this on master or a RC. RC means more overhead to get this started. Regarding 2): if more people agree that this should be fixed, then this is blocking the RC. |
In reply to this post by Vasiliki Kalavri
Thanks Vasia!
We should clearly label Gelly as Work in Progress and at Beta status, then it should be okay. This is very fair, it is the first version, people understand that. In that sense, let us not call Spargel deprecated in favor of Gelly yet, but do that for the next release when Gelly is ready. Stephan On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri <[hidden email] > wrote: > Hi all, > > regarding the 2 gelly issues, I'm sorry but I haven't had time to work on > these. > And most certainly I won't be able to work on these today :S > > In any case, I wouldn't consider them blocker issues, so if you agree, > please go ahead with the release candidate. > > -Vasia. > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: > > > I'll address the remaining documentation issues today. > > > > What about > > > > - > > > > Sync Streaming Java/Scala API > > - Consolidate names across batch/streaming (discussion) > > - Merge static code analysis > > > > and the gelly TODOs > > > > - > > > > FLINK-1522 Add tests for the library methods and examples > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > They seem both unresolved. > > > > Other than that it seems we are good to go. > > Maybe we can manage to get the first release candidate out today? > > > > I'm uncertain whether we should fork off the "release-0.9" branch as part > > of the RC creation or whether we should wait a bit with that. > > > > Forking it of would allow us to merge some of the pull requests (storm > > compat), but we would need to apply many patches to two branches. > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> wrote: > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we will > postpone > > > the batch scheduling to 0.10. It is a crucial feature, but we would not > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > What came up a lot is that there are a good set of additions that are > > close > > > to completion. We will probably bring up the suggestion of a 0.10 > release > > > very soon after the 0.9 release. > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: > > > > > > > Ping. > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > > > > > Critical issues: > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > ) > > > > (I'm on it) > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > I'm investigating the skipped buffer. I vote to not block the release > > on > > > > this though if I can't reproduce it in the next days. > > > > > > > > What about FLINK-2133? > > > > > > > > > Big open issue: > > > > > - There has been consensus to merge the static code analysis PR for > > > this > > > > release, which needs some further testing (I'm on it) > > > > > - Batch scheduling. I think Till and Stephan are working on this. > Can > > > > you guys give an estimate whether we will be able to have it in the > > > release? > > > > > > > > @Till, Stephan: Will batch scheduling make it into the release? :) > > > > > > |
I will address the ExecutionGraphDeadlock today...
On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> wrote: > Thanks Vasia! > > We should clearly label Gelly as Work in Progress and at Beta status, then > it should be okay. This is very fair, it is the first version, people > understand that. > > In that sense, let us not call Spargel deprecated in favor of Gelly yet, > but do that for the next release when Gelly is ready. > > Stephan > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > [hidden email]> wrote: > >> Hi all, >> >> regarding the 2 gelly issues, I'm sorry but I haven't had time to work on >> these. >> And most certainly I won't be able to work on these today :S >> >> In any case, I wouldn't consider them blocker issues, so if you agree, >> please go ahead with the release candidate. >> >> -Vasia. >> >> On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: >> >> > I'll address the remaining documentation issues today. >> > >> > What about >> > >> > - >> > >> > Sync Streaming Java/Scala API >> > - Consolidate names across batch/streaming (discussion) >> > - Merge static code analysis >> > >> > and the gelly TODOs >> > >> > - >> > >> > FLINK-1522 Add tests for the library methods and examples >> > - FLINK-1943 Add Gelly-GSA compiler and translation tests >> > >> > >> > They seem both unresolved. >> > >> > Other than that it seems we are good to go. >> > Maybe we can manage to get the first release candidate out today? >> > >> > I'm uncertain whether we should fork off the "release-0.9" branch as >> part >> > of the RC creation or whether we should wait a bit with that. >> > >> > Forking it of would allow us to merge some of the pull requests (storm >> > compat), but we would need to apply many patches to two branches. >> > >> > >> > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> wrote: >> > >> > > Yes, Till, Max, Ufuk, and me have reached consensus that we will >> postpone >> > > the batch scheduling to 0.10. It is a crucial feature, but we would >> not >> > > like to have it in a highly pre-mature version in there. >> > > >> > > >> > > >> > > What came up a lot is that there are a good set of additions that are >> > close >> > > to completion. We will probably bring up the suggestion of a 0.10 >> release >> > > very soon after the 0.9 release. >> > > >> > > >> > > >> > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: >> > > >> > > > Ping. >> > > > >> > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: >> > > > >> > > > > Critical issues: >> > > > > >> > > > > - Skipped buffer ( >> > > > >> > > >> > >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html >> > > ) >> > > > (I'm on it) >> > > > > - Execution graph deadlock (FLINK-2133) >> > > > >> > > > I'm investigating the skipped buffer. I vote to not block the >> release >> > on >> > > > this though if I can't reproduce it in the next days. >> > > > >> > > > What about FLINK-2133? >> > > > >> > > > > Big open issue: >> > > > > - There has been consensus to merge the static code analysis PR >> for >> > > this >> > > > release, which needs some further testing (I'm on it) >> > > > > - Batch scheduling. I think Till and Stephan are working on this. >> Can >> > > > you guys give an estimate whether we will be able to have it in the >> > > release? >> > > > >> > > > @Till, Stephan: Will batch scheduling make it into the release? :) >> > > >> > >> > > |
In reply to this post by Stephan Ewen
Hi Stephan,
I don't know if I have a saying in this, but I will give it a go :) The two unsolved issues don't affect the functionality at all. Gelly can, at the moment, support anything Spargel could. There is a guide in the documentation explaining how to migrate Spargel code to Gelly. I don't see why Spargel should not be deprecated yet. Just because of those missing tests? On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> wrote: > Thanks Vasia! > > We should clearly label Gelly as Work in Progress and at Beta status, then > it should be okay. This is very fair, it is the first version, people > understand that. > > In that sense, let us not call Spargel deprecated in favor of Gelly yet, > but do that for the next release when Gelly is ready. > > Stephan > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > [hidden email] > > wrote: > > > Hi all, > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time to work on > > these. > > And most certainly I won't be able to work on these today :S > > > > In any case, I wouldn't consider them blocker issues, so if you agree, > > please go ahead with the release candidate. > > > > -Vasia. > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: > > > > > I'll address the remaining documentation issues today. > > > > > > What about > > > > > > - > > > > > > Sync Streaming Java/Scala API > > > - Consolidate names across batch/streaming (discussion) > > > - Merge static code analysis > > > > > > and the gelly TODOs > > > > > > - > > > > > > FLINK-1522 Add tests for the library methods and examples > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > They seem both unresolved. > > > > > > Other than that it seems we are good to go. > > > Maybe we can manage to get the first release candidate out today? > > > > > > I'm uncertain whether we should fork off the "release-0.9" branch as > part > > > of the RC creation or whether we should wait a bit with that. > > > > > > Forking it of would allow us to merge some of the pull requests (storm > > > compat), but we would need to apply many patches to two branches. > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> > wrote: > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we will > > postpone > > > > the batch scheduling to 0.10. It is a crucial feature, but we would > not > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of additions that are > > > close > > > > to completion. We will probably bring up the suggestion of a 0.10 > > release > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> wrote: > > > > > > > > > Ping. > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > > > > > > > Critical issues: > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > ) > > > > > (I'm on it) > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block the > release > > > on > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > Big open issue: > > > > > > - There has been consensus to merge the static code analysis PR > for > > > > this > > > > > release, which needs some further testing (I'm on it) > > > > > > - Batch scheduling. I think Till and Stephan are working on this. > > Can > > > > > you guys give an estimate whether we will be able to have it in the > > > > release? > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the release? :) > > > > > > > > > > |
Okay, I was not aware it is only two missing tests. That is not that big a
deal. I am not very attached to the Spargel Stuff, I just want to make sure we do not deprecate something that works well for something that is still work in progress. On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> wrote: > Hi Stephan, > > I don't know if I have a saying in this, but I will give it a go :) > > The two unsolved issues don't affect the functionality at all. > Gelly can, at the moment, support anything Spargel could. There is a guide > in the documentation explaining how to migrate Spargel code to Gelly. I > don't see why Spargel should not be deprecated yet. Just because of those > missing tests? > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> wrote: > > > Thanks Vasia! > > > > We should clearly label Gelly as Work in Progress and at Beta status, > then > > it should be okay. This is very fair, it is the first version, people > > understand that. > > > > In that sense, let us not call Spargel deprecated in favor of Gelly yet, > > but do that for the next release when Gelly is ready. > > > > Stephan > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > [hidden email] > > > wrote: > > > > > Hi all, > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time to work > on > > > these. > > > And most certainly I won't be able to work on these today :S > > > > > > In any case, I wouldn't consider them blocker issues, so if you agree, > > > please go ahead with the release candidate. > > > > > > -Vasia. > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > What about > > > > > > > > - > > > > > > > > Sync Streaming Java/Scala API > > > > - Consolidate names across batch/streaming (discussion) > > > > - Merge static code analysis > > > > > > > > and the gelly TODOs > > > > > > > > - > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > They seem both unresolved. > > > > > > > > Other than that it seems we are good to go. > > > > Maybe we can manage to get the first release candidate out today? > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" branch as > > part > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > Forking it of would allow us to merge some of the pull requests > (storm > > > > compat), but we would need to apply many patches to two branches. > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> > > wrote: > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we will > > > postpone > > > > > the batch scheduling to 0.10. It is a crucial feature, but we would > > not > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of additions that > are > > > > close > > > > > to completion. We will probably bring up the suggestion of a 0.10 > > > release > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> > wrote: > > > > > > > > > > > Ping. > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > ) > > > > > > (I'm on it) > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block the > > release > > > > on > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > Big open issue: > > > > > > > - There has been consensus to merge the static code analysis PR > > for > > > > > this > > > > > > release, which needs some further testing (I'm on it) > > > > > > > - Batch scheduling. I think Till and Stephan are working on > this. > > > Can > > > > > > you guys give an estimate whether we will be able to have it in > the > > > > > release? > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the release? > :) > > > > > > > > > > > > > > > |
The Pregel-like, vertex-cenric part of Gelly is as stable as it will ever
be. I vote for deprecating Spargel in this release, but keep in mind that this is just an opinion :) On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> wrote: > Okay, I was not aware it is only two missing tests. That is not that big a > deal. > > I am not very attached to the Spargel Stuff, I just want to make sure we do > not deprecate something that works well for something that is still work in > progress. > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> wrote: > > > Hi Stephan, > > > > I don't know if I have a saying in this, but I will give it a go :) > > > > The two unsolved issues don't affect the functionality at all. > > Gelly can, at the moment, support anything Spargel could. There is a > guide > > in the documentation explaining how to migrate Spargel code to Gelly. I > > don't see why Spargel should not be deprecated yet. Just because of those > > missing tests? > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> wrote: > > > > > Thanks Vasia! > > > > > > We should clearly label Gelly as Work in Progress and at Beta status, > > then > > > it should be okay. This is very fair, it is the first version, people > > > understand that. > > > > > > In that sense, let us not call Spargel deprecated in favor of Gelly > yet, > > > but do that for the next release when Gelly is ready. > > > > > > Stephan > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > [hidden email] > > > > wrote: > > > > > > > Hi all, > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time to > work > > on > > > > these. > > > > And most certainly I won't be able to work on these today :S > > > > > > > > In any case, I wouldn't consider them blocker issues, so if you > agree, > > > > please go ahead with the release candidate. > > > > > > > > -Vasia. > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> wrote: > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > What about > > > > > > > > > > - > > > > > > > > > > Sync Streaming Java/Scala API > > > > > - Consolidate names across batch/streaming (discussion) > > > > > - Merge static code analysis > > > > > > > > > > and the gelly TODOs > > > > > > > > > > - > > > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > Other than that it seems we are good to go. > > > > > Maybe we can manage to get the first release candidate out today? > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" branch > as > > > part > > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > > > Forking it of would allow us to merge some of the pull requests > > (storm > > > > > compat), but we would need to apply many patches to two branches. > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> > > > wrote: > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we will > > > > postpone > > > > > > the batch scheduling to 0.10. It is a crucial feature, but we > would > > > not > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of additions that > > are > > > > > close > > > > > > to completion. We will probably bring up the suggestion of a 0.10 > > > > release > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> > > wrote: > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> wrote: > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > ) > > > > > > > (I'm on it) > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block the > > > release > > > > > on > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > Big open issue: > > > > > > > > - There has been consensus to merge the static code analysis > PR > > > for > > > > > > this > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > - Batch scheduling. I think Till and Stephan are working on > > this. > > > > Can > > > > > > > you guys give an estimate whether we will be able to have it in > > the > > > > > > release? > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the release? > > :) > > > > > > > > > > > > > > > > > > > > > |
Hi,
let me clarify: FLINK-1252 is missing a test for PageRank (which might not even be needed, since the implementation is basically identical to the existing Spargel one) and a test for MusicProfiles, which is basically using LabelProgation (and we have a separate test for this). FLINK-1943 is about implementing a compiler and translation test for the recently added Gather-Sum-Apply iteration. IMO, the second would be nice to have, but not a blocker. I could work on it after my paper deadline, in a week. But since I see you're eager to have the release today, we could include this is the first bugfix of 0.9. -Vasia. On 5 June 2015 at 13:59, Andra Lungu <[hidden email]> wrote: > The Pregel-like, vertex-cenric part of Gelly is as stable as it will ever > be. > I vote for deprecating Spargel in this release, but keep in mind that this > is just an opinion :) > > On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> wrote: > > > Okay, I was not aware it is only two missing tests. That is not that big > a > > deal. > > > > I am not very attached to the Spargel Stuff, I just want to make sure we > do > > not deprecate something that works well for something that is still work > in > > progress. > > > > > > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> > wrote: > > > > > Hi Stephan, > > > > > > I don't know if I have a saying in this, but I will give it a go :) > > > > > > The two unsolved issues don't affect the functionality at all. > > > Gelly can, at the moment, support anything Spargel could. There is a > > guide > > > in the documentation explaining how to migrate Spargel code to Gelly. I > > > don't see why Spargel should not be deprecated yet. Just because of > those > > > missing tests? > > > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> wrote: > > > > > > > Thanks Vasia! > > > > > > > > We should clearly label Gelly as Work in Progress and at Beta status, > > > then > > > > it should be okay. This is very fair, it is the first version, people > > > > understand that. > > > > > > > > In that sense, let us not call Spargel deprecated in favor of Gelly > > yet, > > > > but do that for the next release when Gelly is ready. > > > > > > > > Stephan > > > > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > > [hidden email] > > > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time to > > work > > > on > > > > > these. > > > > > And most certainly I won't be able to work on these today :S > > > > > > > > > > In any case, I wouldn't consider them blocker issues, so if you > > agree, > > > > > please go ahead with the release candidate. > > > > > > > > > > -Vasia. > > > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> > wrote: > > > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > > > What about > > > > > > > > > > > > - > > > > > > > > > > > > Sync Streaming Java/Scala API > > > > > > - Consolidate names across batch/streaming (discussion) > > > > > > - Merge static code analysis > > > > > > > > > > > > and the gelly TODOs > > > > > > > > > > > > - > > > > > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > > > Other than that it seems we are good to go. > > > > > > Maybe we can manage to get the first release candidate out today? > > > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" branch > > as > > > > part > > > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > > > > > Forking it of would allow us to merge some of the pull requests > > > (storm > > > > > > compat), but we would need to apply many patches to two branches. > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen <[hidden email]> > > > > wrote: > > > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we > will > > > > > postpone > > > > > > > the batch scheduling to 0.10. It is a crucial feature, but we > > would > > > > not > > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of additions > that > > > are > > > > > > close > > > > > > > to completion. We will probably bring up the suggestion of a > 0.10 > > > > > release > > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email]> > > > wrote: > > > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> > wrote: > > > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > > ) > > > > > > > > (I'm on it) > > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block the > > > > release > > > > > > on > > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > > > Big open issue: > > > > > > > > > - There has been consensus to merge the static code > analysis > > PR > > > > for > > > > > > > this > > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > > - Batch scheduling. I think Till and Stephan are working on > > > this. > > > > > Can > > > > > > > > you guys give an estimate whether we will be able to have it > in > > > the > > > > > > > release? > > > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the > release? > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Fair enough about including the issues into 0.9.1
Concerning Gelly, would you recommend people to use that in production today? If not, it would be nice to have some non-deprecated code where we are confident about that. On Fri, Jun 5, 2015 at 2:08 PM, Vasiliki Kalavri <[hidden email]> wrote: > Hi, > > let me clarify: > > FLINK-1252 is missing a test for PageRank (which might not even be needed, > since the implementation is basically identical to the existing Spargel > one) and a test for MusicProfiles, which is basically using LabelProgation > (and we have a separate test for this). > > FLINK-1943 is about implementing a compiler and translation test for the > recently added Gather-Sum-Apply iteration. > > IMO, the second would be nice to have, but not a blocker. > I could work on it after my paper deadline, in a week. But since I see > you're eager to have the release today, we could include this is the first > bugfix of 0.9. > > -Vasia. > > > On 5 June 2015 at 13:59, Andra Lungu <[hidden email]> wrote: > > > The Pregel-like, vertex-cenric part of Gelly is as stable as it will ever > > be. > > I vote for deprecating Spargel in this release, but keep in mind that > this > > is just an opinion :) > > > > On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> wrote: > > > > > Okay, I was not aware it is only two missing tests. That is not that > big > > a > > > deal. > > > > > > I am not very attached to the Spargel Stuff, I just want to make sure > we > > do > > > not deprecate something that works well for something that is still > work > > in > > > progress. > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> > > wrote: > > > > > > > Hi Stephan, > > > > > > > > I don't know if I have a saying in this, but I will give it a go :) > > > > > > > > The two unsolved issues don't affect the functionality at all. > > > > Gelly can, at the moment, support anything Spargel could. There is a > > > guide > > > > in the documentation explaining how to migrate Spargel code to > Gelly. I > > > > don't see why Spargel should not be deprecated yet. Just because of > > those > > > > missing tests? > > > > > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> > wrote: > > > > > > > > > Thanks Vasia! > > > > > > > > > > We should clearly label Gelly as Work in Progress and at Beta > status, > > > > then > > > > > it should be okay. This is very fair, it is the first version, > people > > > > > understand that. > > > > > > > > > > In that sense, let us not call Spargel deprecated in favor of Gelly > > > yet, > > > > > but do that for the next release when Gelly is ready. > > > > > > > > > > Stephan > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > > > [hidden email] > > > > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time to > > > work > > > > on > > > > > > these. > > > > > > And most certainly I won't be able to work on these today :S > > > > > > > > > > > > In any case, I wouldn't consider them blocker issues, so if you > > > agree, > > > > > > please go ahead with the release candidate. > > > > > > > > > > > > -Vasia. > > > > > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> > > wrote: > > > > > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > > > > > What about > > > > > > > > > > > > > > - > > > > > > > > > > > > > > Sync Streaming Java/Scala API > > > > > > > - Consolidate names across batch/streaming (discussion) > > > > > > > - Merge static code analysis > > > > > > > > > > > > > > and the gelly TODOs > > > > > > > > > > > > > > - > > > > > > > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > > > > > Other than that it seems we are good to go. > > > > > > > Maybe we can manage to get the first release candidate out > today? > > > > > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" > branch > > > as > > > > > part > > > > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > > > > > > > Forking it of would allow us to merge some of the pull requests > > > > (storm > > > > > > > compat), but we would need to apply many patches to two > branches. > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen < > [hidden email]> > > > > > wrote: > > > > > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we > > will > > > > > > postpone > > > > > > > > the batch scheduling to 0.10. It is a crucial feature, but we > > > would > > > > > not > > > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of additions > > that > > > > are > > > > > > > close > > > > > > > > to completion. We will probably bring up the suggestion of a > > 0.10 > > > > > > release > > > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi <[hidden email] > > > > > > wrote: > > > > > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> > > wrote: > > > > > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > > > ) > > > > > > > > > (I'm on it) > > > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block > the > > > > > release > > > > > > > on > > > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > > > > > Big open issue: > > > > > > > > > > - There has been consensus to merge the static code > > analysis > > > PR > > > > > for > > > > > > > > this > > > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > > > - Batch scheduling. I think Till and Stephan are working > on > > > > this. > > > > > > Can > > > > > > > > > you guys give an estimate whether we will be able to have > it > > in > > > > the > > > > > > > > release? > > > > > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the > > release? > > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
I would. Vasia already used it in production at Telefonica :)
On Fri, Jun 5, 2015 at 3:51 PM, Stephan Ewen <[hidden email]> wrote: > Fair enough about including the issues into 0.9.1 > > Concerning Gelly, would you recommend people to use that in production > today? If not, it would be nice to have some non-deprecated code where we > are confident about that. > > On Fri, Jun 5, 2015 at 2:08 PM, Vasiliki Kalavri < > [hidden email]> > wrote: > > > Hi, > > > > let me clarify: > > > > FLINK-1252 is missing a test for PageRank (which might not even be > needed, > > since the implementation is basically identical to the existing Spargel > > one) and a test for MusicProfiles, which is basically using > LabelProgation > > (and we have a separate test for this). > > > > FLINK-1943 is about implementing a compiler and translation test for the > > recently added Gather-Sum-Apply iteration. > > > > IMO, the second would be nice to have, but not a blocker. > > I could work on it after my paper deadline, in a week. But since I see > > you're eager to have the release today, we could include this is the > first > > bugfix of 0.9. > > > > -Vasia. > > > > > > On 5 June 2015 at 13:59, Andra Lungu <[hidden email]> wrote: > > > > > The Pregel-like, vertex-cenric part of Gelly is as stable as it will > ever > > > be. > > > I vote for deprecating Spargel in this release, but keep in mind that > > this > > > is just an opinion :) > > > > > > On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> wrote: > > > > > > > Okay, I was not aware it is only two missing tests. That is not that > > big > > > a > > > > deal. > > > > > > > > I am not very attached to the Spargel Stuff, I just want to make sure > > we > > > do > > > > not deprecate something that works well for something that is still > > work > > > in > > > > progress. > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> > > > wrote: > > > > > > > > > Hi Stephan, > > > > > > > > > > I don't know if I have a saying in this, but I will give it a go :) > > > > > > > > > > The two unsolved issues don't affect the functionality at all. > > > > > Gelly can, at the moment, support anything Spargel could. There is > a > > > > guide > > > > > in the documentation explaining how to migrate Spargel code to > > Gelly. I > > > > > don't see why Spargel should not be deprecated yet. Just because of > > > those > > > > > missing tests? > > > > > > > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> > > wrote: > > > > > > > > > > > Thanks Vasia! > > > > > > > > > > > > We should clearly label Gelly as Work in Progress and at Beta > > status, > > > > > then > > > > > > it should be okay. This is very fair, it is the first version, > > people > > > > > > understand that. > > > > > > > > > > > > In that sense, let us not call Spargel deprecated in favor of > Gelly > > > > yet, > > > > > > but do that for the next release when Gelly is ready. > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > > > > [hidden email] > > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time > to > > > > work > > > > > on > > > > > > > these. > > > > > > > And most certainly I won't be able to work on these today :S > > > > > > > > > > > > > > In any case, I wouldn't consider them blocker issues, so if you > > > > agree, > > > > > > > please go ahead with the release candidate. > > > > > > > > > > > > > > -Vasia. > > > > > > > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> > > > wrote: > > > > > > > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > > > > > > > What about > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > Sync Streaming Java/Scala API > > > > > > > > - Consolidate names across batch/streaming (discussion) > > > > > > > > - Merge static code analysis > > > > > > > > > > > > > > > > and the gelly TODOs > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > > > > > > > Other than that it seems we are good to go. > > > > > > > > Maybe we can manage to get the first release candidate out > > today? > > > > > > > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" > > branch > > > > as > > > > > > part > > > > > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > > > > > > > > > Forking it of would allow us to merge some of the pull > requests > > > > > (storm > > > > > > > > compat), but we would need to apply many patches to two > > branches. > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen < > > [hidden email]> > > > > > > wrote: > > > > > > > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we > > > will > > > > > > > postpone > > > > > > > > > the batch scheduling to 0.10. It is a crucial feature, but > we > > > > would > > > > > > not > > > > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of > additions > > > that > > > > > are > > > > > > > > close > > > > > > > > > to completion. We will probably bring up the suggestion of > a > > > 0.10 > > > > > > > release > > > > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi < > [hidden email] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> > > > wrote: > > > > > > > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > > > > ) > > > > > > > > > > (I'm on it) > > > > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block > > the > > > > > > release > > > > > > > > on > > > > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > > > > > > > Big open issue: > > > > > > > > > > > - There has been consensus to merge the static code > > > analysis > > > > PR > > > > > > for > > > > > > > > > this > > > > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > > > > - Batch scheduling. I think Till and Stephan are > working > > on > > > > > this. > > > > > > > Can > > > > > > > > > > you guys give an estimate whether we will be able to have > > it > > > in > > > > > the > > > > > > > > > release? > > > > > > > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the > > > release? > > > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
In reply to this post by Stephan Ewen
If you want my personal opinion, I'd say yes.
I am currently using Gelly for all my projects. For one of them, we have been running experiments over the last 4 months and we'll be deploying it in production very soon :) Gelly did not change any internals or runtime features; it simply builds on the DataSet API and the delta iterations and these are stable features. The motivation for deprecating Spargel and creating the migration guide was that we don't want to confuse people by having two Graph APIs. If you feel uncomfortable with a deprecated API and one in beta, then go ahead and change that. It's no problem for me. Cheers, -V. On 5 June 2015 at 15:51, Stephan Ewen <[hidden email]> wrote: > Fair enough about including the issues into 0.9.1 > > Concerning Gelly, would you recommend people to use that in production > today? If not, it would be nice to have some non-deprecated code where we > are confident about that. > > On Fri, Jun 5, 2015 at 2:08 PM, Vasiliki Kalavri < > [hidden email]> > wrote: > > > Hi, > > > > let me clarify: > > > > FLINK-1252 is missing a test for PageRank (which might not even be > needed, > > since the implementation is basically identical to the existing Spargel > > one) and a test for MusicProfiles, which is basically using > LabelProgation > > (and we have a separate test for this). > > > > FLINK-1943 is about implementing a compiler and translation test for the > > recently added Gather-Sum-Apply iteration. > > > > IMO, the second would be nice to have, but not a blocker. > > I could work on it after my paper deadline, in a week. But since I see > > you're eager to have the release today, we could include this is the > first > > bugfix of 0.9. > > > > -Vasia. > > > > > > On 5 June 2015 at 13:59, Andra Lungu <[hidden email]> wrote: > > > > > The Pregel-like, vertex-cenric part of Gelly is as stable as it will > ever > > > be. > > > I vote for deprecating Spargel in this release, but keep in mind that > > this > > > is just an opinion :) > > > > > > On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> wrote: > > > > > > > Okay, I was not aware it is only two missing tests. That is not that > > big > > > a > > > > deal. > > > > > > > > I am not very attached to the Spargel Stuff, I just want to make sure > > we > > > do > > > > not deprecate something that works well for something that is still > > work > > > in > > > > progress. > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email]> > > > wrote: > > > > > > > > > Hi Stephan, > > > > > > > > > > I don't know if I have a saying in this, but I will give it a go :) > > > > > > > > > > The two unsolved issues don't affect the functionality at all. > > > > > Gelly can, at the moment, support anything Spargel could. There is > a > > > > guide > > > > > in the documentation explaining how to migrate Spargel code to > > Gelly. I > > > > > don't see why Spargel should not be deprecated yet. Just because of > > > those > > > > > missing tests? > > > > > > > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> > > wrote: > > > > > > > > > > > Thanks Vasia! > > > > > > > > > > > > We should clearly label Gelly as Work in Progress and at Beta > > status, > > > > > then > > > > > > it should be okay. This is very fair, it is the first version, > > people > > > > > > understand that. > > > > > > > > > > > > In that sense, let us not call Spargel deprecated in favor of > Gelly > > > > yet, > > > > > > but do that for the next release when Gelly is ready. > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > > > > [hidden email] > > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had time > to > > > > work > > > > > on > > > > > > > these. > > > > > > > And most certainly I won't be able to work on these today :S > > > > > > > > > > > > > > In any case, I wouldn't consider them blocker issues, so if you > > > > agree, > > > > > > > please go ahead with the release candidate. > > > > > > > > > > > > > > -Vasia. > > > > > > > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email]> > > > wrote: > > > > > > > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > > > > > > > What about > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > Sync Streaming Java/Scala API > > > > > > > > - Consolidate names across batch/streaming (discussion) > > > > > > > > - Merge static code analysis > > > > > > > > > > > > > > > > and the gelly TODOs > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > FLINK-1522 Add tests for the library methods and examples > > > > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation tests > > > > > > > > > > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > > > > > > > Other than that it seems we are good to go. > > > > > > > > Maybe we can manage to get the first release candidate out > > today? > > > > > > > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" > > branch > > > > as > > > > > > part > > > > > > > > of the RC creation or whether we should wait a bit with that. > > > > > > > > > > > > > > > > Forking it of would allow us to merge some of the pull > requests > > > > > (storm > > > > > > > > compat), but we would need to apply many patches to two > > branches. > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen < > > [hidden email]> > > > > > > wrote: > > > > > > > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that we > > > will > > > > > > > postpone > > > > > > > > > the batch scheduling to 0.10. It is a crucial feature, but > we > > > > would > > > > > > not > > > > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of > additions > > > that > > > > > are > > > > > > > > close > > > > > > > > > to completion. We will probably bring up the suggestion of > a > > > 0.10 > > > > > > > release > > > > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi < > [hidden email] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> > > > wrote: > > > > > > > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > > > > ) > > > > > > > > > > (I'm on it) > > > > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not block > > the > > > > > > release > > > > > > > > on > > > > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > > > > > > > Big open issue: > > > > > > > > > > > - There has been consensus to merge the static code > > > analysis > > > > PR > > > > > > for > > > > > > > > > this > > > > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > > > > - Batch scheduling. I think Till and Stephan are > working > > on > > > > > this. > > > > > > > Can > > > > > > > > > > you guys give an estimate whether we will be able to have > > it > > > in > > > > > the > > > > > > > > > release? > > > > > > > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the > > > release? > > > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Okay, then I agree with you!
Thanks for clarifying that :-) On Fri, Jun 5, 2015 at 4:13 PM, Vasiliki Kalavri <[hidden email]> wrote: > If you want my personal opinion, I'd say yes. > > I am currently using Gelly for all my projects. For one of them, we have > been running experiments over the last 4 months and we'll be deploying it > in production very soon :) > > Gelly did not change any internals or runtime features; it simply builds on > the DataSet API and the delta iterations and these are stable features. > > The motivation for deprecating Spargel and creating the migration guide was > that we don't want to confuse people by having two Graph APIs. If you feel > uncomfortable with a deprecated API and one in beta, then go ahead and > change that. It's no problem for me. > > Cheers, > -V. > > On 5 June 2015 at 15:51, Stephan Ewen <[hidden email]> wrote: > > > Fair enough about including the issues into 0.9.1 > > > > Concerning Gelly, would you recommend people to use that in production > > today? If not, it would be nice to have some non-deprecated code where we > > are confident about that. > > > > On Fri, Jun 5, 2015 at 2:08 PM, Vasiliki Kalavri < > > [hidden email]> > > wrote: > > > > > Hi, > > > > > > let me clarify: > > > > > > FLINK-1252 is missing a test for PageRank (which might not even be > > needed, > > > since the implementation is basically identical to the existing Spargel > > > one) and a test for MusicProfiles, which is basically using > > LabelProgation > > > (and we have a separate test for this). > > > > > > FLINK-1943 is about implementing a compiler and translation test for > the > > > recently added Gather-Sum-Apply iteration. > > > > > > IMO, the second would be nice to have, but not a blocker. > > > I could work on it after my paper deadline, in a week. But since I see > > > you're eager to have the release today, we could include this is the > > first > > > bugfix of 0.9. > > > > > > -Vasia. > > > > > > > > > On 5 June 2015 at 13:59, Andra Lungu <[hidden email]> wrote: > > > > > > > The Pregel-like, vertex-cenric part of Gelly is as stable as it will > > ever > > > > be. > > > > I vote for deprecating Spargel in this release, but keep in mind that > > > this > > > > is just an opinion :) > > > > > > > > On Fri, Jun 5, 2015 at 1:50 PM, Stephan Ewen <[hidden email]> > wrote: > > > > > > > > > Okay, I was not aware it is only two missing tests. That is not > that > > > big > > > > a > > > > > deal. > > > > > > > > > > I am not very attached to the Spargel Stuff, I just want to make > sure > > > we > > > > do > > > > > not deprecate something that works well for something that is still > > > work > > > > in > > > > > progress. > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu <[hidden email] > > > > > > wrote: > > > > > > > > > > > Hi Stephan, > > > > > > > > > > > > I don't know if I have a saying in this, but I will give it a go > :) > > > > > > > > > > > > The two unsolved issues don't affect the functionality at all. > > > > > > Gelly can, at the moment, support anything Spargel could. There > is > > a > > > > > guide > > > > > > in the documentation explaining how to migrate Spargel code to > > > Gelly. I > > > > > > don't see why Spargel should not be deprecated yet. Just because > of > > > > those > > > > > > missing tests? > > > > > > > > > > > > On Fri, Jun 5, 2015 at 1:40 PM, Stephan Ewen <[hidden email]> > > > wrote: > > > > > > > > > > > > > Thanks Vasia! > > > > > > > > > > > > > > We should clearly label Gelly as Work in Progress and at Beta > > > status, > > > > > > then > > > > > > > it should be okay. This is very fair, it is the first version, > > > people > > > > > > > understand that. > > > > > > > > > > > > > > In that sense, let us not call Spargel deprecated in favor of > > Gelly > > > > > yet, > > > > > > > but do that for the next release when Gelly is ready. > > > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 12:10 PM, Vasiliki Kalavri < > > > > > > > [hidden email] > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > regarding the 2 gelly issues, I'm sorry but I haven't had > time > > to > > > > > work > > > > > > on > > > > > > > > these. > > > > > > > > And most certainly I won't be able to work on these today :S > > > > > > > > > > > > > > > > In any case, I wouldn't consider them blocker issues, so if > you > > > > > agree, > > > > > > > > please go ahead with the release candidate. > > > > > > > > > > > > > > > > -Vasia. > > > > > > > > > > > > > > > > On 5 June 2015 at 11:46, Robert Metzger <[hidden email] > > > > > > wrote: > > > > > > > > > > > > > > > > > I'll address the remaining documentation issues today. > > > > > > > > > > > > > > > > > > What about > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > > > Sync Streaming Java/Scala API > > > > > > > > > - Consolidate names across batch/streaming (discussion) > > > > > > > > > - Merge static code analysis > > > > > > > > > > > > > > > > > > and the gelly TODOs > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > > > > > > FLINK-1522 Add tests for the library methods and > examples > > > > > > > > > - FLINK-1943 Add Gelly-GSA compiler and translation > tests > > > > > > > > > > > > > > > > > > > > > > > > > > > They seem both unresolved. > > > > > > > > > > > > > > > > > > Other than that it seems we are good to go. > > > > > > > > > Maybe we can manage to get the first release candidate out > > > today? > > > > > > > > > > > > > > > > > > I'm uncertain whether we should fork off the "release-0.9" > > > branch > > > > > as > > > > > > > part > > > > > > > > > of the RC creation or whether we should wait a bit with > that. > > > > > > > > > > > > > > > > > > Forking it of would allow us to merge some of the pull > > requests > > > > > > (storm > > > > > > > > > compat), but we would need to apply many patches to two > > > branches. > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:56 AM, Stephan Ewen < > > > [hidden email]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Yes, Till, Max, Ufuk, and me have reached consensus that > we > > > > will > > > > > > > > postpone > > > > > > > > > > the batch scheduling to 0.10. It is a crucial feature, > but > > we > > > > > would > > > > > > > not > > > > > > > > > > like to have it in a highly pre-mature version in there. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What came up a lot is that there are a good set of > > additions > > > > that > > > > > > are > > > > > > > > > close > > > > > > > > > > to completion. We will probably bring up the suggestion > of > > a > > > > 0.10 > > > > > > > > release > > > > > > > > > > very soon after the 0.9 release. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 5, 2015 at 10:50 AM, Ufuk Celebi < > > [hidden email] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Ping. > > > > > > > > > > > > > > > > > > > > > > On 04 Jun 2015, at 14:11, Ufuk Celebi <[hidden email]> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Critical issues: > > > > > > > > > > > > > > > > > > > > > > > > - Skipped buffer ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html > > > > > > > > > > ) > > > > > > > > > > > (I'm on it) > > > > > > > > > > > > - Execution graph deadlock (FLINK-2133) > > > > > > > > > > > > > > > > > > > > > > I'm investigating the skipped buffer. I vote to not > block > > > the > > > > > > > release > > > > > > > > > on > > > > > > > > > > > this though if I can't reproduce it in the next days. > > > > > > > > > > > > > > > > > > > > > > What about FLINK-2133? > > > > > > > > > > > > > > > > > > > > > > > Big open issue: > > > > > > > > > > > > - There has been consensus to merge the static code > > > > analysis > > > > > PR > > > > > > > for > > > > > > > > > > this > > > > > > > > > > > release, which needs some further testing (I'm on it) > > > > > > > > > > > > - Batch scheduling. I think Till and Stephan are > > working > > > on > > > > > > this. > > > > > > > > Can > > > > > > > > > > > you guys give an estimate whether we will be able to > have > > > it > > > > in > > > > > > the > > > > > > > > > > release? > > > > > > > > > > > > > > > > > > > > > > @Till, Stephan: Will batch scheduling make it into the > > > > release? > > > > > > :) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
In reply to this post by Ufuk Celebi-2
Hi everyone,
I'm excited about the upcoming release. I think a few issues still need to be addressed. At least for me, those were - fixing errors messages on builds with the JDK8 - removing Apache thrift dependencies as of https://issues.apache.org/jira/browse/FLINK-1635 - Possibly fix an issue in the delta iterations: https://issues.apache.org/jira/browse/FLINK-1916 @Ufuk: If you're too busy, I could also act as a release manager. Cheers, Max On Thu, Jun 4, 2015 at 2:11 PM, Ufuk Celebi <[hidden email]> wrote: > > On 03 Jun 2015, at 17:00, Robert Metzger <[hidden email]> wrote: > > > What is the status of the 0.9 release planning. > > > > It seems like many of the open issues from the document have been closed. > > When do you think are we able to fork off the "release-0.9" branch and > > create the first RC ? > > It would be great to do it in the next days. If there are no objections, I > would like to act as release manager. :-) > > Critical issues: > > - Skipped buffer ( > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Buffer-re-ordering-problem-td6009.html) > (I'm on it) > - Execution graph deadlock (FLINK-2133) > > Big open issue: > - There has been consensus to merge the static code analysis PR for this > release, which needs some further testing (I'm on it) > - Batch scheduling. I think Till and Stephan are working on this. Can you > guys give an estimate whether we will be able to have it in the release? > > The other big issues have been postponed. > > – Ufuk |
Free forum by Nabble | Edit this page |