[ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
Locked
12
12
[ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
 
|
Hi Flink devs,
As previously announced by Kurt [1], the release branch for 1.9.0 has been cut [2] and we've now started the testing phase for this release, as well as resolving remaining blockers. I want to quickly provide an overview of our progress here. Also, over the course of the testing phase, we will update this mail thread every 2-3 days with the overall progress of the release to keep you updated. *1. Remaining blockers and critical issues* You can find a link here [3] for a release Kanban board that provides an overview of the remaining blockers and critical issues for releasing 1.9.0. The issues listed there are high priority for the release, so any help with reviewing or fixing them is highly appreciated! If you do assign yourself to any unassigned issue and start working on it, please make sure to pull it to the "In Progress" column to let others be aware of this. *2. Creating RC 0 for 1.9.0* We will create RC0 now to drive forward the testing efforts. This should be ready by tomorrow morning (July 16, 8am CET). Note that we will not have an official vote for RC0, as this is mainly to drive testing efforts. RC1 with an official vote will be created once the blockers listed in [3] are resolved. Cheers, Gordon [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html [2] https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 [3] https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Update: RC0 for 1.9.0 has been created. Please see [1] for the preview
source / binary releases and Maven artifacts. Cheers, Gordon [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote: > Hi Flink devs, > > As previously announced by Kurt [1], the release branch for 1.9.0 has been > cut [2] and we've now started the testing phase for this release, as well > as resolving remaining blockers. > > I want to quickly provide an overview of our progress here. > Also, over the course of the testing phase, we will update this mail > thread every 2-3 days with the overall progress of the release to keep you > updated. > > *1. Remaining blockers and critical issues* > You can find a link here [3] for a release Kanban board that provides an > overview of the remaining blockers and critical issues for releasing 1.9.0. > The issues listed there are high priority for the release, so any help > with reviewing or fixing them is highly appreciated! > If you do assign yourself to any unassigned issue and start working on it, > please make sure to pull it to the "In Progress" column to let others be > aware of this. > > *2. Creating RC 0 for 1.9.0* > We will create RC0 now to drive forward the testing efforts. > This should be ready by tomorrow morning (July 16, 8am CET). > Note that we will not have an official vote for RC0, as this is mainly to > drive testing efforts. > RC1 with an official vote will be created once the blockers listed in [3] > are resolved. > > Cheers, > Gordon > > [1] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > [2] > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > [3] > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi all,
It's been a while since our last update for the release testing of 1.9.0, so I want to bring attention to the current status of the release. We are approaching RC1 soon, waiting on the following specific last ongoing threads to be closed: - FLINK-13241: This fixes a problem where when using YARN, slot allocation requests may be ignored [1] - FLINK-13371: Potential partitions resource leak in case of producer restarts [2] - FLINK-13350: Distinguish between temporary tables and persisted tables [3]. Strictly speaking this would be a new feature, but there was a discussion here [4] to include a workaround for now in 1.9.0, and a proper solution later on in 1.10.x. - FLINK-12858: Potential distributed deadlock in case of synchronous savepoint failure [5] The above is the critical path for moving forward with an RC1 for official voting. All of them have PRs already, and are currently being reviewed or close to being merged. Cheers, Gordon [1] https://issues.apache.org/jira/browse/FLINK-13241 [2] https://issues.apache.org/jira/browse/FLINK-13371 [3] https://issues.apache.org/jira/browse/FLINK-13350 [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html [5] https://issues.apache.org/jira/browse/FLINK-12858 On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai <[hidden email]> wrote: > Update: RC0 for 1.9.0 has been created. Please see [1] for the preview > source / binary releases and Maven artifacts. > > Cheers, > Gordon > > [1] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai <[hidden email]> > wrote: > >> Hi Flink devs, >> >> As previously announced by Kurt [1], the release branch for 1.9.0 has >> been cut [2] and we've now started the testing phase for this release, as >> well as resolving remaining blockers. >> >> I want to quickly provide an overview of our progress here. >> Also, over the course of the testing phase, we will update this mail >> thread every 2-3 days with the overall progress of the release to keep you >> updated. >> >> *1. Remaining blockers and critical issues* >> You can find a link here [3] for a release Kanban board that provides an >> overview of the remaining blockers and critical issues for releasing 1.9.0. >> The issues listed there are high priority for the release, so any help >> with reviewing or fixing them is highly appreciated! >> If you do assign yourself to any unassigned issue and start working on >> it, please make sure to pull it to the "In Progress" column to let others >> be aware of this. >> >> *2. Creating RC 0 for 1.9.0* >> We will create RC0 now to drive forward the testing efforts. >> This should be ready by tomorrow morning (July 16, 8am CET). >> Note that we will not have an official vote for RC0, as this is mainly to >> drive testing efforts. >> RC1 with an official vote will be created once the blockers listed in [3] >> are resolved. >> >> Cheers, >> Gordon >> >> [1] >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html >> [2] >> https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 >> [3] >> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 >> > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi Gordon,
Thanks for the following updates of current progress. In addition, it might be better to also cover the fix of network resource leak in jira ticket [1] which would be merged soon I think. [1] FLINK-13245: This fixes the leak of releasing reader/view with partition in network stack. Best, Zhijiang ------------------------------------------------------------------ From:Tzu-Li (Gordon) Tai <[hidden email]> Send Time:2019年7月27日(星期六) 10:41 To:dev <[hidden email]> Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release Hi all, It's been a while since our last update for the release testing of 1.9.0, so I want to bring attention to the current status of the release. We are approaching RC1 soon, waiting on the following specific last ongoing threads to be closed: - FLINK-13241: This fixes a problem where when using YARN, slot allocation requests may be ignored [1] - FLINK-13371: Potential partitions resource leak in case of producer restarts [2] - FLINK-13350: Distinguish between temporary tables and persisted tables [3]. Strictly speaking this would be a new feature, but there was a discussion here [4] to include a workaround for now in 1.9.0, and a proper solution later on in 1.10.x. - FLINK-12858: Potential distributed deadlock in case of synchronous savepoint failure [5] The above is the critical path for moving forward with an RC1 for official voting. All of them have PRs already, and are currently being reviewed or close to being merged. Cheers, Gordon [1] https://issues.apache.org/jira/browse/FLINK-13241 [2] https://issues.apache.org/jira/browse/FLINK-13371 [3] https://issues.apache.org/jira/browse/FLINK-13350 [4] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html [5] https://issues.apache.org/jira/browse/FLINK-12858 On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai <[hidden email]> wrote: > Update: RC0 for 1.9.0 has been created. Please see [1] for the preview > source / binary releases and Maven artifacts. > > Cheers, > Gordon > > [1] > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai <[hidden email]> > wrote: > >> Hi Flink devs, >> >> As previously announced by Kurt [1], the release branch for 1.9.0 has >> been cut [2] and we've now started the testing phase for this release, as >> well as resolving remaining blockers. >> >> I want to quickly provide an overview of our progress here. >> Also, over the course of the testing phase, we will update this mail >> thread every 2-3 days with the overall progress of the release to keep you >> updated. >> >> *1. Remaining blockers and critical issues* >> You can find a link here [3] for a release Kanban board that provides an >> overview of the remaining blockers and critical issues for releasing 1.9.0. >> The issues listed there are high priority for the release, so any help >> with reviewing or fixing them is highly appreciated! >> If you do assign yourself to any unassigned issue and start working on >> it, please make sure to pull it to the "In Progress" column to let others >> be aware of this. >> >> *2. Creating RC 0 for 1.9.0* >> We will create RC0 now to drive forward the testing efforts. >> This should be ready by tomorrow morning (July 16, 8am CET). >> Note that we will not have an official vote for RC0, as this is mainly to >> drive testing efforts. >> RC1 with an official vote will be created once the blockers listed in [3] >> are resolved. >> >> Cheers, >> Gordon >> >> [1] >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html >> [2] >> https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 >> [3] >> https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 >> > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi all,
There are quite a few instabilities in our builds right now (master + release-1.9), some of which are directed or suspiciously related to the 1.9 release. I'll categorize the instabilities into ones which we were already tracking in the 1.9 Burndown Kanban board [1] prior to this email, and which ones seems to be new or were not monitored so that we draw additional attention to them: *Instabilities that were already being tracked* - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod fails on Travis [2] A fix for this is coming with FLINK-13408 (Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired) [3] *New discovered instabilities that we should also start monitoring* - FLINK-13484: ConnectedComponents E2E fails with ResourceNotAvailableException [4] - FLINK-13487: TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall failed on Travis [5]. FLINK-13476 (Partitions not being properly released on cancel) could be the cause [6]. - FLINK-13488: flink-python fails to build on Travis due to Python 3.3 install failure [7] - FLINK-13489: Heavy deployment E2E fails quite consistently on Travis with TM heartbeat timeout [8] - FLINK-9900: ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles deadlocks [9] - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching outputs (could just be that the SQL query tested on Travis is indeterministic) [10] Cheers, Gordon [1] https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 [2] https://issues.apache.org/jira/browse/FLINK-13242 [3] https://issues.apache.org/jira/browse/FLINK-13408 [4] https://issues.apache.org/jira/browse/FLINK-13484 [5] https://issues.apache.org/jira/browse/FLINK-13487 [6] https://issues.apache.org/jira/browse/FLINK-13476 [7] https://issues.apache.org/jira/browse/FLINK-13488 [8] https://issues.apache.org/jira/browse/FLINK-13489 [9] https://issues.apache.org/jira/browse/FLINK-9900 [10] https://issues.apache.org/jira/browse/FLINK-13377 On Sun, Jul 28, 2019 at 6:14 AM zhijiang <[hidden email]> wrote: > Hi Gordon, > > Thanks for the following updates of current progress. > In addition, it might be better to also cover the fix of network resource > leak in jira ticket [1] which would be merged soon I think. > > [1] FLINK-13245: This fixes the leak of releasing reader/view with > partition in network stack. > > Best, > Zhijiang > ------------------------------------------------------------------ > From:Tzu-Li (Gordon) Tai <[hidden email]> > Send Time:2019年7月27日(星期六) 10:41 > To:dev <[hidden email]> > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release > > Hi all, > > It's been a while since our last update for the release testing of 1.9.0, > so I want to bring attention to the current status of the release. > > We are approaching RC1 soon, waiting on the following specific last ongoing > threads to be closed: > - FLINK-13241: This fixes a problem where when using YARN, slot allocation > requests may be ignored [1] > - FLINK-13371: Potential partitions resource leak in case of producer > restarts [2] > - FLINK-13350: Distinguish between temporary tables and persisted tables > [3]. Strictly speaking this would be a new feature, but there was a > discussion here [4] to include a workaround for now in 1.9.0, and a proper > solution later on in 1.10.x. > - FLINK-12858: Potential distributed deadlock in case of synchronous > savepoint failure [5] > > The above is the critical path for moving forward with an RC1 for official > voting. > All of them have PRs already, and are currently being reviewed or close to > being merged. > > Cheers, > Gordon > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > [2] https://issues.apache.org/jira/browse/FLINK-13371 > [3] https://issues.apache.org/jira/browse/FLINK-13350 > [4] > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai <[hidden email]> > wrote: > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the preview > > source / binary releases and Maven artifacts. > > > > Cheers, > > Gordon > > > > [1] > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai <[hidden email] > > > > wrote: > > > >> Hi Flink devs, > >> > >> As previously announced by Kurt [1], the release branch for 1.9.0 has > >> been cut [2] and we've now started the testing phase for this release, > as > >> well as resolving remaining blockers. > >> > >> I want to quickly provide an overview of our progress here. > >> Also, over the course of the testing phase, we will update this mail > >> thread every 2-3 days with the overall progress of the release to keep > you > >> updated. > >> > >> *1. Remaining blockers and critical issues* > >> You can find a link here [3] for a release Kanban board that provides an > >> overview of the remaining blockers and critical issues for releasing > 1.9.0. > >> The issues listed there are high priority for the release, so any help > >> with reviewing or fixing them is highly appreciated! > >> If you do assign yourself to any unassigned issue and start working on > >> it, please make sure to pull it to the "In Progress" column to let > others > >> be aware of this. > >> > >> *2. Creating RC 0 for 1.9.0* > >> We will create RC0 now to drive forward the testing efforts. > >> This should be ready by tomorrow morning (July 16, 8am CET). > >> Note that we will not have an official vote for RC0, as this is mainly > to > >> drive testing efforts. > >> RC1 with an official vote will be created once the blockers listed in > [3] > >> are resolved. > >> > >> Cheers, > >> Gordon > >> > >> [1] > >> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > >> [2] > >> > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > >> [3] > >> > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > >> > > > > |
|
|
Hi Gordon,
Thanks for updating progress. Currently I'm working on FLINK-9900. I need a committer to assign the ticket to me. Tzu-Li (Gordon) Tai <[hidden email]>于2019年7月30日 周二13:01写道： > Hi all, > > There are quite a few instabilities in our builds right now (master + > release-1.9), some of which are directed or suspiciously related to the 1.9 > release. > > I'll categorize the instabilities into ones which we were already tracking > in the 1.9 Burndown Kanban board [1] prior to this email, and which ones > seems to be new or were not monitored so that we draw additional attention > to them: > > *Instabilities that were already being tracked* > > - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod fails on > Travis [2] > A fix for this is coming with FLINK-13408 (Schedule > StandaloneResourceManager.setFailUnfulfillableRequest whenever the > leadership is acquired) [3] > > *New discovered instabilities that we should also start monitoring* > > - FLINK-13484: ConnectedComponents E2E fails with > ResourceNotAvailableException [4] > - FLINK-13487: > TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall > failed on Travis [5]. FLINK-13476 (Partitions not being properly released > on cancel) could be the cause [6]. > - FLINK-13488: flink-python fails to build on Travis due to Python 3.3 > install failure [7] > - FLINK-13489: Heavy deployment E2E fails quite consistently on Travis with > TM heartbeat timeout [8] > - FLINK-9900: > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles > deadlocks [9] > - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching outputs > (could just be that the SQL query tested on Travis is indeterministic) [10] > > Cheers, > Gordon > > [1] > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > [2] https://issues.apache.org/jira/browse/FLINK-13242 > [3] https://issues.apache.org/jira/browse/FLINK-13408 > [4] https://issues.apache.org/jira/browse/FLINK-13484 > [5] https://issues.apache.org/jira/browse/FLINK-13487 > [6] https://issues.apache.org/jira/browse/FLINK-13476 > [7] https://issues.apache.org/jira/browse/FLINK-13488 > [8] https://issues.apache.org/jira/browse/FLINK-13489 > [9] https://issues.apache.org/jira/browse/FLINK-9900 > [10] https://issues.apache.org/jira/browse/FLINK-13377 > > On Sun, Jul 28, 2019 at 6:14 AM zhijiang <[hidden email] > .invalid> > wrote: > > > Hi Gordon, > > > > Thanks for the following updates of current progress. > > In addition, it might be better to also cover the fix of network resource > > leak in jira ticket [1] which would be merged soon I think. > > > > [1] FLINK-13245: This fixes the leak of releasing reader/view with > > partition in network stack. > > > > Best, > > Zhijiang > > ------------------------------------------------------------------ > > From:Tzu-Li (Gordon) Tai <[hidden email]> > > Send Time:2019年7月27日(星期六) 10:41 > > To:dev <[hidden email]> > > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release > > > > Hi all, > > > > It's been a while since our last update for the release testing of 1.9.0, > > so I want to bring attention to the current status of the release. > > > > We are approaching RC1 soon, waiting on the following specific last > ongoing > > threads to be closed: > > - FLINK-13241: This fixes a problem where when using YARN, slot > allocation > > requests may be ignored [1] > > - FLINK-13371: Potential partitions resource leak in case of producer > > restarts [2] > > - FLINK-13350: Distinguish between temporary tables and persisted tables > > [3]. Strictly speaking this would be a new feature, but there was a > > discussion here [4] to include a workaround for now in 1.9.0, and a > proper > > solution later on in 1.10.x. > > - FLINK-12858: Potential distributed deadlock in case of synchronous > > savepoint failure [5] > > > > The above is the critical path for moving forward with an RC1 for > official > > voting. > > All of them have PRs already, and are currently being reviewed or close > to > > being merged. > > > > Cheers, > > Gordon > > > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > > [2] https://issues.apache.org/jira/browse/FLINK-13371 > > [3] https://issues.apache.org/jira/browse/FLINK-13350 > > [4] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai <[hidden email] > > > > wrote: > > > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the preview > > > source / binary releases and Maven artifacts. > > > > > > Cheers, > > > Gordon > > > > > > [1] > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai < > [hidden email] > > > > > > wrote: > > > > > >> Hi Flink devs, > > >> > > >> As previously announced by Kurt [1], the release branch for 1.9.0 has > > >> been cut [2] and we've now started the testing phase for this release, > > as > > >> well as resolving remaining blockers. > > >> > > >> I want to quickly provide an overview of our progress here. > > >> Also, over the course of the testing phase, we will update this mail > > >> thread every 2-3 days with the overall progress of the release to keep > > you > > >> updated. > > >> > > >> *1. Remaining blockers and critical issues* > > >> You can find a link here [3] for a release Kanban board that provides > an > > >> overview of the remaining blockers and critical issues for releasing > > 1.9.0. > > >> The issues listed there are high priority for the release, so any help > > >> with reviewing or fixing them is highly appreciated! > > >> If you do assign yourself to any unassigned issue and start working on > > >> it, please make sure to pull it to the "In Progress" column to let > > others > > >> be aware of this. > > >> > > >> *2. Creating RC 0 for 1.9.0* > > >> We will create RC0 now to drive forward the testing efforts. > > >> This should be ready by tomorrow morning (July 16, 8am CET). > > >> Note that we will not have an official vote for RC0, as this is mainly > > to > > >> drive testing efforts. > > >> RC1 with an official vote will be created once the blockers listed in > > [3] > > >> are resolved. > > >> > > >> Cheers, > > >> Gordon > > >> > > >> [1] > > >> > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > > >> [2] > > >> > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > > >> [3] > > >> > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > >> > > > > > > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi Biao,
Thanks for working on FLINK-9900. The ticket is already assigned to you now. Cheers, Gordon On Tue, Jul 30, 2019 at 2:31 PM Biao Liu <[hidden email]> wrote: > Hi Gordon, > > Thanks for updating progress. > > Currently I'm working on FLINK-9900. I need a committer to assign the > ticket to me. > > Tzu-Li (Gordon) Tai <[hidden email]>于2019年7月30日 周二13:01写道： > > > Hi all, > > > > There are quite a few instabilities in our builds right now (master + > > release-1.9), some of which are directed or suspiciously related to the > 1.9 > > release. > > > > I'll categorize the instabilities into ones which we were already > tracking > > in the 1.9 Burndown Kanban board [1] prior to this email, and which ones > > seems to be new or were not monitored so that we draw additional > attention > > to them: > > > > *Instabilities that were already being tracked* > > > > - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod fails on > > Travis [2] > > A fix for this is coming with FLINK-13408 (Schedule > > StandaloneResourceManager.setFailUnfulfillableRequest whenever the > > leadership is acquired) [3] > > > > *New discovered instabilities that we should also start monitoring* > > > > - FLINK-13484: ConnectedComponents E2E fails with > > ResourceNotAvailableException [4] > > - FLINK-13487: > > TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall > > failed on Travis [5]. FLINK-13476 (Partitions not being properly released > > on cancel) could be the cause [6]. > > - FLINK-13488: flink-python fails to build on Travis due to Python 3.3 > > install failure [7] > > - FLINK-13489: Heavy deployment E2E fails quite consistently on Travis > with > > TM heartbeat timeout [8] > > - FLINK-9900: > > > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles > > deadlocks [9] > > - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching outputs > > (could just be that the SQL query tested on Travis is indeterministic) > [10] > > > > Cheers, > > Gordon > > > > [1] > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > [2] https://issues.apache.org/jira/browse/FLINK-13242 > > [3] https://issues.apache.org/jira/browse/FLINK-13408 > > [4] https://issues.apache.org/jira/browse/FLINK-13484 > > [5] https://issues.apache.org/jira/browse/FLINK-13487 > > [6] https://issues.apache.org/jira/browse/FLINK-13476 > > [7] https://issues.apache.org/jira/browse/FLINK-13488 > > [8] https://issues.apache.org/jira/browse/FLINK-13489 > > [9] https://issues.apache.org/jira/browse/FLINK-9900 > > [10] https://issues.apache.org/jira/browse/FLINK-13377 > > > > On Sun, Jul 28, 2019 at 6:14 AM zhijiang <[hidden email] > > .invalid> > > wrote: > > > > > Hi Gordon, > > > > > > Thanks for the following updates of current progress. > > > In addition, it might be better to also cover the fix of network > resource > > > leak in jira ticket [1] which would be merged soon I think. > > > > > > [1] FLINK-13245: This fixes the leak of releasing reader/view with > > > partition in network stack. > > > > > > Best, > > > Zhijiang > > > ------------------------------------------------------------------ > > > From:Tzu-Li (Gordon) Tai <[hidden email]> > > > Send Time:2019年7月27日(星期六) 10:41 > > > To:dev <[hidden email]> > > > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release > > > > > > Hi all, > > > > > > It's been a while since our last update for the release testing of > 1.9.0, > > > so I want to bring attention to the current status of the release. > > > > > > We are approaching RC1 soon, waiting on the following specific last > > ongoing > > > threads to be closed: > > > - FLINK-13241: This fixes a problem where when using YARN, slot > > allocation > > > requests may be ignored [1] > > > - FLINK-13371: Potential partitions resource leak in case of producer > > > restarts [2] > > > - FLINK-13350: Distinguish between temporary tables and persisted > tables > > > [3]. Strictly speaking this would be a new feature, but there was a > > > discussion here [4] to include a workaround for now in 1.9.0, and a > > proper > > > solution later on in 1.10.x. > > > - FLINK-12858: Potential distributed deadlock in case of synchronous > > > savepoint failure [5] > > > > > > The above is the critical path for moving forward with an RC1 for > > official > > > voting. > > > All of them have PRs already, and are currently being reviewed or close > > to > > > being merged. > > > > > > Cheers, > > > Gordon > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > > > [2] https://issues.apache.org/jira/browse/FLINK-13371 > > > [3] https://issues.apache.org/jira/browse/FLINK-13350 > > > [4] > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > > > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > > > > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai < > [hidden email] > > > > > > wrote: > > > > > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the > preview > > > > source / binary releases and Maven artifacts. > > > > > > > > Cheers, > > > > Gordon > > > > > > > > [1] > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai < > > [hidden email] > > > > > > > > wrote: > > > > > > > >> Hi Flink devs, > > > >> > > > >> As previously announced by Kurt [1], the release branch for 1.9.0 > has > > > >> been cut [2] and we've now started the testing phase for this > release, > > > as > > > >> well as resolving remaining blockers. > > > >> > > > >> I want to quickly provide an overview of our progress here. > > > >> Also, over the course of the testing phase, we will update this mail > > > >> thread every 2-3 days with the overall progress of the release to > keep > > > you > > > >> updated. > > > >> > > > >> *1. Remaining blockers and critical issues* > > > >> You can find a link here [3] for a release Kanban board that > provides > > an > > > >> overview of the remaining blockers and critical issues for releasing > > > 1.9.0. > > > >> The issues listed there are high priority for the release, so any > help > > > >> with reviewing or fixing them is highly appreciated! > > > >> If you do assign yourself to any unassigned issue and start working > on > > > >> it, please make sure to pull it to the "In Progress" column to let > > > others > > > >> be aware of this. > > > >> > > > >> *2. Creating RC 0 for 1.9.0* > > > >> We will create RC0 now to drive forward the testing efforts. > > > >> This should be ready by tomorrow morning (July 16, 8am CET). > > > >> Note that we will not have an official vote for RC0, as this is > mainly > > > to > > > >> drive testing efforts. > > > >> RC1 with an official vote will be created once the blockers listed > in > > > [3] > > > >> are resolved. > > > >> > > > >> Cheers, > > > >> Gordon > > > >> > > > >> [1] > > > >> > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > > > >> [2] > > > >> > > > > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > > > >> [3] > > > >> > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > >> > > > > > > > > > > > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Update: RC1 for 1.9.0 has been created. Please see [1] for the preview
source / binary releases and Maven artifacts. Best, Kurt [1] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-1-td31233.html On Tue, Jul 30, 2019 at 2:36 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote: > Hi Biao, > > Thanks for working on FLINK-9900. The ticket is already assigned to you > now. > > Cheers, > Gordon > > On Tue, Jul 30, 2019 at 2:31 PM Biao Liu <[hidden email]> wrote: > > > Hi Gordon, > > > > Thanks for updating progress. > > > > Currently I'm working on FLINK-9900. I need a committer to assign the > > ticket to me. > > > > Tzu-Li (Gordon) Tai <[hidden email]>于2019年7月30日 周二13:01写道： > > > > > Hi all, > > > > > > There are quite a few instabilities in our builds right now (master + > > > release-1.9), some of which are directed or suspiciously related to the > > 1.9 > > > release. > > > > > > I'll categorize the instabilities into ones which we were already > > tracking > > > in the 1.9 Burndown Kanban board [1] prior to this email, and which > ones > > > seems to be new or were not monitored so that we draw additional > > attention > > > to them: > > > > > > *Instabilities that were already being tracked* > > > > > > - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod fails on > > > Travis [2] > > > A fix for this is coming with FLINK-13408 (Schedule > > > StandaloneResourceManager.setFailUnfulfillableRequest whenever the > > > leadership is acquired) [3] > > > > > > *New discovered instabilities that we should also start monitoring* > > > > > > - FLINK-13484: ConnectedComponents E2E fails with > > > ResourceNotAvailableException [4] > > > - FLINK-13487: > > > TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall > > > failed on Travis [5]. FLINK-13476 (Partitions not being properly > released > > > on cancel) could be the cause [6]. > > > - FLINK-13488: flink-python fails to build on Travis due to Python 3.3 > > > install failure [7] > > > - FLINK-13489: Heavy deployment E2E fails quite consistently on Travis > > with > > > TM heartbeat timeout [8] > > > - FLINK-9900: > > > > > > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles > > > deadlocks [9] > > > - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching > outputs > > > (could just be that the SQL query tested on Travis is indeterministic) > > [10] > > > > > > Cheers, > > > Gordon > > > > > > [1] > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-13242 > > > [3] https://issues.apache.org/jira/browse/FLINK-13408 > > > [4] https://issues.apache.org/jira/browse/FLINK-13484 > > > [5] https://issues.apache.org/jira/browse/FLINK-13487 > > > [6] https://issues.apache.org/jira/browse/FLINK-13476 > > > [7] https://issues.apache.org/jira/browse/FLINK-13488 > > > [8] https://issues.apache.org/jira/browse/FLINK-13489 > > > [9] https://issues.apache.org/jira/browse/FLINK-9900 > > > [10] https://issues.apache.org/jira/browse/FLINK-13377 > > > > > > On Sun, Jul 28, 2019 at 6:14 AM zhijiang <[hidden email] > > > .invalid> > > > wrote: > > > > > > > Hi Gordon, > > > > > > > > Thanks for the following updates of current progress. > > > > In addition, it might be better to also cover the fix of network > > resource > > > > leak in jira ticket [1] which would be merged soon I think. > > > > > > > > [1] FLINK-13245: This fixes the leak of releasing reader/view with > > > > partition in network stack. > > > > > > > > Best, > > > > Zhijiang > > > > ------------------------------------------------------------------ > > > > From:Tzu-Li (Gordon) Tai <[hidden email]> > > > > Send Time:2019年7月27日(星期六) 10:41 > > > > To:dev <[hidden email]> > > > > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 > release > > > > > > > > Hi all, > > > > > > > > It's been a while since our last update for the release testing of > > 1.9.0, > > > > so I want to bring attention to the current status of the release. > > > > > > > > We are approaching RC1 soon, waiting on the following specific last > > > ongoing > > > > threads to be closed: > > > > - FLINK-13241: This fixes a problem where when using YARN, slot > > > allocation > > > > requests may be ignored [1] > > > > - FLINK-13371: Potential partitions resource leak in case of producer > > > > restarts [2] > > > > - FLINK-13350: Distinguish between temporary tables and persisted > > tables > > > > [3]. Strictly speaking this would be a new feature, but there was a > > > > discussion here [4] to include a workaround for now in 1.9.0, and a > > > proper > > > > solution later on in 1.10.x. > > > > - FLINK-12858: Potential distributed deadlock in case of synchronous > > > > savepoint failure [5] > > > > > > > > The above is the critical path for moving forward with an RC1 for > > > official > > > > voting. > > > > All of them have PRs already, and are currently being reviewed or > close > > > to > > > > being merged. > > > > > > > > Cheers, > > > > Gordon > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > > > > [2] https://issues.apache.org/jira/browse/FLINK-13371 > > > > [3] https://issues.apache.org/jira/browse/FLINK-13350 > > > > [4] > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > > > > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > > > > > > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai < > > [hidden email] > > > > > > > > wrote: > > > > > > > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the > > preview > > > > > source / binary releases and Maven artifacts. > > > > > > > > > > Cheers, > > > > > Gordon > > > > > > > > > > [1] > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > > > > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai < > > > [hidden email] > > > > > > > > > > wrote: > > > > > > > > > >> Hi Flink devs, > > > > >> > > > > >> As previously announced by Kurt [1], the release branch for 1.9.0 > > has > > > > >> been cut [2] and we've now started the testing phase for this > > release, > > > > as > > > > >> well as resolving remaining blockers. > > > > >> > > > > >> I want to quickly provide an overview of our progress here. > > > > >> Also, over the course of the testing phase, we will update this > > > > >> thread every 2-3 days with the overall progress of the release to > > keep > > > > you > > > > >> updated. > > > > >> > > > > >> *1. Remaining blockers and critical issues* > > > > >> You can find a link here [3] for a release Kanban board that > > provides > > > an > > > > >> overview of the remaining blockers and critical issues for > releasing > > > > 1.9.0. > > > > >> The issues listed there are high priority for the release, so any > > help > > > > >> with reviewing or fixing them is highly appreciated! > > > > >> If you do assign yourself to any unassigned issue and start > working > > on > > > > >> it, please make sure to pull it to the "In Progress" column to let > > > > others > > > > >> be aware of this. > > > > >> > > > > >> *2. Creating RC 0 for 1.9.0* > > > > >> We will create RC0 now to drive forward the testing efforts. > > > > >> This should be ready by tomorrow morning (July 16, 8am CET). > > > > >> Note that we will not have an official vote for RC0, as this is > > mainly > > > > to > > > > >> drive testing efforts. > > > > >> RC1 with an official vote will be created once the blockers listed > > in > > > > [3] > > > > >> are resolved. > > > > >> > > > > >> Cheers, > > > > >> Gordon > > > > >> > > > > >> [1] > > > > >> > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > > > > >> [2] > > > > >> > > > > > > > > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > > > > >> [3] > > > > >> > > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > >> > > > > > > > > > > > > > > > > > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi all,
According to the 1.9.x burndown board [1], we're approaching a releasable state for 1.9.0. Thanks to everyone who participated in the work for fixing the blockers so far, especially Till who has been coordinating a lot of the efforts. Below is a summary of the current state of the few remaining blockers: Pending bugs to be fixed - - *FLINK-13159 - Restored PojoSerializer not using correct classloader for deserialization [2] STATUS: *PR opened and reviewed, waiting for Travis run before merging *NOTES:* this bug is not specific to 1.9.0 only; will be backported to 1.8.x as well. It was made a blocker for 1.9.0 as well since the fix is relatively low-effort. - *FLINK-13593 - Prevent failing the wrong execution attempt in CheckpointFailureManager [3] STATUS:* PR opened, some final passes of reviews pending Additional tests to be added - - *FLINK-13441 - Add batch sql E2E test which runs with fewer slots than parallelism to test the newly introduced batch scheduling modes [4]* *STATUS:* PR opened and being reviewed. *NOTES:* The TPC-H E2E test has also been modified to cover this scenario. Unstable tests: - *FLINK-13489 - Heavy deployment E2E test fails on Travis (agreed to make this a non-blocker) [5] STATUS:* The cause of this isn't a critical issue, and it is agreed that this would not be a blocker for the release. - *FLINK-13581 - BatchFineGrainedRecoveryITCase failed on Travis [6] STATUS: *PR opened and review is in progress - *FLINK-13527 - Unstable KafkaProducerExactlyOnceITCase fails [7] STATUS:* Blocked by FLINK-13593 (blocker issue mentioned above) *NOTES: *Yu Li already mentioned that with the fix in FLINK-13593, this test no longer fails - *FLINK-13607 - TCP-H E2E tests fails on Travis [8] STATUS:* Awaiting final confirmations on whether or not the instability still exists. *NOTES:* Kurt is also running a variation of this with multiple TMs and high parallelism (10-20 TMs, ~1000 DoP) internally in Alibaba. So, from the looks of things, it should be safe to say that we can aim for creating the first voting RC (RC2) by the end of this week (August 9th)! An official voting thread for RC2 will be established once it is ready. Cheers, Gordon [1] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=328 [2] https://issues.apache.org/jira/browse/FLINK-13159 [3] https://issues.apache.org/jira/browse/FLINK-13593 [4] https://issues.apache.org/jira/browse/FLINK-13441 [5] https://issues.apache.org/jira/browse/FLINK-13489 [6] https://issues.apache.org/jira/browse/FLINK-13581 [7] https://issues.apache.org/jira/browse/FLINK-13527 [8] https://issues.apache.org/jira/browse/FLINK-13607 On Thu, Aug 1, 2019 at 3:03 PM Kurt Young <[hidden email]> wrote: > Update: RC1 for 1.9.0 has been created. Please see [1] for the preview > source / binary releases and Maven artifacts. > > Best, > Kurt > > [1] > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-1-td31233.html > > > On Tue, Jul 30, 2019 at 2:36 PM Tzu-Li (Gordon) Tai <[hidden email]> > wrote: > > > Hi Biao, > > > > Thanks for working on FLINK-9900. The ticket is already assigned to you > > now. > > > > Cheers, > > Gordon > > > > On Tue, Jul 30, 2019 at 2:31 PM Biao Liu <[hidden email]> wrote: > > > > > Hi Gordon, > > > > > > Thanks for updating progress. > > > > > > Currently I'm working on FLINK-9900. I need a committer to assign the > > > ticket to me. > > > > > > Tzu-Li (Gordon) Tai <[hidden email]>于2019年7月30日 周二13:01写道： > > > > > > > Hi all, > > > > > > > > There are quite a few instabilities in our builds right now (master + > > > > release-1.9), some of which are directed or suspiciously related to > the > > > 1.9 > > > > release. > > > > > > > > I'll categorize the instabilities into ones which we were already > > > tracking > > > > in the 1.9 Burndown Kanban board [1] prior to this email, and which > > ones > > > > seems to be new or were not monitored so that we draw additional > > > attention > > > > to them: > > > > > > > > *Instabilities that were already being tracked* > > > > > > > > - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod fails > on > > > > Travis [2] > > > > A fix for this is coming with FLINK-13408 (Schedule > > > > StandaloneResourceManager.setFailUnfulfillableRequest whenever the > > > > leadership is acquired) [3] > > > > > > > > *New discovered instabilities that we should also start monitoring* > > > > > > > > - FLINK-13484: ConnectedComponents E2E fails with > > > > ResourceNotAvailableException [4] > > > > - FLINK-13487: > > > > > TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall > > > > failed on Travis [5]. FLINK-13476 (Partitions not being properly > > released > > > > on cancel) could be the cause [6]. > > > > - FLINK-13488: flink-python fails to build on Travis due to Python > 3.3 > > > > install failure [7] > > > > - FLINK-13489: Heavy deployment E2E fails quite consistently on > Travis > > > with > > > > TM heartbeat timeout [8] > > > > - FLINK-9900: > > > > > > > > > > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles > > > > deadlocks [9] > > > > - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching > > outputs > > > > (could just be that the SQL query tested on Travis is > indeterministic) > > > [10] > > > > > > > > Cheers, > > > > Gordon > > > > > > > > [1] > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-13242 > > > > [3] https://issues.apache.org/jira/browse/FLINK-13408 > > > > [4] https://issues.apache.org/jira/browse/FLINK-13484 > > > > [5] https://issues.apache.org/jira/browse/FLINK-13487 > > > > [6] https://issues.apache.org/jira/browse/FLINK-13476 > > > > [7] https://issues.apache.org/jira/browse/FLINK-13488 > > > > [8] https://issues.apache.org/jira/browse/FLINK-13489 > > > > [9] https://issues.apache.org/jira/browse/FLINK-9900 > > > > [10] https://issues.apache.org/jira/browse/FLINK-13377 > > > > > > > > On Sun, Jul 28, 2019 at 6:14 AM zhijiang <[hidden email] > > > > .invalid> > > > > wrote: > > > > > > > > > Hi Gordon, > > > > > > > > > > Thanks for the following updates of current progress. > > > > > In addition, it might be better to also cover the fix of network > > > resource > > > > > leak in jira ticket [1] which would be merged soon I think. > > > > > > > > > > [1] FLINK-13245: This fixes the leak of releasing reader/view with > > > > > partition in network stack. > > > > > > > > > > Best, > > > > > Zhijiang > > > > > ------------------------------------------------------------------ > > > > > From:Tzu-Li (Gordon) Tai <[hidden email]> > > > > > Send Time:2019年7月27日(星期六) 10:41 > > > > > To:dev <[hidden email]> > > > > > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 > > release > > > > > > > > > > Hi all, > > > > > > > > > > It's been a while since our last update for the release testing of > > > 1.9.0, > > > > > so I want to bring attention to the current status of the release. > > > > > > > > > > We are approaching RC1 soon, waiting on the following specific last > > > > ongoing > > > > > threads to be closed: > > > > > - FLINK-13241: This fixes a problem where when using YARN, slot > > > > allocation > > > > > requests may be ignored [1] > > > > > - FLINK-13371: Potential partitions resource leak in case of > producer > > > > > restarts [2] > > > > > - FLINK-13350: Distinguish between temporary tables and persisted > > > tables > > > > > [3]. Strictly speaking this would be a new feature, but there was a > > > > > discussion here [4] to include a workaround for now in 1.9.0, and a > > > > proper > > > > > solution later on in 1.10.x. > > > > > - FLINK-12858: Potential distributed deadlock in case of > synchronous > > > > > savepoint failure [5] > > > > > > > > > > The above is the critical path for moving forward with an RC1 for > > > > official > > > > > voting. > > > > > All of them have PRs already, and are currently being reviewed or > > close > > > > to > > > > > being merged. > > > > > > > > > > Cheers, > > > > > Gordon > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > > > > > [2] https://issues.apache.org/jira/browse/FLINK-13371 > > > > > [3] https://issues.apache.org/jira/browse/FLINK-13350 > > > > > [4] > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > > > > > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > > > > > > > > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai < > > > [hidden email] > > > > > > > > > > wrote: > > > > > > > > > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the > > > preview > > > > > > source / binary releases and Maven artifacts. > > > > > > > > > > > > Cheers, > > > > > > Gordon > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > > > > > > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai < > > > > [hidden email] > > > > > > > > > > > > wrote: > > > > > > > > > > > >> Hi Flink devs, > > > > > >> > > > > > >> As previously announced by Kurt [1], the release branch for > 1.9.0 > > > has > > > > > >> been cut [2] and we've now started the testing phase for this > > > release, > > > > > as > > > > > >> well as resolving remaining blockers. > > > > > >> > > > > > >> I want to quickly provide an overview of our progress here. > > > > > >> Also, over the course of the testing phase, we will update this > > > > > >> thread every 2-3 days with the overall progress of the release > to > > > keep > > > > > you > > > > > >> updated. > > > > > >> > > > > > >> *1. Remaining blockers and critical issues* > > > > > >> You can find a link here [3] for a release Kanban board that > > > provides > > > > an > > > > > >> overview of the remaining blockers and critical issues for > > releasing > > > > > 1.9.0. > > > > > >> The issues listed there are high priority for the release, so > any > > > help > > > > > >> with reviewing or fixing them is highly appreciated! > > > > > >> If you do assign yourself to any unassigned issue and start > > working > > > on > > > > > >> it, please make sure to pull it to the "In Progress" column to > let > > > > > others > > > > > >> be aware of this. > > > > > >> > > > > > >> *2. Creating RC 0 for 1.9.0* > > > > > >> We will create RC0 now to drive forward the testing efforts. > > > > > >> This should be ready by tomorrow morning (July 16, 8am CET). > > > > > >> Note that we will not have an official vote for RC0, as this is > > > mainly > > > > > to > > > > > >> drive testing efforts. > > > > > >> RC1 with an official vote will be created once the blockers > listed > > > in > > > > > [3] > > > > > >> are resolved. > > > > > >> > > > > > >> Cheers, > > > > > >> Gordon > > > > > >> > > > > > >> [1] > > > > > >> > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > > > > > >> [2] > > > > > >> > > > > > > > > > > > > > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > > > > > >> [3] > > > > > >> > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi everyone,
in the context of FLIP-42 Fabian and myself were working on a docker-based playground as part of the "Getting Started" section of our documentation. The PR [1] was merged into master today. Besides documentation, this also adds an additional example to `flink-streaming-examples`. For the playground to work with the Apache Flink 1.9 this example needs to become part of the distribution. Would it be possible to still include this PR in the 1.9.0 release. As far as I know, documentation can generally be added after the feature freeze, but since this also touches the examples, this is kind of a corner case, I suppose. What do you think? Best, Konstantin [1] https://github.com/apache/flink/pull/9192 On Wed, Aug 7, 2019 at 2:51 PM Tzu-Li (Gordon) Tai <[hidden email]> wrote: > Hi all, > > According to the 1.9.x burndown board [1], we're approaching a releasable > state for 1.9.0. > Thanks to everyone who participated in the work for fixing the blockers so > far, especially Till who has been coordinating a lot of the efforts. > > Below is a summary of the current state of the few remaining blockers: > > Pending bugs to be fixed - > > - > *FLINK-13159 - Restored PojoSerializer not using correct classloader for > deserialization [2] STATUS: *PR opened and reviewed, waiting for Travis > run before merging > *NOTES:* this bug is not specific to 1.9.0 only; will be backported to > 1.8.x as well. It was made a blocker for 1.9.0 as well since the fix is > relatively low-effort. > - > *FLINK-13593 - Prevent failing the wrong execution attempt in > CheckpointFailureManager [3] STATUS:* PR opened, some final passes of > reviews pending > > Additional tests to be added - > > - *FLINK-13441 - Add batch sql E2E test which runs with fewer slots than > parallelism to test the newly introduced batch scheduling modes [4]* > *STATUS:* PR opened and being reviewed. > *NOTES:* The TPC-H E2E test has also been modified to cover this > scenario. > > Unstable tests: > > - > *FLINK-13489 - Heavy deployment E2E test fails on Travis (agreed to make > this a non-blocker) [5] STATUS:* The cause of this isn't a critical > issue, and it is agreed that this would not be a blocker for the > release. > - > *FLINK-13581 - BatchFineGrainedRecoveryITCase failed on Travis [6] STATUS: > *PR > opened and review is in progress > - > *FLINK-13527 - Unstable KafkaProducerExactlyOnceITCase fails [7] > STATUS:* Blocked > by FLINK-13593 (blocker issue mentioned above) > *NOTES: *Yu Li already mentioned that with the fix in FLINK-13593, this > test no longer fails > - > *FLINK-13607 - TCP-H E2E tests fails on Travis [8] STATUS:* Awaiting final > confirmations on whether or not the instability still exists. > *NOTES:* Kurt is also running a variation of this with multiple TMs and > high parallelism (10-20 TMs, ~1000 DoP) internally in Alibaba. > > So, from the looks of things, it should be safe to say that we can aim for > creating the first voting RC (RC2) by the end of this week (August 9th)! > An official voting thread for RC2 will be established once it is ready. > > Cheers, > Gordon > > [1] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=328 > [2] https://issues.apache.org/jira/browse/FLINK-13159 > [3] https://issues.apache.org/jira/browse/FLINK-13593 > [4] https://issues.apache.org/jira/browse/FLINK-13441 > [5] https://issues.apache.org/jira/browse/FLINK-13489 > [6] https://issues.apache.org/jira/browse/FLINK-13581 > [7] https://issues.apache.org/jira/browse/FLINK-13527 > [8] https://issues.apache.org/jira/browse/FLINK-13607 > > On Thu, Aug 1, 2019 at 3:03 PM Kurt Young <[hidden email]> wrote: > > > Update: RC1 for 1.9.0 has been created. Please see [1] for the preview > > source / binary releases and Maven artifacts. > > > > Best, > > Kurt > > > > [1] > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-1-td31233.html > > > > > > On Tue, Jul 30, 2019 at 2:36 PM Tzu-Li (Gordon) Tai <[hidden email] > > > > wrote: > > > > > Hi Biao, > > > > > > Thanks for working on FLINK-9900. The ticket is already assigned to you > > > now. > > > > > > Cheers, > > > Gordon > > > > > > On Tue, Jul 30, 2019 at 2:31 PM Biao Liu <[hidden email]> wrote: > > > > > > > Hi Gordon, > > > > > > > > Thanks for updating progress. > > > > > > > > Currently I'm working on FLINK-9900. I need a committer to assign the > > > > ticket to me. > > > > > > > > Tzu-Li (Gordon) Tai <[hidden email]>于2019年7月30日 周二13:01写道： > > > > > > > > > Hi all, > > > > > > > > > > There are quite a few instabilities in our builds right now > (master + > > > > > release-1.9), some of which are directed or suspiciously related to > > the > > > > 1.9 > > > > > release. > > > > > > > > > > I'll categorize the instabilities into ones which we were already > > > > tracking > > > > > in the 1.9 Burndown Kanban board [1] prior to this email, and which > > > ones > > > > > seems to be new or were not monitored so that we draw additional > > > > attention > > > > > to them: > > > > > > > > > > *Instabilities that were already being tracked* > > > > > > > > > > - FLINK-13242: StandaloneResourceManagerTest.testStartupPeriod > fails > > on > > > > > Travis [2] > > > > > A fix for this is coming with FLINK-13408 (Schedule > > > > > StandaloneResourceManager.setFailUnfulfillableRequest whenever the > > > > > leadership is acquired) [3] > > > > > > > > > > *New discovered instabilities that we should also start monitoring* > > > > > > > > > > - FLINK-13484: ConnectedComponents E2E fails with > > > > > ResourceNotAvailableException [4] > > > > > - FLINK-13487: > > > > > > > TaskExecutorPartitionLifecycleTest.testPartitionReleaseAfterReleaseCall > > > > > failed on Travis [5]. FLINK-13476 (Partitions not being properly > > > released > > > > > on cancel) could be the cause [6]. > > > > > - FLINK-13488: flink-python fails to build on Travis due to Python > > 3.3 > > > > > install failure [7] > > > > > - FLINK-13489: Heavy deployment E2E fails quite consistently on > > Travis > > > > with > > > > > TM heartbeat timeout [8] > > > > > - FLINK-9900: > > > > > > > > > > > > > > > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles > > > > > deadlocks [9] > > > > > - FLINK-13377: Streaming SQ E2E fails on Travis with mismatching > > > outputs > > > > > (could just be that the SQL query tested on Travis is > > indeterministic) > > > > [10] > > > > > > > > > > Cheers, > > > > > Gordon > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-13242 > > > > > [3] https://issues.apache.org/jira/browse/FLINK-13408 > > > > > [4] https://issues.apache.org/jira/browse/FLINK-13484 > > > > > [5] https://issues.apache.org/jira/browse/FLINK-13487 > > > > > [6] https://issues.apache.org/jira/browse/FLINK-13476 > > > > > [7] https://issues.apache.org/jira/browse/FLINK-13488 > > > > > [8] https://issues.apache.org/jira/browse/FLINK-13489 > > > > > [9] https://issues.apache.org/jira/browse/FLINK-9900 > > > > > [10] https://issues.apache.org/jira/browse/FLINK-13377 > > > > > > > > > > On Sun, Jul 28, 2019 at 6:14 AM zhijiang < > [hidden email] > > > > > .invalid> > > > > > wrote: > > > > > > > > > > > Hi Gordon, > > > > > > > > > > > > Thanks for the following updates of current progress. > > > > > > In addition, it might be better to also cover the fix of network > > > > resource > > > > > > leak in jira ticket [1] which would be merged soon I think. > > > > > > > > > > > > [1] FLINK-13245: This fixes the leak of releasing reader/view > with > > > > > > partition in network stack. > > > > > > > > > > > > Best, > > > > > > Zhijiang > > > > > > > ------------------------------------------------------------------ > > > > > > From:Tzu-Li (Gordon) Tai <[hidden email]> > > > > > > Send Time:2019年7月27日(星期六) 10:41 > > > > > > To:dev <[hidden email]> > > > > > > Subject:Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 > > > release > > > > > > > > > > > > Hi all, > > > > > > > > > > > > It's been a while since our last update for the release testing > of > > > > 1.9.0, > > > > > > so I want to bring attention to the current status of the > release. > > > > > > > > > > > > We are approaching RC1 soon, waiting on the following specific > last > > > > > ongoing > > > > > > threads to be closed: > > > > > > - FLINK-13241: This fixes a problem where when using YARN, slot > > > > > allocation > > > > > > requests may be ignored [1] > > > > > > - FLINK-13371: Potential partitions resource leak in case of > > producer > > > > > > restarts [2] > > > > > > - FLINK-13350: Distinguish between temporary tables and persisted > > > > tables > > > > > > [3]. Strictly speaking this would be a new feature, but there > was a > > > > > > discussion here [4] to include a workaround for now in 1.9.0, > and a > > > > > proper > > > > > > solution later on in 1.10.x. > > > > > > - FLINK-12858: Potential distributed deadlock in case of > > synchronous > > > > > > savepoint failure [5] > > > > > > > > > > > > The above is the critical path for moving forward with an RC1 for > > > > > official > > > > > > voting. > > > > > > All of them have PRs already, and are currently being reviewed or > > > close > > > > > to > > > > > > being merged. > > > > > > > > > > > > Cheers, > > > > > > Gordon > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-13241 > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-13371 > > > > > > [3] https://issues.apache.org/jira/browse/FLINK-13350 > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Support-temporary-tables-in-SQL-API-td30831.html > > > > > > [5] https://issues.apache.org/jira/browse/FLINK-12858 > > > > > > > > > > > > On Tue, Jul 16, 2019 at 5:26 AM Tzu-Li (Gordon) Tai < > > > > [hidden email] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Update: RC0 for 1.9.0 has been created. Please see [1] for the > > > > preview > > > > > > > source / binary releases and Maven artifacts. > > > > > > > > > > > > > > Cheers, > > > > > > > Gordon > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PREVIEW-Apache-Flink-1-9-0-release-candidate-0-td30583.html > > > > > > > > > > > > > > On Mon, Jul 15, 2019 at 6:39 PM Tzu-Li (Gordon) Tai < > > > > > [hidden email] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > >> Hi Flink devs, > > > > > > >> > > > > > > >> As previously announced by Kurt [1], the release branch for > > 1.9.0 > > > > has > > > > > > >> been cut [2] and we've now started the testing phase for this > > > > release, > > > > > > as > > > > > > >> well as resolving remaining blockers. > > > > > > >> > > > > > > >> I want to quickly provide an overview of our progress here. > > > > > > >> Also, over the course of the testing phase, we will update > this > > > > > > >> thread every 2-3 days with the overall progress of the release > > to > > > > keep > > > > > > you > > > > > > >> updated. > > > > > > >> > > > > > > >> *1. Remaining blockers and critical issues* > > > > > > >> You can find a link here [3] for a release Kanban board that > > > > provides > > > > > an > > > > > > >> overview of the remaining blockers and critical issues for > > > releasing > > > > > > 1.9.0. > > > > > > >> The issues listed there are high priority for the release, so > > any > > > > help > > > > > > >> with reviewing or fixing them is highly appreciated! > > > > > > >> If you do assign yourself to any unassigned issue and start > > > working > > > > on > > > > > > >> it, please make sure to pull it to the "In Progress" column to > > let > > > > > > others > > > > > > >> be aware of this. > > > > > > >> > > > > > > >> *2. Creating RC 0 for 1.9.0* > > > > > > >> We will create RC0 now to drive forward the testing efforts. > > > > > > >> This should be ready by tomorrow morning (July 16, 8am CET). > > > > > > >> Note that we will not have an official vote for RC0, as this > is > > > > mainly > > > > > > to > > > > > > >> drive testing efforts. > > > > > > >> RC1 with an official vote will be created once the blockers > > listed > > > > in > > > > > > [3] > > > > > > >> are resolved. > > > > > > >> > > > > > > >> Cheers, > > > > > > >> Gordon > > > > > > >> > > > > > > >> [1] > > > > > > >> > > > > > > > > > > > > > > > > > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ANNOUNCE-Flink-1-9-release-branch-has-been-created-td30500.html > > > > > > >> [2] > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://gitbox.apache.org/repos/asf?p=flink.git;a=shortlog;h=refs/heads/release-1.9 > > > > > > >> [3] > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/RapidBoard.jspa?projectKey=FLINK&rapidView=328 > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Konstantin Knauf | Solutions Architect +49 160 91394525 Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
I've spent time exploring this playground and its accompanying
documentation, and found it to be a big step forward in making it easy for folks to experience some of Flink's key features firsthand. From a training and educational perspective, I'd love to see this in 1.9. *David Anderson* | Training Coordinator Follow us @VervericaData -- Join Flink Forward - The Apache Flink Conference Stream Processing | Event Driven | Real Time |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi,
I worked with Konstantin and reviewed the PR. I think the playground is a great way to get started with Flink and explore it's recovery mechanism and unique features like savepoints. I'm in favor of adding the required streaming example program for the 1.9 release unless there's a good technical argument against it. Best, Fabian |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
+1 to include this in 1.9.0, adding some examples doesn't look like new
feature to me. BTW, I am also trying this tutorial based on release-1.9 branch, but blocked by: git clone --branch release-1.10-SNAPSHOT [hidden email]:apache/flink-playgrounds.git Neither 1.10 nor 1.9 exists in flink-playground yet. Best, Kurt On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> wrote: > Hi, > I worked with Konstantin and reviewed the PR. > I think the playground is a great way to get started with Flink and explore > it's recovery mechanism and unique features like savepoints. > > I'm in favor of adding the required streaming example program for the 1.9 > release unless there's a good technical argument against it. > > Best, Fabian > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
Before backporting the playground PR to the release-1.9, I'd like to
understand why the ClickEventCount job needs to be part of the Flink distribution. Looking at the example, it seems to only work in combination with a Kafka cluster. Since it is not self-contained, it does not add much value for a user who does not want to use the playgrounds. Moreover, we already have the StateMachineExample job which can be used to read from Kafka if a Kafka cluster is available. So my question would be why don't we include the example job in the docker images for the playground? This would be in my opinion a better separation of concerns. I've cross posted my question on the original PR as well. Cheers, Till On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: > +1 to include this in 1.9.0, adding some examples doesn't look like new > feature to me. > BTW, I am also trying this tutorial based on release-1.9 branch, but > blocked by: > > git clone --branch release-1.10-SNAPSHOT > [hidden email]:apache/flink-playgrounds.git > > Neither 1.10 nor 1.9 exists in flink-playground yet. > > Best, > Kurt > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> wrote: > > > Hi, > > I worked with Konstantin and reviewed the PR. > > I think the playground is a great way to get started with Flink and > explore > > it's recovery mechanism and unique features like savepoints. > > > > I'm in favor of adding the required streaming example program for the 1.9 > > release unless there's a good technical argument against it. > > > > Best, Fabian > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
The motivation for including the job as an example is to not have to
maintain a separate Docker image. We would like to use the regular Flink 1.9 image for the playground and avoid to maintain an image that is slightly different from the regular 1.9 image. Maintaining the job in a different repository or somewhere else would mean, that we need to have a proper release cycle for it as well. Having it among the other examples means it's included in the regular release. Best, Fabian Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < [hidden email]>: > Before backporting the playground PR to the release-1.9, I'd like to > understand why the ClickEventCount job needs to be part of the Flink > distribution. Looking at the example, it seems to only work in combination > with a Kafka cluster. Since it is not self-contained, it does not add much > value for a user who does not want to use the playgrounds. Moreover, we > already have the StateMachineExample job which can be used to read from > Kafka if a Kafka cluster is available. So my question would be why don't we > include the example job in the docker images for the playground? This would > be in my opinion a better separation of concerns. > > I've cross posted my question on the original PR as well. > > Cheers, > Till > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: > > > +1 to include this in 1.9.0, adding some examples doesn't look like new > > feature to me. > > BTW, I am also trying this tutorial based on release-1.9 branch, but > > blocked by: > > > > git clone --branch release-1.10-SNAPSHOT > > [hidden email]:apache/flink-playgrounds.git > > > > Neither 1.10 nor 1.9 exists in flink-playground yet. > > > > Best, > > Kurt > > > > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> wrote: > > > > > Hi, > > > I worked with Konstantin and reviewed the PR. > > > I think the playground is a great way to get started with Flink and > > explore > > > it's recovery mechanism and unique features like savepoints. > > > > > > I'm in favor of adding the required streaming example program for the > 1.9 > > > release unless there's a good technical argument against it. > > > > > > Best, Fabian > > > > > > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi Till,
as Fabian said, we considered the option you mentioned, but in the end decided that not maintaining a separate images has more advantages. In the context of FLIP-42 we are also revisiting the examples in general and want to clean these up a bit. So, for what it's worth, there will be an opportunity for revisiting this topic soon. Best, Konstantin On Thu, Aug 8, 2019 at 11:43 AM Fabian Hueske <[hidden email]> wrote: > The motivation for including the job as an example is to not have to > maintain a separate Docker image. > We would like to use the regular Flink 1.9 image for the playground and > avoid to maintain an image that is slightly different from the regular 1.9 > image. > > Maintaining the job in a different repository or somewhere else would mean, > that we need to have a proper release cycle for it as well. > Having it among the other examples means it's included in the regular > release. > > Best, Fabian > > > Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < > [hidden email]>: > > > Before backporting the playground PR to the release-1.9, I'd like to > > understand why the ClickEventCount job needs to be part of the Flink > > distribution. Looking at the example, it seems to only work in > combination > > with a Kafka cluster. Since it is not self-contained, it does not add > much > > value for a user who does not want to use the playgrounds. Moreover, we > > already have the StateMachineExample job which can be used to read from > > Kafka if a Kafka cluster is available. So my question would be why don't > we > > include the example job in the docker images for the playground? This > would > > be in my opinion a better separation of concerns. > > > > I've cross posted my question on the original PR as well. > > > > Cheers, > > Till > > > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: > > > > > +1 to include this in 1.9.0, adding some examples doesn't look like new > > > feature to me. > > > BTW, I am also trying this tutorial based on release-1.9 branch, but > > > blocked by: > > > > > > git clone --branch release-1.10-SNAPSHOT > > > [hidden email]:apache/flink-playgrounds.git > > > > > > Neither 1.10 nor 1.9 exists in flink-playground yet. > > > > > > Best, > > > Kurt > > > > > > > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> > wrote: > > > > > > > Hi, > > > > I worked with Konstantin and reviewed the PR. > > > > I think the playground is a great way to get started with Flink and > > > explore > > > > it's recovery mechanism and unique features like savepoints. > > > > > > > > I'm in favor of adding the required streaming example program for the > > 1.9 > > > > release unless there's a good technical argument against it. > > > > > > > > Best, Fabian > > > > > > > > > > -- Konstantin Knauf | Solutions Architect +49 160 91394525 Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
I see that keeping the playground job in the Flink repository has a couple
of advantages, among other things that it's easier to keep up to date. However, in particular in the light of the potential repository split where we want to separate connectors from Flink core, it seems very problematic to put the ClickEventCount which depends on Flink's Kafka connector in Flink's distribution. To me it seems that this was the path of least resistance but I'm not sure whether it stays like this. I think it would have been cleaner to separate the playground project from Flink core. Cheers, Till On Thu, Aug 8, 2019 at 1:28 PM Konstantin Knauf <[hidden email]> wrote: > Hi Till, > > as Fabian said, we considered the option you mentioned, but in the end > decided that not maintaining a separate images has more advantages. > > In the context of FLIP-42 we are also revisiting the examples in general > and want to clean these up a bit. So, for what it's worth, there will be an > opportunity for revisiting this topic soon. > > Best, > > Konstantin > > > > On Thu, Aug 8, 2019 at 11:43 AM Fabian Hueske <[hidden email]> wrote: > > > The motivation for including the job as an example is to not have to > > maintain a separate Docker image. > > We would like to use the regular Flink 1.9 image for the playground and > > avoid to maintain an image that is slightly different from the regular > 1.9 > > image. > > > > Maintaining the job in a different repository or somewhere else would > mean, > > that we need to have a proper release cycle for it as well. > > Having it among the other examples means it's included in the regular > > release. > > > > Best, Fabian > > > > > > Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < > > [hidden email]>: > > > > > Before backporting the playground PR to the release-1.9, I'd like to > > > understand why the ClickEventCount job needs to be part of the Flink > > > distribution. Looking at the example, it seems to only work in > > combination > > > with a Kafka cluster. Since it is not self-contained, it does not add > > much > > > value for a user who does not want to use the playgrounds. Moreover, we > > > already have the StateMachineExample job which can be used to read from > > > Kafka if a Kafka cluster is available. So my question would be why > don't > > we > > > include the example job in the docker images for the playground? This > > would > > > be in my opinion a better separation of concerns. > > > > > > I've cross posted my question on the original PR as well. > > > > > > Cheers, > > > Till > > > > > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: > > > > > > > +1 to include this in 1.9.0, adding some examples doesn't look like > new > > > > feature to me. > > > > BTW, I am also trying this tutorial based on release-1.9 branch, but > > > > blocked by: > > > > > > > > git clone --branch release-1.10-SNAPSHOT > > > > [hidden email]:apache/flink-playgrounds.git > > > > > > > > Neither 1.10 nor 1.9 exists in flink-playground yet. > > > > > > > > Best, > > > > Kurt > > > > > > > > > > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> > > wrote: > > > > > > > > > Hi, > > > > > I worked with Konstantin and reviewed the PR. > > > > > I think the playground is a great way to get started with Flink and > > > > explore > > > > > it's recovery mechanism and unique features like savepoints. > > > > > > > > > > I'm in favor of adding the required streaming example program for > the > > > 1.9 > > > > > release unless there's a good technical argument against it. > > > > > > > > > > Best, Fabian > > > > > > > > > > > > > > > > > -- > > Konstantin Knauf | Solutions Architect > > +49 160 91394525 > > > Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 > > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Just as a short addendum, there are also benefits of having the
ClickEventCount job not being part of the Flink repository. Assume there is a bug in the job, then you would have to wait for the next Flink release to fix it. On Thu, Aug 8, 2019 at 2:24 PM Till Rohrmann <[hidden email]> wrote: > I see that keeping the playground job in the Flink repository has a couple > of advantages, among other things that it's easier to keep up to date. > However, in particular in the light of the potential repository split where > we want to separate connectors from Flink core, it seems very problematic > to put the ClickEventCount which depends on Flink's Kafka connector in > Flink's distribution. To me it seems that this was the path of least > resistance but I'm not sure whether it stays like this. I think it would > have been cleaner to separate the playground project from Flink core. > > Cheers, > Till > > On Thu, Aug 8, 2019 at 1:28 PM Konstantin Knauf <[hidden email]> > wrote: > >> Hi Till, >> >> as Fabian said, we considered the option you mentioned, but in the end >> decided that not maintaining a separate images has more advantages. >> >> In the context of FLIP-42 we are also revisiting the examples in general >> and want to clean these up a bit. So, for what it's worth, there will be >> an >> opportunity for revisiting this topic soon. >> >> Best, >> >> Konstantin >> >> >> >> On Thu, Aug 8, 2019 at 11:43 AM Fabian Hueske <[hidden email]> wrote: >> >> > The motivation for including the job as an example is to not have to >> > maintain a separate Docker image. >> > We would like to use the regular Flink 1.9 image for the playground and >> > avoid to maintain an image that is slightly different from the regular >> 1.9 >> > image. >> > >> > Maintaining the job in a different repository or somewhere else would >> mean, >> > that we need to have a proper release cycle for it as well. >> > Having it among the other examples means it's included in the regular >> > release. >> > >> > Best, Fabian >> > >> > >> > Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < >> > [hidden email]>: >> > >> > > Before backporting the playground PR to the release-1.9, I'd like to >> > > understand why the ClickEventCount job needs to be part of the Flink >> > > distribution. Looking at the example, it seems to only work in >> > combination >> > > with a Kafka cluster. Since it is not self-contained, it does not add >> > much >> > > value for a user who does not want to use the playgrounds. Moreover, >> we >> > > already have the StateMachineExample job which can be used to read >> from >> > > Kafka if a Kafka cluster is available. So my question would be why >> don't >> > we >> > > include the example job in the docker images for the playground? This >> > would >> > > be in my opinion a better separation of concerns. >> > > >> > > I've cross posted my question on the original PR as well. >> > > >> > > Cheers, >> > > Till >> > > >> > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: >> > > >> > > > +1 to include this in 1.9.0, adding some examples doesn't look like >> new >> > > > feature to me. >> > > > BTW, I am also trying this tutorial based on release-1.9 branch, but >> > > > blocked by: >> > > > >> > > > git clone --branch release-1.10-SNAPSHOT >> > > > [hidden email]:apache/flink-playgrounds.git >> > > > >> > > > Neither 1.10 nor 1.9 exists in flink-playground yet. >> > > > >> > > > Best, >> > > > Kurt >> > > > >> > > > >> > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> >> > wrote: >> > > > >> > > > > Hi, >> > > > > I worked with Konstantin and reviewed the PR. >> > > > > I think the playground is a great way to get started with Flink >> and >> > > > explore >> > > > > it's recovery mechanism and unique features like savepoints. >> > > > > >> > > > > I'm in favor of adding the required streaming example program for >> the >> > > 1.9 >> > > > > release unless there's a good technical argument against it. >> > > > > >> > > > > Best, Fabian >> > > > > >> > > > >> > > >> > >> >> >> -- >> >> Konstantin Knauf | Solutions Architect >> >> +49 160 91394525 >> >> >> Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 >> >> >> -- >> >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >> >> -- >> >> Ververica GmbH >> Registered at Amtsgericht Charlottenburg: HRB 158244 B >> Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen >> > |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
Hi Till,
we will try to find another way to make the playground available for users soon. The discussion of and how to split up the Flink Repository started only after we discussed the playground and flink-playgrounds repositories. I think, this is the reason we went this way, not necessarily convenience. Cheers, Konstantin On Thu, Aug 8, 2019 at 2:25 PM Till Rohrmann <[hidden email]> wrote: > Just as a short addendum, there are also benefits of having the > ClickEventCount job not being part of the Flink repository. Assume there is > a bug in the job, then you would have to wait for the next Flink release to > fix it. > > On Thu, Aug 8, 2019 at 2:24 PM Till Rohrmann <[hidden email]> wrote: > > > I see that keeping the playground job in the Flink repository has a > couple > > of advantages, among other things that it's easier to keep up to date. > > However, in particular in the light of the potential repository split > where > > we want to separate connectors from Flink core, it seems very problematic > > to put the ClickEventCount which depends on Flink's Kafka connector in > > Flink's distribution. To me it seems that this was the path of least > > resistance but I'm not sure whether it stays like this. I think it would > > have been cleaner to separate the playground project from Flink core. > > > > Cheers, > > Till > > > > On Thu, Aug 8, 2019 at 1:28 PM Konstantin Knauf < > [hidden email]> > > wrote: > > > >> Hi Till, > >> > >> as Fabian said, we considered the option you mentioned, but in the end > >> decided that not maintaining a separate images has more advantages. > >> > >> In the context of FLIP-42 we are also revisiting the examples in general > >> and want to clean these up a bit. So, for what it's worth, there will be > >> an > >> opportunity for revisiting this topic soon. > >> > >> Best, > >> > >> Konstantin > >> > >> > >> > >> On Thu, Aug 8, 2019 at 11:43 AM Fabian Hueske <[hidden email]> > wrote: > >> > >> > The motivation for including the job as an example is to not have to > >> > maintain a separate Docker image. > >> > We would like to use the regular Flink 1.9 image for the playground > and > >> > avoid to maintain an image that is slightly different from the regular > >> 1.9 > >> > image. > >> > > >> > Maintaining the job in a different repository or somewhere else would > >> mean, > >> > that we need to have a proper release cycle for it as well. > >> > Having it among the other examples means it's included in the regular > >> > release. > >> > > >> > Best, Fabian > >> > > >> > > >> > Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < > >> > [hidden email]>: > >> > > >> > > Before backporting the playground PR to the release-1.9, I'd like to > >> > > understand why the ClickEventCount job needs to be part of the Flink > >> > > distribution. Looking at the example, it seems to only work in > >> > combination > >> > > with a Kafka cluster. Since it is not self-contained, it does not > add > >> > much > >> > > value for a user who does not want to use the playgrounds. Moreover, > >> we > >> > > already have the StateMachineExample job which can be used to read > >> from > >> > > Kafka if a Kafka cluster is available. So my question would be why > >> don't > >> > we > >> > > include the example job in the docker images for the playground? > This > >> > would > >> > > be in my opinion a better separation of concerns. > >> > > > >> > > I've cross posted my question on the original PR as well. > >> > > > >> > > Cheers, > >> > > Till > >> > > > >> > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> wrote: > >> > > > >> > > > +1 to include this in 1.9.0, adding some examples doesn't look > like > >> new > >> > > > feature to me. > >> > > > BTW, I am also trying this tutorial based on release-1.9 branch, > but > >> > > > blocked by: > >> > > > > >> > > > git clone --branch release-1.10-SNAPSHOT > >> > > > [hidden email]:apache/flink-playgrounds.git > >> > > > > >> > > > Neither 1.10 nor 1.9 exists in flink-playground yet. > >> > > > > >> > > > Best, > >> > > > Kurt > >> > > > > >> > > > > >> > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email]> > >> > wrote: > >> > > > > >> > > > > Hi, > >> > > > > I worked with Konstantin and reviewed the PR. > >> > > > > I think the playground is a great way to get started with Flink > >> and > >> > > > explore > >> > > > > it's recovery mechanism and unique features like savepoints. > >> > > > > > >> > > > > I'm in favor of adding the required streaming example program > for > >> the > >> > > 1.9 > >> > > > > release unless there's a good technical argument against it. > >> > > > > > >> > > > > Best, Fabian > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >> -- > >> > >> Konstantin Knauf | Solutions Architect > >> > >> +49 160 91394525 > >> > >> > >> Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 > >> > >> > >> -- > >> > >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > >> > >> -- > >> > >> Ververica GmbH > >> Registered at Amtsgericht Charlottenburg: HRB 158244 B > >> Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > >> > > > -- Konstantin Knauf | Solutions Architect +49 160 91394525 Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen |
Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release
|
|
OK, let's stop the discussion about the playground in the release 1.9
thread. I've started a new thread on [hidden email] to continue the discussion [1]. Best, Fabian [1] https://lists.apache.org/thread.html/4f54c0b4162e3db8626afdca5c354050282282d3cc229d01f2d8ca3e@%3Cdev.flink.apache.org%3E Am Do., 8. Aug. 2019 um 15:23 Uhr schrieb Konstantin Knauf < [hidden email]>: > Hi Till, > > we will try to find another way to make the playground available for users > soon. The discussion of and how to split up the Flink Repository started > only after we discussed the playground and flink-playgrounds repositories. > I think, this is the reason we went this way, not necessarily convenience. > > Cheers, > > Konstantin > > On Thu, Aug 8, 2019 at 2:25 PM Till Rohrmann <[hidden email]> wrote: > > > Just as a short addendum, there are also benefits of having the > > ClickEventCount job not being part of the Flink repository. Assume there > is > > a bug in the job, then you would have to wait for the next Flink release > to > > fix it. > > > > On Thu, Aug 8, 2019 at 2:24 PM Till Rohrmann <[hidden email]> > wrote: > > > > > I see that keeping the playground job in the Flink repository has a > > couple > > > of advantages, among other things that it's easier to keep up to date. > > > However, in particular in the light of the potential repository split > > where > > > we want to separate connectors from Flink core, it seems very > problematic > > > to put the ClickEventCount which depends on Flink's Kafka connector in > > > Flink's distribution. To me it seems that this was the path of least > > > resistance but I'm not sure whether it stays like this. I think it > would > > > have been cleaner to separate the playground project from Flink core. > > > > > > Cheers, > > > Till > > > > > > On Thu, Aug 8, 2019 at 1:28 PM Konstantin Knauf < > > [hidden email]> > > > wrote: > > > > > >> Hi Till, > > >> > > >> as Fabian said, we considered the option you mentioned, but in the end > > >> decided that not maintaining a separate images has more advantages. > > >> > > >> In the context of FLIP-42 we are also revisiting the examples in > general > > >> and want to clean these up a bit. So, for what it's worth, there will > be > > >> an > > >> opportunity for revisiting this topic soon. > > >> > > >> Best, > > >> > > >> Konstantin > > >> > > >> > > >> > > >> On Thu, Aug 8, 2019 at 11:43 AM Fabian Hueske <[hidden email]> > > wrote: > > >> > > >> > The motivation for including the job as an example is to not have to > > >> > maintain a separate Docker image. > > >> > We would like to use the regular Flink 1.9 image for the playground > > and > > >> > avoid to maintain an image that is slightly different from the > regular > > >> 1.9 > > >> > image. > > >> > > > >> > Maintaining the job in a different repository or somewhere else > would > > >> mean, > > >> > that we need to have a proper release cycle for it as well. > > >> > Having it among the other examples means it's included in the > regular > > >> > release. > > >> > > > >> > Best, Fabian > > >> > > > >> > > > >> > Am Do., 8. Aug. 2019 um 09:47 Uhr schrieb Till Rohrmann < > > >> > [hidden email]>: > > >> > > > >> > > Before backporting the playground PR to the release-1.9, I'd like > to > > >> > > understand why the ClickEventCount job needs to be part of the > Flink > > >> > > distribution. Looking at the example, it seems to only work in > > >> > combination > > >> > > with a Kafka cluster. Since it is not self-contained, it does not > > add > > >> > much > > >> > > value for a user who does not want to use the playgrounds. > Moreover, > > >> we > > >> > > already have the StateMachineExample job which can be used to read > > >> from > > >> > > Kafka if a Kafka cluster is available. So my question would be why > > >> don't > > >> > we > > >> > > include the example job in the docker images for the playground? > > This > > >> > would > > >> > > be in my opinion a better separation of concerns. > > >> > > > > >> > > I've cross posted my question on the original PR as well. > > >> > > > > >> > > Cheers, > > >> > > Till > > >> > > > > >> > > On Thu, Aug 8, 2019 at 9:23 AM Kurt Young <[hidden email]> > wrote: > > >> > > > > >> > > > +1 to include this in 1.9.0, adding some examples doesn't look > > like > > >> new > > >> > > > feature to me. > > >> > > > BTW, I am also trying this tutorial based on release-1.9 branch, > > but > > >> > > > blocked by: > > >> > > > > > >> > > > git clone --branch release-1.10-SNAPSHOT > > >> > > > [hidden email]:apache/flink-playgrounds.git > > >> > > > > > >> > > > Neither 1.10 nor 1.9 exists in flink-playground yet. > > >> > > > > > >> > > > Best, > > >> > > > Kurt > > >> > > > > > >> > > > > > >> > > > On Thu, Aug 8, 2019 at 3:18 PM Fabian Hueske <[hidden email] > > > > >> > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > I worked with Konstantin and reviewed the PR. > > >> > > > > I think the playground is a great way to get started with > Flink > > >> and > > >> > > > explore > > >> > > > > it's recovery mechanism and unique features like savepoints. > > >> > > > > > > >> > > > > I'm in favor of adding the required streaming example program > > for > > >> the > > >> > > 1.9 > > >> > > > > release unless there's a good technical argument against it. > > >> > > > > > > >> > > > > Best, Fabian > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >> -- > > >> > > >> Konstantin Knauf | Solutions Architect > > >> > > >> +49 160 91394525 > > >> > > >> > > >> Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 > > >> > > >> > > >> -- > > >> > > >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > >> > > >> -- > > >> > > >> Ververica GmbH > > >> Registered at Amtsgericht Charlottenburg: HRB 158244 B > > >> Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > > >> > > > > > > > > -- > > Konstantin Knauf | Solutions Architect > > +49 160 91394525 > > > Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2019 > > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > |
«
Return to (DEPRECATED) Apache Flink Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |