[DISCUSS] Releasing Flink 1.8.3

[jincheng sun]

Hi Flink devs,

It has been more than 2 months since the 1.8.2 released. So, What do you
think about releasing Flink 1.8.3 soon?

We already have many important bug fixes in the release-1.8 branch (29
resolved issues).

Most notable fixes are:

- FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM is
shut down
- FLINK-14315 NPE with JobMaster.disconnectTaskManager
- FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
- FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
- FLINK-14589 Redundant slot requests with the same AllocationID leads to
inconsistent slot table

Furthermore, the following critical issues is in progress, maybe we can
wait for it if it is not too much effort.

- FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's main
thread

Please let me know what you think?

Best,
Jincheng

[Dian Fu-2]

Hi Jincheng,

Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.

Regards,
Dian

On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]>
wrote:

> Hi Flink devs,
>
> It has been more than 2 months since the 1.8.2 released. So, What do you
> think about releasing Flink 1.8.3 soon?
>
> We already have many important bug fixes in the release-1.8 branch (29
> resolved issues).
>
> Most notable fixes are:
>
> - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM is
> shut down
> - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> - FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
> - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> - FLINK-14589 Redundant slot requests with the same AllocationID leads to
> inconsistent slot table
>
> Furthermore, the following critical issues is in progress, maybe we can
> wait for it if it is not too much effort.
>
> - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's main
> thread
>
> Please let me know what you think?
>
> Best,
> Jincheng
>

[Hequn Cheng]

Hi Jincheng,

Thanks for kicking this discussion off!

+1 to the 1.8.3 release as it would be nice to have these important fixes
and also two
months have passed since the last release.

Besides, I wonder if I can be the release manager of 1.8.3 or work with you
together @Jincheng?
It's always exciting to help the community as much as possible.

Best, Hequn

On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:

> Hi Jincheng,
>
> Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
>
> Regards,
> Dian
>
> On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]>
> wrote:
>
> > Hi Flink devs,
> >
> > It has been more than 2 months since the 1.8.2 released. So, What do you
> > think about releasing Flink 1.8.3 soon?
> >
> > We already have many important bug fixes in the release-1.8 branch (29
> > resolved issues).
> >
> > Most notable fixes are:
> >
> > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
> is
> > shut down
> > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > - FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
> > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > - FLINK-14589 Redundant slot requests with the same AllocationID leads to
> > inconsistent slot table
> >
> > Furthermore, the following critical issues is in progress, maybe we can
> > wait for it if it is not too much effort.
> >
> > - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
> main
> > thread
> >
> > Please let me know what you think?
> >
> > Best,
> > Jincheng
> >
>

[Jark Wu-2]

+1 for the 1.8.3 release and for Hequn being the RM.
Thanks Jincheng for the effort and help on the releasing.

Best,
Jark



On Sat, 9 Nov 2019 at 15:59, Hequn Cheng <[hidden email]> wrote:

> Hi Jincheng,
>
> Thanks for kicking this discussion off!
>
> +1 to the 1.8.3 release as it would be nice to have these important fixes
> and also two
> months have passed since the last release.
>
> Besides, I wonder if I can be the release manager of 1.8.3 or work with you
> together @Jincheng?
> It's always exciting to help the community as much as possible.
>
> Best, Hequn
>
> On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:
>
> > Hi Jincheng,
> >
> > Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
> >
> > Regards,
> > Dian
> >
> > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]>
> > wrote:
> >
> > > Hi Flink devs,
> > >
> > > It has been more than 2 months since the 1.8.2 released. So, What do
> you
> > > think about releasing Flink 1.8.3 soon?
> > >
> > > We already have many important bug fixes in the release-1.8 branch (29
> > > resolved issues).
> > >
> > > Most notable fixes are:
> > >
> > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
> > is
> > > shut down
> > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> fieldsNames
> > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > - FLINK-14589 Redundant slot requests with the same AllocationID leads
> to
> > > inconsistent slot table
> > >
> > > Furthermore, the following critical issues is in progress, maybe we can
> > > wait for it if it is not too much effort.
> > >
> > > - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
> > main
> > > thread
> > >
> > > Please let me know what you think?
> > >
> > > Best,
> > > Jincheng
> > >
> >
>

[Yu Li]

+1 for starting the 1.8.3 release cycle. Thanks for watching and driving
this forward jincheng!

Also +1 for Hequn to be the 1.8.3 RM.

Best Regards,
Yu


On Sat, 9 Nov 2019 at 23:36, Jark Wu <[hidden email]> wrote:

> +1 for the 1.8.3 release and for Hequn being the RM.
> Thanks Jincheng for the effort and help on the releasing.
>
> Best,
> Jark
>
>
>
> On Sat, 9 Nov 2019 at 15:59, Hequn Cheng <[hidden email]> wrote:
>
> > Hi Jincheng,
> >
> > Thanks for kicking this discussion off!
> >
> > +1 to the 1.8.3 release as it would be nice to have these important fixes
> > and also two
> > months have passed since the last release.
> >
> > Besides, I wonder if I can be the release manager of 1.8.3 or work with
> you
> > together @Jincheng?
> > It's always exciting to help the community as much as possible.
> >
> > Best, Hequn
> >
> > On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:
> >
> > > Hi Jincheng,
> > >
> > > Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
> > >
> > > Regards,
> > > Dian
> > >
> > > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]
> >
> > > wrote:
> > >
> > > > Hi Flink devs,
> > > >
> > > > It has been more than 2 months since the 1.8.2 released. So, What do
> > you
> > > > think about releasing Flink 1.8.3 soon?
> > > >
> > > > We already have many important bug fixes in the release-1.8 branch
> (29
> > > > resolved issues).
> > > >
> > > > Most notable fixes are:
> > > >
> > > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when
> AM
> > > is
> > > > shut down
> > > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> > fieldsNames
> > > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > > - FLINK-14589 Redundant slot requests with the same AllocationID
> leads
> > to
> > > > inconsistent slot table
> > > >
> > > > Furthermore, the following critical issues is in progress, maybe we
> can
> > > > wait for it if it is not too much effort.
> > > >
> > > > - FLINK-13184 Starting a TaskExecutor blocks the
> YarnResourceManager's
> > > main
> > > > thread
> > > >
> > > > Please let me know what you think?
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > >
> >
>

[Yang Wang]

Hi Jincheng, Hequn

I am now working on FLINK-13184 and already attach a PR to fix this issue.
I think it is
important for large scale deployment on yarn(1000+ containers). It
will accelerate TaskExecutor
launch and reduce the pressure of hdfs. I hope it could be merge to 1.8.3.


Best,
Yang

Yu Li <[hidden email]> 于2019年11月10日周日 上午10:58写道：

> +1 for starting the 1.8.3 release cycle. Thanks for watching and driving
> this forward jincheng!
>
> Also +1 for Hequn to be the 1.8.3 RM.
>
> Best Regards,
> Yu
>
>
> On Sat, 9 Nov 2019 at 23:36, Jark Wu <[hidden email]> wrote:
>
> > +1 for the 1.8.3 release and for Hequn being the RM.
> > Thanks Jincheng for the effort and help on the releasing.
> >
> > Best,
> > Jark
> >
> >
> >
> > On Sat, 9 Nov 2019 at 15:59, Hequn Cheng <[hidden email]> wrote:
> >
> > > Hi Jincheng,
> > >
> > > Thanks for kicking this discussion off!
> > >
> > > +1 to the 1.8.3 release as it would be nice to have these important
> fixes
> > > and also two
> > > months have passed since the last release.
> > >
> > > Besides, I wonder if I can be the release manager of 1.8.3 or work with
> > you
> > > together @Jincheng?
> > > It's always exciting to help the community as much as possible.
> > >
> > > Best, Hequn
> > >
> > > On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:
> > >
> > > > Hi Jincheng,
> > > >
> > > > Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
> > > >
> > > > Regards,
> > > > Dian
> > > >
> > > > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Hi Flink devs,
> > > > >
> > > > > It has been more than 2 months since the 1.8.2 released. So, What
> do
> > > you
> > > > > think about releasing Flink 1.8.3 soon?
> > > > >
> > > > > We already have many important bug fixes in the release-1.8 branch
> > (29
> > > > > resolved issues).
> > > > >
> > > > > Most notable fixes are:
> > > > >
> > > > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership
> when
> > AM
> > > > is
> > > > > shut down
> > > > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> > > fieldsNames
> > > > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > > > - FLINK-14589 Redundant slot requests with the same AllocationID
> > leads
> > > to
> > > > > inconsistent slot table
> > > > >
> > > > > Furthermore, the following critical issues is in progress, maybe we
> > can
> > > > > wait for it if it is not too much effort.
> > > > >
> > > > > - FLINK-13184 Starting a TaskExecutor blocks the
> > YarnResourceManager's
> > > > main
> > > > > thread
> > > > >
> > > > > Please let me know what you think?
> > > > >
> > > > > Best,
> > > > > Jincheng
> > > > >
> > > >
> > >
> >
>

[jincheng sun]

Thanks for all of your feedback!

Hi Yang, great thanks for  working on FLINK-13184 and advancing this issue,
we hope that more problems can be solved in 1.8.3.

Hi Hequn, glad to hear that you want to be the Release Manager of flink
1.8.3.

I believe that you will be a great RM, and I am very willing to help you
with the final release in the final stages. :)

The release of Apache Flink involves a number of tasks. For details, you
can consult the documentation [1]. If you have any questions, please let me
know and let us work together.

[1]
https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release

Best,
Jincheng

Yang Wang <[hidden email]> 于2019年11月10日周日 下午8:09写道：

> Hi Jincheng, Hequn
>
> I am now working on FLINK-13184 and already attach a PR to fix this issue.
> I think it is
> important for large scale deployment on yarn(1000+ containers). It
> will accelerate TaskExecutor
> launch and reduce the pressure of hdfs. I hope it could be merge to 1.8.3.
>
>
> Best,
> Yang
>
> Yu Li <[hidden email]> 于2019年11月10日周日 上午10:58写道：
>
> > +1 for starting the 1.8.3 release cycle. Thanks for watching and driving
> > this forward jincheng!
> >
> > Also +1 for Hequn to be the 1.8.3 RM.
> >
> > Best Regards,
> > Yu
> >
> >
> > On Sat, 9 Nov 2019 at 23:36, Jark Wu <[hidden email]> wrote:
> >
> > > +1 for the 1.8.3 release and for Hequn being the RM.
> > > Thanks Jincheng for the effort and help on the releasing.
> > >
> > > Best,
> > > Jark
> > >
> > >
> > >
> > > On Sat, 9 Nov 2019 at 15:59, Hequn Cheng <[hidden email]> wrote:
> > >
> > > > Hi Jincheng,
> > > >
> > > > Thanks for kicking this discussion off!
> > > >
> > > > +1 to the 1.8.3 release as it would be nice to have these important
> > fixes
> > > > and also two
> > > > months have passed since the last release.
> > > >
> > > > Besides, I wonder if I can be the release manager of 1.8.3 or work
> with
> > > you
> > > > together @Jincheng?
> > > > It's always exciting to help the community as much as possible.
> > > >
> > > > Best, Hequn
> > > >
> > > > On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]>
> wrote:
> > > >
> > > > > Hi Jincheng,
> > > > >
> > > > > Thanks a lot for bringing up this discussion. +1 for releasing
> 1.8.3.
> > > > >
> > > > > Regards,
> > > > > Dian
> > > > >
> > > > > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Flink devs,
> > > > > >
> > > > > > It has been more than 2 months since the 1.8.2 released. So, What
> > do
> > > > you
> > > > > > think about releasing Flink 1.8.3 soon?
> > > > > >
> > > > > > We already have many important bug fixes in the release-1.8
> branch
> > > (29
> > > > > > resolved issues).
> > > > > >
> > > > > > Most notable fixes are:
> > > > > >
> > > > > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership
> > when
> > > AM
> > > > > is
> > > > > > shut down
> > > > > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > > > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> > > > fieldsNames
> > > > > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > > > > - FLINK-14589 Redundant slot requests with the same AllocationID
> > > leads
> > > > to
> > > > > > inconsistent slot table
> > > > > >
> > > > > > Furthermore, the following critical issues is in progress, maybe
> we
> > > can
> > > > > > wait for it if it is not too much effort.
> > > > > >
> > > > > > - FLINK-13184 Starting a TaskExecutor blocks the
> > > YarnResourceManager's
> > > > > main
> > > > > > thread
> > > > > >
> > > > > > Please let me know what you think?
> > > > > >
> > > > > > Best,
> > > > > > Jincheng
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Danny Chan]

Thanks Hequn to be in change of the release !

Best,
Danny Chan
在 2019年11月9日 +0800 PM3:59，Hequn Cheng <[hidden email]>，写道：

> Hi Jincheng,
>
> Thanks for kicking this discussion off!
>
> +1 to the 1.8.3 release as it would be nice to have these important fixes
> and also two
> months have passed since the last release.
>
> Besides, I wonder if I can be the release manager of 1.8.3 or work with you
> together @Jincheng?
> It's always exciting to help the community as much as possible.
>
> Best, Hequn
>
> On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:
>
> > Hi Jincheng,
> >
> > Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
> >
> > Regards,
> > Dian
> >
> > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]>
> > wrote:
> >
> > > Hi Flink devs,
> > >
> > > It has been more than 2 months since the 1.8.2 released. So, What do you
> > > think about releasing Flink 1.8.3 soon?
> > >
> > > We already have many important bug fixes in the release-1.8 branch (29
> > > resolved issues).
> > >
> > > Most notable fixes are:
> > >
> > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
> > is
> > > shut down
> > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > - FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
> > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > - FLINK-14589 Redundant slot requests with the same AllocationID leads to
> > > inconsistent slot table
> > >
> > > Furthermore, the following critical issues is in progress, maybe we can
> > > wait for it if it is not too much effort.
> > >
> > > - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
> > main
> > > thread
> > >
> > > Please let me know what you think?
> > >
> > > Best,
> > > Jincheng
> > >
> >

[Hequn Cheng]

Hi all,

Thanks a lot for your kindly reply and feedbacks.

@Jincheng Thank you very much for providing the release help.
@Yang Wang Thanks a lot for your PR. I think it is an important fix! I left
some comments in FLINK-13184 and we can discuss whether this is a blocker
there.

I have gone over all the issues of 1.8.3. Currently, only one critical
issue:
[Critical]:
[FLINK-13184] Starting a TaskExecutor blocks the YarnResourceManager's main
thread(@Yang Wang is working on it, reviewed by @Rongrong and @Tison)

All issues of this release can also be tracked here:
https://issues.apache.org/jira/projects/FLINK/versions/12346112

Ideally, we can kick off the release vote for the first RC early next week.
If there are some other critical issues for 1.8.3, please let me know here
to account for it for the 1.8.3 release.

Best, Hequn

On Mon, Nov 11, 2019 at 2:27 PM Danny Chan <[hidden email]> wrote:

> Thanks Hequn to be in change of the release !
>
> Best,
> Danny Chan
> 在 2019年11月9日 +0800 PM3:59，Hequn Cheng <[hidden email]>，写道：
> > Hi Jincheng,
> >
> > Thanks for kicking this discussion off!
> >
> > +1 to the 1.8.3 release as it would be nice to have these important fixes
> > and also two
> > months have passed since the last release.
> >
> > Besides, I wonder if I can be the release manager of 1.8.3 or work with
> you
> > together @Jincheng?
> > It's always exciting to help the community as much as possible.
> >
> > Best, Hequn
> >
> > On Sat, Nov 9, 2019 at 12:34 PM Dian Fu <[hidden email]> wrote:
> >
> > > Hi Jincheng,
> > >
> > > Thanks a lot for bringing up this discussion. +1 for releasing 1.8.3.
> > >
> > > Regards,
> > > Dian
> > >
> > > On Sat, Nov 9, 2019 at 12:11 PM jincheng sun <[hidden email]
> >
> > > wrote:
> > >
> > > > Hi Flink devs,
> > > >
> > > > It has been more than 2 months since the 1.8.2 released. So, What do
> you
> > > > think about releasing Flink 1.8.3 soon?
> > > >
> > > > We already have many important bug fixes in the release-1.8 branch
> (29
> > > > resolved issues).
> > > >
> > > > Most notable fixes are:
> > > >
> > > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when
> AM
> > > is
> > > > shut down
> > > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> fieldsNames
> > > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > > - FLINK-14589 Redundant slot requests with the same AllocationID
> leads to
> > > > inconsistent slot table
> > > >
> > > > Furthermore, the following critical issues is in progress, maybe we
> can
> > > > wait for it if it is not too much effort.
> > > >
> > > > - FLINK-13184 Starting a TaskExecutor blocks the
> YarnResourceManager's
> > > main
> > > > thread
> > > >
> > > > Please let me know what you think?
> > > >
> > > > Best,
> > > > Jincheng
> > > >
> > >
>

[Chesnay Schepler-3]

It would be great if you could give me a day or 2 to check how easy it
would be to bump the various jackson dependencies to eliminate a few
security vulnerabilities.

On 09/11/2019 05:10, jincheng sun wrote:

> Hi Flink devs,
>
> It has been more than 2 months since the 1.8.2 released. So, What do you
> think about releasing Flink 1.8.3 soon?
>
> We already have many important bug fixes in the release-1.8 branch (29
> resolved issues).
>
> Most notable fixes are:
>
> - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM is
> shut down
> - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> - FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
> - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> - FLINK-14589 Redundant slot requests with the same AllocationID leads to
> inconsistent slot table
>
> Furthermore, the following critical issues is in progress, maybe we can
> wait for it if it is not too much effort.
>
> - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's main
> thread
>
> Please let me know what you think?
>
> Best,
> Jincheng
>

[jincheng sun]

+1 for try to eliminate the security vulnerabilities. Great thanks for
doing this important work, Chesnay!
What do you think Hequn ?

Best,
Jincheng

Chesnay Schepler <[hidden email]> 于2019年11月13日周三 下午5:17写道：

> It would be great if you could give me a day or 2 to check how easy it
> would be to bump the various jackson dependencies to eliminate a few
> security vulnerabilities.
>
> On 09/11/2019 05:10, jincheng sun wrote:
> > Hi Flink devs,
> >
> > It has been more than 2 months since the 1.8.2 released. So, What do you
> > think about releasing Flink 1.8.3 soon?
> >
> > We already have many important bug fixes in the release-1.8 branch (29
> > resolved issues).
> >
> > Most notable fixes are:
> >
> > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
> is
> > shut down
> > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > - FLINK-12848 Method equals() in RowTypeInfo should consider fieldsNames
> > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > - FLINK-14589 Redundant slot requests with the same AllocationID leads to
> > inconsistent slot table
> >
> > Furthermore, the following critical issues is in progress, maybe we can
> > wait for it if it is not too much effort.
> >
> > - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
> main
> > thread
> >
> > Please let me know what you think?
> >
> > Best,
> > Jincheng
> >
>
>

[Hequn Cheng]

Hi Chesnay, Jincheng

Sure, I think it's good to have these fixes.
Thanks a lot for providing the information about the security
vulnerabilities! @Chesnay

Best, Hequn

On Thu, Nov 14, 2019 at 10:07 AM jincheng sun <[hidden email]>
wrote:

> +1 for try to eliminate the security vulnerabilities. Great thanks for
> doing this important work, Chesnay!
> What do you think Hequn ?
>
> Best,
> Jincheng
>
> Chesnay Schepler <[hidden email]> 于2019年11月13日周三 下午5:17写道：
>
> > It would be great if you could give me a day or 2 to check how easy it
> > would be to bump the various jackson dependencies to eliminate a few
> > security vulnerabilities.
> >
> > On 09/11/2019 05:10, jincheng sun wrote:
> > > Hi Flink devs,
> > >
> > > It has been more than 2 months since the 1.8.2 released. So, What do
> you
> > > think about releasing Flink 1.8.3 soon?
> > >
> > > We already have many important bug fixes in the release-1.8 branch (29
> > > resolved issues).
> > >
> > > Most notable fixes are:
> > >
> > > - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
> > is
> > > shut down
> > > - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> > > - FLINK-12848 Method equals() in RowTypeInfo should consider
> fieldsNames
> > > - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> > > - FLINK-14589 Redundant slot requests with the same AllocationID leads
> to
> > > inconsistent slot table
> > >
> > > Furthermore, the following critical issues is in progress, maybe we can
> > > wait for it if it is not too much effort.
> > >
> > > - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
> > main
> > > thread
> > >
> > > Please let me know what you think?
> > >
> > > Best,
> > > Jincheng
> > >
> >
> >
>

[Chesnay Schepler-3]

So here's the state of things:


The master of flink-shaded now uses jackson 2.10.1, which eliminates a
whole category of security vulnerabilities.
The flink master works perfectly fine with that version; 1.9 will likely
do so too and 1.8 would require a minor adjustment.

Hence, there may be value in first doing a flink-shaded release so we
can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .


As for other jackson dependencies (coming from calcite, kafka, kinesis),
I ran the unit and end-to-end tests of master yesterday will /all
/jackson dependencies set to 2.10.1, and they passed. I will open a PR
soon-ish for making this change on master.

The question now is whether we want to backport this change to 1.8/1.9 .
Some code paths /may /not be covered by our tests, and transitive
jackson users /might /run into issues.
Alternatively, we could set this up as an opt-in upgrade, by adding a
separate profile that bumps the versions. This would present
users/providers who are concerned about the vulnerabilities an easy
workaround, at the risk of /some /things /maybe /not working.

On 14/11/2019 03:16, Hequn Cheng wrote:

> Hi Chesnay, Jincheng
>
> Sure, I think it's good to have these fixes.
> Thanks a lot for providing the information about the security
> vulnerabilities! @Chesnay
>
> Best, Hequn
>
> On Thu, Nov 14, 2019 at 10:07 AM jincheng sun <[hidden email]>
> wrote:
>
>> +1 for try to eliminate the security vulnerabilities. Great thanks for
>> doing this important work, Chesnay!
>> What do you think Hequn ?
>>
>> Best,
>> Jincheng
>>
>> Chesnay Schepler <[hidden email]> 于2019年11月13日周三 下午5:17写道：
>>
>>> It would be great if you could give me a day or 2 to check how easy it
>>> would be to bump the various jackson dependencies to eliminate a few
>>> security vulnerabilities.
>>>
>>> On 09/11/2019 05:10, jincheng sun wrote:
>>>> Hi Flink devs,
>>>>
>>>> It has been more than 2 months since the 1.8.2 released. So, What do
>> you
>>>> think about releasing Flink 1.8.3 soon?
>>>>
>>>> We already have many important bug fixes in the release-1.8 branch (29
>>>> resolved issues).
>>>>
>>>> Most notable fixes are:
>>>>
>>>> - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
>>> is
>>>> shut down
>>>> - FLINK-14315 NPE with JobMaster.disconnectTaskManager
>>>> - FLINK-12848 Method equals() in RowTypeInfo should consider
>> fieldsNames
>>>> - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
>>>> - FLINK-14589 Redundant slot requests with the same AllocationID leads
>> to
>>>> inconsistent slot table
>>>>
>>>> Furthermore, the following critical issues is in progress, maybe we can
>>>> wait for it if it is not too much effort.
>>>>
>>>> - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
>>> main
>>>> thread
>>>>
>>>> Please let me know what you think?
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>

[Hequn Cheng]

Hi Chesnay,

Great to hear that jackson-2.10.1 works well on master. Really a good job!

- Whether backport this change to 1.8/1.9
I had taken a quick look at the security vulnerabilities, some of them seem
can lead to high-security problems, thus from my point of view, I'm in
favor of adding the fix into 1.9/1.8. However, I would like to trust your
judgment as you are more professional at this problem.

- How to port this change to 1.8/1.9
I think providing an opt-in upgrade is a good idea. Another question here
is whether do we plan to support multi jackson versions that have
eliminated the security vulnerabilities. If we only plan to support 2.10.1,
I would like to make it a non-opt-in upgrade. As an option, users can
downgrade the flink version if meet problems using the new version. Of
course, we will try our best to make the new release out of question.
Another concern of making it an opt-in upgrade is, it will make our build
unlikely convergence as more and more build options will be added when we
upgrade a commonly used lib like this one.

What do you think?

Best, Hequn

On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]> wrote:

> So here's the state of things:
>
>
> The master of flink-shaded now uses jackson 2.10.1, which eliminates a
> whole category of security vulnerabilities.
> The flink master works perfectly fine with that version; 1.9 will likely
> do so too and 1.8 would require a minor adjustment.
>
> Hence, there may be value in first doing a flink-shaded release so we can
> eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
>
>
> As for other jackson dependencies (coming from calcite, kafka, kinesis), I
> ran the unit and end-to-end tests of master yesterday will *all *jackson
> dependencies set to 2.10.1, and they passed. I will open a PR soon-ish for
> making this change on master.
>
> The question now is whether we want to backport this change to 1.8/1.9 .
> Some code paths *may *not be covered by our tests, and transitive jackson
> users *might *run into issues.
> Alternatively, we could set this up as an opt-in upgrade, by adding a
> separate profile that bumps the versions. This would present
> users/providers who are concerned about the vulnerabilities an easy
> workaround, at the risk of *some *things *maybe *not working.
>
> On 14/11/2019 03:16, Hequn Cheng wrote:
>
> Hi Chesnay, Jincheng
>
> Sure, I think it's good to have these fixes.
> Thanks a lot for providing the information about the security
> vulnerabilities! @Chesnay
>
> Best, Hequn
>
> On Thu, Nov 14, 2019 at 10:07 AM jincheng sun <[hidden email]> <[hidden email]>
> wrote:
>
>
> +1 for try to eliminate the security vulnerabilities. Great thanks for
> doing this important work, Chesnay!
> What do you think Hequn ?
>
> Best,
> Jincheng
>
> Chesnay Schepler <[hidden email]> <[hidden email]> 于2019年11月13日周三 下午5:17写道：
>
>
> It would be great if you could give me a day or 2 to check how easy it
> would be to bump the various jackson dependencies to eliminate a few
> security vulnerabilities.
>
> On 09/11/2019 05:10, jincheng sun wrote:
>
> Hi Flink devs,
>
> It has been more than 2 months since the 1.8.2 released. So, What do
>
> you
>
> think about releasing Flink 1.8.3 soon?
>
> We already have many important bug fixes in the release-1.8 branch (29
> resolved issues).
>
> Most notable fixes are:
>
> - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
>
> is
>
> shut down
> - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> - FLINK-12848 Method equals() in RowTypeInfo should consider
>
> fieldsNames
>
> - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> - FLINK-14589 Redundant slot requests with the same AllocationID leads
>
> to
>
> inconsistent slot table
>
> Furthermore, the following critical issues is in progress, maybe we can
> wait for it if it is not too much effort.
>
> - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
>
> main
>
> thread
>
> Please let me know what you think?
>
> Best,
> Jincheng
>
>
>
>

[Chesnay Schepler-3]

The opt-in approach would only be used for 1.8.3 / 1.9.2; on master (and
thus starting from 1.10.0) it's not opt-in.

I have only proposed it as an opt-in because a) we usually do not bump
dependencies in bugfix releases and b) it's a short-term change that we
aren't allowing to mature properly.
In contrast, the 1.10 release is significantly further away, hence no
opt-in.

Hence, I'm not concerned about such kind of ugprades being more common
in the future.

We can certainly support every jackson version that fixes these
vulnerabilities; individual modules can always use a different version
(that hopefully includes the fixes).
Ideally of course we'd only be using 1 version, but that may or may not
be feasible.

On 15/11/2019 04:07, Hequn Cheng wrote:

> Hi Chesnay,
>
> Great to hear that jackson-2.10.1 works well on master. Really a good job!
>
> - Whether backport this change to 1.8/1.9
> I had taken a quick look at the security vulnerabilities, some of them
> seem can lead to high-security problems, thus from my point of view,
> I'm in favor of adding the fix into 1.9/1.8. However, I would like to
> trust your judgment as you are more professional at this problem.
>
> - How to port this change to 1.8/1.9
> I think providing an opt-in upgrade is a good idea. Another question
> here is whether do we plan to support multi jackson versions that have
> eliminated the security vulnerabilities. If we only plan to support
> 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
> users can downgrade the flink version if meet problems using the new
> version. Of course, we will try our best to make the new release out
> of question.
> Another concern of making it an opt-in upgrade is, it will make our
> build unlikely convergence as more and more build options will be
> added when we upgrade a commonly used lib like this one.
>
> What do you think?
>
> Best, Hequn
>
> On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     So here's the state of things:
>
>
>     The master of flink-shaded now uses jackson 2.10.1, which
>     eliminates a whole category of security vulnerabilities.
>     The flink master works perfectly fine with that version; 1.9 will
>     likely do so too and 1.8 would require a minor adjustment.
>
>     Hence, there may be value in first doing a flink-shaded release so
>     we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
>
>
>     As for other jackson dependencies (coming from calcite, kafka,
>     kinesis), I ran the unit and end-to-end tests of master yesterday
>     will /all /jackson dependencies set to 2.10.1, and they passed. I
>     will open a PR soon-ish for making this change on master.
>
>     The question now is whether we want to backport this change to
>     1.8/1.9 .
>     Some code paths /may /not be covered by our tests, and transitive
>     jackson users /might /run into issues.
>     Alternatively, we could set this up as an opt-in upgrade, by
>     adding a separate profile that bumps the versions. This would
>     present users/providers who are concerned about the
>     vulnerabilities an easy workaround, at the risk of /some /things
>     /maybe /not working.
>
>     On 14/11/2019 03:16, Hequn Cheng wrote:
>>     Hi Chesnay, Jincheng
>>
>>     Sure, I think it's good to have these fixes.
>>     Thanks a lot for providing the information about the security
>>     vulnerabilities! @Chesnay
>>
>>     Best, Hequn
>>
>>     On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<[hidden email]>  <mailto:[hidden email]>
>>     wrote:
>>
>>>     +1 for try to eliminate the security vulnerabilities. Great thanks for
>>>     doing this important work, Chesnay!
>>>     What do you think Hequn ?
>>>
>>>     Best,
>>>     Jincheng
>>>
>>>     Chesnay Schepler<[hidden email]>  <mailto:[hidden email]>  于2019年11月13日周三 下午5:17写道：
>>>
>>>>     It would be great if you could give me a day or 2 to check how easy it
>>>>     would be to bump the various jackson dependencies to eliminate a few
>>>>     security vulnerabilities.
>>>>
>>>>     On 09/11/2019 05:10, jincheng sun wrote:
>>>>>     Hi Flink devs,
>>>>>
>>>>>     It has been more than 2 months since the 1.8.2 released. So, What do
>>>     you
>>>>>     think about releasing Flink 1.8.3 soon?
>>>>>
>>>>>     We already have many important bug fixes in the release-1.8 branch (29
>>>>>     resolved issues).
>>>>>
>>>>>     Most notable fixes are:
>>>>>
>>>>>     - FLINK-14010 Dispatcher & JobManagers don't give up leadership when AM
>>>>     is
>>>>>     shut down
>>>>>     - FLINK-14315 NPE with JobMaster.disconnectTaskManager
>>>>>     - FLINK-12848 Method equals() in RowTypeInfo should consider
>>>     fieldsNames
>>>>>     - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
>>>>>     - FLINK-14589 Redundant slot requests with the same AllocationID leads
>>>     to
>>>>>     inconsistent slot table
>>>>>
>>>>>     Furthermore, the following critical issues is in progress, maybe we can
>>>>>     wait for it if it is not too much effort.
>>>>>
>>>>>     - FLINK-13184 Starting a TaskExecutor blocks the YarnResourceManager's
>>>>     main
>>>>>     thread
>>>>>
>>>>>     Please let me know what you think?
>>>>>
>>>>>     Best,
>>>>>     Jincheng
>>>>>
>

[Ufuk Celebi-2]

The opt-in approach seems reasonable to me. +1 to include the profiles in
1.8 and 1.9 without changing the default versions (including the default
version of flink-shaded).

As far as I can tell, the next steps would be:

1) Release flink-shaded with upgraded Jackson
2a) Bump the flink-shaded version by default in master
2b) Create opt-in profiles for 1.8 and 1.9 (the opt-in profiles should also
cover the upgrade to the most recent flink-shaded version)

@Chesnay: is this a correct summary?

Note this would block the 1.8.3 release on step 1. As an upside, we might
get some additional feedback until the 1.10 release with these profiles in
case users make use of them with 1.8/1.9.

– Ufuk

On Fri, Nov 15, 2019 at 12:08 PM Chesnay Schepler <[hidden email]>
wrote:

>
> The opt-in approach would only be used for 1.8.3 / 1.9.2; on master (and
> thus starting from 1.10.0) it's not opt-in.
>
> I have only proposed it as an opt-in because a) we usually do not bump
> dependencies in bugfix releases and b) it's a short-term change that we
> aren't allowing to mature properly.
> In contrast, the 1.10 release is significantly further away, hence no
> opt-in.
>
> Hence, I'm not concerned about such kind of ugprades being more common
> in the future.
>
> We can certainly support every jackson version that fixes these
> vulnerabilities; individual modules can always use a different version
> (that hopefully includes the fixes).
> Ideally of course we'd only be using 1 version, but that may or may not
> be feasible.
>
> On 15/11/2019 04:07, Hequn Cheng wrote:
> > Hi Chesnay,
> >
> > Great to hear that jackson-2.10.1 works well on master. Really a good

job!

> >
> > - Whether backport this change to 1.8/1.9
> > I had taken a quick look at the security vulnerabilities, some of them
> > seem can lead to high-security problems, thus from my point of view,
> > I'm in favor of adding the fix into 1.9/1.8. However, I would like to
> > trust your judgment as you are more professional at this problem.
> >
> > - How to port this change to 1.8/1.9
> > I think providing an opt-in upgrade is a good idea. Another question
> > here is whether do we plan to support multi jackson versions that have
> > eliminated the security vulnerabilities. If we only plan to support
> > 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
> > users can downgrade the flink version if meet problems using the new
> > version. Of course, we will try our best to make the new release out
> > of question.
> > Another concern of making it an opt-in upgrade is, it will make our
> > build unlikely convergence as more and more build options will be
> > added when we upgrade a commonly used lib like this one.
> >
> > What do you think?
> >
> > Best, Hequn
> >
> > On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >
> >     So here's the state of things:
> >
> >
> >     The master of flink-shaded now uses jackson 2.10.1, which
> >     eliminates a whole category of security vulnerabilities.
> >     The flink master works perfectly fine with that version; 1.9 will
> >     likely do so too and 1.8 would require a minor adjustment.
> >
> >     Hence, there may be value in first doing a flink-shaded release so
> >     we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
> >
> >
> >     As for other jackson dependencies (coming from calcite, kafka,
> >     kinesis), I ran the unit and end-to-end tests of master yesterday
> >     will /all /jackson dependencies set to 2.10.1, and they passed. I
> >     will open a PR soon-ish for making this change on master.
> >
> >     The question now is whether we want to backport this change to
> >     1.8/1.9 .
> >     Some code paths /may /not be covered by our tests, and transitive
> >     jackson users /might /run into issues.
> >     Alternatively, we could set this up as an opt-in upgrade, by
> >     adding a separate profile that bumps the versions. This would
> >     present users/providers who are concerned about the
> >     vulnerabilities an easy workaround, at the risk of /some /things
> >     /maybe /not working.
> >
> >     On 14/11/2019 03:16, Hequn Cheng wrote:
> >>     Hi Chesnay, Jincheng
> >>
> >>     Sure, I think it's good to have these fixes.
> >>     Thanks a lot for providing the information about the security
> >>     vulnerabilities! @Chesnay
> >>
> >>     Best, Hequn
> >>
> >>     On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<

[hidden email]>  <mailto:[hidden email]>
> >>     wrote:
> >>
> >>>     +1 for try to eliminate the security vulnerabilities. Great
thanks for
> >>>     doing this important work, Chesnay!
> >>>     What do you think Hequn ?
> >>>
> >>>     Best,
> >>>     Jincheng
> >>>
> >>>     Chesnay Schepler<[hidden email]>  <mailto:[hidden email]>
 于2019年11月13日周三 下午5:17写道：
> >>>
> >>>>     It would be great if you could give me a day or 2 to check how
easy it
> >>>>     would be to bump the various jackson dependencies to eliminate a
few
> >>>>     security vulnerabilities.
> >>>>
> >>>>     On 09/11/2019 05:10, jincheng sun wrote:
> >>>>>     Hi Flink devs,
> >>>>>
> >>>>>     It has been more than 2 months since the 1.8.2 released. So,
What do
> >>>     you
> >>>>>     think about releasing Flink 1.8.3 soon?
> >>>>>
> >>>>>     We already have many important bug fixes in the release-1.8
branch (29
> >>>>>     resolved issues).
> >>>>>
> >>>>>     Most notable fixes are:
> >>>>>
> >>>>>     - FLINK-14010 Dispatcher & JobManagers don't give up leadership
when AM
> >>>>     is
> >>>>>     shut down
> >>>>>     - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> >>>>>     - FLINK-12848 Method equals() in RowTypeInfo should consider
> >>>     fieldsNames
> >>>>>     - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
> >>>>>     - FLINK-14589 Redundant slot requests with the same
AllocationID leads
> >>>     to
> >>>>>     inconsistent slot table
> >>>>>
> >>>>>     Furthermore, the following critical issues is in progress,
maybe we can
> >>>>>     wait for it if it is not too much effort.
> >>>>>
> >>>>>     - FLINK-13184 Starting a TaskExecutor blocks the
YarnResourceManager's
> >>>>     main
> >>>>>     thread
> >>>>>
> >>>>>     Please let me know what you think?
> >>>>>
> >>>>>     Best,
> >>>>>     Jincheng
> >>>>>
> >

[Chesnay Schepler-3]

Ufuk's summary is correct.

There's a slight caveat in that we'd also have to bump the shade-plugin
to 3.1.1 since it otherwise fails on jackson,
but I have no concerns about this change.

On 15/11/2019 12:19, Ufuk Celebi wrote:

> The opt-in approach seems reasonable to me. +1 to include the profiles in
> 1.8 and 1.9 without changing the default versions (including the default
> version of flink-shaded).
>
> As far as I can tell, the next steps would be:
>
> 1) Release flink-shaded with upgraded Jackson
> 2a) Bump the flink-shaded version by default in master
> 2b) Create opt-in profiles for 1.8 and 1.9 (the opt-in profiles should also
> cover the upgrade to the most recent flink-shaded version)
>
> @Chesnay: is this a correct summary?
>
> Note this would block the 1.8.3 release on step 1. As an upside, we might
> get some additional feedback until the 1.10 release with these profiles in
> case users make use of them with 1.8/1.9.
>
> – Ufuk
>
> On Fri, Nov 15, 2019 at 12:08 PM Chesnay Schepler <[hidden email]>
> wrote:
>> The opt-in approach would only be used for 1.8.3 / 1.9.2; on master (and
>> thus starting from 1.10.0) it's not opt-in.
>>
>> I have only proposed it as an opt-in because a) we usually do not bump
>> dependencies in bugfix releases and b) it's a short-term change that we
>> aren't allowing to mature properly.
>> In contrast, the 1.10 release is significantly further away, hence no
>> opt-in.
>>
>> Hence, I'm not concerned about such kind of ugprades being more common
>> in the future.
>>
>> We can certainly support every jackson version that fixes these
>> vulnerabilities; individual modules can always use a different version
>> (that hopefully includes the fixes).
>> Ideally of course we'd only be using 1 version, but that may or may not
>> be feasible.
>>
>> On 15/11/2019 04:07, Hequn Cheng wrote:
>>> Hi Chesnay,
>>>
>>> Great to hear that jackson-2.10.1 works well on master. Really a good
> job!
>>> - Whether backport this change to 1.8/1.9
>>> I had taken a quick look at the security vulnerabilities, some of them
>>> seem can lead to high-security problems, thus from my point of view,
>>> I'm in favor of adding the fix into 1.9/1.8. However, I would like to
>>> trust your judgment as you are more professional at this problem.
>>>
>>> - How to port this change to 1.8/1.9
>>> I think providing an opt-in upgrade is a good idea. Another question
>>> here is whether do we plan to support multi jackson versions that have
>>> eliminated the security vulnerabilities. If we only plan to support
>>> 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
>>> users can downgrade the flink version if meet problems using the new
>>> version. Of course, we will try our best to make the new release out
>>> of question.
>>> Another concern of making it an opt-in upgrade is, it will make our
>>> build unlikely convergence as more and more build options will be
>>> added when we upgrade a commonly used lib like this one.
>>>
>>> What do you think?
>>>
>>> Best, Hequn
>>>
>>> On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>      So here's the state of things:
>>>
>>>
>>>      The master of flink-shaded now uses jackson 2.10.1, which
>>>      eliminates a whole category of security vulnerabilities.
>>>      The flink master works perfectly fine with that version; 1.9 will
>>>      likely do so too and 1.8 would require a minor adjustment.
>>>
>>>      Hence, there may be value in first doing a flink-shaded release so
>>>      we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
>>>
>>>
>>>      As for other jackson dependencies (coming from calcite, kafka,
>>>      kinesis), I ran the unit and end-to-end tests of master yesterday
>>>      will /all /jackson dependencies set to 2.10.1, and they passed. I
>>>      will open a PR soon-ish for making this change on master.
>>>
>>>      The question now is whether we want to backport this change to
>>>      1.8/1.9 .
>>>      Some code paths /may /not be covered by our tests, and transitive
>>>      jackson users /might /run into issues.
>>>      Alternatively, we could set this up as an opt-in upgrade, by
>>>      adding a separate profile that bumps the versions. This would
>>>      present users/providers who are concerned about the
>>>      vulnerabilities an easy workaround, at the risk of /some /things
>>>      /maybe /not working.
>>>
>>>      On 14/11/2019 03:16, Hequn Cheng wrote:
>>>>      Hi Chesnay, Jincheng
>>>>
>>>>      Sure, I think it's good to have these fixes.
>>>>      Thanks a lot for providing the information about the security
>>>>      vulnerabilities! @Chesnay
>>>>
>>>>      Best, Hequn
>>>>
>>>>      On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<
> [hidden email]>  <mailto:[hidden email]>
>>>>      wrote:
>>>>
>>>>>      +1 for try to eliminate the security vulnerabilities. Great
> thanks for
>>>>>      doing this important work, Chesnay!
>>>>>      What do you think Hequn ?
>>>>>
>>>>>      Best,
>>>>>      Jincheng
>>>>>
>>>>>      Chesnay Schepler<[hidden email]>  <mailto:[hidden email]>
>   于2019年11月13日周三 下午5:17写道：
>>>>>>      It would be great if you could give me a day or 2 to check how
> easy it
>>>>>>      would be to bump the various jackson dependencies to eliminate a
> few
>>>>>>      security vulnerabilities.
>>>>>>
>>>>>>      On 09/11/2019 05:10, jincheng sun wrote:
>>>>>>>      Hi Flink devs,
>>>>>>>
>>>>>>>      It has been more than 2 months since the 1.8.2 released. So,
> What do
>>>>>      you
>>>>>>>      think about releasing Flink 1.8.3 soon?
>>>>>>>
>>>>>>>      We already have many important bug fixes in the release-1.8
> branch (29
>>>>>>>      resolved issues).
>>>>>>>
>>>>>>>      Most notable fixes are:
>>>>>>>
>>>>>>>      - FLINK-14010 Dispatcher & JobManagers don't give up leadership
> when AM
>>>>>>      is
>>>>>>>      shut down
>>>>>>>      - FLINK-14315 NPE with JobMaster.disconnectTaskManager
>>>>>>>      - FLINK-12848 Method equals() in RowTypeInfo should consider
>>>>>      fieldsNames
>>>>>>>      - FLINK-12342 Yarn Resource Manager Acquires Too Many Containers
>>>>>>>      - FLINK-14589 Redundant slot requests with the same
> AllocationID leads
>>>>>      to
>>>>>>>      inconsistent slot table
>>>>>>>
>>>>>>>      Furthermore, the following critical issues is in progress,
> maybe we can
>>>>>>>      wait for it if it is not too much effort.
>>>>>>>
>>>>>>>      - FLINK-13184 Starting a TaskExecutor blocks the
> YarnResourceManager's
>>>>>>      main
>>>>>>>      thread
>>>>>>>
>>>>>>>      Please let me know what you think?
>>>>>>>
>>>>>>>      Best,
>>>>>>>      Jincheng
>>>>>>>

[Chesnay Schepler-3]

One small modification: the flink-shaded upgrade does not have to be
part of the profile; since it is only intended for internal use anyway
(and thus has limited exposure) we can be pretty sure this doesn't break
anything.

On 15/11/2019 12:23, Chesnay Schepler wrote:

> Ufuk's summary is correct.
>
> There's a slight caveat in that we'd also have to bump the
> shade-plugin to 3.1.1 since it otherwise fails on jackson,
> but I have no concerns about this change.
>
> On 15/11/2019 12:19, Ufuk Celebi wrote:
>> The opt-in approach seems reasonable to me. +1 to include the
>> profiles in
>> 1.8 and 1.9 without changing the default versions (including the default
>> version of flink-shaded).
>>
>> As far as I can tell, the next steps would be:
>>
>> 1) Release flink-shaded with upgraded Jackson
>> 2a) Bump the flink-shaded version by default in master
>> 2b) Create opt-in profiles for 1.8 and 1.9 (the opt-in profiles
>> should also
>> cover the upgrade to the most recent flink-shaded version)
>>
>> @Chesnay: is this a correct summary?
>>
>> Note this would block the 1.8.3 release on step 1. As an upside, we
>> might
>> get some additional feedback until the 1.10 release with these
>> profiles in
>> case users make use of them with 1.8/1.9.
>>
>> – Ufuk
>>
>> On Fri, Nov 15, 2019 at 12:08 PM Chesnay Schepler <[hidden email]>
>> wrote:
>>> The opt-in approach would only be used for 1.8.3 / 1.9.2; on master
>>> (and
>>> thus starting from 1.10.0) it's not opt-in.
>>>
>>> I have only proposed it as an opt-in because a) we usually do not bump
>>> dependencies in bugfix releases and b) it's a short-term change that we
>>> aren't allowing to mature properly.
>>> In contrast, the 1.10 release is significantly further away, hence no
>>> opt-in.
>>>
>>> Hence, I'm not concerned about such kind of ugprades being more common
>>> in the future.
>>>
>>> We can certainly support every jackson version that fixes these
>>> vulnerabilities; individual modules can always use a different version
>>> (that hopefully includes the fixes).
>>> Ideally of course we'd only be using 1 version, but that may or may not
>>> be feasible.
>>>
>>> On 15/11/2019 04:07, Hequn Cheng wrote:
>>>> Hi Chesnay,
>>>>
>>>> Great to hear that jackson-2.10.1 works well on master. Really a good
>> job!
>>>> - Whether backport this change to 1.8/1.9
>>>> I had taken a quick look at the security vulnerabilities, some of them
>>>> seem can lead to high-security problems, thus from my point of view,
>>>> I'm in favor of adding the fix into 1.9/1.8. However, I would like to
>>>> trust your judgment as you are more professional at this problem.
>>>>
>>>> - How to port this change to 1.8/1.9
>>>> I think providing an opt-in upgrade is a good idea. Another question
>>>> here is whether do we plan to support multi jackson versions that have
>>>> eliminated the security vulnerabilities. If we only plan to support
>>>> 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
>>>> users can downgrade the flink version if meet problems using the new
>>>> version. Of course, we will try our best to make the new release out
>>>> of question.
>>>> Another concern of making it an opt-in upgrade is, it will make our
>>>> build unlikely convergence as more and more build options will be
>>>> added when we upgrade a commonly used lib like this one.
>>>>
>>>> What do you think?
>>>>
>>>> Best, Hequn
>>>>
>>>> On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
>>>> <mailto:[hidden email]>> wrote:
>>>>
>>>>      So here's the state of things:
>>>>
>>>>
>>>>      The master of flink-shaded now uses jackson 2.10.1, which
>>>>      eliminates a whole category of security vulnerabilities.
>>>>      The flink master works perfectly fine with that version; 1.9 will
>>>>      likely do so too and 1.8 would require a minor adjustment.
>>>>
>>>>      Hence, there may be value in first doing a flink-shaded
>>>> release so
>>>>      we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
>>>>
>>>>
>>>>      As for other jackson dependencies (coming from calcite, kafka,
>>>>      kinesis), I ran the unit and end-to-end tests of master yesterday
>>>>      will /all /jackson dependencies set to 2.10.1, and they passed. I
>>>>      will open a PR soon-ish for making this change on master.
>>>>
>>>>      The question now is whether we want to backport this change to
>>>>      1.8/1.9 .
>>>>      Some code paths /may /not be covered by our tests, and transitive
>>>>      jackson users /might /run into issues.
>>>>      Alternatively, we could set this up as an opt-in upgrade, by
>>>>      adding a separate profile that bumps the versions. This would
>>>>      present users/providers who are concerned about the
>>>>      vulnerabilities an easy workaround, at the risk of /some /things
>>>>      /maybe /not working.
>>>>
>>>>      On 14/11/2019 03:16, Hequn Cheng wrote:
>>>>>      Hi Chesnay, Jincheng
>>>>>
>>>>>      Sure, I think it's good to have these fixes.
>>>>>      Thanks a lot for providing the information about the security
>>>>>      vulnerabilities! @Chesnay
>>>>>
>>>>>      Best, Hequn
>>>>>
>>>>>      On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<
>> [hidden email]> <mailto:[hidden email]>
>>>>>      wrote:
>>>>>
>>>>>>      +1 for try to eliminate the security vulnerabilities. Great
>> thanks for
>>>>>>      doing this important work, Chesnay!
>>>>>>      What do you think Hequn ?
>>>>>>
>>>>>>      Best,
>>>>>>      Jincheng
>>>>>>
>>>>>>      Chesnay Schepler<[hidden email]>
>>>>>> <mailto:[hidden email]>
>>   于2019年11月13日周三 下午5:17写道：
>>>>>>>      It would be great if you could give me a day or 2 to check how
>> easy it
>>>>>>>      would be to bump the various jackson dependencies to
>>>>>>> eliminate a
>> few
>>>>>>>      security vulnerabilities.
>>>>>>>
>>>>>>>      On 09/11/2019 05:10, jincheng sun wrote:
>>>>>>>>      Hi Flink devs,
>>>>>>>>
>>>>>>>>      It has been more than 2 months since the 1.8.2 released. So,
>> What do
>>>>>>      you
>>>>>>>>      think about releasing Flink 1.8.3 soon?
>>>>>>>>
>>>>>>>>      We already have many important bug fixes in the release-1.8
>> branch (29
>>>>>>>>      resolved issues).
>>>>>>>>
>>>>>>>>      Most notable fixes are:
>>>>>>>>
>>>>>>>>      - FLINK-14010 Dispatcher & JobManagers don't give up
>>>>>>>> leadership
>> when AM
>>>>>>>      is
>>>>>>>>      shut down
>>>>>>>>      - FLINK-14315 NPE with JobMaster.disconnectTaskManager
>>>>>>>>      - FLINK-12848 Method equals() in RowTypeInfo should consider
>>>>>>      fieldsNames
>>>>>>>>      - FLINK-12342 Yarn Resource Manager Acquires Too Many
>>>>>>>> Containers
>>>>>>>>      - FLINK-14589 Redundant slot requests with the same
>> AllocationID leads
>>>>>>      to
>>>>>>>>      inconsistent slot table
>>>>>>>>
>>>>>>>>      Furthermore, the following critical issues is in progress,
>> maybe we can
>>>>>>>>      wait for it if it is not too much effort.
>>>>>>>>
>>>>>>>>      - FLINK-13184 Starting a TaskExecutor blocks the
>> YarnResourceManager's
>>>>>>>      main
>>>>>>>>      thread
>>>>>>>>
>>>>>>>>      Please let me know what you think?
>>>>>>>>
>>>>>>>>      Best,
>>>>>>>>      Jincheng
>>>>>>>>
>
>

[Hequn Cheng]

Hi,

@Chesnay Thanks a lot for the explanation. +1 to the opt-in approach for
1.8/1.9.
@Ufuk Thank you for the nice summary.

Looks good so far except that we need to postpone 1.8.3 a bit to first do a
flink-shaded release.
BTW, @chesnay when would we plan to release the flink-shaded with upgraded
Jackson?

Best, Hequn

On Fri, Nov 15, 2019 at 7:43 PM Chesnay Schepler <[hidden email]> wrote:

> One small modification: the flink-shaded upgrade does not have to be
> part of the profile; since it is only intended for internal use anyway
> (and thus has limited exposure) we can be pretty sure this doesn't break
> anything.
>
> On 15/11/2019 12:23, Chesnay Schepler wrote:
> > Ufuk's summary is correct.
> >
> > There's a slight caveat in that we'd also have to bump the
> > shade-plugin to 3.1.1 since it otherwise fails on jackson,
> > but I have no concerns about this change.
> >
> > On 15/11/2019 12:19, Ufuk Celebi wrote:
> >> The opt-in approach seems reasonable to me. +1 to include the
> >> profiles in
> >> 1.8 and 1.9 without changing the default versions (including the default
> >> version of flink-shaded).
> >>
> >> As far as I can tell, the next steps would be:
> >>
> >> 1) Release flink-shaded with upgraded Jackson
> >> 2a) Bump the flink-shaded version by default in master
> >> 2b) Create opt-in profiles for 1.8 and 1.9 (the opt-in profiles
> >> should also
> >> cover the upgrade to the most recent flink-shaded version)
> >>
> >> @Chesnay: is this a correct summary?
> >>
> >> Note this would block the 1.8.3 release on step 1. As an upside, we
> >> might
> >> get some additional feedback until the 1.10 release with these
> >> profiles in
> >> case users make use of them with 1.8/1.9.
> >>
> >> – Ufuk
> >>
> >> On Fri, Nov 15, 2019 at 12:08 PM Chesnay Schepler <[hidden email]>
> >> wrote:
> >>> The opt-in approach would only be used for 1.8.3 / 1.9.2; on master
> >>> (and
> >>> thus starting from 1.10.0) it's not opt-in.
> >>>
> >>> I have only proposed it as an opt-in because a) we usually do not bump
> >>> dependencies in bugfix releases and b) it's a short-term change that we
> >>> aren't allowing to mature properly.
> >>> In contrast, the 1.10 release is significantly further away, hence no
> >>> opt-in.
> >>>
> >>> Hence, I'm not concerned about such kind of ugprades being more common
> >>> in the future.
> >>>
> >>> We can certainly support every jackson version that fixes these
> >>> vulnerabilities; individual modules can always use a different version
> >>> (that hopefully includes the fixes).
> >>> Ideally of course we'd only be using 1 version, but that may or may not
> >>> be feasible.
> >>>
> >>> On 15/11/2019 04:07, Hequn Cheng wrote:
> >>>> Hi Chesnay,
> >>>>
> >>>> Great to hear that jackson-2.10.1 works well on master. Really a good
> >> job!
> >>>> - Whether backport this change to 1.8/1.9
> >>>> I had taken a quick look at the security vulnerabilities, some of them
> >>>> seem can lead to high-security problems, thus from my point of view,
> >>>> I'm in favor of adding the fix into 1.9/1.8. However, I would like to
> >>>> trust your judgment as you are more professional at this problem.
> >>>>
> >>>> - How to port this change to 1.8/1.9
> >>>> I think providing an opt-in upgrade is a good idea. Another question
> >>>> here is whether do we plan to support multi jackson versions that have
> >>>> eliminated the security vulnerabilities. If we only plan to support
> >>>> 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
> >>>> users can downgrade the flink version if meet problems using the new
> >>>> version. Of course, we will try our best to make the new release out
> >>>> of question.
> >>>> Another concern of making it an opt-in upgrade is, it will make our
> >>>> build unlikely convergence as more and more build options will be
> >>>> added when we upgrade a commonly used lib like this one.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Best, Hequn
> >>>>
> >>>> On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
> >>>> <mailto:[hidden email]>> wrote:
> >>>>
> >>>>      So here's the state of things:
> >>>>
> >>>>
> >>>>      The master of flink-shaded now uses jackson 2.10.1, which
> >>>>      eliminates a whole category of security vulnerabilities.
> >>>>      The flink master works perfectly fine with that version; 1.9 will
> >>>>      likely do so too and 1.8 would require a minor adjustment.
> >>>>
> >>>>      Hence, there may be value in first doing a flink-shaded
> >>>> release so
> >>>>      we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
> >>>>
> >>>>
> >>>>      As for other jackson dependencies (coming from calcite, kafka,
> >>>>      kinesis), I ran the unit and end-to-end tests of master yesterday
> >>>>      will /all /jackson dependencies set to 2.10.1, and they passed. I
> >>>>      will open a PR soon-ish for making this change on master.
> >>>>
> >>>>      The question now is whether we want to backport this change to
> >>>>      1.8/1.9 .
> >>>>      Some code paths /may /not be covered by our tests, and transitive
> >>>>      jackson users /might /run into issues.
> >>>>      Alternatively, we could set this up as an opt-in upgrade, by
> >>>>      adding a separate profile that bumps the versions. This would
> >>>>      present users/providers who are concerned about the
> >>>>      vulnerabilities an easy workaround, at the risk of /some /things
> >>>>      /maybe /not working.
> >>>>
> >>>>      On 14/11/2019 03:16, Hequn Cheng wrote:
> >>>>>      Hi Chesnay, Jincheng
> >>>>>
> >>>>>      Sure, I think it's good to have these fixes.
> >>>>>      Thanks a lot for providing the information about the security
> >>>>>      vulnerabilities! @Chesnay
> >>>>>
> >>>>>      Best, Hequn
> >>>>>
> >>>>>      On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<
> >> [hidden email]> <mailto:[hidden email]>
> >>>>>      wrote:
> >>>>>
> >>>>>>      +1 for try to eliminate the security vulnerabilities. Great
> >> thanks for
> >>>>>>      doing this important work, Chesnay!
> >>>>>>      What do you think Hequn ?
> >>>>>>
> >>>>>>      Best,
> >>>>>>      Jincheng
> >>>>>>
> >>>>>>      Chesnay Schepler<[hidden email]>
> >>>>>> <mailto:[hidden email]>
> >>   于2019年11月13日周三 下午5:17写道：
> >>>>>>>      It would be great if you could give me a day or 2 to check how
> >> easy it
> >>>>>>>      would be to bump the various jackson dependencies to
> >>>>>>> eliminate a
> >> few
> >>>>>>>      security vulnerabilities.
> >>>>>>>
> >>>>>>>      On 09/11/2019 05:10, jincheng sun wrote:
> >>>>>>>>      Hi Flink devs,
> >>>>>>>>
> >>>>>>>>      It has been more than 2 months since the 1.8.2 released. So,
> >> What do
> >>>>>>      you
> >>>>>>>>      think about releasing Flink 1.8.3 soon?
> >>>>>>>>
> >>>>>>>>      We already have many important bug fixes in the release-1.8
> >> branch (29
> >>>>>>>>      resolved issues).
> >>>>>>>>
> >>>>>>>>      Most notable fixes are:
> >>>>>>>>
> >>>>>>>>      - FLINK-14010 Dispatcher & JobManagers don't give up
> >>>>>>>> leadership
> >> when AM
> >>>>>>>      is
> >>>>>>>>      shut down
> >>>>>>>>      - FLINK-14315 NPE with JobMaster.disconnectTaskManager
> >>>>>>>>      - FLINK-12848 Method equals() in RowTypeInfo should consider
> >>>>>>      fieldsNames
> >>>>>>>>      - FLINK-12342 Yarn Resource Manager Acquires Too Many
> >>>>>>>> Containers
> >>>>>>>>      - FLINK-14589 Redundant slot requests with the same
> >> AllocationID leads
> >>>>>>      to
> >>>>>>>>      inconsistent slot table
> >>>>>>>>
> >>>>>>>>      Furthermore, the following critical issues is in progress,
> >> maybe we can
> >>>>>>>>      wait for it if it is not too much effort.
> >>>>>>>>
> >>>>>>>>      - FLINK-13184 Starting a TaskExecutor blocks the
> >> YarnResourceManager's
> >>>>>>>      main
> >>>>>>>>      thread
> >>>>>>>>
> >>>>>>>>      Please let me know what you think?
> >>>>>>>>
> >>>>>>>>      Best,
> >>>>>>>>      Jincheng
> >>>>>>>>
> >
> >
>
>

[Chesnay Schepler-3]

I'm not aware of any more planned changes to flink-shaded; so we could
start the release right away.

On 15/11/2019 13:44, Hequn Cheng wrote:

> Hi,
>
> @Chesnay Thanks a lot for the explanation. +1 to the opt-in approach for
> 1.8/1.9.
> @Ufuk Thank you for the nice summary.
>
> Looks good so far except that we need to postpone 1.8.3 a bit to first do a
> flink-shaded release.
> BTW, @chesnay when would we plan to release the flink-shaded with upgraded
> Jackson?
>
> Best, Hequn
>
> On Fri, Nov 15, 2019 at 7:43 PM Chesnay Schepler <[hidden email]> wrote:
>
>> One small modification: the flink-shaded upgrade does not have to be
>> part of the profile; since it is only intended for internal use anyway
>> (and thus has limited exposure) we can be pretty sure this doesn't break
>> anything.
>>
>> On 15/11/2019 12:23, Chesnay Schepler wrote:
>>> Ufuk's summary is correct.
>>>
>>> There's a slight caveat in that we'd also have to bump the
>>> shade-plugin to 3.1.1 since it otherwise fails on jackson,
>>> but I have no concerns about this change.
>>>
>>> On 15/11/2019 12:19, Ufuk Celebi wrote:
>>>> The opt-in approach seems reasonable to me. +1 to include the
>>>> profiles in
>>>> 1.8 and 1.9 without changing the default versions (including the default
>>>> version of flink-shaded).
>>>>
>>>> As far as I can tell, the next steps would be:
>>>>
>>>> 1) Release flink-shaded with upgraded Jackson
>>>> 2a) Bump the flink-shaded version by default in master
>>>> 2b) Create opt-in profiles for 1.8 and 1.9 (the opt-in profiles
>>>> should also
>>>> cover the upgrade to the most recent flink-shaded version)
>>>>
>>>> @Chesnay: is this a correct summary?
>>>>
>>>> Note this would block the 1.8.3 release on step 1. As an upside, we
>>>> might
>>>> get some additional feedback until the 1.10 release with these
>>>> profiles in
>>>> case users make use of them with 1.8/1.9.
>>>>
>>>> – Ufuk
>>>>
>>>> On Fri, Nov 15, 2019 at 12:08 PM Chesnay Schepler <[hidden email]>
>>>> wrote:
>>>>> The opt-in approach would only be used for 1.8.3 / 1.9.2; on master
>>>>> (and
>>>>> thus starting from 1.10.0) it's not opt-in.
>>>>>
>>>>> I have only proposed it as an opt-in because a) we usually do not bump
>>>>> dependencies in bugfix releases and b) it's a short-term change that we
>>>>> aren't allowing to mature properly.
>>>>> In contrast, the 1.10 release is significantly further away, hence no
>>>>> opt-in.
>>>>>
>>>>> Hence, I'm not concerned about such kind of ugprades being more common
>>>>> in the future.
>>>>>
>>>>> We can certainly support every jackson version that fixes these
>>>>> vulnerabilities; individual modules can always use a different version
>>>>> (that hopefully includes the fixes).
>>>>> Ideally of course we'd only be using 1 version, but that may or may not
>>>>> be feasible.
>>>>>
>>>>> On 15/11/2019 04:07, Hequn Cheng wrote:
>>>>>> Hi Chesnay,
>>>>>>
>>>>>> Great to hear that jackson-2.10.1 works well on master. Really a good
>>>> job!
>>>>>> - Whether backport this change to 1.8/1.9
>>>>>> I had taken a quick look at the security vulnerabilities, some of them
>>>>>> seem can lead to high-security problems, thus from my point of view,
>>>>>> I'm in favor of adding the fix into 1.9/1.8. However, I would like to
>>>>>> trust your judgment as you are more professional at this problem.
>>>>>>
>>>>>> - How to port this change to 1.8/1.9
>>>>>> I think providing an opt-in upgrade is a good idea. Another question
>>>>>> here is whether do we plan to support multi jackson versions that have
>>>>>> eliminated the security vulnerabilities. If we only plan to support
>>>>>> 2.10.1, I would like to make it a non-opt-in upgrade. As an option,
>>>>>> users can downgrade the flink version if meet problems using the new
>>>>>> version. Of course, we will try our best to make the new release out
>>>>>> of question.
>>>>>> Another concern of making it an opt-in upgrade is, it will make our
>>>>>> build unlikely convergence as more and more build options will be
>>>>>> added when we upgrade a commonly used lib like this one.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Best, Hequn
>>>>>>
>>>>>> On Thu, Nov 14, 2019 at 6:00 PM Chesnay Schepler <[hidden email]
>>>>>> <mailto:[hidden email]>> wrote:
>>>>>>
>>>>>>       So here's the state of things:
>>>>>>
>>>>>>
>>>>>>       The master of flink-shaded now uses jackson 2.10.1, which
>>>>>>       eliminates a whole category of security vulnerabilities.
>>>>>>       The flink master works perfectly fine with that version; 1.9 will
>>>>>>       likely do so too and 1.8 would require a minor adjustment.
>>>>>>
>>>>>>       Hence, there may be value in first doing a flink-shaded
>>>>>> release so
>>>>>>       we can eliminate these vulnerabilities in 1.8.3 and 1.9.2 .
>>>>>>
>>>>>>
>>>>>>       As for other jackson dependencies (coming from calcite, kafka,
>>>>>>       kinesis), I ran the unit and end-to-end tests of master yesterday
>>>>>>       will /all /jackson dependencies set to 2.10.1, and they passed. I
>>>>>>       will open a PR soon-ish for making this change on master.
>>>>>>
>>>>>>       The question now is whether we want to backport this change to
>>>>>>       1.8/1.9 .
>>>>>>       Some code paths /may /not be covered by our tests, and transitive
>>>>>>       jackson users /might /run into issues.
>>>>>>       Alternatively, we could set this up as an opt-in upgrade, by
>>>>>>       adding a separate profile that bumps the versions. This would
>>>>>>       present users/providers who are concerned about the
>>>>>>       vulnerabilities an easy workaround, at the risk of /some /things
>>>>>>       /maybe /not working.
>>>>>>
>>>>>>       On 14/11/2019 03:16, Hequn Cheng wrote:
>>>>>>>       Hi Chesnay, Jincheng
>>>>>>>
>>>>>>>       Sure, I think it's good to have these fixes.
>>>>>>>       Thanks a lot for providing the information about the security
>>>>>>>       vulnerabilities! @Chesnay
>>>>>>>
>>>>>>>       Best, Hequn
>>>>>>>
>>>>>>>       On Thu, Nov 14, 2019 at 10:07 AM jincheng sun<
>>>> [hidden email]> <mailto:[hidden email]>
>>>>>>>       wrote:
>>>>>>>
>>>>>>>>       +1 for try to eliminate the security vulnerabilities. Great
>>>> thanks for
>>>>>>>>       doing this important work, Chesnay!
>>>>>>>>       What do you think Hequn ?
>>>>>>>>
>>>>>>>>       Best,
>>>>>>>>       Jincheng
>>>>>>>>
>>>>>>>>       Chesnay Schepler<[hidden email]>
>>>>>>>> <mailto:[hidden email]>
>>>>    于2019年11月13日周三 下午5:17写道：
>>>>>>>>>       It would be great if you could give me a day or 2 to check how
>>>> easy it
>>>>>>>>>       would be to bump the various jackson dependencies to
>>>>>>>>> eliminate a
>>>> few
>>>>>>>>>       security vulnerabilities.
>>>>>>>>>
>>>>>>>>>       On 09/11/2019 05:10, jincheng sun wrote:
>>>>>>>>>>       Hi Flink devs,
>>>>>>>>>>
>>>>>>>>>>       It has been more than 2 months since the 1.8.2 released. So,
>>>> What do
>>>>>>>>       you
>>>>>>>>>>       think about releasing Flink 1.8.3 soon?
>>>>>>>>>>
>>>>>>>>>>       We already have many important bug fixes in the release-1.8
>>>> branch (29
>>>>>>>>>>       resolved issues).
>>>>>>>>>>
>>>>>>>>>>       Most notable fixes are:
>>>>>>>>>>
>>>>>>>>>>       - FLINK-14010 Dispatcher & JobManagers don't give up
>>>>>>>>>> leadership
>>>> when AM
>>>>>>>>>       is
>>>>>>>>>>       shut down
>>>>>>>>>>       - FLINK-14315 NPE with JobMaster.disconnectTaskManager
>>>>>>>>>>       - FLINK-12848 Method equals() in RowTypeInfo should consider
>>>>>>>>       fieldsNames
>>>>>>>>>>       - FLINK-12342 Yarn Resource Manager Acquires Too Many
>>>>>>>>>> Containers
>>>>>>>>>>       - FLINK-14589 Redundant slot requests with the same
>>>> AllocationID leads
>>>>>>>>       to
>>>>>>>>>>       inconsistent slot table
>>>>>>>>>>
>>>>>>>>>>       Furthermore, the following critical issues is in progress,
>>>> maybe we can
>>>>>>>>>>       wait for it if it is not too much effort.
>>>>>>>>>>
>>>>>>>>>>       - FLINK-13184 Starting a TaskExecutor blocks the
>>>> YarnResourceManager's
>>>>>>>>>       main
>>>>>>>>>>       thread
>>>>>>>>>>
>>>>>>>>>>       Please let me know what you think?
>>>>>>>>>>
>>>>>>>>>>       Best,
>>>>>>>>>>       Jincheng
>>>>>>>>>>
>>>
>>