[DISCUSS] Releasing Flink 1.4.1

[Tzu-Li (Gordon) Tai]

Hi all,

I would like to bring up the discussion of releasing Flink 1.4.1.
We’ve collected a few issues over the past few weeks, some of which is quite
critical.

Here’s a list of issues, and their current status, that I’m aware of and IMO
should be included:
- [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
(pending PR)
- [FLINK-8421] HeapInternalTimerService does not reconfigure serializers on
restore (assigned, no PR)
- [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
- [FLINK-8419] No metrics registered for dynamically discovered Kafka
partitions (assigned, no PR)
- [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
- [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)

Also FYI, here [1] is a list of tickets that were marked as a blocker for
1.4.1 on JIRA.
[1]
https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC

If you think something is missing and should be added for 1.4.1, could you
please add them to the JIRA and let others know here?

Cheers,
Gordon



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[Maciek Próchniak]

Hi Gordon,

do you think I'd be possible to include
https://issues.apache.org/jira/browse/FLINK-7949?

thanks,
maciek

On 15/01/2018 11:49, Tzu-Li (Gordon) Tai wrote:

> Hi all,
>
> I would like to bring up the discussion of releasing Flink 1.4.1.
> We’ve collected a few issues over the past few weeks, some of which is quite
> critical.
>
> Here’s a list of issues, and their current status, that I’m aware of and IMO
> should be included:
> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
> (pending PR)
> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers on
> restore (assigned, no PR)
> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
> partitions (assigned, no PR)
> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>
> Also FYI, here [1] is a list of tickets that were marked as a blocker for
> 1.4.1 on JIRA.
> [1]
> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>
> If you think something is missing and should be added for 1.4.1, could you
> please add them to the JIRA and let others know here?
>
> Cheers,
> Gordon
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

[Till Rohrmann]

Thanks for starting the discussion Gordon. I think we already have some
good improvements in the 1.4.1 branch. Thus, +1 for releasing 1.4.1 soonish.

@Maciek, I'll backport FLINK-7949 to the 1.4.1 release branch.

Cheers,
Till

On Mon, Jan 15, 2018 at 11:56 AM, Maciek Próchniak <[hidden email]> wrote:

> Hi Gordon,
>
> do you think I'd be possible to include https://issues.apache.org/jira
> /browse/FLINK-7949?
>
> thanks,
> maciek
>
>
> On 15/01/2018 11:49, Tzu-Li (Gordon) Tai wrote:
>
>> Hi all,
>>
>> I would like to bring up the discussion of releasing Flink 1.4.1.
>> We’ve collected a few issues over the past few weeks, some of which is
>> quite
>> critical.
>>
>> Here’s a list of issues, and their current status, that I’m aware of and
>> IMO
>> should be included:
>> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
>> (pending PR)
>> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers
>> on
>> restore (assigned, no PR)
>> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
>> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
>> partitions (assigned, no PR)
>> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
>> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>>
>> Also FYI, here [1] is a list of tickets that were marked as a blocker for
>> 1.4.1 on JIRA.
>> [1]
>> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&
>> jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%
>> 20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>>
>> If you think something is missing and should be added for 1.4.1, could you
>> please add them to the JIRA and let others know here?
>>
>> Cheers,
>> Gordon
>>
>>
>>
>> --
>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.
>> com/
>>
>>
>

[Maciek Próchniak]

thanks Till,

maciek



On 15/01/2018 12:35, Till Rohrmann wrote:

> Thanks for starting the discussion Gordon. I think we already have some
> good improvements in the 1.4.1 branch. Thus, +1 for releasing 1.4.1 soonish.
>
> @Maciek, I'll backport FLINK-7949 to the 1.4.1 release branch.
>
> Cheers,
> Till
>
> On Mon, Jan 15, 2018 at 11:56 AM, Maciek Próchniak <[hidden email]> wrote:
>
>> Hi Gordon,
>>
>> do you think I'd be possible to include https://issues.apache.org/jira
>> /browse/FLINK-7949?
>>
>> thanks,
>> maciek
>>
>>
>> On 15/01/2018 11:49, Tzu-Li (Gordon) Tai wrote:
>>
>>> Hi all,
>>>
>>> I would like to bring up the discussion of releasing Flink 1.4.1.
>>> We’ve collected a few issues over the past few weeks, some of which is
>>> quite
>>> critical.
>>>
>>> Here’s a list of issues, and their current status, that I’m aware of and
>>> IMO
>>> should be included:
>>> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
>>> (pending PR)
>>> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers
>>> on
>>> restore (assigned, no PR)
>>> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
>>> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
>>> partitions (assigned, no PR)
>>> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
>>> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>>>
>>> Also FYI, here [1] is a list of tickets that were marked as a blocker for
>>> 1.4.1 on JIRA.
>>> [1]
>>> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&
>>> jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%
>>> 20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>>>
>>> If you think something is missing and should be added for 1.4.1, could you
>>> please add them to the JIRA and let others know here?
>>>
>>> Cheers,
>>> Gordon
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.
>>> com/
>>>
>>>

[vishal]

Hello folks,  
 
    https://issues.apache.org/jira/browse/FLINK-8226 . We would want a
schedule on the 1.4.1 version. Without checkpointing and state, CEP is in
effect not production ready IMHO. Is there any time line on 1.4.1 ? The
earlier the better.

Thank you and Regards,

Vishal



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[Kyle Hamlin]

Can we add this to 1.4.1 release:

https://issues.apache.org/jira/browse/FLINK-8406

On Mon, Jan 15, 2018 at 9:42 AM Maciek Próchniak <[hidden email]> wrote:

> thanks Till,
>
> maciek
>
>
>
> On 15/01/2018 12:35, Till Rohrmann wrote:
> > Thanks for starting the discussion Gordon. I think we already have some
> > good improvements in the 1.4.1 branch. Thus, +1 for releasing 1.4.1
> soonish.
> >
> > @Maciek, I'll backport FLINK-7949 to the 1.4.1 release branch.
> >
> > Cheers,
> > Till
> >
> > On Mon, Jan 15, 2018 at 11:56 AM, Maciek Próchniak <[hidden email]> wrote:
> >
> >> Hi Gordon,
> >>
> >> do you think I'd be possible to include https://issues.apache.org/jira
> >> /browse/FLINK-7949?
> >>
> >> thanks,
> >> maciek
> >>
> >>
> >> On 15/01/2018 11:49, Tzu-Li (Gordon) Tai wrote:
> >>
> >>> Hi all,
> >>>
> >>> I would like to bring up the discussion of releasing Flink 1.4.1.
> >>> We’ve collected a few issues over the past few weeks, some of which is
> >>> quite
> >>> critical.
> >>>
> >>> Here’s a list of issues, and their current status, that I’m aware of
> and
> >>> IMO
> >>> should be included:
> >>> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
> >>> (pending PR)
> >>> - [FLINK-8421] HeapInternalTimerService does not reconfigure
> serializers
> >>> on
> >>> restore (assigned, no PR)
> >>> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending
> PR)
> >>> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
> >>> partitions (assigned, no PR)
> >>> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
> >>> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
> >>>
> >>> Also FYI, here [1] is a list of tickets that were marked as a blocker
> for
> >>> 1.4.1 on JIRA.
> >>> [1]
> >>> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&
> >>> jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%
> >>> 20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
> >>>
> >>> If you think something is missing and should be added for 1.4.1, could
> you
> >>> please add them to the JIRA and let others know here?
> >>>
> >>> Cheers,
> >>> Gordon
> >>>
> >>>
> >>>
> >>> --
> >>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.
> >>> com/
> >>>
> >>>
>
>

[Shuyi Chen]

Can we add this to 1.4.1 release? The flink-yarn-kerberos deployment is not
working in 1.4.0.

https://issues.apache.org/jira/browse/FLINK-8275

Thanks a lot.

On Mon, Jan 15, 2018 at 7:53 AM, vishal <[hidden email]> wrote:

> Hello folks,
>
>     https://issues.apache.org/jira/browse/FLINK-8226 . We would want a
> schedule on the 1.4.1 version. Without checkpointing and state, CEP is in
> effect not production ready IMHO. Is there any time line on 1.4.1 ? The
> earlier the better.
>
> Thank you and Regards,
>
> Vishal
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

--
"So you have to trust that the dots will somehow connect in your future."

[Chesnay Schepler-3]

Taken from another ML thread
<http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/save-points-through-REST-API-not-supported-td17066.html>
some users would appreciate it if we included the savepoint trigger
handler i implemented here
<https://github.com/zentol/flink/tree/release-1.4-custom>. This one is a
slightly modified version of our cancel-with-savepoint handler.

On 15.01.2018 16:59, Kyle Hamlin wrote:

> Can we add this to 1.4.1 release:
>
> https://issues.apache.org/jira/browse/FLINK-8406
>
> On Mon, Jan 15, 2018 at 9:42 AM Maciek Próchniak <[hidden email]> wrote:
>
>> thanks Till,
>>
>> maciek
>>
>>
>>
>> On 15/01/2018 12:35, Till Rohrmann wrote:
>>> Thanks for starting the discussion Gordon. I think we already have some
>>> good improvements in the 1.4.1 branch. Thus, +1 for releasing 1.4.1
>> soonish.
>>> @Maciek, I'll backport FLINK-7949 to the 1.4.1 release branch.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Mon, Jan 15, 2018 at 11:56 AM, Maciek Próchniak <[hidden email]> wrote:
>>>
>>>> Hi Gordon,
>>>>
>>>> do you think I'd be possible to include https://issues.apache.org/jira
>>>> /browse/FLINK-7949?
>>>>
>>>> thanks,
>>>> maciek
>>>>
>>>>
>>>> On 15/01/2018 11:49, Tzu-Li (Gordon) Tai wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I would like to bring up the discussion of releasing Flink 1.4.1.
>>>>> We’ve collected a few issues over the past few weeks, some of which is
>>>>> quite
>>>>> critical.
>>>>>
>>>>> Here’s a list of issues, and their current status, that I’m aware of
>> and
>>>>> IMO
>>>>> should be included:
>>>>> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
>>>>> (pending PR)
>>>>> - [FLINK-8421] HeapInternalTimerService does not reconfigure
>> serializers
>>>>> on
>>>>> restore (assigned, no PR)
>>>>> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending
>> PR)
>>>>> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
>>>>> partitions (assigned, no PR)
>>>>> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
>>>>> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>>>>>
>>>>> Also FYI, here [1] is a list of tickets that were marked as a blocker
>> for
>>>>> 1.4.1 on JIRA.
>>>>> [1]
>>>>> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&
>>>>> jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%
>>>>> 20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>>>>>
>>>>> If you think something is missing and should be added for 1.4.1, could
>> you
>>>>> please add them to the JIRA and let others know here?
>>>>>
>>>>> Cheers,
>>>>> Gordon
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.
>>>>> com/
>>>>>
>>>>>
>>

[shashank734]

Please include these two bugs :

https://issues.apache.org/jira/browse/FLINK-7756

https://issues.apache.org/jira/browse/FLINK-7760

Unable to restore from savepoint with CEP.



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[jincheng sun]

Thanks for starting the discussion Gordon.

I think it is better to add the commits of
`https://issues.apache.org/jira/browse/FLINK-8355`
and
 `https://issues.apache.org/jira/browse/FLINK-8325` for table API & SQL.

Because these two issues fixed commonly used `count (*)` bug and improve of
non-grouped aggregate for DataSet.
What do you think @Fabian @Timo ?

Thanks, Jincheng

2018-01-16 23:03 GMT+08:00 shashank734 <[hidden email]>:

> Please include these two bugs :
>
> https://issues.apache.org/jira/browse/FLINK-7756
>
> https://issues.apache.org/jira/browse/FLINK-7760
>
> Unable to restore from savepoint with CEP.
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

[Timo Walther-2]

@Jincheng: Yes, I think we should include the two Table API PRs.

Am 1/16/18 um 5:28 PM schrieb jincheng sun:

> Thanks for starting the discussion Gordon.
>
> I think it is better to add the commits of
> `https://issues.apache.org/jira/browse/FLINK-8355`
> and
>   `https://issues.apache.org/jira/browse/FLINK-8325` for table API & SQL.
>
> Because these two issues fixed commonly used `count (*)` bug and improve of
> non-grouped aggregate for DataSet.
> What do you think @Fabian @Timo ?
>
> Thanks, Jincheng
>
> 2018-01-16 23:03 GMT+08:00 shashank734 <[hidden email]>:
>
>> Please include these two bugs :
>>
>> https://issues.apache.org/jira/browse/FLINK-7756
>>
>> https://issues.apache.org/jira/browse/FLINK-7760
>>
>> Unable to restore from savepoint with CEP.
>>
>>
>>
>> --
>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>>

[Steven Wu]

can we add these two? They can make fine-grained recovery more consistent.
https://issues.apache.org/jira/browse/FLINK-8042
https://issues.apache.org/jira/browse/FLINK-8043



On Tue, Jan 16, 2018 at 8:35 AM, Timo Walther <[hidden email]> wrote:

> @Jincheng: Yes, I think we should include the two Table API PRs.
>
> Am 1/16/18 um 5:28 PM schrieb jincheng sun:
>
> Thanks for starting the discussion Gordon.
>>
>> I think it is better to add the commits of
>> `https://issues.apache.org/jira/browse/FLINK-8355`
>> <https://issues.apache.org/jira/browse/FLINK-8355>
>> and
>>   `https://issues.apache.org/jira/browse/FLINK-8325`
>> <https://issues.apache.org/jira/browse/FLINK-8325> for table API & SQL.
>>
>> Because these two issues fixed commonly used `count (*)` bug and improve
>> of
>> non-grouped aggregate for DataSet.
>> What do you think @Fabian @Timo ?
>>
>> Thanks, Jincheng
>>
>> 2018-01-16 23:03 GMT+08:00 shashank734 <[hidden email]>:
>>
>> Please include these two bugs :
>>>
>>> https://issues.apache.org/jira/browse/FLINK-7756
>>>
>>> https://issues.apache.org/jira/browse/FLINK-7760
>>>
>>> Unable to restore from savepoint with CEP.
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.
>>> com/
>>>
>>>
>

[Chesnay Schepler-3]

Here's a list of all issues raised in this thread and their respective
state:

Unassigned:

[FLINK-7756] RocksDB state backend Checkpointing (Async and Incremental) is not working with CEP.
[FLINK-7760] Restore failing from external checkpointing metadata.
[FLINK-8042] retry individual failover-strategy for some time first before reverting to full job restart

PR open:

[FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
[FLINK-8318] Shade all Elasticsearch connector dependencies
[FLINK-8409] KafkaConsumerThread potential NPE
[FLINK-8419] No metrics registered for dynamically discovered Kafka partitions
[FLINK-8421] HeapInternalTimerService does not reconfigure serializers on restore

Resolved:

[FLINK-7949] AsyncWaitOperator is not restarting when queue is full
[FLINK-8226] Dangling reference generated after NFA clean up timed out SharedBufferEntry
[FLINK-8295] Cassandra connector has incorrect Netty shading
[FLINK-8325] Add COUNT AGG support constant parameter, i.e. COUNT(*), COUNT(1)
[FLINK-8355] DataSet Should not union a NULL row for AGG without GROUP BY clause.
[FLINK-8406] BucketingSink does not detect hadoop file systems
[FLINK-8043] change fullRestarts (for fine grained recovery) from guage to counter

On 15.01.2018 11:49, Tzu-Li (Gordon) Tai wrote:

> Hi all,
>
> I would like to bring up the discussion of releasing Flink 1.4.1.
> We’ve collected a few issues over the past few weeks, some of which is quite
> critical.
>
> Here’s a list of issues, and their current status, that I’m aware of and IMO
> should be included:
> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
> (pending PR)
> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers on
> restore (assigned, no PR)
> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
> partitions (assigned, no PR)
> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>
> Also FYI, here [1] is a list of tickets that were marked as a blocker for
> 1.4.1 on JIRA.
> [1]
> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>
> If you think something is missing and should be added for 1.4.1, could you
> please add them to the JIRA and let others know here?
>
> Cheers,
> Gordon
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>

[Tzu-Li (Gordon) Tai]

Thanks a lot for the wrap-up Chesnay.

For open issues with a PR already:

[FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path 
[FLINK-8318] Shade all Elasticsearch connector dependencies 
I am currently reviewing these two.

[FLINK-8409] KafkaConsumerThread potential NPE 
[FLINK-8419] No metrics registered for dynamically discovered Kafka partitions 
These have been reviewed and is mergeable.

[FLINK-8421] HeapInternalTimerService does not reconfigure serializers on restore 

This is still waiting for a review, but I think Aljoscha will start reviewing it soon.


For unassigned PRs, I recommend that we start kicking off a 1.4.1 RC by the end of this week with what we have, and leave those for a fast 1.4.2 release afterwards.

The reason is we’ve already collected quite a few very critical fixes for 1.4.1 that users have reported and are waiting on, so blocking the 1.4.1 release any longer on still un-assigned issues might not be a good idea.

I would also be happy to be the release manager for the 1.4.1 release.

Cheers,
Gordon

On 31 January 2018 at 11:55:06 AM, Chesnay Schepler ([hidden email]) wrote:

Here's a list of all issues raised in this thread and their respective  
state:  

Unassigned:  

[FLINK-7756] RocksDB state backend Checkpointing (Async and Incremental) is not working with CEP.  
[FLINK-7760] Restore failing from external checkpointing metadata.  
[FLINK-8042] retry individual failover-strategy for some time first before reverting to full job restart  

PR open:  

[FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path  
[FLINK-8318] Shade all Elasticsearch connector dependencies  
[FLINK-8409] KafkaConsumerThread potential NPE  
[FLINK-8419] No metrics registered for dynamically discovered Kafka partitions  
[FLINK-8421] HeapInternalTimerService does not reconfigure serializers on restore  

Resolved:  

[FLINK-7949] AsyncWaitOperator is not restarting when queue is full  
[FLINK-8226] Dangling reference generated after NFA clean up timed out SharedBufferEntry  
[FLINK-8295] Cassandra connector has incorrect Netty shading  
[FLINK-8325] Add COUNT AGG support constant parameter, i.e. COUNT(*), COUNT(1)  
[FLINK-8355] DataSet Should not union a NULL row for AGG without GROUP BY clause.  
[FLINK-8406] BucketingSink does not detect hadoop file systems  
[FLINK-8043] change fullRestarts (for fine grained recovery) from guage to counter  

On 15.01.2018 11:49, Tzu-Li (Gordon) Tai wrote:  

> Hi all,  
>  
> I would like to bring up the discussion of releasing Flink 1.4.1.  
> We’ve collected a few issues over the past few weeks, some of which is quite  
> critical.  
>  
> Here’s a list of issues, and their current status, that I’m aware of and IMO  
> should be included:  
> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path  
> (pending PR)  
> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers on  
> restore (assigned, no PR)  
> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)  
> - [FLINK-8419] No metrics registered for dynamically discovered Kafka  
> partitions (assigned, no PR)  
> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)  
> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)  
>  
> Also FYI, here [1] is a list of tickets that were marked as a blocker for  
> 1.4.1 on JIRA.  
> [1]  
> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC 
>  
> If you think something is missing and should be added for 1.4.1, could you  
> please add them to the JIRA and let others know here?  
>  
> Cheers,  
> Gordon  
>  
>  
>  
> --  
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ 
>  

[Tzu-Li (Gordon) Tai]

Status update:

We're waiting for the PR for [FLINK-8318] Shade all Elasticsearch connector dependencies to be reviewed and merged [1].
Once that happens, I’ll kick off RC1 for Flink 1.4.1.

Cheers,
Gordon

[1] https://github.com/apache/flink/pull/5426

On 31 January 2018 at 12:15:09 PM, Tzu-Li (Gordon) Tai ([hidden email]) wrote:

Thanks a lot for the wrap-up Chesnay.

For open issues with a PR already:

[FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path 
[FLINK-8318] Shade all Elasticsearch connector dependencies 
I am currently reviewing these two.

[FLINK-8409] KafkaConsumerThread potential NPE 
[FLINK-8419] No metrics registered for dynamically discovered Kafka partitions 
These have been reviewed and is mergeable.

[FLINK-8421] HeapInternalTimerService does not reconfigure serializers on restore 

This is still waiting for a review, but I think Aljoscha will start reviewing it soon.


For unassigned PRs, I recommend that we start kicking off a 1.4.1 RC by the end of this week with what we have, and leave those for a fast 1.4.2 release afterwards.

The reason is we’ve already collected quite a few very critical fixes for 1.4.1 that users have reported and are waiting on, so blocking the 1.4.1 release any longer on still un-assigned issues might not be a good idea.

I would also be happy to be the release manager for the 1.4.1 release.

Cheers,
Gordon

On 31 January 2018 at 11:55:06 AM, Chesnay Schepler ([hidden email]) wrote:

Here's a list of all issues raised in this thread and their respective
state:

Unassigned:

[FLINK-7756] RocksDB state backend Checkpointing (Async and Incremental) is not working with CEP.
[FLINK-7760] Restore failing from external checkpointing metadata.
[FLINK-8042] retry individual failover-strategy for some time first before reverting to full job restart

PR open:

[FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
[FLINK-8318] Shade all Elasticsearch connector dependencies
[FLINK-8409] KafkaConsumerThread potential NPE
[FLINK-8419] No metrics registered for dynamically discovered Kafka partitions
[FLINK-8421] HeapInternalTimerService does not reconfigure serializers on restore

Resolved:

[FLINK-7949] AsyncWaitOperator is not restarting when queue is full
[FLINK-8226] Dangling reference generated after NFA clean up timed out SharedBufferEntry
[FLINK-8295] Cassandra connector has incorrect Netty shading
[FLINK-8325] Add COUNT AGG support constant parameter, i.e. COUNT(*), COUNT(1)
[FLINK-8355] DataSet Should not union a NULL row for AGG without GROUP BY clause.
[FLINK-8406] BucketingSink does not detect hadoop file systems
[FLINK-8043] change fullRestarts (for fine grained recovery) from guage to counter

On 15.01.2018 11:49, Tzu-Li (Gordon) Tai wrote:

> Hi all,
>
> I would like to bring up the discussion of releasing Flink 1.4.1.
> We’ve collected a few issues over the past few weeks, some of which is quite
> critical.
>
> Here’s a list of issues, and their current status, that I’m aware of and IMO
> should be included:
> - [FLINK-8275] Flink YARN TMs using wrong local Kerberos keytab path
> (pending PR)
> - [FLINK-8421] HeapInternalTimerService does not reconfigure serializers on
> restore (assigned, no PR)
> - [FLINK-8318] Shade all Elasticsearch connector dependencies (pending PR)
> - [FLINK-8419] No metrics registered for dynamically discovered Kafka
> partitions (assigned, no PR)
> - [FLINK-8409] KafkaConsumerThread potential NPE (assigned, no PR)
> - [FLINK-8295] Cassandra connector has incorrect Netty shading (merged)
>
> Also FYI, here [1] is a list of tickets that were marked as a blocker for
> 1.4.1 on JIRA.
> [1]
> https://issues.apache.org/jira/browse/FLINK-8263?filter=-1&jql=project%20%3D%20FLINK%20AND%20priority%20%3D%20Blocker%20AND%20fixVersion%20%3D%201.4.1%20order%20by%20updated%20DESC
>
> If you think something is missing and should be added for 1.4.1, could you
> please add them to the JIRA and let others know here?
>
> Cheers,
> Gordon
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>