Hi Everyone,
After the recent Flink 1.6 release the people working on Flink at data Artisans came together to talk about what we want to work on for Flink 1.7. The following is a list of high-level directions that we will be working on for the next couple of months. This doesn't mean that other things are not important or maybe more important, so please chime in. That being said, here's the high-level list: - make the Rest API versioned - provide docker-compose based quickstarts for Flink SQL - support writing to S3 in the new streaming file sink - add a new type of join that allows "joining streams with tables" - Scala 2.12 support - improvements to resource scheduling, local recovery - improved support for running Flink in containers, Flink dynamically reacting to changes in the container deployment - automatic rescaling policies - initial support for state migration, i.e. changing the schema/TypeSerializer of Flink State This is also an invitation for others to post what they would like to work on and also to point out glaring omissions. Best, Aljoscha |
Hi:
Can we include task manager isolation for jobs in 1.7.0? This is a feature that blocks us from deploying flink more widely in our compan? I've opened issues <https://issues.apache.org/jira/browse/FLINK-9662> for that and want to help to contribute that, but lacks guidance and reviews from community. On Thu, Aug 23, 2018 at 4:12 PM Aljoscha Krettek <[hidden email]> wrote: > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data > Artisans came together to talk about what we want to work on for Flink 1.7. > The following is a list of high-level directions that we will be working on > for the next couple of months. This doesn't mean that other things are not > important or maybe more important, so please chime in. > > That being said, here's the high-level list: > - make the Rest API versioned > - provide docker-compose based quickstarts for Flink SQL > - support writing to S3 in the new streaming file sink > - add a new type of join that allows "joining streams with tables" > - Scala 2.12 support > - improvements to resource scheduling, local recovery > - improved support for running Flink in containers, Flink dynamically > reacting to changes in the container deployment > - automatic rescaling policies > - initial support for state migration, i.e. changing the > schema/TypeSerializer of Flink State > > This is also an invitation for others to post what they would like to work > on and also to point out glaring omissions. > > Best, > Aljoscha -- Liu, Renjie Software Engineer, MVAD |
In reply to this post by Aljoscha Krettek-2
Hi,
I am concerned about the progress of CEP library development, Can the following two main feature be kicked off and be involed in Flink1.7? 1) integration of CEP & SQL 2) dynamic change of CEP pattern without the downtime And i am willing to contribute to this, thx. Aljoscha Krettek <[hidden email]> 于2018年8月23日周四 下午4:12写道: > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data > Artisans came together to talk about what we want to work on for Flink 1.7. > The following is a list of high-level directions that we will be working on > for the next couple of months. This doesn't mean that other things are not > important or maybe more important, so please chime in. > > That being said, here's the high-level list: > - make the Rest API versioned > - provide docker-compose based quickstarts for Flink SQL > - support writing to S3 in the new streaming file sink > - add a new type of join that allows "joining streams with tables" > - Scala 2.12 support > - improvements to resource scheduling, local recovery > - improved support for running Flink in containers, Flink dynamically > reacting to changes in the container deployment > - automatic rescaling policies > - initial support for state migration, i.e. changing the > schema/TypeSerializer of Flink State > > This is also an invitation for others to post what they would like to work > on and also to point out glaring omissions. > > Best, > Aljoscha |
Hi,
I think Kafka 1.0/1.1/2.0 connector can be part of the Flink 1.7 release. The current kafka 1.0 connector PR has been submitted. I am refactoring the existing kafka connector test code to reduce the amount of duplicate code. Thanks, vino. Coo1 min <[hidden email]> 于2018年8月23日周四 下午5:59写道: > Hi, > > I am concerned about the progress of CEP library development, Can the > following two main feature be kicked off and be involed in Flink1.7? > > 1) integration of CEP & SQL > 2) dynamic change of CEP pattern without the downtime > > And i am willing to contribute to this, thx. > > Aljoscha Krettek <[hidden email]> 于2018年8月23日周四 下午4:12写道: > > > Hi Everyone, > > > > After the recent Flink 1.6 release the people working on Flink at data > > Artisans came together to talk about what we want to work on for Flink > 1.7. > > The following is a list of high-level directions that we will be working > on > > for the next couple of months. This doesn't mean that other things are > not > > important or maybe more important, so please chime in. > > > > That being said, here's the high-level list: > > - make the Rest API versioned > > - provide docker-compose based quickstarts for Flink SQL > > - support writing to S3 in the new streaming file sink > > - add a new type of join that allows "joining streams with tables" > > - Scala 2.12 support > > - improvements to resource scheduling, local recovery > > - improved support for running Flink in containers, Flink dynamically > > reacting to changes in the container deployment > > - automatic rescaling policies > > - initial support for state migration, i.e. changing the > > schema/TypeSerializer of Flink State > > > > This is also an invitation for others to post what they would like to > work > > on and also to point out glaring omissions. > > > > Best, > > Aljoscha > |
In reply to this post by Aljoscha Krettek-2
I would be nice to see the state TTL work finished. At the moment state
will not expire if a key is not accessed while the job is running, which means jobs that require state expiration must continue using timers to expire data. On Thu, Aug 23, 2018 at 1:12 AM Aljoscha Krettek <[hidden email]> wrote: > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data > Artisans came together to talk about what we want to work on for Flink 1.7. > The following is a list of high-level directions that we will be working on > for the next couple of months. This doesn't mean that other things are not > important or maybe more important, so please chime in. > > That being said, here's the high-level list: > - make the Rest API versioned > - provide docker-compose based quickstarts for Flink SQL > - support writing to S3 in the new streaming file sink > - add a new type of join that allows "joining streams with tables" > - Scala 2.12 support > - improvements to resource scheduling, local recovery > - improved support for running Flink in containers, Flink dynamically > reacting to changes in the container deployment > - automatic rescaling policies > - initial support for state migration, i.e. changing the > schema/TypeSerializer of Flink State > > This is also an invitation for others to post what they would like to work > on and also to point out glaring omissions. > > Best, > Aljoscha |
In reply to this post by Aljoscha Krettek-2
Any plans to advance the "splittable source" idea? It probably won't be
ready to implement, but important to get on the radar. Thanks, Thomas On Thu, Aug 23, 2018 at 1:12 AM Aljoscha Krettek <[hidden email]> wrote: > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data > Artisans came together to talk about what we want to work on for Flink 1.7. > The following is a list of high-level directions that we will be working on > for the next couple of months. This doesn't mean that other things are not > important or maybe more important, so please chime in. > > That being said, here's the high-level list: > - make the Rest API versioned > - provide docker-compose based quickstarts for Flink SQL > - support writing to S3 in the new streaming file sink > - add a new type of join that allows "joining streams with tables" > - Scala 2.12 support > - improvements to resource scheduling, local recovery > - improved support for running Flink in containers, Flink dynamically > reacting to changes in the container deployment > - automatic rescaling policies > - initial support for state migration, i.e. changing the > schema/TypeSerializer of Flink State > > This is also an invitation for others to post what they would like to work > on and also to point out glaring omissions. > > Best, > Aljoscha |
In reply to this post by Aljoscha Krettek-2
I would like to have https://issues.apache.org/jira/browse/FLINK-7289 in the
release. It's problematic to control memory usage of rocksdb backend. And many people have issue on it. Best Yan Aljoscha Krettek-2 wrote > Hi Everyone, > > After the recent Flink 1.6 release the people working on Flink at data > Artisans came together to talk about what we want to work on for Flink > 1.7. The following is a list of high-level directions that we will be > working on for the next couple of months. This doesn't mean that other > things are not important or maybe more important, so please chime in. > > That being said, here's the high-level list: > - make the Rest API versioned > - provide docker-compose based quickstarts for Flink SQL > - support writing to S3 in the new streaming file sink > - add a new type of join that allows "joining streams with tables" > - Scala 2.12 support > - improvements to resource scheduling, local recovery > - improved support for running Flink in containers, Flink dynamically > reacting to changes in the container deployment > - automatic rescaling policies > - initial support for state migration, i.e. changing the > schema/TypeSerializer of Flink State > > This is also an invitation for others to post what they would like to work > on and also to point out glaring omissions. > > Best, > Aljoscha -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ |
Free forum by Nabble | Edit this page |