[DISCUSS] FLIP-32: Restructure flink-table for future contributions
Locked
[DISCUSS] FLIP-32: Restructure flink-table for future contributions
 
|
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
Hi Timo,
Thanks a lot for bringing up the discussion and the detailed implementation plan. The implementation plan makes much sense to me. +1 to the FLIP. Regards, Dian > 在 2019年1月25日，下午11:00，Jark Wu <[hidden email]> 写道： > > Hi Timo, > > Thanks for the detailed design. +1 for the FLIP. > > It's very nice to have Blink planner and runtime as a plugin in the early > stages. This will keep flink-table stable as much as possible. > > Best, > Jark > > > On Fri, 25 Jan 2019 at 22:26, Hequn Cheng <[hidden email]> wrote: > >> Hi Timo, >> >> +1 for the FLIP! >> >> Great work and thanks a lot for the detailed document! The task dependency >> graph is very helpful for teasing out relationships between tasks. >> Looking forward to the JIRAs and hoping to contribute to it! >> >> Best, >> Hequn >> >> >> On Fri, Jan 25, 2019 at 7:54 PM Piotr Nowojski <[hidden email]> >> wrote: >> >>> Hi, >>> >>> +1 from my side for this plan. The proposed decoupling TypeSystem and >> most >>> interface (TableSource/Sinks, UDFs, Catalogs, …) discussions from the of >>> merging MVP Blink's runtime gives us best shot of handling Blink merging >> as >>> fluently and as painless as possible. I’m also looking forward to the >>> follow up discussions that will need to take place in order to achieve >> the >>> final goal :) >>> >>> Piotrek >>> >>>> On 25 Jan 2019, at 07:53, jincheng sun <[hidden email]> >> wrote: >>>> >>>> Hi Timo, >>>> >>>> Thanks a lot for bring up the FLIP-32 discussion and the very detailed >>>> implementation plan document ! >>>> >>>> Restructure `flink-table` is an important part of flink merge blink, >>>> looking forward to the JIRAs which will be opened ! >>>> >>>> Cheers， >>>> Jincheng >>>> >>>> >>>> Timo Walther <[hidden email]> 于2019年1月24日周四 下午9:06写道： >>>> >>>>> Hi everyone, >>>>> >>>>> as Stephan already announced on the mailing list [1], the Flink >>>>> community will receive a big code contribution from Alibaba. The >>>>> flink-table module is one of the biggest parts that will receive many >>>>> new features and major architectural improvements. Instead of waiting >>>>> until the next major version of Flink or introducing big API-breaking >>>>> changes, we would like to gradually build up the Blink-based planner >> and >>>>> runtime while keeping the Table & SQL API mostly stable. Users will be >>>>> able to play around with the current merge status of the new planner >> or >>>>> fall back to the old planner until the new one is stable. >>>>> >>>>> We have prepared a design document that discusses a restructuring of >> the >>>>> flink-table module and suggests a rough implementation plan: >>>>> >>>>> >>>>> >>> >> https://docs.google.com/document/d/1Tfl2dBqBV3qSBy7oV3qLYvRRDbUOasvA1lhvYWWljQw/edit?usp=sharing >>>>> >>>>> I will briefly summarize the steps we would like to do: >>>>> >>>>> - Split the flink-table module similar to the proposal of FLIP-28 [3] >>>>> which is outdated. This is a preparation to separate API from core >>>>> (targeted for Flink 1.8). >>>>> - Perform minor API changes to separate API from actual implementation >>>>> (targeted for Flink 1.8). >>>>> - Merge a MVP Blink SQL planner given that necessary Flink >> core/runtime >>>>> changes have been completed. >>>>> The merging will happen in stages (e.g. basic planner framework, >> then >>>>> operator by operator). The exact merging plan still needs to be >>> determined. >>>>> - Rework the type system in order to unblock work on unified table >>>>> environments, UDFs, sources/sinks, and catalog. >>>>> - Enable full end-to-end batch and stream execution features. >>>>> >>>>> Our mid-term goal: >>>>> >>>>> Run full TPC-DS on a unified batch/streaming runtime. Initially, we >> will >>>>> only support ingesting data coming from the DataStream API. Once we >>>>> reworked the sources/sink interfaces, we will target full end-to-end >>>>> TPC-DS query execution with table connectors. >>>>> >>>>> A rough task dependency graph is illustrated in the design document. A >>>>> more detailed task dependency structure will be added to JIRA after we >>>>> agreed on this FLIP. >>>>> >>>>> Looking forward to any feedback. >>>>> >>>>> Thanks, >>>>> Timo >>>>> >>>>> [1] >>>>> >>>>> >>> >> https://lists.apache.org/thread.html/2f7330e85d702a53b4a2b361149930b50f2e89d8e8a572f8ee2a0e6d@%3Cdev.flink.apache.org%3E >>>>> [2] >>>>> >>>>> >>> >> https://lists.apache.org/thread.html/6066abd0f09fc1c41190afad67770ede8efd0bebc36f00938eecc118@%3Cdev.flink.apache.org%3E >>>>> [3] >>>>> >>>>> >>> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-28%3A+Long-term+goal+of+making+flink-table+Scala-free >>>>> >>>>> >>> >>> >> |
smime.p7s (3K)
Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions
|
|
| Free forum by Nabble | Edit this page |