Hi everyone,
Recently, I drafted a proposal about inner join in Flink ( http://goo.gl/4AdR7h). This document reviews some related work on the Table/SQL topic and it provides a relatively complete view about the inner join semantics and implementation. Besides, I also share my (objective) thoughts about unifying the batch/stream query processing. I know there are lots of developers who are interested in this subject. Please share your ideas and all suggestions are welcome. Thanks, Xingcan |
Hello Xingcan,
Thanks for the proposal. It seems (I may miss something) the proposed semantics&approach for unbounded inner join is similar as the one proposed in FLINK-5878. I did not create the PR for FLINK-5878, as the implementation for inner join is closely associated with "Retract" (proposed in FLINK-6047). I have not completely read through your doc, but the window-joins that you mentioned are definitely the topics that we are also interested. Will read it carefully and left comments on your doc. Thanks! Regards, Shaoxuan On Wed, May 17, 2017 at 8:56 PM, Xingcan Cui <[hidden email]> wrote: > Hi everyone, > > Recently, I drafted a proposal about inner join in Flink ( > http://goo.gl/4AdR7h). > > This document reviews some related work on the Table/SQL topic and it > provides a relatively complete view about the inner join semantics and > implementation. Besides, I also share my (objective) thoughts about > unifying the batch/stream query processing. > > I know there are lots of developers who are interested in this subject. > Please share your ideas and all suggestions are welcome. > > Thanks, > Xingcan > |
Hi Xingcan,
Thanks for the proposal. I have glanced at the design document but not detailedly. The semantics of Record-to-window Join is already in progress in FLINK-5725,. It would be great if you can share your ideas about the implementions. Thanks very much. Yuhong -----邮件原件----- 发件人: Shaoxuan Wang [mailto:[hidden email]] 发送时间: 2017年5月17日 22:48 收件人: Dev 主题: Re: Proposal about inner join in Flink Hello Xingcan, Thanks for the proposal. It seems (I may miss something) the proposed semantics&approach for unbounded inner join is similar as the one proposed in FLINK-5878. I did not create the PR for FLINK-5878, as the implementation for inner join is closely associated with "Retract" (proposed in FLINK-6047). I have not completely read through your doc, but the window-joins that you mentioned are definitely the topics that we are also interested. Will read it carefully and left comments on your doc. Thanks! Regards, Shaoxuan On Wed, May 17, 2017 at 8:56 PM, Xingcan Cui <[hidden email]> wrote: > Hi everyone, > > Recently, I drafted a proposal about inner join in Flink ( > http://goo.gl/4AdR7h). > > This document reviews some related work on the Table/SQL topic and it > provides a relatively complete view about the inner join semantics and > implementation. Besides, I also share my (objective) thoughts about > unifying the batch/stream query processing. > > I know there are lots of developers who are interested in this subject. > Please share your ideas and all suggestions are welcome. > > Thanks, > Xingcan > |
Hi Shaoxuan and Yuhong,
thanks for your feedbacks. @Shaoxuan, actually the unbounded inner join is just inspired by your document and the discussions. I've spent a lot of time arranging the references, but still put the link at a wrong please. Sorry for that and had it done now. Go back to work... I think there's no need to retract something for unbounded inner joins (well, maybe we need it when taking the join as a sub-query). What's your opinion? As for the window-joins, I think the most difficult part is how to consolidate the semantics with batch environment. I present a naive solution (by system UDF) for that and not sure if it will work. @Yuhong, I've reviewed the PR for FLINK-5725. For the SQL translation part, I think it's clear and suitable. For the execution part I suppose it can work and generate precise results. The only point I concerned is the parallelism. Will it only be executed in a single thread? Thanks, Xingcan On Thu, May 18, 2017 at 11:40 AM, Hongyuhong <[hidden email]> wrote: > Hi Xingcan, > Thanks for the proposal. > I have glanced at the design document but not detailedly. The semantics of > Record-to-window Join is already in progress in FLINK-5725,. > It would be great if you can share your ideas about the implementions. > > Thanks very much. > Yuhong > > -----邮件原件----- > 发件人: Shaoxuan Wang [mailto:[hidden email]] > 发送时间: 2017年5月17日 22:48 > 收件人: Dev > 主题: Re: Proposal about inner join in Flink > > Hello Xingcan, > Thanks for the proposal. It seems (I may miss something) the proposed > semantics&approach for unbounded inner join is similar as the one proposed > in FLINK-5878. > I did not create the PR for FLINK-5878, as the implementation for inner > join is closely associated with "Retract" (proposed in FLINK-6047). > I have not completely read through your doc, but the window-joins that you > mentioned are definitely the topics that we are also interested. Will read > it carefully and left comments on your doc. Thanks! > > Regards, > Shaoxuan > > > On Wed, May 17, 2017 at 8:56 PM, Xingcan Cui <[hidden email]> wrote: > > > Hi everyone, > > > > Recently, I drafted a proposal about inner join in Flink ( > > http://goo.gl/4AdR7h). > > > > This document reviews some related work on the Table/SQL topic and it > > provides a relatively complete view about the inner join semantics and > > implementation. Besides, I also share my (objective) thoughts about > > unifying the batch/stream query processing. > > > > I know there are lots of developers who are interested in this subject. > > Please share your ideas and all suggestions are welcome. > > > > Thanks, > > Xingcan > > > |
Free forum by Nabble | Edit this page |