[VOTE] FLIP-88: PyFlink User-Defined Function Resource Management

[Dian Fu-2]

Hi all,

I'd like to start the vote of FLIP-88 [1] since that we have reached an agreement on the design in the discussion thread [2].

This vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have received sufficient votes.

Regards,
Dian

[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
[2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html

[Jingsong Lee]

Hi Dian:

Thanks for your driving. I have some questions:

- Where should these configurations belong? You have mentioned
tableApi/SQL, so should in TableConfig?
- If just in table/sql, whether it should be called: table.python.****,
because in table, all config options are called table.***.
- What should table module do? So in CommonPythonCalc, we should read
options from table config, and set resources to OneInputTransformation?
- Are all buffer.memory off-heap memory? I took a look
to AbstractPythonScalarFunctionOperator, there is a forwardedInputQueue, is
this one a heap queue? So we need heap memory too?

Hope to get your reply.

Best,
Jingsong Lee

On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]> wrote:

> Hi all,
>
> I'd like to start the vote of FLIP-88 [1] since that we have reached an
> agreement on the design in the discussion thread [2].
>
> This vote will be open for at least 72 hours. Unless there is an
> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have
> received sufficient votes.
>
> Regards,
> Dian
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
> [2]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html

--
Best, Jingsong Lee

[Jingsong Li]

Sorry for bothering your voting.
Let's discuss in discussion thread.

Best,
Jingsong Lee

On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <[hidden email]> wrote:

> Hi Dian:
>
> Thanks for your driving. I have some questions:
>
> - Where should these configurations belong? You have mentioned
> tableApi/SQL, so should in TableConfig?
> - If just in table/sql, whether it should be called: table.python.****,
> because in table, all config options are called table.***.
> - What should table module do? So in CommonPythonCalc, we should read
> options from table config, and set resources to OneInputTransformation?
> - Are all buffer.memory off-heap memory? I took a look
> to AbstractPythonScalarFunctionOperator, there is a forwardedInputQueue, is
> this one a heap queue? So we need heap memory too?
>
> Hope to get your reply.
>
> Best,
> Jingsong Lee
>
> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]> wrote:
>
>> Hi all,
>>
>> I'd like to start the vote of FLIP-88 [1] since that we have reached an
>> agreement on the design in the discussion thread [2].
>>
>> This vote will be open for at least 72 hours. Unless there is an
>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have
>> received sufficient votes.
>>
>> Regards,
>> Dian
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
>> [2]
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
>
>
>
> --
> Best, Jingsong Lee
>

--
Best, Jingsong Lee

[Dian Fu-2]

Hi Jingsong,

It's fine. :)  Appreciated the comments!

I have replied you in the discussion thread as I also think it's better to discuss these in the discussion thread.

Thanks,
Dian

> 在 2019年12月2日，下午3:47，Jingsong Li <[hidden email]> 写道：
>
> Sorry for bothering your voting.
> Let's discuss in discussion thread.
>
> Best,
> Jingsong Lee
>
> On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <[hidden email]> wrote:
>
>> Hi Dian:
>>
>> Thanks for your driving. I have some questions:
>>
>> - Where should these configurations belong? You have mentioned
>> tableApi/SQL, so should in TableConfig?
>> - If just in table/sql, whether it should be called: table.python.****,
>> because in table, all config options are called table.***.
>> - What should table module do? So in CommonPythonCalc, we should read
>> options from table config, and set resources to OneInputTransformation?
>> - Are all buffer.memory off-heap memory? I took a look
>> to AbstractPythonScalarFunctionOperator, there is a forwardedInputQueue, is
>> this one a heap queue? So we need heap memory too?
>>
>> Hope to get your reply.
>>
>> Best,
>> Jingsong Lee
>>
>> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]> wrote:
>>
>>> Hi all,
>>>
>>> I'd like to start the vote of FLIP-88 [1] since that we have reached an
>>> agreement on the design in the discussion thread [2].
>>>
>>> This vote will be open for at least 72 hours. Unless there is an
>>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have
>>> received sufficient votes.
>>>
>>> Regards,
>>> Dian
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
>>> [2]
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
>>
>>
>>
>> --
>> Best, Jingsong Lee
>>
>
>
> --
> Best, Jingsong Lee

[Jingsong Li]

+1 (non-binding)

Best,
Jingsong Lee

On Mon, Dec 2, 2019 at 5:30 PM Dian Fu <[hidden email]> wrote:

> Hi Jingsong,
>
> It's fine. :)  Appreciated the comments!
>
> I have replied you in the discussion thread as I also think it's better to
> discuss these in the discussion thread.
>
> Thanks,
> Dian
>
> > 在 2019年12月2日，下午3:47，Jingsong Li <[hidden email]> 写道：
> >
> > Sorry for bothering your voting.
> > Let's discuss in discussion thread.
> >
> > Best,
> > Jingsong Lee
> >
> > On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <[hidden email]>
> wrote:
> >
> >> Hi Dian:
> >>
> >> Thanks for your driving. I have some questions:
> >>
> >> - Where should these configurations belong? You have mentioned
> >> tableApi/SQL, so should in TableConfig?
> >> - If just in table/sql, whether it should be called: table.python.****,
> >> because in table, all config options are called table.***.
> >> - What should table module do? So in CommonPythonCalc, we should read
> >> options from table config, and set resources to OneInputTransformation?
> >> - Are all buffer.memory off-heap memory? I took a look
> >> to AbstractPythonScalarFunctionOperator, there is a
> forwardedInputQueue, is
> >> this one a heap queue? So we need heap memory too?
> >>
> >> Hope to get your reply.
> >>
> >> Best,
> >> Jingsong Lee
> >>
> >> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I'd like to start the vote of FLIP-88 [1] since that we have reached an
> >>> agreement on the design in the discussion thread [2].
> >>>
> >>> This vote will be open for at least 72 hours. Unless there is an
> >>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have
> >>> received sufficient votes.
> >>>
> >>> Regards,
> >>> Dian
> >>>
> >>> [1]
> >>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
> >>> [2]
> >>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
> >>
> >>
> >>
> >> --
> >> Best, Jingsong Lee
> >>
> >
> >
> > --
> > Best, Jingsong Lee
>
>

--
Best, Jingsong Lee

[jincheng sun]

+1（binding)

Best,
Jincheng

Jingsong Li <[hidden email]> 于2019年12月3日周二 下午7:30写道：

> +1 (non-binding)
>
> Best,
> Jingsong Lee
>
> On Mon, Dec 2, 2019 at 5:30 PM Dian Fu <[hidden email]> wrote:
>
> > Hi Jingsong,
> >
> > It's fine. :)  Appreciated the comments!
> >
> > I have replied you in the discussion thread as I also think it's better
> to
> > discuss these in the discussion thread.
> >
> > Thanks,
> > Dian
> >
> > > 在 2019年12月2日，下午3:47，Jingsong Li <[hidden email]> 写道：
> > >
> > > Sorry for bothering your voting.
> > > Let's discuss in discussion thread.
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <[hidden email]>
> > wrote:
> > >
> > >> Hi Dian:
> > >>
> > >> Thanks for your driving. I have some questions:
> > >>
> > >> - Where should these configurations belong? You have mentioned
> > >> tableApi/SQL, so should in TableConfig?
> > >> - If just in table/sql, whether it should be called:
> table.python.****,
> > >> because in table, all config options are called table.***.
> > >> - What should table module do? So in CommonPythonCalc, we should read
> > >> options from table config, and set resources to
> OneInputTransformation?
> > >> - Are all buffer.memory off-heap memory? I took a look
> > >> to AbstractPythonScalarFunctionOperator, there is a
> > forwardedInputQueue, is
> > >> this one a heap queue? So we need heap memory too?
> > >>
> > >> Hope to get your reply.
> > >>
> > >> Best,
> > >> Jingsong Lee
> > >>
> > >> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]> wrote:
> > >>
> > >>> Hi all,
> > >>>
> > >>> I'd like to start the vote of FLIP-88 [1] since that we have reached
> an
> > >>> agreement on the design in the discussion thread [2].
> > >>>
> > >>> This vote will be open for at least 72 hours. Unless there is an
> > >>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we have
> > >>> received sufficient votes.
> > >>>
> > >>> Regards,
> > >>> Dian
> > >>>
> > >>> [1]
> > >>>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
> > >>> [2]
> > >>>
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
> > >>
> > >>
> > >>
> > >> --
> > >> Best, Jingsong Lee
> > >>
> > >
> > >
> > > --
> > > Best, Jingsong Lee
> >
> >
>
> --
> Best, Jingsong Lee
>

[Hequn Cheng]

+1 (binding)

Best,
Hequn

On Thu, Dec 5, 2019 at 4:41 PM jincheng sun <[hidden email]>
wrote:

> +1（binding)
>
> Best,
> Jincheng
>
> Jingsong Li <[hidden email]> 于2019年12月3日周二 下午7:30写道：
>
> > +1 (non-binding)
> >
> > Best,
> > Jingsong Lee
> >
> > On Mon, Dec 2, 2019 at 5:30 PM Dian Fu <[hidden email]> wrote:
> >
> > > Hi Jingsong,
> > >
> > > It's fine. :)  Appreciated the comments!
> > >
> > > I have replied you in the discussion thread as I also think it's better
> > to
> > > discuss these in the discussion thread.
> > >
> > > Thanks,
> > > Dian
> > >
> > > > 在 2019年12月2日，下午3:47，Jingsong Li <[hidden email]> 写道：
> > > >
> > > > Sorry for bothering your voting.
> > > > Let's discuss in discussion thread.
> > > >
> > > > Best,
> > > > Jingsong Lee
> > > >
> > > > On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <[hidden email]
> >
> > > wrote:
> > > >
> > > >> Hi Dian:
> > > >>
> > > >> Thanks for your driving. I have some questions:
> > > >>
> > > >> - Where should these configurations belong? You have mentioned
> > > >> tableApi/SQL, so should in TableConfig?
> > > >> - If just in table/sql, whether it should be called:
> > table.python.****,
> > > >> because in table, all config options are called table.***.
> > > >> - What should table module do? So in CommonPythonCalc, we should
> read
> > > >> options from table config, and set resources to
> > OneInputTransformation?
> > > >> - Are all buffer.memory off-heap memory? I took a look
> > > >> to AbstractPythonScalarFunctionOperator, there is a
> > > forwardedInputQueue, is
> > > >> this one a heap queue? So we need heap memory too?
> > > >>
> > > >> Hope to get your reply.
> > > >>
> > > >> Best,
> > > >> Jingsong Lee
> > > >>
> > > >> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]>
> wrote:
> > > >>
> > > >>> Hi all,
> > > >>>
> > > >>> I'd like to start the vote of FLIP-88 [1] since that we have
> reached
> > an
> > > >>> agreement on the design in the discussion thread [2].
> > > >>>
> > > >>> This vote will be open for at least 72 hours. Unless there is an
> > > >>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we
> have
> > > >>> received sufficient votes.
> > > >>>
> > > >>> Regards,
> > > >>> Dian
> > > >>>
> > > >>> [1]
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
> > > >>> [2]
> > > >>>
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Best, Jingsong Lee
> > > >>
> > > >
> > > >
> > > > --
> > > > Best, Jingsong Lee
> > >
> > >
> >
> > --
> > Best, Jingsong Lee
> >
>

[Jark Wu]

Thanks for driving this Dian! The FLIP looks good to me.

+1 (binding)

Best,
Jark

On Thu, 5 Dec 2019 at 16:44, Hequn Cheng <[hidden email]> wrote:

> +1 (binding)
>
> Best,
> Hequn
>
> On Thu, Dec 5, 2019 at 4:41 PM jincheng sun <[hidden email]>
> wrote:
>
> > +1（binding)
> >
> > Best,
> > Jincheng
> >
> > Jingsong Li <[hidden email]> 于2019年12月3日周二 下午7:30写道：
> >
> > > +1 (non-binding)
> > >
> > > Best,
> > > Jingsong Lee
> > >
> > > On Mon, Dec 2, 2019 at 5:30 PM Dian Fu <[hidden email]> wrote:
> > >
> > > > Hi Jingsong,
> > > >
> > > > It's fine. :)  Appreciated the comments!
> > > >
> > > > I have replied you in the discussion thread as I also think it's
> better
> > > to
> > > > discuss these in the discussion thread.
> > > >
> > > > Thanks,
> > > > Dian
> > > >
> > > > > 在 2019年12月2日，下午3:47，Jingsong Li <[hidden email]> 写道：
> > > > >
> > > > > Sorry for bothering your voting.
> > > > > Let's discuss in discussion thread.
> > > > >
> > > > > Best,
> > > > > Jingsong Lee
> > > > >
> > > > > On Mon, Dec 2, 2019 at 3:32 PM Jingsong Lee <
> [hidden email]
> > >
> > > > wrote:
> > > > >
> > > > >> Hi Dian:
> > > > >>
> > > > >> Thanks for your driving. I have some questions:
> > > > >>
> > > > >> - Where should these configurations belong? You have mentioned
> > > > >> tableApi/SQL, so should in TableConfig?
> > > > >> - If just in table/sql, whether it should be called:
> > > table.python.****,
> > > > >> because in table, all config options are called table.***.
> > > > >> - What should table module do? So in CommonPythonCalc, we should
> > read
> > > > >> options from table config, and set resources to
> > > OneInputTransformation?
> > > > >> - Are all buffer.memory off-heap memory? I took a look
> > > > >> to AbstractPythonScalarFunctionOperator, there is a
> > > > forwardedInputQueue, is
> > > > >> this one a heap queue? So we need heap memory too?
> > > > >>
> > > > >> Hope to get your reply.
> > > > >>
> > > > >> Best,
> > > > >> Jingsong Lee
> > > > >>
> > > > >> On Mon, Dec 2, 2019 at 2:34 PM Dian Fu <[hidden email]>
> > wrote:
> > > > >>
> > > > >>> Hi all,
> > > > >>>
> > > > >>> I'd like to start the vote of FLIP-88 [1] since that we have
> > reached
> > > an
> > > > >>> agreement on the design in the discussion thread [2].
> > > > >>>
> > > > >>> This vote will be open for at least 72 hours. Unless there is an
> > > > >>> objection, I will try to close it by Dec 5, 2019 08:00 UTC if we
> > have
> > > > >>> received sufficient votes.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Dian
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-88%3A+PyFlink+User-Defined+Function+Resource+Management
> > > > >>> [2]
> > > > >>>
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-PyFlink-User-Defined-Function-Resource-Management-tt34631.html
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Best, Jingsong Lee
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Best, Jingsong Lee
> > > >
> > > >
> > >
> > > --
> > > Best, Jingsong Lee
> > >
> >
>