(DEPRECATED) Apache Flink Mailing List archive.

[DISCUSS] Drop stale class Program

Classic

List

Threaded

14 messages Options

tison

[DISCUSS] Drop stale class Program

Hi devs,

Participating the thread "Flink client api enhancement"[1], I just notice
that inside submission codepath of Flink we always has a branch dealing
with the case that main class of user program is a subclass of
o.a.f.api.common.Program, which is defined as

@PublicEvolving
public interface Program {
Plan getPhan(String... args);
}

This class, as user-facing interface, asks user to implement #getPlan
which return a almost Flink internal class. FLINK-10862[2] pointed out
this confusion.

Since our codebase contains quite a bit code handling this stale class,
and also it obstructs the effort enhancing Flink cilent api,
I'd like to propose dropping it. Or we can start a survey on user list
to see if there is any user depending on this class.

best,
tison.

[1]
https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
[2] https://issues.apache.org/jira/browse/FLINK-10862

Biao Liu

Re: [DISCUSS] Drop stale class Program

Hi Zili,

Thank you for bring us this discussion.

My gut feeling is +1 for dropping it.
Usually it costs some time to deprecate a public (actually it's
`PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
Then it might be abandoned it in some later version.

I'm not sure how big the burden is to make it compatible with the enhanced
client API. If it's a critical blocker, I support dropping it radically in
1.10. Of course a survey is necessary. And the result of survey is
acceptable.

Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：

> Hi devs,
>
> Participating the thread "Flink client api enhancement"[1], I just notice
> that inside submission codepath of Flink we always has a branch dealing
> with the case that main class of user program is a subclass of
> o.a.f.api.common.Program, which is defined as
>
> @PublicEvolving
> public interface Program {
> Plan getPhan(String... args);
> }
>
> This class, as user-facing interface, asks user to implement #getPlan
> which return a almost Flink internal class. FLINK-10862[2] pointed out
> this confusion.
>
> Since our codebase contains quite a bit code handling this stale class,
> and also it obstructs the effort enhancing Flink cilent api,
> I'd like to propose dropping it. Or we can start a survey on user list
> to see if there is any user depending on this class.
>
> best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> [2] https://issues.apache.org/jira/browse/FLINK-10862
>

Flavio Pompermaier

Re: [DISCUSS] Drop stale class Program

In my experience a basic "official" (but optional) program description
would be very useful indeed (in order to ease the integration with other
frameworks).

Of course it should be extended and integrated with the REST services and
the Web UI (when defined) in order to be useful..
It ease to show to the user what a job does and which parameters it
requires (optional or mandatory) and with a proper help description.
Indeed, when we write a Flink job we implement the following interface:

public interface FlinkJob {
String getDescription();
List<FlinkJobParameter> getParameters();
boolean isStreamingOrBatch();
}

public class ClusterJobParameter {

private String paramName;
private String paramType = "string";
private String paramDesc;
private String paramDefaultValue;
private Set<String> choices;
private boolean mandatory;
}

This really helps to launch a Flink job by a frontend (if the rest services
returns back those infos).

Best,
Flavio

On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:

> Hi Zili,
>
> Thank you for bring us this discussion.
>
> My gut feeling is +1 for dropping it.
> Usually it costs some time to deprecate a public (actually it's
> `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> Then it might be abandoned it in some later version.
>
> I'm not sure how big the burden is to make it compatible with the enhanced
> client API. If it's a critical blocker, I support dropping it radically in
> 1.10. Of course a survey is necessary. And the result of survey is
> acceptable.
>
>
>
> Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
>
> > Hi devs,
> >
> > Participating the thread "Flink client api enhancement"[1], I just notice
> > that inside submission codepath of Flink we always has a branch dealing
> > with the case that main class of user program is a subclass of
> > o.a.f.api.common.Program, which is defined as
> >
> > @PublicEvolving
> > public interface Program {
> > Plan getPhan(String... args);
> > }
> >
> > This class, as user-facing interface, asks user to implement #getPlan
> > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > this confusion.
> >
> > Since our codebase contains quite a bit code handling this stale class,
> > and also it obstructs the effort enhancing Flink cilent api,
> > I'd like to propose dropping it. Or we can start a survey on user list
> > to see if there is any user depending on this class.
> >
> > best,
> > tison.
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > [2] https://issues.apache.org/jira/browse/FLINK-10862
> >
>

tison

Re: [DISCUSS] Drop stale class Program

Hi Biao,

Thanks for your reply.

For the "burden" part, inside PackagedProgram and ClusterClient
we currently contains branches handling whether the mainClass of
user job jar is subclass of Program. Any effort under client api
enhancement should be compatible with such codepaths unless we
drop it.

The class Program is only active in batch codepath, while the
so-called interactive mode, i.e., executing the main method of
user's main class, is using widely and valid in both batch and
streaming codepath.

Previously we have drop flink-storm directly and I'm afraid this
Program class is much less known and used than flink-storm.
Keeping compatible with such class is an unnecessary overhead.

However, the enhancement of client api is still under discussion
so it would be ok to deprecate at first and propose removal when
necessary.

Either drop it or deprecate it prevents users from being confused
how to use this stale class. But given the community has dropped
stale flink-storm, flink-libarary/python|ml, I'd like to propose
drop this class if in the survey we receiving no usage report.

Best,
tison.

Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：

> In my experience a basic "official" (but optional) program description
> would be very useful indeed (in order to ease the integration with other
> frameworks).
>
> Of course it should be extended and integrated with the REST services and
> the Web UI (when defined) in order to be useful..
> It ease to show to the user what a job does and which parameters it
> requires (optional or mandatory) and with a proper help description.
> Indeed, when we write a Flink job we implement the following interface:
>
> public interface FlinkJob {
> String getDescription();
> List<FlinkJobParameter> getParameters();
> boolean isStreamingOrBatch();
> }
>
> public class ClusterJobParameter {
>
> private String paramName;
> private String paramType = "string";
> private String paramDesc;
> private String paramDefaultValue;
> private Set<String> choices;
> private boolean mandatory;
> }
>
> This really helps to launch a Flink job by a frontend (if the rest services
> returns back those infos).
>
> Best,
> Flavio
>
> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:
>
> > Hi Zili,
> >
> > Thank you for bring us this discussion.
> >
> > My gut feeling is +1 for dropping it.
> > Usually it costs some time to deprecate a public (actually it's
> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> > Then it might be abandoned it in some later version.
> >
> > I'm not sure how big the burden is to make it compatible with the
> enhanced
> > client API. If it's a critical blocker, I support dropping it radically
> in
> > 1.10. Of course a survey is necessary. And the result of survey is
> > acceptable.
> >
> >
> >
> > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> >
> > > Hi devs,
> > >
> > > Participating the thread "Flink client api enhancement"[1], I just
> notice
> > > that inside submission codepath of Flink we always has a branch dealing
> > > with the case that main class of user program is a subclass of
> > > o.a.f.api.common.Program, which is defined as
> > >
> > > @PublicEvolving
> > > public interface Program {
> > > Plan getPhan(String... args);
> > > }
> > >
> > > This class, as user-facing interface, asks user to implement #getPlan
> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > > this confusion.
> > >
> > > Since our codebase contains quite a bit code handling this stale class,
> > > and also it obstructs the effort enhancing Flink cilent api,
> > > I'd like to propose dropping it. Or we can start a survey on user list
> > > to see if there is any user depending on this class.
> > >
> > > best,
> > > tison.
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >
> >
>

Biao Liu

Re: [DISCUSS] Drop stale class Program

In reply to this post by Flavio Pompermaier

To Flavio, good point for the integration suggestion.

I think it should be considered in the "Flink client api enhancement"
discussion. But the outdated API should be deprecated somehow.

Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：

tison

Re: [DISCUSS] Drop stale class Program

In reply to this post by tison

Hi Flavio,

Thanks for your reply and share.

I agree with that an "official" program description
would be helpful as you described.

However, this thread mainly focuses on drop the stale
class Program.

For proposing a better program description from Flink
side, feel free to start a new thread on dev or user
list(I would regard this as a user requirement).

Best,
tison.

Zili Chen <[hidden email]> 于2019年7月19日周五下午5:10写道：

> Hi Biao,
>
> Thanks for your reply.
>
> For the "burden" part, inside PackagedProgram and ClusterClient
> we currently contains branches handling whether the mainClass of
> user job jar is subclass of Program. Any effort under client api
> enhancement should be compatible with such codepaths unless we
> drop it.
>
> The class Program is only active in batch codepath, while the
> so-called interactive mode, i.e., executing the main method of
> user's main class, is using widely and valid in both batch and
> streaming codepath.
>
> Previously we have drop flink-storm directly and I'm afraid this
> Program class is much less known and used than flink-storm.
> Keeping compatible with such class is an unnecessary overhead.
>
> However, the enhancement of client api is still under discussion
> so it would be ok to deprecate at first and propose removal when
> necessary.
>
> Either drop it or deprecate it prevents users from being confused
> how to use this stale class. But given the community has dropped
> stale flink-storm, flink-libarary/python|ml, I'd like to propose
> drop this class if in the survey we receiving no usage report.
>
> Best,
> tison.
>
>
> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
>
>> In my experience a basic "official" (but optional) program description
>> would be very useful indeed (in order to ease the integration with other
>> frameworks).
>>
>> Of course it should be extended and integrated with the REST services and
>> the Web UI (when defined) in order to be useful..
>> It ease to show to the user what a job does and which parameters it
>> requires (optional or mandatory) and with a proper help description.
>> Indeed, when we write a Flink job we implement the following interface:
>>
>> public interface FlinkJob {
>> String getDescription();
>> List<FlinkJobParameter> getParameters();
>> boolean isStreamingOrBatch();
>> }
>>
>> public class ClusterJobParameter {
>>
>> private String paramName;
>> private String paramType = "string";
>> private String paramDesc;
>> private String paramDefaultValue;
>> private Set<String> choices;
>> private boolean mandatory;
>> }
>>
>> This really helps to launch a Flink job by a frontend (if the rest
>> services
>> returns back those infos).
>>
>> Best,
>> Flavio
>>
>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:
>>
>> > Hi Zili,
>> >
>> > Thank you for bring us this discussion.
>> >
>> > My gut feeling is +1 for dropping it.
>> > Usually it costs some time to deprecate a public (actually it's
>> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>> first.
>> > Then it might be abandoned it in some later version.
>> >
>> > I'm not sure how big the burden is to make it compatible with the
>> enhanced
>> > client API. If it's a critical blocker, I support dropping it radically
>> in
>> > 1.10. Of course a survey is necessary. And the result of survey is
>> > acceptable.
>> >
>> >
>> >
>> > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
>> >
>> > > Hi devs,
>> > >
>> > > Participating the thread "Flink client api enhancement"[1], I just
>> notice
>> > > that inside submission codepath of Flink we always has a branch
>> dealing
>> > > with the case that main class of user program is a subclass of
>> > > o.a.f.api.common.Program, which is defined as
>> > >
>> > > @PublicEvolving
>> > > public interface Program {
>> > > Plan getPhan(String... args);
>> > > }
>> > >
>> > > This class, as user-facing interface, asks user to implement #getPlan
>> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
>> > > this confusion.
>> > >
>> > > Since our codebase contains quite a bit code handling this stale
>> class,
>> > > and also it obstructs the effort enhancing Flink cilent api,
>> > > I'd like to propose dropping it. Or we can start a survey on user list
>> > > to see if there is any user depending on this class.
>> > >
>> > > best,
>> > > tison.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
>> > >
>> >
>>
>

Flavio Pompermaier

Re: [DISCUSS] Drop stale class Program

In reply to this post by Biao Liu

+1 to remove directly the Program class (I think nobody use it and it's not
supported at all by REST services and UI).
Moreover it requires a lot of transitive dependencies while it should be a
very simple thing..
+1 to add this discussion to "Flink client api enhancement"

On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:

> To Flavio, good point for the integration suggestion.
>
> I think it should be considered in the "Flink client api enhancement"
> discussion. But the outdated API should be deprecated somehow.
>
> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
>
> > In my experience a basic "official" (but optional) program description
> > would be very useful indeed (in order to ease the integration with other
> > frameworks).
> >
> > Of course it should be extended and integrated with the REST services and
> > the Web UI (when defined) in order to be useful..
> > It ease to show to the user what a job does and which parameters it
> > requires (optional or mandatory) and with a proper help description.
> > Indeed, when we write a Flink job we implement the following interface:
> >
> > public interface FlinkJob {
> > String getDescription();
> > List<FlinkJobParameter> getParameters();
> > boolean isStreamingOrBatch();
> > }
> >
> > public class ClusterJobParameter {
> >
> > private String paramName;
> > private String paramType = "string";
> > private String paramDesc;
> > private String paramDefaultValue;
> > private Set<String> choices;
> > private boolean mandatory;
> > }
> >
> > This really helps to launch a Flink job by a frontend (if the rest
> services
> > returns back those infos).
> >
> > Best,
> > Flavio
> >
> > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:
> >
> > > Hi Zili,
> > >
> > > Thank you for bring us this discussion.
> > >
> > > My gut feeling is +1 for dropping it.
> > > Usually it costs some time to deprecate a public (actually it's
> > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> first.
> > > Then it might be abandoned it in some later version.
> > >
> > > I'm not sure how big the burden is to make it compatible with the
> > enhanced
> > > client API. If it's a critical blocker, I support dropping it radically
> > in
> > > 1.10. Of course a survey is necessary. And the result of survey is
> > > acceptable.
> > >
> > >
> > >
> > > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> > >
> > > > Hi devs,
> > > >
> > > > Participating the thread "Flink client api enhancement"[1], I just
> > notice
> > > > that inside submission codepath of Flink we always has a branch
> dealing
> > > > with the case that main class of user program is a subclass of
> > > > o.a.f.api.common.Program, which is defined as
> > > >
> > > > @PublicEvolving
> > > > public interface Program {
> > > > Plan getPhan(String... args);
> > > > }
> > > >
> > > > This class, as user-facing interface, asks user to implement #getPlan
> > > > which return a almost Flink internal class. FLINK-10862[2] pointed
> out
> > > > this confusion.
> > > >
> > > > Since our codebase contains quite a bit code handling this stale
> class,
> > > > and also it obstructs the effort enhancing Flink cilent api,
> > > > I'd like to propose dropping it. Or we can start a survey on user
> list
> > > > to see if there is any user depending on this class.
> > > >
> > > > best,
> > > > tison.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > >
> > >
> >
>

tison

Re: [DISCUSS] Drop stale class Program

Hi,

I created a thread for survey in user list[1]. Please take participate in
if interested.

Best,
tison.

[1]
https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E

Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：

> +1 to remove directly the Program class (I think nobody use it and it's not
> supported at all by REST services and UI).
> Moreover it requires a lot of transitive dependencies while it should be a
> very simple thing..
> +1 to add this discussion to "Flink client api enhancement"
>
> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:
>
> > To Flavio, good point for the integration suggestion.
> >
> > I think it should be considered in the "Flink client api enhancement"
> > discussion. But the outdated API should be deprecated somehow.
> >
> > Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
> >
> > > In my experience a basic "official" (but optional) program description
> > > would be very useful indeed (in order to ease the integration with
> other
> > > frameworks).
> > >
> > > Of course it should be extended and integrated with the REST services
> and
> > > the Web UI (when defined) in order to be useful..
> > > It ease to show to the user what a job does and which parameters it
> > > requires (optional or mandatory) and with a proper help description.
> > > Indeed, when we write a Flink job we implement the following interface:
> > >
> > > public interface FlinkJob {
> > > String getDescription();
> > > List<FlinkJobParameter> getParameters();
> > > boolean isStreamingOrBatch();
> > > }
> > >
> > > public class ClusterJobParameter {
> > >
> > > private String paramName;
> > > private String paramType = "string";
> > > private String paramDesc;
> > > private String paramDefaultValue;
> > > private Set<String> choices;
> > > private boolean mandatory;
> > > }
> > >
> > > This really helps to launch a Flink job by a frontend (if the rest
> > services
> > > returns back those infos).
> > >
> > > Best,
> > > Flavio
> > >
> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:
> > >
> > > > Hi Zili,
> > > >
> > > > Thank you for bring us this discussion.
> > > >
> > > > My gut feeling is +1 for dropping it.
> > > > Usually it costs some time to deprecate a public (actually it's
> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> > first.
> > > > Then it might be abandoned it in some later version.
> > > >
> > > > I'm not sure how big the burden is to make it compatible with the
> > > enhanced
> > > > client API. If it's a critical blocker, I support dropping it
> radically
> > > in
> > > > 1.10. Of course a survey is necessary. And the result of survey is
> > > > acceptable.
> > > >
> > > >
> > > >
> > > > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> > > >
> > > > > Hi devs,
> > > > >
> > > > > Participating the thread "Flink client api enhancement"[1], I just
> > > notice
> > > > > that inside submission codepath of Flink we always has a branch
> > dealing
> > > > > with the case that main class of user program is a subclass of
> > > > > o.a.f.api.common.Program, which is defined as
> > > > >
> > > > > @PublicEvolving
> > > > > public interface Program {
> > > > > Plan getPhan(String... args);
> > > > > }
> > > > >
> > > > > This class, as user-facing interface, asks user to implement
> #getPlan
> > > > > which return a almost Flink internal class. FLINK-10862[2] pointed
> > out
> > > > > this confusion.
> > > > >
> > > > > Since our codebase contains quite a bit code handling this stale
> > class,
> > > > > and also it obstructs the effort enhancing Flink cilent api,
> > > > > I'd like to propose dropping it. Or we can start a survey on user
> > list
> > > > > to see if there is any user depending on this class.
> > > > >
> > > > > best,
> > > > > tison.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > > >
> > > >
> > >
> >
>

tison

Re: [DISCUSS] Drop stale class Program

Hi,

With a one-week survey in user list[1], nobody expect Flavio and Jeff
participant the thread.

Flavio shared his experience with a revised Program like interface.
This could be regraded as downstream integration and in client api
enhancements document we propose rich interface for this integration.
Anyway, the Flink scope Program is less functional than it should be.

With no objection I'd like to push on this thread. We need a committer
participant this thread to shepherd the removal/deprecation of Program, a
@PublicEvolving interface. Anybody has spare time? Or anything I can do
to make progress?

Best,
tison.

[1]
https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E

Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：

> Hi,
>
> I created a thread for survey in user list[1]. Please take participate in
> if interested.
>
> Best,
> tison.
>
> [1]
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>
>
> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
>
>> +1 to remove directly the Program class (I think nobody use it and it's
>> not
>> supported at all by REST services and UI).
>> Moreover it requires a lot of transitive dependencies while it should be a
>> very simple thing..
>> +1 to add this discussion to "Flink client api enhancement"
>>
>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:
>>
>> > To Flavio, good point for the integration suggestion.
>> >
>> > I think it should be considered in the "Flink client api enhancement"
>> > discussion. But the outdated API should be deprecated somehow.
>> >
>> > Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
>> >
>> > > In my experience a basic "official" (but optional) program description
>> > > would be very useful indeed (in order to ease the integration with
>> other
>> > > frameworks).
>> > >
>> > > Of course it should be extended and integrated with the REST services
>> and
>> > > the Web UI (when defined) in order to be useful..
>> > > It ease to show to the user what a job does and which parameters it
>> > > requires (optional or mandatory) and with a proper help description.
>> > > Indeed, when we write a Flink job we implement the following
>> interface:
>> > >
>> > > public interface FlinkJob {
>> > > String getDescription();
>> > > List<FlinkJobParameter> getParameters();
>> > > boolean isStreamingOrBatch();
>> > > }
>> > >
>> > > public class ClusterJobParameter {
>> > >
>> > > private String paramName;
>> > > private String paramType = "string";
>> > > private String paramDesc;
>> > > private String paramDefaultValue;
>> > > private Set<String> choices;
>> > > private boolean mandatory;
>> > > }
>> > >
>> > > This really helps to launch a Flink job by a frontend (if the rest
>> > services
>> > > returns back those infos).
>> > >
>> > > Best,
>> > > Flavio
>> > >
>> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]> wrote:
>> > >
>> > > > Hi Zili,
>> > > >
>> > > > Thank you for bring us this discussion.
>> > > >
>> > > > My gut feeling is +1 for dropping it.
>> > > > Usually it costs some time to deprecate a public (actually it's
>> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>> > first.
>> > > > Then it might be abandoned it in some later version.
>> > > >
>> > > > I'm not sure how big the burden is to make it compatible with the
>> > > enhanced
>> > > > client API. If it's a critical blocker, I support dropping it
>> radically
>> > > in
>> > > > 1.10. Of course a survey is necessary. And the result of survey is
>> > > > acceptable.
>> > > >
>> > > >
>> > > >
>> > > > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
>> > > >
>> > > > > Hi devs,
>> > > > >
>> > > > > Participating the thread "Flink client api enhancement"[1], I just
>> > > notice
>> > > > > that inside submission codepath of Flink we always has a branch
>> > dealing
>> > > > > with the case that main class of user program is a subclass of
>> > > > > o.a.f.api.common.Program, which is defined as
>> > > > >
>> > > > > @PublicEvolving
>> > > > > public interface Program {
>> > > > > Plan getPhan(String... args);
>> > > > > }
>> > > > >
>> > > > > This class, as user-facing interface, asks user to implement
>> #getPlan
>> > > > > which return a almost Flink internal class. FLINK-10862[2] pointed
>> > out
>> > > > > this confusion.
>> > > > >
>> > > > > Since our codebase contains quite a bit code handling this stale
>> > class,
>> > > > > and also it obstructs the effort enhancing Flink cilent api,
>> > > > > I'd like to propose dropping it. Or we can start a survey on user
>> > list
>> > > > > to see if there is any user depending on this class.
>> > > > >
>> > > > > best,
>> > > > > tison.
>> > > > >
>> > > > > [1]
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
>> > > > >
>> > > >
>> > >
>> >
>>
>

Kostas Kloudas-4

Re: [DISCUSS] Drop stale class Program

Hi all,

It is nice to have this discussion.

I am totally up for removing the unused Program interface, as this will
simplify a lot of other code paths in the ClusterClient and elsewhere.

Also about the easier integration of Flink with other frameworks, there
is another discussion in the mailing list with exactly this topic:
[DISCUSS] Flink client api enhancement for downstream project

Cheers,
Kostas

On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <[hidden email]> wrote:

> Hi,
>
> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> participant the thread.
>
> Flavio shared his experience with a revised Program like interface.
> This could be regraded as downstream integration and in client api
> enhancements document we propose rich interface for this integration.
> Anyway, the Flink scope Program is less functional than it should be.
>
> With no objection I'd like to push on this thread. We need a committer
> participant this thread to shepherd the removal/deprecation of Program, a
> @PublicEvolving interface. Anybody has spare time? Or anything I can do
> to make progress?
>
> Best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>
>
> Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：
>
> > Hi,
> >
> > I created a thread for survey in user list[1]. Please take participate in
> > if interested.
> >
> > Best,
> > tison.
> >
> > [1]
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >
> >
> > Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
> >
> >> +1 to remove directly the Program class (I think nobody use it and it's
> >> not
> >> supported at all by REST services and UI).
> >> Moreover it requires a lot of transitive dependencies while it should
> be a
> >> very simple thing..
> >> +1 to add this discussion to "Flink client api enhancement"
> >>
> >> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:
> >>
> >> > To Flavio, good point for the integration suggestion.
> >> >
> >> > I think it should be considered in the "Flink client api enhancement"
> >> > discussion. But the outdated API should be deprecated somehow.
> >> >
> >> > Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
> >> >
> >> > > In my experience a basic "official" (but optional) program
> description
> >> > > would be very useful indeed (in order to ease the integration with
> >> other
> >> > > frameworks).
> >> > >
> >> > > Of course it should be extended and integrated with the REST
> services
> >> and
> >> > > the Web UI (when defined) in order to be useful..
> >> > > It ease to show to the user what a job does and which parameters it
> >> > > requires (optional or mandatory) and with a proper help description.
> >> > > Indeed, when we write a Flink job we implement the following
> >> interface:
> >> > >
> >> > > public interface FlinkJob {
> >> > > String getDescription();
> >> > > List<FlinkJobParameter> getParameters();
> >> > > boolean isStreamingOrBatch();
> >> > > }
> >> > >
> >> > > public class ClusterJobParameter {
> >> > >
> >> > > private String paramName;
> >> > > private String paramType = "string";
> >> > > private String paramDesc;
> >> > > private String paramDefaultValue;
> >> > > private Set<String> choices;
> >> > > private boolean mandatory;
> >> > > }
> >> > >
> >> > > This really helps to launch a Flink job by a frontend (if the rest
> >> > services
> >> > > returns back those infos).
> >> > >
> >> > > Best,
> >> > > Flavio
> >> > >
> >> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]>
> wrote:
> >> > >
> >> > > > Hi Zili,
> >> > > >
> >> > > > Thank you for bring us this discussion.
> >> > > >
> >> > > > My gut feeling is +1 for dropping it.
> >> > > > Usually it costs some time to deprecate a public (actually it's
> >> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> >> > first.
> >> > > > Then it might be abandoned it in some later version.
> >> > > >
> >> > > > I'm not sure how big the burden is to make it compatible with the
> >> > > enhanced
> >> > > > client API. If it's a critical blocker, I support dropping it
> >> radically
> >> > > in
> >> > > > 1.10. Of course a survey is necessary. And the result of survey is
> >> > > > acceptable.
> >> > > >
> >> > > >
> >> > > >
> >> > > > Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> >> > > >
> >> > > > > Hi devs,
> >> > > > >
> >> > > > > Participating the thread "Flink client api enhancement"[1], I
> just
> >> > > notice
> >> > > > > that inside submission codepath of Flink we always has a branch
> >> > dealing
> >> > > > > with the case that main class of user program is a subclass of
> >> > > > > o.a.f.api.common.Program, which is defined as
> >> > > > >
> >> > > > > @PublicEvolving
> >> > > > > public interface Program {
> >> > > > > Plan getPhan(String... args);
> >> > > > > }
> >> > > > >
> >> > > > > This class, as user-facing interface, asks user to implement
> >> #getPlan
> >> > > > > which return a almost Flink internal class. FLINK-10862[2]
> pointed
> >> > out
> >> > > > > this confusion.
> >> > > > >
> >> > > > > Since our codebase contains quite a bit code handling this stale
> >> > class,
> >> > > > > and also it obstructs the effort enhancing Flink cilent api,
> >> > > > > I'd like to propose dropping it. Or we can start a survey on
> user
> >> > list
> >> > > > > to see if there is any user depending on this class.
> >> > > > >
> >> > > > > best,
> >> > > > > tison.
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> >> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Aljoscha Krettek-2

Re: [DISCUSS] Drop stale class Program

Hi,

I would be in favour of removing Program (and the code paths that support it) for Flink 1.10. Most users of Flink don’t actually know it exists and it is only making our code more complicated. Going forward with the new Client API discussions will be a lot easier without it as well.

Best,
Aljoscha

> On 14. Aug 2019, at 11:08, Kostas Kloudas <[hidden email]> wrote:
>
> Hi all,
>
> It is nice to have this discussion.
>
> I am totally up for removing the unused Program interface, as this will
> simplify a lot of other code paths in the ClusterClient and elsewhere.
>
> Also about the easier integration of Flink with other frameworks, there
> is another discussion in the mailing list with exactly this topic:
> [DISCUSS] Flink client api enhancement for downstream project
>
> Cheers,
> Kostas
>
>
> On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <[hidden email]> wrote:
>
>> Hi,
>>
>> With a one-week survey in user list[1], nobody expect Flavio and Jeff
>> participant the thread.
>>
>> Flavio shared his experience with a revised Program like interface.
>> This could be regraded as downstream integration and in client api
>> enhancements document we propose rich interface for this integration.
>> Anyway, the Flink scope Program is less functional than it should be.
>>
>> With no objection I'd like to push on this thread. We need a committer
>> participant this thread to shepherd the removal/deprecation of Program, a
>> @PublicEvolving interface. Anybody has spare time? Or anything I can do
>> to make progress?
>>
>> Best,
>> tison.
>>
>> [1]
>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>
>>
>> Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：
>>
>>> Hi,
>>>
>>> I created a thread for survey in user list[1]. Please take participate in
>>> if interested.
>>>
>>> Best,
>>> tison.
>>>
>>> [1]
>>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>>
>>>
>>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
>>>
>>>> +1 to remove directly the Program class (I think nobody use it and it's
>>>> not
>>>> supported at all by REST services and UI).
>>>> Moreover it requires a lot of transitive dependencies while it should
>> be a
>>>> very simple thing..
>>>> +1 to add this discussion to "Flink client api enhancement"
>>>>
>>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:
>>>>
>>>>> To Flavio, good point for the integration suggestion.
>>>>>
>>>>> I think it should be considered in the "Flink client api enhancement"
>>>>> discussion. But the outdated API should be deprecated somehow.
>>>>>
>>>>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
>>>>>
>>>>>> In my experience a basic "official" (but optional) program
>> description
>>>>>> would be very useful indeed (in order to ease the integration with
>>>> other
>>>>>> frameworks).
>>>>>>
>>>>>> Of course it should be extended and integrated with the REST
>> services
>>>> and
>>>>>> the Web UI (when defined) in order to be useful..
>>>>>> It ease to show to the user what a job does and which parameters it
>>>>>> requires (optional or mandatory) and with a proper help description.
>>>>>> Indeed, when we write a Flink job we implement the following
>>>> interface:
>>>>>>
>>>>>> public interface FlinkJob {
>>>>>> String getDescription();
>>>>>> List<FlinkJobParameter> getParameters();
>>>>>> boolean isStreamingOrBatch();
>>>>>> }
>>>>>>
>>>>>> public class ClusterJobParameter {
>>>>>>
>>>>>> private String paramName;
>>>>>> private String paramType = "string";
>>>>>> private String paramDesc;
>>>>>> private String paramDefaultValue;
>>>>>> private Set<String> choices;
>>>>>> private boolean mandatory;
>>>>>> }
>>>>>>
>>>>>> This really helps to launch a Flink job by a frontend (if the rest
>>>>> services
>>>>>> returns back those infos).
>>>>>>
>>>>>> Best,
>>>>>> Flavio
>>>>>>
>>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]>
>> wrote:
>>>>>>
>>>>>>> Hi Zili,
>>>>>>>
>>>>>>> Thank you for bring us this discussion.
>>>>>>>
>>>>>>> My gut feeling is +1 for dropping it.
>>>>>>> Usually it costs some time to deprecate a public (actually it's
>>>>>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>>>>> first.
>>>>>>> Then it might be abandoned it in some later version.
>>>>>>>
>>>>>>> I'm not sure how big the burden is to make it compatible with the
>>>>>> enhanced
>>>>>>> client API. If it's a critical blocker, I support dropping it
>>>> radically
>>>>>> in
>>>>>>> 1.10. Of course a survey is necessary. And the result of survey is
>>>>>>> acceptable.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
>>>>>>>
>>>>>>>> Hi devs,
>>>>>>>>
>>>>>>>> Participating the thread "Flink client api enhancement"[1], I
>> just
>>>>>> notice
>>>>>>>> that inside submission codepath of Flink we always has a branch
>>>>> dealing
>>>>>>>> with the case that main class of user program is a subclass of
>>>>>>>> o.a.f.api.common.Program, which is defined as
>>>>>>>>
>>>>>>>> @PublicEvolving
>>>>>>>> public interface Program {
>>>>>>>> Plan getPhan(String... args);
>>>>>>>> }
>>>>>>>>
>>>>>>>> This class, as user-facing interface, asks user to implement
>>>> #getPlan
>>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
>> pointed
>>>>> out
>>>>>>>> this confusion.
>>>>>>>>
>>>>>>>> Since our codebase contains quite a bit code handling this stale
>>>>> class,
>>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
>>>>>>>> I'd like to propose dropping it. Or we can start a survey on
>> user
>>>>> list
>>>>>>>> to see if there is any user depending on this class.
>>>>>>>>
>>>>>>>> best,
>>>>>>>> tison.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>

Stephan Ewen

Re: [DISCUSS] Drop stale class Program

+1 to drop it.

It's one of the oldest pieces of legacy.

On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
>
> I would be in favour of removing Program (and the code paths that support
> it) for Flink 1.10. Most users of Flink don’t actually know it exists and
> it is only making our code more complicated. Going forward with the new
> Client API discussions will be a lot easier without it as well.
>
> Best,
> Aljoscha
>
> > On 14. Aug 2019, at 11:08, Kostas Kloudas <[hidden email]> wrote:
> >
> > Hi all,
> >
> > It is nice to have this discussion.
> >
> > I am totally up for removing the unused Program interface, as this will
> > simplify a lot of other code paths in the ClusterClient and elsewhere.
> >
> > Also about the easier integration of Flink with other frameworks, there
> > is another discussion in the mailing list with exactly this topic:
> > [DISCUSS] Flink client api enhancement for downstream project
> >
> > Cheers,
> > Kostas
> >
> >
> > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <[hidden email]> wrote:
> >
> >> Hi,
> >>
> >> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> >> participant the thread.
> >>
> >> Flavio shared his experience with a revised Program like interface.
> >> This could be regraded as downstream integration and in client api
> >> enhancements document we propose rich interface for this integration.
> >> Anyway, the Flink scope Program is less functional than it should be.
> >>
> >> With no objection I'd like to push on this thread. We need a committer
> >> participant this thread to shepherd the removal/deprecation of Program,
> a
> >> @PublicEvolving interface. Anybody has spare time? Or anything I can do
> >> to make progress?
> >>
> >> Best,
> >> tison.
> >>
> >> [1]
> >>
> >>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >>
> >>
> >> Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：
> >>
> >>> Hi,
> >>>
> >>> I created a thread for survey in user list[1]. Please take participate
> in
> >>> if interested.
> >>>
> >>> Best,
> >>> tison.
> >>>
> >>> [1]
> >>>
> >>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >>>
> >>>
> >>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
> >>>
> >>>> +1 to remove directly the Program class (I think nobody use it and
> it's
> >>>> not
> >>>> supported at all by REST services and UI).
> >>>> Moreover it requires a lot of transitive dependencies while it should
> >> be a
> >>>> very simple thing..
> >>>> +1 to add this discussion to "Flink client api enhancement"
> >>>>
> >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]> wrote:
> >>>>
> >>>>> To Flavio, good point for the integration suggestion.
> >>>>>
> >>>>> I think it should be considered in the "Flink client api enhancement"
> >>>>> discussion. But the outdated API should be deprecated somehow.
> >>>>>
> >>>>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
> >>>>>
> >>>>>> In my experience a basic "official" (but optional) program
> >> description
> >>>>>> would be very useful indeed (in order to ease the integration with
> >>>> other
> >>>>>> frameworks).
> >>>>>>
> >>>>>> Of course it should be extended and integrated with the REST
> >> services
> >>>> and
> >>>>>> the Web UI (when defined) in order to be useful..
> >>>>>> It ease to show to the user what a job does and which parameters it
> >>>>>> requires (optional or mandatory) and with a proper help description.
> >>>>>> Indeed, when we write a Flink job we implement the following
> >>>> interface:
> >>>>>>
> >>>>>> public interface FlinkJob {
> >>>>>> String getDescription();
> >>>>>> List<FlinkJobParameter> getParameters();
> >>>>>> boolean isStreamingOrBatch();
> >>>>>> }
> >>>>>>
> >>>>>> public class ClusterJobParameter {
> >>>>>>
> >>>>>> private String paramName;
> >>>>>> private String paramType = "string";
> >>>>>> private String paramDesc;
> >>>>>> private String paramDefaultValue;
> >>>>>> private Set<String> choices;
> >>>>>> private boolean mandatory;
> >>>>>> }
> >>>>>>
> >>>>>> This really helps to launch a Flink job by a frontend (if the rest
> >>>>> services
> >>>>>> returns back those infos).
> >>>>>>
> >>>>>> Best,
> >>>>>> Flavio
> >>>>>>
> >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]>
> >> wrote:
> >>>>>>
> >>>>>>> Hi Zili,
> >>>>>>>
> >>>>>>> Thank you for bring us this discussion.
> >>>>>>>
> >>>>>>> My gut feeling is +1 for dropping it.
> >>>>>>> Usually it costs some time to deprecate a public (actually it's
> >>>>>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> >>>>> first.
> >>>>>>> Then it might be abandoned it in some later version.
> >>>>>>>
> >>>>>>> I'm not sure how big the burden is to make it compatible with the
> >>>>>> enhanced
> >>>>>>> client API. If it's a critical blocker, I support dropping it
> >>>> radically
> >>>>>> in
> >>>>>>> 1.10. Of course a survey is necessary. And the result of survey is
> >>>>>>> acceptable.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> >>>>>>>
> >>>>>>>> Hi devs,
> >>>>>>>>
> >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> >> just
> >>>>>> notice
> >>>>>>>> that inside submission codepath of Flink we always has a branch
> >>>>> dealing
> >>>>>>>> with the case that main class of user program is a subclass of
> >>>>>>>> o.a.f.api.common.Program, which is defined as
> >>>>>>>>
> >>>>>>>> @PublicEvolving
> >>>>>>>> public interface Program {
> >>>>>>>> Plan getPhan(String... args);
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> This class, as user-facing interface, asks user to implement
> >>>> #getPlan
> >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> >> pointed
> >>>>> out
> >>>>>>>> this confusion.
> >>>>>>>>
> >>>>>>>> Since our codebase contains quite a bit code handling this stale
> >>>>> class,
> >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> >> user
> >>>>> list
> >>>>>>>> to see if there is any user depending on this class.
> >>>>>>>>
> >>>>>>>> best,
> >>>>>>>> tison.
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Kostas Kloudas-4

Re: [DISCUSS] Drop stale class Program

I already opened a JIRA for the removal and I will also create a (short)
FLIP, as it is a PublicEvolving interface and its removal should go through
a FLIP.

The JIRA can be found here https://issues.apache.org/jira/browse/FLINK-13713

Cheers,
Kostas

On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen <[hidden email]> wrote:

> +1 to drop it.
>
> It's one of the oldest pieces of legacy.
>
> On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I would be in favour of removing Program (and the code paths that support
> > it) for Flink 1.10. Most users of Flink don’t actually know it exists and
> > it is only making our code more complicated. Going forward with the new
> > Client API discussions will be a lot easier without it as well.
> >
> > Best,
> > Aljoscha
> >
> > > On 14. Aug 2019, at 11:08, Kostas Kloudas <[hidden email]> wrote:
> > >
> > > Hi all,
> > >
> > > It is nice to have this discussion.
> > >
> > > I am totally up for removing the unused Program interface, as this will
> > > simplify a lot of other code paths in the ClusterClient and elsewhere.
> > >
> > > Also about the easier integration of Flink with other frameworks, there
> > > is another discussion in the mailing list with exactly this topic:
> > > [DISCUSS] Flink client api enhancement for downstream project
> > >
> > > Cheers,
> > > Kostas
> > >
> > >
> > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <[hidden email]>
> wrote:
> > >
> > >> Hi,
> > >>
> > >> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> > >> participant the thread.
> > >>
> > >> Flavio shared his experience with a revised Program like interface.
> > >> This could be regraded as downstream integration and in client api
> > >> enhancements document we propose rich interface for this integration.
> > >> Anyway, the Flink scope Program is less functional than it should be.
> > >>
> > >> With no objection I'd like to push on this thread. We need a committer
> > >> participant this thread to shepherd the removal/deprecation of
> Program,
> > a
> > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> do
> > >> to make progress?
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>
> > >>
> > >> Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：
> > >>
> > >>> Hi,
> > >>>
> > >>> I created a thread for survey in user list[1]. Please take
> participate
> > in
> > >>> if interested.
> > >>>
> > >>> Best,
> > >>> tison.
> > >>>
> > >>> [1]
> > >>>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>>
> > >>>
> > >>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
> > >>>
> > >>>> +1 to remove directly the Program class (I think nobody use it and
> > it's
> > >>>> not
> > >>>> supported at all by REST services and UI).
> > >>>> Moreover it requires a lot of transitive dependencies while it
> should
> > >> be a
> > >>>> very simple thing..
> > >>>> +1 to add this discussion to "Flink client api enhancement"
> > >>>>
> > >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]>
> wrote:
> > >>>>
> > >>>>> To Flavio, good point for the integration suggestion.
> > >>>>>
> > >>>>> I think it should be considered in the "Flink client api
> enhancement"
> > >>>>> discussion. But the outdated API should be deprecated somehow.
> > >>>>>
> > >>>>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午4:21写道：
> > >>>>>
> > >>>>>> In my experience a basic "official" (but optional) program
> > >> description
> > >>>>>> would be very useful indeed (in order to ease the integration with
> > >>>> other
> > >>>>>> frameworks).
> > >>>>>>
> > >>>>>> Of course it should be extended and integrated with the REST
> > >> services
> > >>>> and
> > >>>>>> the Web UI (when defined) in order to be useful..
> > >>>>>> It ease to show to the user what a job does and which parameters
> it
> > >>>>>> requires (optional or mandatory) and with a proper help
> description.
> > >>>>>> Indeed, when we write a Flink job we implement the following
> > >>>> interface:
> > >>>>>>
> > >>>>>> public interface FlinkJob {
> > >>>>>> String getDescription();
> > >>>>>> List<FlinkJobParameter> getParameters();
> > >>>>>> boolean isStreamingOrBatch();
> > >>>>>> }
> > >>>>>>
> > >>>>>> public class ClusterJobParameter {
> > >>>>>>
> > >>>>>> private String paramName;
> > >>>>>> private String paramType = "string";
> > >>>>>> private String paramDesc;
> > >>>>>> private String paramDefaultValue;
> > >>>>>> private Set<String> choices;
> > >>>>>> private boolean mandatory;
> > >>>>>> }
> > >>>>>>
> > >>>>>> This really helps to launch a Flink job by a frontend (if the rest
> > >>>>> services
> > >>>>>> returns back those infos).
> > >>>>>>
> > >>>>>> Best,
> > >>>>>> Flavio
> > >>>>>>
> > >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]>
> > >> wrote:
> > >>>>>>
> > >>>>>>> Hi Zili,
> > >>>>>>>
> > >>>>>>> Thank you for bring us this discussion.
> > >>>>>>>
> > >>>>>>> My gut feeling is +1 for dropping it.
> > >>>>>>> Usually it costs some time to deprecate a public (actually it's
> > >>>>>>> `PublicEvolving`) API. Ideally it should be marked as
> `Deprecated`
> > >>>>> first.
> > >>>>>>> Then it might be abandoned it in some later version.
> > >>>>>>>
> > >>>>>>> I'm not sure how big the burden is to make it compatible with the
> > >>>>>> enhanced
> > >>>>>>> client API. If it's a critical blocker, I support dropping it
> > >>>> radically
> > >>>>>> in
> > >>>>>>> 1.10. Of course a survey is necessary. And the result of survey
> is
> > >>>>>>> acceptable.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> > >>>>>>>
> > >>>>>>>> Hi devs,
> > >>>>>>>>
> > >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> > >> just
> > >>>>>> notice
> > >>>>>>>> that inside submission codepath of Flink we always has a branch
> > >>>>> dealing
> > >>>>>>>> with the case that main class of user program is a subclass of
> > >>>>>>>> o.a.f.api.common.Program, which is defined as
> > >>>>>>>>
> > >>>>>>>> @PublicEvolving
> > >>>>>>>> public interface Program {
> > >>>>>>>> Plan getPhan(String... args);
> > >>>>>>>> }
> > >>>>>>>>
> > >>>>>>>> This class, as user-facing interface, asks user to implement
> > >>>> #getPlan
> > >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> > >> pointed
> > >>>>> out
> > >>>>>>>> this confusion.
> > >>>>>>>>
> > >>>>>>>> Since our codebase contains quite a bit code handling this stale
> > >>>>> class,
> > >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> > >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> > >> user
> > >>>>> list
> > >>>>>>>> to see if there is any user depending on this class.
> > >>>>>>>>
> > >>>>>>>> best,
> > >>>>>>>> tison.
> > >>>>>>>>
> > >>>>>>>> [1]
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

tison

Re: [DISCUSS] Drop stale class Program

Thanks for your attentions!

Thank Kostas for creating the JIRA and drafting the FLIP.
I would volunteer to help review it :-)

It's good to see that we make progress on this thread.

Best,
tison.

Kostas Kloudas <[hidden email]> 于2019年8月14日周三下午6:39写道：

> I already opened a JIRA for the removal and I will also create a (short)
> FLIP, as it is a PublicEvolving interface and its removal should go through
> a FLIP.
>
> The JIRA can be found here
> https://issues.apache.org/jira/browse/FLINK-13713
>
> Cheers,
> Kostas
>
> On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen <[hidden email]> wrote:
>
> > +1 to drop it.
> >
> > It's one of the oldest pieces of legacy.
> >
> > On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> > > Hi,
> > >
> > > I would be in favour of removing Program (and the code paths that
> support
> > > it) for Flink 1.10. Most users of Flink don’t actually know it exists
> and
> > > it is only making our code more complicated. Going forward with the new
> > > Client API discussions will be a lot easier without it as well.
> > >
> > > Best,
> > > Aljoscha
> > >
> > > > On 14. Aug 2019, at 11:08, Kostas Kloudas <[hidden email]>
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > It is nice to have this discussion.
> > > >
> > > > I am totally up for removing the unused Program interface, as this
> will
> > > > simplify a lot of other code paths in the ClusterClient and
> elsewhere.
> > > >
> > > > Also about the easier integration of Flink with other frameworks,
> there
> > > > is another discussion in the mailing list with exactly this topic:
> > > > [DISCUSS] Flink client api enhancement for downstream project
> > > >
> > > > Cheers,
> > > > Kostas
> > > >
> > > >
> > > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <[hidden email]>
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> With a one-week survey in user list[1], nobody expect Flavio and
> Jeff
> > > >> participant the thread.
> > > >>
> > > >> Flavio shared his experience with a revised Program like interface.
> > > >> This could be regraded as downstream integration and in client api
> > > >> enhancements document we propose rich interface for this
> integration.
> > > >> Anyway, the Flink scope Program is less functional than it should
> be.
> > > >>
> > > >> With no objection I'd like to push on this thread. We need a
> committer
> > > >> participant this thread to shepherd the removal/deprecation of
> > Program,
> > > a
> > > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> > do
> > > >> to make progress?
> > > >>
> > > >> Best,
> > > >> tison.
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>
> > > >>
> > > >> Zili Chen <[hidden email]> 于2019年7月22日周一下午8:38写道：
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> I created a thread for survey in user list[1]. Please take
> > participate
> > > in
> > > >>> if interested.
> > > >>>
> > > >>> Best,
> > > >>> tison.
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>>
> > > >>>
> > > >>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五下午5:18写道：
> > > >>>
> > > >>>> +1 to remove directly the Program class (I think nobody use it and
> > > it's
> > > >>>> not
> > > >>>> supported at all by REST services and UI).
> > > >>>> Moreover it requires a lot of transitive dependencies while it
> > should
> > > >> be a
> > > >>>> very simple thing..
> > > >>>> +1 to add this discussion to "Flink client api enhancement"
> > > >>>>
> > > >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <[hidden email]>
> > wrote:
> > > >>>>
> > > >>>>> To Flavio, good point for the integration suggestion.
> > > >>>>>
> > > >>>>> I think it should be considered in the "Flink client api
> > enhancement"
> > > >>>>> discussion. But the outdated API should be deprecated somehow.
> > > >>>>>
> > > >>>>> Flavio Pompermaier <[hidden email]> 于2019年7月19日周五
> 下午4:21写道：
> > > >>>>>
> > > >>>>>> In my experience a basic "official" (but optional) program
> > > >> description
> > > >>>>>> would be very useful indeed (in order to ease the integration
> with
> > > >>>> other
> > > >>>>>> frameworks).
> > > >>>>>>
> > > >>>>>> Of course it should be extended and integrated with the REST
> > > >> services
> > > >>>> and
> > > >>>>>> the Web UI (when defined) in order to be useful..
> > > >>>>>> It ease to show to the user what a job does and which parameters
> > it
> > > >>>>>> requires (optional or mandatory) and with a proper help
> > description.
> > > >>>>>> Indeed, when we write a Flink job we implement the following
> > > >>>> interface:
> > > >>>>>>
> > > >>>>>> public interface FlinkJob {
> > > >>>>>> String getDescription();
> > > >>>>>> List<FlinkJobParameter> getParameters();
> > > >>>>>> boolean isStreamingOrBatch();
> > > >>>>>> }
> > > >>>>>>
> > > >>>>>> public class ClusterJobParameter {
> > > >>>>>>
> > > >>>>>> private String paramName;
> > > >>>>>> private String paramType = "string";
> > > >>>>>> private String paramDesc;
> > > >>>>>> private String paramDefaultValue;
> > > >>>>>> private Set<String> choices;
> > > >>>>>> private boolean mandatory;
> > > >>>>>> }
> > > >>>>>>
> > > >>>>>> This really helps to launch a Flink job by a frontend (if the
> rest
> > > >>>>> services
> > > >>>>>> returns back those infos).
> > > >>>>>>
> > > >>>>>> Best,
> > > >>>>>> Flavio
> > > >>>>>>
> > > >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <[hidden email]>
> > > >> wrote:
> > > >>>>>>
> > > >>>>>>> Hi Zili,
> > > >>>>>>>
> > > >>>>>>> Thank you for bring us this discussion.
> > > >>>>>>>
> > > >>>>>>> My gut feeling is +1 for dropping it.
> > > >>>>>>> Usually it costs some time to deprecate a public (actually it's
> > > >>>>>>> `PublicEvolving`) API. Ideally it should be marked as
> > `Deprecated`
> > > >>>>> first.
> > > >>>>>>> Then it might be abandoned it in some later version.
> > > >>>>>>>
> > > >>>>>>> I'm not sure how big the burden is to make it compatible with
> the
> > > >>>>>> enhanced
> > > >>>>>>> client API. If it's a critical blocker, I support dropping it
> > > >>>> radically
> > > >>>>>> in
> > > >>>>>>> 1.10. Of course a survey is necessary. And the result of survey
> > is
> > > >>>>>>> acceptable.
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> Zili Chen <[hidden email]> 于2019年7月19日周五下午1:44写道：
> > > >>>>>>>
> > > >>>>>>>> Hi devs,
> > > >>>>>>>>
> > > >>>>>>>> Participating the thread "Flink client api enhancement"[1], I
> > > >> just
> > > >>>>>> notice
> > > >>>>>>>> that inside submission codepath of Flink we always has a
> branch
> > > >>>>> dealing
> > > >>>>>>>> with the case that main class of user program is a subclass of
> > > >>>>>>>> o.a.f.api.common.Program, which is defined as
> > > >>>>>>>>
> > > >>>>>>>> @PublicEvolving
> > > >>>>>>>> public interface Program {
> > > >>>>>>>> Plan getPhan(String... args);
> > > >>>>>>>> }
> > > >>>>>>>>
> > > >>>>>>>> This class, as user-facing interface, asks user to implement
> > > >>>> #getPlan
> > > >>>>>>>> which return a almost Flink internal class. FLINK-10862[2]
> > > >> pointed
> > > >>>>> out
> > > >>>>>>>> this confusion.
> > > >>>>>>>>
> > > >>>>>>>> Since our codebase contains quite a bit code handling this
> stale
> > > >>>>> class,
> > > >>>>>>>> and also it obstructs the effort enhancing Flink cilent api,
> > > >>>>>>>> I'd like to propose dropping it. Or we can start a survey on
> > > >> user
> > > >>>>> list
> > > >>>>>>>> to see if there is any user depending on this class.
> > > >>>>>>>>
> > > >>>>>>>> best,
> > > >>>>>>>> tison.
> > > >>>>>>>>
> > > >>>>>>>> [1]
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>