Flink Low level APIs How To’s

[Ramesh Byndoor]

Team,

would like to create jobs from DAG's with details which is maintained
externally(Filters, transform and Stream SQL). Idea is to create JobGraph
and submit to Flink. I have started looking into JobGraph, Job, Vertex,
JobGenerator from flink low level API's, Looks promising but is there any
guidelines documented or any better practices for the same. Are these used
by any in the community.?

[Till Rohrmann]

Hi Ramesh,

the JobGraph, JobVertex, JobGraphGenerator, etc. are all internal classes
for which the community does not give any stability guarantees. Thus, they
can change from one version to another breaking all your programs.
Therefore, I would not recommend using these primitives if you are not
willing to resolve potential version upgrade problems.

Apart from that, there is only little documentation for these components.
The best documentation is actually the code.

Cheers,
Till

On Tue, Oct 2, 2018 at 3:21 PM Ramesh Byndoor <[hidden email]> wrote:

> Team,
>
> would like to create jobs from DAG's with details which is maintained
> externally(Filters, transform and Stream SQL). Idea is to create JobGraph
> and submit to Flink. I have started looking into JobGraph, Job, Vertex,
> JobGenerator from flink low level API's, Looks promising but is there any
> guidelines documented or any better practices for the same. Are these used
> by any in the community.?
>

[tison]

Hi Ramesh,

Apart from Till recommended, internal classes go with document around the
source code define them,
and for the "practices" part, maybe you can take a look at their tests.

But after all, these are all about Flink dev topics and so unstable to be
used in production codes.

Best,
tison.


Till Rohrmann <[hidden email]> 于2018年10月2日周二 下午9:51写道：

> Hi Ramesh,
>
> the JobGraph, JobVertex, JobGraphGenerator, etc. are all internal classes
> for which the community does not give any stability guarantees. Thus, they
> can change from one version to another breaking all your programs.
> Therefore, I would not recommend using these primitives if you are not
> willing to resolve potential version upgrade problems.
>
> Apart from that, there is only little documentation for these components.
> The best documentation is actually the code.
>
> Cheers,
> Till
>
> On Tue, Oct 2, 2018 at 3:21 PM Ramesh Byndoor <[hidden email]>
> wrote:
>
> > Team,
> >
> > would like to create jobs from DAG's with details which is maintained
> > externally(Filters, transform and Stream SQL). Idea is to create JobGraph
> > and submit to Flink. I have started looking into JobGraph, Job, Vertex,
> > JobGenerator from flink low level API's, Looks promising but is there any
> > guidelines documented or any better practices for the same. Are these
> used
> > by any in the community.?
> >
>