[DISCUSS] @Public libraries

[Greg Hogan]

Hi all,

Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated
@Public or restricted to use of @PublicEvolving?

We would ensure that library APIs do not add restrictions to the core APIs.
Libraries could use @PublicEvolving or @Internal core APIs within @Public
or @PublicEvolving components as long as the functionality could be adapted
or rewritten as necessary. An example: CombineHint is @Internal. Gelly
could use CombineHint in a @Public method but could not accept CombineHint
as a parameter to a @Public method.

Greg

[Aljoscha Krettek-2]

I would be for also annotating library methods/classes. Maybe Robert has a
stronger opinion on this because he introduced these annotations.

On Tue, 22 Nov 2016 at 18:56 Greg Hogan <[hidden email]> wrote:

> Hi all,
>
> Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated
> @Public or restricted to use of @PublicEvolving?
>
> We would ensure that library APIs do not add restrictions to the core APIs.
> Libraries could use @PublicEvolving or @Internal core APIs within @Public
> or @PublicEvolving components as long as the functionality could be adapted
> or rewritten as necessary. An example: CombineHint is @Internal. Gelly
> could use CombineHint in a @Public method but could not accept CombineHint
> as a parameter to a @Public method.
>
> Greg
>

[Till Rohrmann]

I think in general annotating library methods/classes is a good idea. The
question is just which APIs are going to be marked stable.

In the past we've seen that we might have marked some of Flink's APIs
stable too early. As a consequence we have to carry them along for quite
some time (at the very least until Flink 2.0).

If we now come to the conclusion that the library APIs are not yet stable
enough to mark them Public, then we will probably mark a lot of the API
PublicEvolving which adds only little value for the user.

Cheers,
Till

On Wed, Nov 23, 2016 at 2:25 PM, Aljoscha Krettek <[hidden email]>
wrote:

> I would be for also annotating library methods/classes. Maybe Robert has a
> stronger opinion on this because he introduced these annotations.
>
> On Tue, 22 Nov 2016 at 18:56 Greg Hogan <[hidden email]> wrote:
>
> > Hi all,
> >
> > Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated
> > @Public or restricted to use of @PublicEvolving?
> >
> > We would ensure that library APIs do not add restrictions to the core
> APIs.
> > Libraries could use @PublicEvolving or @Internal core APIs within @Public
> > or @PublicEvolving components as long as the functionality could be
> adapted
> > or rewritten as necessary. An example: CombineHint is @Internal. Gelly
> > could use CombineHint in a @Public method but could not accept
> CombineHint
> > as a parameter to a @Public method.
> >
> > Greg
> >
>

[Theodore Vasiloudis]

What Till said is true for FlinkML, until all the moving parts are in place
there's not much point in annotating any as Public. The Spark project has
the @Experimental tag IIRC, that would fit our case better.

On Wed, Nov 23, 2016 at 4:09 PM, Till Rohrmann <[hidden email]> wrote:

> I think in general annotating library methods/classes is a good idea. The
> question is just which APIs are going to be marked stable.
>
> In the past we've seen that we might have marked some of Flink's APIs
> stable too early. As a consequence we have to carry them along for quite
> some time (at the very least until Flink 2.0).
>
> If we now come to the conclusion that the library APIs are not yet stable
> enough to mark them Public, then we will probably mark a lot of the API
> PublicEvolving which adds only little value for the user.
>
> Cheers,
> Till
>
> On Wed, Nov 23, 2016 at 2:25 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > I would be for also annotating library methods/classes. Maybe Robert has
> a
> > stronger opinion on this because he introduced these annotations.
> >
> > On Tue, 22 Nov 2016 at 18:56 Greg Hogan <[hidden email]> wrote:
> >
> > > Hi all,
> > >
> > > Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated
> > > @Public or restricted to use of @PublicEvolving?
> > >
> > > We would ensure that library APIs do not add restrictions to the core
> > APIs.
> > > Libraries could use @PublicEvolving or @Internal core APIs within
> @Public
> > > or @PublicEvolving components as long as the functionality could be
> > adapted
> > > or rewritten as necessary. An example: CombineHint is @Internal. Gelly
> > > could use CombineHint in a @Public method but could not accept
> > CombineHint
> > > as a parameter to a @Public method.
> > >
> > > Greg
> > >
> >
>

[Robert Metzger]

In the initial discussion / proposal for interface annotations we decided
to annotate only the very core APIs with @Public and give the libraries
more freedom to evolve over time.
I think we should not have a general rule for the libraries and decide this
case-by-case. If you feel that Gelly is mature enough that methods become
@Public, we can do that. For CEP and ML, I have the feeling that bigger
refactorings might take place, so I would not annotate.

Maybe a good rule of thumb for making an interface as @Public is when users
are complaining that it broke between releases.
If a library doesn't have many users yet, there is no point in restricting
yourself too early.

The @PublicEvolving annotation was mainly added to be used inside a class
marked @Public.
If a class is not annotated at all, it is not considered stable.
However, sometimes you want to mark something within a @Public class as
@Internal or @PublicEvolving (=might change in the future, but is
considered for public use for now).
So marking a class as @PublicEvolving doesn't do anything for the "japicmp"
tool.






On Wed, Nov 23, 2016 at 5:21 PM, Theodore Vasiloudis <
[hidden email]> wrote:

> What Till said is true for FlinkML, until all the moving parts are in place
> there's not much point in annotating any as Public. The Spark project has
> the @Experimental tag IIRC, that would fit our case better.
>
> On Wed, Nov 23, 2016 at 4:09 PM, Till Rohrmann <[hidden email]>
> wrote:
>
> > I think in general annotating library methods/classes is a good idea. The
> > question is just which APIs are going to be marked stable.
> >
> > In the past we've seen that we might have marked some of Flink's APIs
> > stable too early. As a consequence we have to carry them along for quite
> > some time (at the very least until Flink 2.0).
> >
> > If we now come to the conclusion that the library APIs are not yet stable
> > enough to mark them Public, then we will probably mark a lot of the API
> > PublicEvolving which adds only little value for the user.
> >
> > Cheers,
> > Till
> >
> > On Wed, Nov 23, 2016 at 2:25 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> > > I would be for also annotating library methods/classes. Maybe Robert
> has
> > a
> > > stronger opinion on this because he introduced these annotations.
> > >
> > > On Tue, 22 Nov 2016 at 18:56 Greg Hogan <[hidden email]> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Should stable APIs in Flink's CEP, ML, and Gelly libraries be
> annotated
> > > > @Public or restricted to use of @PublicEvolving?
> > > >
> > > > We would ensure that library APIs do not add restrictions to the core
> > > APIs.
> > > > Libraries could use @PublicEvolving or @Internal core APIs within
> > @Public
> > > > or @PublicEvolving components as long as the functionality could be
> > > adapted
> > > > or rewritten as necessary. An example: CombineHint is @Internal.
> Gelly
> > > > could use CombineHint in a @Public method but could not accept
> > > CombineHint
> > > > as a parameter to a @Public method.
> > > >
> > > > Greg
> > > >
> > >
> >
>