DataStream API - What does "Beta" mean?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

DataStream API - What does "Beta" mean?

ankurcha
Hi,

This is a more of a semantics / scope question. What does it mean by the "beta" status?

Main questions are around:

* API stability - How stable is the API?
* Regression / testing coverage?
* Feature completeness?
* Production readiness?

Also, a pointer to the relevant roadmaps and/or planning documents would be very helpful.

-- Ankur Chauhan
Reply | Threaded
Open this post in threaded view
|

Re: DataStream API - What does "Beta" mean?

Stephan Ewen
Hi!

The beta status refers exactly to API stability and test coverage.

Full status APIs, like the batch API, are very stable. For example, we have
not made any breaking changes in the last two releases (except for rare
corner case flags). The test coverage is also pretty good.

Beta APIs (like streaming, or ML) may undergo some breaking changes. The
streaming API is currently finalized and will most likely go out of beta
(to full status) in the next release in some weeks. Then the API will not
be broken again as well.


Feature completeness is a bit of a vague question. I think all core
features are implemented (features you need for production use), in that
sense it is feature complete. Still, every release adds new features to
almost every API, be that convenience features or new constructs for
efficiency, useful semantics, ...


Concerning production readiness, people run this in production (both batch
and streaming).


Greetings,
Stephan


On Thu, Aug 20, 2015 at 8:16 AM, ankurcha <[hidden email]> wrote:

> Hi,
>
> This is a more of a semantics / scope question. What does it mean by the
> "beta" status?
>
> Main questions are around:
>
> * API stability - How stable is the API?
> * Regression / testing coverage?
> * Feature completeness?
> * Production readiness?
>
> Also, a pointer to the relevant roadmaps and/or planning documents would be
> very helpful.
>
> -- Ankur Chauhan
>
>
>
> --
> View this message in context:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DataStream-API-What-does-Beta-mean-tp7582.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: DataStream API - What does "Beta" mean?

Kostas Tzoumas-2
Stephan was very fast :-)

I will just add that production readiness has a lot to do with the runtime
code, which is largely shared by the DataSet and the Data Stream API. The
beta status does not refer to production-readiness, but rather the ability
to still make breaking changes.

And the roadmap is to graduate DataStream out of beta at the next (0.10)
release and maintain backwards compatibility after that (minus corner cases
as Stephan said).

Kostas

On Thu, Aug 20, 2015 at 9:59 AM, Stephan Ewen <[hidden email]> wrote:

> Hi!
>
> The beta status refers exactly to API stability and test coverage.
>
> Full status APIs, like the batch API, are very stable. For example, we have
> not made any breaking changes in the last two releases (except for rare
> corner case flags). The test coverage is also pretty good.
>
> Beta APIs (like streaming, or ML) may undergo some breaking changes. The
> streaming API is currently finalized and will most likely go out of beta
> (to full status) in the next release in some weeks. Then the API will not
> be broken again as well.
>
>
> Feature completeness is a bit of a vague question. I think all core
> features are implemented (features you need for production use), in that
> sense it is feature complete. Still, every release adds new features to
> almost every API, be that convenience features or new constructs for
> efficiency, useful semantics, ...
>
>
> Concerning production readiness, people run this in production (both batch
> and streaming).
>
>
> Greetings,
> Stephan
>
>
> On Thu, Aug 20, 2015 at 8:16 AM, ankurcha <[hidden email]> wrote:
>
> > Hi,
> >
> > This is a more of a semantics / scope question. What does it mean by the
> > "beta" status?
> >
> > Main questions are around:
> >
> > * API stability - How stable is the API?
> > * Regression / testing coverage?
> > * Feature completeness?
> > * Production readiness?
> >
> > Also, a pointer to the relevant roadmaps and/or planning documents would
> be
> > very helpful.
> >
> > -- Ankur Chauhan
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DataStream-API-What-does-Beta-mean-tp7582.html
> > Sent from the Apache Flink Mailing List archive. mailing list archive at
> > Nabble.com.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: DataStream API - What does "Beta" mean?

Aljoscha Krettek-2
Hi,
on the wiki (
https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home) we
have design documents for how we want to bring the streaming API into the
final shape for the 0.10 release.

The changes will change the names of some of the streaming API methods (and
classes) and introduce some restrictions to make parallel processing
(especially of windows) more fool-proof. The overall "feel" of the API will
not change however. It will still be a functional API where you transform
distributed collections by using functions. Most of the examples and
written code should already work out of box.

Please let us know if you have any input on that. Suggestions/comments are
always welcome. :D

Cheers,
Aljoscha

On Thu, 20 Aug 2015 at 10:04 Kostas Tzoumas <[hidden email]> wrote:

> Stephan was very fast :-)
>
> I will just add that production readiness has a lot to do with the runtime
> code, which is largely shared by the DataSet and the Data Stream API. The
> beta status does not refer to production-readiness, but rather the ability
> to still make breaking changes.
>
> And the roadmap is to graduate DataStream out of beta at the next (0.10)
> release and maintain backwards compatibility after that (minus corner cases
> as Stephan said).
>
> Kostas
>
> On Thu, Aug 20, 2015 at 9:59 AM, Stephan Ewen <[hidden email]> wrote:
>
> > Hi!
> >
> > The beta status refers exactly to API stability and test coverage.
> >
> > Full status APIs, like the batch API, are very stable. For example, we
> have
> > not made any breaking changes in the last two releases (except for rare
> > corner case flags). The test coverage is also pretty good.
> >
> > Beta APIs (like streaming, or ML) may undergo some breaking changes. The
> > streaming API is currently finalized and will most likely go out of beta
> > (to full status) in the next release in some weeks. Then the API will not
> > be broken again as well.
> >
> >
> > Feature completeness is a bit of a vague question. I think all core
> > features are implemented (features you need for production use), in that
> > sense it is feature complete. Still, every release adds new features to
> > almost every API, be that convenience features or new constructs for
> > efficiency, useful semantics, ...
> >
> >
> > Concerning production readiness, people run this in production (both
> batch
> > and streaming).
> >
> >
> > Greetings,
> > Stephan
> >
> >
> > On Thu, Aug 20, 2015 at 8:16 AM, ankurcha <[hidden email]> wrote:
> >
> > > Hi,
> > >
> > > This is a more of a semantics / scope question. What does it mean by
> the
> > > "beta" status?
> > >
> > > Main questions are around:
> > >
> > > * API stability - How stable is the API?
> > > * Regression / testing coverage?
> > > * Feature completeness?
> > > * Production readiness?
> > >
> > > Also, a pointer to the relevant roadmaps and/or planning documents
> would
> > be
> > > very helpful.
> > >
> > > -- Ankur Chauhan
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DataStream-API-What-does-Beta-mean-tp7582.html
> > > Sent from the Apache Flink Mailing List archive. mailing list archive
> at
> > > Nabble.com.
> > >
> >
>