Planning Flink release 0.7-incubating

[Fabian Hueske]

Done with reverting and merging the individual commits.

Sorry, didn't wanted to sound too negative. I just think merged commits are
an important incentive for contributors.
And thanks for merging of course ;-)

2014-10-08 12:25 GMT+02:00 Robert Metzger <[hidden email]>:

> Oh, damn. You're right. I've been using the "merge_pull_request.sh" script
> which is squashing together the commits automatically. In this particular
> case, its quite unfortunate because it contains part of the work of Artems
> GSoC summer project, and also another pull request from Timo.
>
> I agree if you revert and push the commits individually in this case.
>
> On Wed, Oct 8, 2014 at 12:19 PM, Fabian Hueske <[hidden email]> wrote:
>
> > I checked the merged Hadoop function compatibility pull request.
> > The PR consisted of four individual commits of three authors. All commits
> > have been squashed into a single commit such that two authors basically
> > lost their contributions.
> > I think this is not correct and should never be done. It might cause
> > frustration among contributors and eventually cause contributors to leave
> > the community.
> >
> > I propose to squash commits in pull requests only if it is really
> necessary
> > and in that case only commits that have the same author.
> >
> > For the current issue with the Hadoop function compat PR, I will prepare
> a
> > revert patch for the master and add the four commits of the PR on top.
> >
> > Fabian
> >
> > 2014-10-08 12:06 GMT+02:00 Robert Metzger <[hidden email]>:
> >
> > > I merged the Hadoop compat and POJOs, currently rebasing stephan's
> fault
> > > tolerance ....
> > >
> > > On Wed, Oct 8, 2014 at 10:50 AM, Fabian Hueske <[hidden email]>
> > wrote:
> > >
> > > > I think, the question is which PRs to merge into master before
> forking
> > of
> > > > the release candidate branch.
> > > > You did already merge #126 into the master branch, so it will be in
> ;-)
> > > >
> > > > 2014-10-08 10:37 GMT+02:00 Till Rohrmann <[hidden email]>:
> > > >
> > > > > We can also include the blob manager. The corresponding PR is #126.
> > > > >
> > > > > On Wed, Oct 8, 2014 at 1:51 AM, Márton Balassi <
> > > [hidden email]
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Pushed the resolution for FLINK-1103 and the streaming bugfix
> > commit.
> > > > > >
> > > > > > We are good for now on our side for an rc. Maybe I should add the
> > > > > streaming
> > > > > > connectors dependency fix that Stephan suggested here an the
> > mailing
> > > > list
> > > > > > soon.
> > > > > >
> > > > > > Marton
> > > > > >
> > > > > > On Tue, Oct 7, 2014 at 11:44 PM, Stephan Ewen <[hidden email]>
> > > > wrote:
> > > > > >
> > > > > > > We can add it and keep it initially undocumented (experimental)
> > > until
> > > > > > > further tests...
> > > > > > >
> > > > > > > On Tue, Oct 7, 2014 at 10:40 PM, Robert Metzger <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I would add #142.
> > > > > > > >
> > > > > > > > On Tue, Oct 7, 2014 at 10:29 PM, Stephan Ewen <
> > [hidden email]>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > I agree, having a release candidate out would be nice.
> > > > > > > > >
> > > > > > > > > What is your opinion on issue #142? Fault tolerance is
> > inactive
> > > > by
> > > > > > > > default,
> > > > > > > > > but can be activated through the configuration.
> > > > > > > > >
> > > > > > > > > On Tue, Oct 7, 2014 at 7:18 PM, Robert Metzger <
> > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Oh. I didn't know this. The last mail from Gyula sounded
> > like
> > > > the
> > > > > > > > > Streaming
> > > > > > > > > > part is ready2release.
> > > > > > > > > >
> > > > > > > > > > My personal goal was to prepare a release candiate today
> so
> > > > that
> > > > > we
> > > > > > > > have
> > > > > > > > > a
> > > > > > > > > > reference point against which we can test (which would
> also
> > > > mean
> > > > > > > > forking
> > > > > > > > > of
> > > > > > > > > > a 0.7-release branch and basically doing a feature
> freeze).
> > > > > > > > > > I also though that it might be a good idea to have a
> > release
> > > > > > > candidate
> > > > > > > > by
> > > > > > > > > > tomorrow because it could be of use for the Hackathon
> > > tomorrow
> > > > in
> > > > > > > > > > Stockholm.
> > > > > > > > > > But since the Scala POJO changes and the streaming
> examples
> > > are
> > > > > > both
> > > > > > > > > > outstanding, we can probably scratch that.
> > > > > > > > > >
> > > > > > > > > > Doing a fork, release candidate and a feature freeze
> > doesn't
> > > > mean
> > > > > > we
> > > > > > > > can
> > > > > > > > > > not supply bugfixes anymore for the 0.7-release ;)
> > > > > > > > > >
> > > > > > > > > > What is your best-case scenario for FLINK-1103? (as far
> as
> > I
> > > > can
> > > > > > see
> > > > > > > > the
> > > > > > > > > > only outstanding streaming feature for the release)
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Robert
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Oct 7, 2014 at 7:07 PM, Márton Balassi <
> > > > > > > > [hidden email]
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > As for the streaming side we would like to push some
> > bugfix
> > > > > > commits
> > > > > > > > and
> > > > > > > > > > the
> > > > > > > > > > > resolution of the FLINK-1103 JIRA issue.
> > > > > > > > > > >
> > > > > > > > > > > These are more or less ready, hopefully will be
> available
> > > at
> > > > > the
> > > > > > > end
> > > > > > > > of
> > > > > > > > > > > this week.
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Oct 7, 2014 at 6:50 PM, Robert Metzger <
> > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I've merged the record api deprecation already.
> > > > > > > > > > > > I'll merge #141 once Aljoscha provided his Scala
> > changes.
> > > > > > > > > > > >
> > > > > > > > > > > > We certainly should merge #136 and #143 as well.
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Oct 7, 2014 at 5:20 PM, Fabian Hueske <
> > > > > > > [hidden email]>
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > So which PRs will be included in the candidate?
> > > > > > > > > > > > >
> > > > > > > > > > > > > #141 POJOs
> > > > > > > > > > > > > #144 Deprecate Record API
> > > > > > > > > > > > > #136 Fixed example quickstart
> > > > > > > > > > > > > #143(?) Hadoop Compat: Documentation + Hadoop
> > function
> > > > > > wrappers
> > > > > > > > > > > (includes
> > > > > > > > > > > > > PR #131)
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2014-10-07 16:26 GMT+02:00 Stephan Ewen <
> > > > [hidden email]
> > > > > >:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Great news, looking forward to seeing this in the
> > > > master!
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Oct 7, 2014 at 1:53 PM, Robert Metzger <
> > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > As an update for everyone: My POJO feature is
> > > > finished,
> > > > > > > > > including
> > > > > > > > > > > > > > > documentation.
> > > > > > > > > > > > > > > Aljoscha is currently adopting the Scala API to
> > > have
> > > > > > > support
> > > > > > > > > for
> > > > > > > > > > > > > (nested)
> > > > > > > > > > > > > > > POJOs as well.
> > > > > > > > > > > > > > > Once that is done, I'll merge everything and
> > > create a
> > > > > > first
> > > > > > > > > > > candidate
> > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > we can use for testing.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Tue, Sep 30, 2014 at 10:49 PM, Fabian
> Hueske <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I just checked the "Run Example" quickstart
> and
> > > it
> > > > > > needs
> > > > > > > a
> > > > > > > > > bit
> > > > > > > > > > of
> > > > > > > > > > > > > work.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2014-09-30 22:41 GMT+02:00 Robert Metzger <
> > > > > > > > > [hidden email]
> > > > > > > > > > >:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I'm working hard on getting the POJOs
> ready.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > We also should do a pass over our
> > > documentation,
> > > > > the
> > > > > > > > > > > quickstarts
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > website to see if everything is in a good
> > state
> > > > > (for
> > > > > > > > > example
> > > > > > > > > > > the
> > > > > > > > > > > > > > > > > collection-based execution needs
> > documentation
> > > as
> > > > > > > well).
> > > > > > > > We
> > > > > > > > > > > > should
> > > > > > > > > > > > > > also
> > > > > > > > > > > > > > > > > finally document the hadoop-input format
> > > wrappers
> > > > > (I
> > > > > > > > think
> > > > > > > > > > Timo
> > > > > > > > > > > > is
> > > > > > > > > > > > > > > > working
> > > > > > > > > > > > > > > > > on a pull request for that).
> > > > > > > > > > > > > > > > > This page mentions the
> > LocalDistributedExecutor
> > > > and
> > > > > > > > > contains
> > > > > > > > > > > some
> > > > > > > > > > > > > > (most
> > > > > > > > > > > > > > > > > likely outdated) scala code:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://flink.incubator.apache.org/docs/0.7-incubating/local_execution.html
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > We also need to deprecate the old record
> api
> > (
> > > > > > > > > > > > > > > > >
> > > https://issues.apache.org/jira/browse/FLINK-1106
> > > > ).
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Tue, Sep 30, 2014 at 9:17 PM, Stephan
> > Ewen <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I think we are approaching ready state.
> > > > > > > > > > > > > > > > > > Last issues are going in and we started
> > > working
> > > > > on
> > > > > > > > > > > dependencies
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > test
> > > > > > > > > > > > > > > > > > platform diversity in order to make
> > > stabilizing
> > > > > > > phase.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > We should have an official feature freeze
> > > soon
> > > > > and
> > > > > > > fork
> > > > > > > > > the
> > > > > > > > > > > > > > > 0.7-release
> > > > > > > > > > > > > > > > > > branch. I personally vote to include the
> > POJO
> > > > > > support
> > > > > > > > (I
> > > > > > > > > > > think
> > > > > > > > > > > > > > Robert
> > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > sorting that one out and is close to
> > > > completion),
> > > > > > > and I
> > > > > > > > > > want
> > > > > > > > > > > to
> > > > > > > > > > > > > add
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > collection-based execution (today or
> > > tomorrow).
> > > > > > Till
> > > > > > > > has
> > > > > > > > > > the
> > > > > > > > > > > > BLOB
> > > > > > > > > > > > > > > > manager
> > > > > > > > > > > > > > > > > > ready, which would be good to include
> > (better
> > > > > > support
> > > > > > > > > large
> > > > > > > > > > > > > > libraries
> > > > > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > > fat jars).
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > After that, I vote to freeze.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Tue, Sep 30, 2014 at 9:12 PM, Gyula
> > Fora <
> > > > > > > > > > > [hidden email]
> > > > > > > > > > > > >
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hey,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > So what is the current decision
> regarding
> > > the
> > > > > > time
> > > > > > > of
> > > > > > > > > the
> > > > > > > > > > > > > > upcoming
> > > > > > > > > > > > > > > > > > release?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > As for the streaming component, we
> > included
> > > > all
> > > > > > the
> > > > > > > > > > > features
> > > > > > > > > > > > we
> > > > > > > > > > > > > > > > wanted,
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > will start to test everything tomorrow,
> > > > making
> > > > > > sure
> > > > > > > > > that
> > > > > > > > > > > all
> > > > > > > > > > > > > > works
> > > > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > > > intended.
> > > > > > > > > > > > > > > > > > > We are also almost finished with
> cleaning
> > > up
> > > > > the
> > > > > > > > > > connector
> > > > > > > > > > > > > > > > dependencies
> > > > > > > > > > > > > > > > > > > that Stephan pointed out, should be
> > > finished
> > > > by
> > > > > > > > > tomorrow.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Gyula
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > On 26 Sep 2014, at 10:49, Fabian
> Hueske <
> > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > +1 to manage this on JIRA (if
> possible)
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > 2014-09-26 10:40 GMT+02:00 Aljoscha
> > > > Krettek <
> > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >> Can we not manage this stuff on
> Jira?
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > >> On Fri, Sep 26, 2014 at 10:16 AM,
> > > Stephan
> > > > > > Ewen <
> > > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>> I personally like the idea of SOFT
> > > > > time-based
> > > > > > > > > feature
> > > > > > > > > > > > > > freezes.
> > > > > > > > > > > > > > > > > > > Otherwise,
> > > > > > > > > > > > > > > > > > > >>> releases will get delayed again and
> > > > again,
> > > > > > > > because
> > > > > > > > > of
> > > > > > > > > > > > > > features
> > > > > > > > > > > > > > > > that
> > > > > > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > > > >> try
> > > > > > > > > > > > > > > > > > > >>> to squeeze in.
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> Having reached the freeze point
> > > already,
> > > > we
> > > > > > > could
> > > > > > > > > > still
> > > > > > > > > > > > > > include
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>> features that are pending ready
> state
> > > in
> > > > > the
> > > > > > > next
> > > > > > > > > > days
> > > > > > > > > > > > > > > > (streaming,
> > > > > > > > > > > > > > > > > > blob
> > > > > > > > > > > > > > > > > > > >>> Manager, POJOs), but otherwise head
> > > for a
> > > > > > > release
> > > > > > > > > > > state.
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>> We had a mail listing the issues to
> > > > include
> > > > > > > into
> > > > > > > > > 0.7,
> > > > > > > > > > > > but a
> > > > > > > > > > > > > > > wiki
> > > > > > > > > > > > > > > > > page
> > > > > > > > > > > > > > > > > > > >> would
> > > > > > > > > > > > > > > > > > > >>> probably be better. In that sense,
> we
> > > > could
> > > > > > > start
> > > > > > > > > > > > > collecting
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > issues
> > > > > > > > > > > > > > > > > > > >> for
> > > > > > > > > > > > > > > > > > > >>> the next release from now on.
> > > > > > > > > > > > > > > > > > > >>> Am 26.09.2014 09:17 schrieb
> "Daniel
> > > > > > Warneke" <
> > > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > >>>> Hi,
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>>> I like Fabian's idea. Is there a
> > wiki
> > > > page
> > > > > > (or
> > > > > > > > > > > something
> > > > > > > > > > > > > > > > similar)
> > > > > > > > > > > > > > > > > > > where
> > > > > > > > > > > > > > > > > > > >> we
> > > > > > > > > > > > > > > > > > > >>>> can collect the proposed JIRAs?
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>>> Best regards,
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>>>    Daniel
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>>> Am 24.09.2014 23:03, schrieb
> Fabian
> > > > > Hueske:
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>>>> I agree, a hard feature stop
> > deadline
> > > > > might
> > > > > > > not
> > > > > > > > > be
> > > > > > > > > > > the
> > > > > > > > > > > > > best
> > > > > > > > > > > > > > > > > > practice.
> > > > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > > > >>>>> How about the following
> procedure:
> > > > > > > > > > > > > > > > > > > >>>>> We decide two (or three) weeks
> > > before a
> > > > > > > > targeted
> > > > > > > > > > > > release
> > > > > > > > > > > > > > date
> > > > > > > > > > > > > > > > > about
> > > > > > > > > > > > > > > > > > > >> which
> > > > > > > > > > > > > > > > > > > >>>>> JIRAs to include. JIRAs that are
> > > > selected
> > > > > > > for a
> > > > > > > > > > > release
> > > > > > > > > > > > > > > should
> > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >>>>> completed
> > > > > > > > > > > > > > > > > > > >>>>> or really close to completion
> (via
> > > > > progress
> > > > > > > > > > estimates
> > > > > > > > > > > > in
> > > > > > > > > > > > > > > JIRA).
> > > > > > > > > > > > > > > > > > > >>>>> After we decided which JIRAs to
> > > include
> > > > > in
> > > > > > a
> > > > > > > > > > release,
> > > > > > > > > > > > we
> > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > use
> > > > > > > > > > > > > > > > > > JIRA
> > > > > > > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > > > > > > >>>>> track the progress and dedicate
> > > another
> > > > > > week
> > > > > > > > > > > > exclusively
> > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > testing
> > > > > > > > > > > > > > > > > > > >> after
> > > > > > > > > > > > > > > > > > > >>>>> the last feature was completed.
> > > > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > > > >>>>> 2014-09-24 19:10 GMT+02:00 Márton
> > > > > Balassi <
> > > > > > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > > > >>>>> As for the streaming team we're
> > also
> > > > > > getting
> > > > > > > > > ready
> > > > > > > > > > > for
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > release,
> > > > > > > > > > > > > > > > > > > >> but a
> > > > > > > > > > > > > > > > > > > >>>>>> couple of days will be needed to
> > > > finish
> > > > > > the
> > > > > > > > > > features
> > > > > > > > > > > > > that
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > > > >> like
> > > > > > > > > > > > > > > > > > > >>>>>> to
> > > > > > > > > > > > > > > > > > > >>>>>> include.
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>    - A little work is still
> needed
> > > for
> > > > > > > reduce
> > > > > > > > > > > > operations
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > >>>>>>    groups/connected streams (any
> > > > comment
> > > > > > on
> > > > > > > > > > Gyula's
> > > > > > > > > > > > > recent
> > > > > > > > > > > > > > > > > e-mail
> > > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > > > >>>>>> really
> > > > > > > > > > > > > > > > > > > >>>>>>    appreciated :) )
> > > > > > > > > > > > > > > > > > > >>>>>>    - The examples are being
> > updated
> > > to
> > > > > > match
> > > > > > > > the
> > > > > > > > > > > > > standard,
> > > > > > > > > > > > > > > > check
> > > > > > > > > > > > > > > > > > out
> > > > > > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > > > > > >>>>>>    WordCount. (
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > https://github.com/mbalassi/incubator-flink/blob/
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > streaming-new/flink-addons/flink-streaming/flink-
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > streaming-examples/src/main/java/org/apache/flink/
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > streaming/examples/wordcount/WordCount.java
> > > > > > > > > > > > > > > > > > > >>>>>> )
> > > > > > > > > > > > > > > > > > > >>>>>>    Hopefully it gives you some
> > deja
> > > > vu.
> > > > > :)
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>> On Wed, Sep 24, 2014 at 6:53 PM,
> > > Ufuk
> > > > > > > Celebi <
> > > > > > > > > > > > > > > [hidden email]>
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>> On 24 Sep 2014, at 18:37, Robert
> > > > > Metzger <
> > > > > > > > > > > > > > > [hidden email]
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> Hey guys,
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> exactly 3 weeks ago, we
> > discussed
> > > to
> > > > > do
> > > > > > a
> > > > > > > > > > feature
> > > > > > > > > > > > > freeze
> > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>>>>>>> 0.7-incubating release today.
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> From our initial feature list:
> > > > > > > > > > > > > > > > > > > >>>>>>>> - *Flink Streaming* "Beta
> > > Preview".
> > > > I
> > > > > > > would
> > > > > > > > > > > suggest
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > ship
> > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> streaming,
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> but clearly mark it as a
> preview
> > > in
> > > > > the
> > > > > > > > > > > > documentation.
> > > > > > > > > > > > > > > > > > > >>>>>>>> -* Java API Pojo
> improvements*:
> > > Code
> > > > > > > > > generation,
> > > > > > > > > > > key
> > > > > > > > > > > > > > > > selection
> > > > > > > > > > > > > > > > > > > >> using a
> > > > > > > > > > > > > > > > > > > >>>>>>>> string-expression:
> > > > > > > > > > > > > > > > > > >
> > > > > https://issues.apache.org/jira/browse/FLINK-1032
> > > > > > > > > > > > > > > > > > > >>>>>>>>  - *Reworked Scala API*. Bring
> > the
> > > > > Scala
> > > > > > > API
> > > > > > > > > in
> > > > > > > > > > > sync
> > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >> latest
> > > > > > > > > > > > > > > > > > > >>>>>>>> developments in the Java API:
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > https://issues.apache.org/jira/browse/FLINK-641
> > > > > > > > > > > > > > > > > > > >>>>>>>>  -* Akka-based RPC service*:
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > https://issues.apache.org/jira/browse/FLINK-1019
> > > > > > > > > > > > > > > > > > > >>>>>>>>  - *Kryo-based serialization*.
> > > This
> > > > > > > feature
> > > > > > > > > has
> > > > > > > > > > > been
> > > > > > > > > > > > > > > > requested
> > > > > > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > > > > > >> many
> > > > > > > > > > > > > > > > > > > >>>>>>>> users. Mostly because they
> > wanted
> > > to
> > > > > use
> > > > > > > > > > > Collections
> > > > > > > > > > > > > > > inside
> > > > > > > > > > > > > > > > > > POJOs:
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > https://issues.apache.org/jira/browse/FLINK-610
> > > > > > > > > > > > > > > > > > > >>>>>>>> - Rework JobManager internals
> to
> > > > > support
> > > > > > > > > > > incremental
> > > > > > > > > > > > > > > program
> > > > > > > > > > > > > > > > > > > >> rollout &
> > > > > > > > > > > > > > > > > > > >>>>>>>> execution
> > > > > > > > > > > > > > > > > > > >>>>>>>> - First parts of dynamic
> memory
> > > > > > > assignments
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> The following features are in
> > the
> > > > > > master,
> > > > > > > as
> > > > > > > > > of
> > > > > > > > > > > > today:
> > > > > > > > > > > > > > > > > > > >>>>>>>> - *Flink Streaming*
> > > > > > > > > > > > > > > > > > > >>>>>>>> - *Reworked Scala API*
> > > > > > > > > > > > > > > > > > > >>>>>>>> -* New Scheduler*
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> We certainly need some days to
> > > test
> > > > > > > > everything
> > > > > > > > > > > until
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > > start
> > > > > > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> vote.
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> Based on our experience with
> the
> > > > last
> > > > > > > major
> > > > > > > > > > > > release, I
> > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > > > really
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> like
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> to
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> do the testing and bugfixing
> > > BEFORE
> > > > > the
> > > > > > > > first
> > > > > > > > > > > > release
> > > > > > > > > > > > > > > > > candidate.
> > > > > > > > > > > > > > > > > > > For
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> the
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> 0.6-incubating release, we had
> 6
> > > > > > > candidates)
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> How do you guys feel about
> this?
> > > > > Should
> > > > > > we
> > > > > > > > > wait
> > > > > > > > > > a
> > > > > > > > > > > > few
> > > > > > > > > > > > > > more
> > > > > > > > > > > > > > > > > days
> > > > > > > > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > > > > > > >>>>>>>> release so that a few more
> > > features
> > > > > make
> > > > > > > it
> > > > > > > > > into
> > > > > > > > > > > the
> > > > > > > > > > > > > > > > release?
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> I'm undecided on this. On the
> > one
> > > > > hand,
> > > > > > > its
> > > > > > > > > > really
> > > > > > > > > > > > > nice
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > release
> > > > > > > > > > > > > > > > > > > >> on a
> > > > > > > > > > > > > > > > > > > >>>>>>>> regular schedule, but it also
> > eats
> > > > up
> > > > > > some
> > > > > > > > > time
> > > > > > > > > > > and
> > > > > > > > > > > > > > causes
> > > > > > > > > > > > > > > > > > > overhead
> > > > > > > > > > > > > > > > > > > >>>>>>>> (different branches etc.).
> > > > > > > > > > > > > > > > > > > >>>>>>>> I would really like to have
> the
> > > Java
> > > > > API
> > > > > > > > Pojo
> > > > > > > > > > > > > > improvements
> > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> release.
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> I think I can finish it until
> > end
> > > of
> > > > > > this
> > > > > > > > > week.
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>>> Opinions?
> > > > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> I agree that the finished
> > features
> > > > > > > > (especially
> > > > > > > > > > the
> > > > > > > > > > > > > Scala
> > > > > > > > > > > > > > > API)
> > > > > > > > > > > > > > > > > are
> > > > > > > > > > > > > > > > > > > >> nice
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>> for
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> a new release, but still I
> would
> > > like
> > > > > to
> > > > > > > > wait a
> > > > > > > > > > few
> > > > > > > > > > > > > more
> > > > > > > > > > > > > > > > days.
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>> Some of the missing features
> are
> > on
> > > > the
> > > > > > > brink
> > > > > > > > > of
> > > > > > > > > > > > being
> > > > > > > > > > > > > > > > finished
> > > > > > > > > > > > > > > > > > > (e.g.
> > > > > > > > > > > > > > > > > > > >>>>>>> the
> > > > > > > > > > > > > > > > > > > >>>>>>> Pojo improvements). I wouldn't
> > want
> > > > to
> > > > > > > > invest a
> > > > > > > > > > > week
> > > > > > > > > > > > in
> > > > > > > > > > > > > > bug
> > > > > > > > > > > > > > > > > > fixing
> > > > > > > > > > > > > > > > > > > >> and
> > > > > > > > > > > > > > > > > > > >>>>>>> doing the release vote, when
> the
> > > new
> > > > > > > features
> > > > > > > > > are
> > > > > > > > > > > > > likely
> > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > >> finished
> > > > > > > > > > > > > > > > > > > >>>>>>> just a few days afterwards. And
> > the
> > > > > > > upcoming
> > > > > > > > > > > features
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > > >> definitely be
> > > > > > > > > > > > > > > > > > > >>>>>>> worth a release, so users can
> > work
> > > > with
> > > > > > > them.
> > > > > > > > > ;)
> > > > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>