Please help testing the 0.7.0-incubating release candidate

[Robert Metzger]

Hi guys,

I've created a first release candidate for the 0.7-incubating major release:

http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/


This thread is not an official release vote thread, rather a discussion
thread to collect all open issues that we want to address until we create
the voting release candidate.


Best,
Robert

[Robert Metzger]

Staging repository:
https://repository.apache.org/content/repositories/orgapacheflink-1014

On Wed, Oct 8, 2014 at 1:15 PM, Robert Metzger <[hidden email]> wrote:

> Hi guys,
>
> I've created a first release candidate for the 0.7-incubating major
> release:
>
> http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
>
>
> This thread is not an official release vote thread, rather a discussion
> thread to collect all open issues that we want to address until we create
> the voting release candidate.
>
>
> Best,
> Robert
>

[Leonidas Fegaras]

I tested flink-0.7.0-incubating-bin-hadoop2-yarn.tgz.
It works fine for Apache MRQL in Flink mode:
1) It passed all query tests for MRQL in Flink local mode (25 queries).
2) It was tested successfully for PageRank on an 1.4GB XML file (DBLP
bibliography) using 20 slots on a 40-slot yarn cluster.

I had to make the following changes to the MRQL code:
1) changed some Java imports (eg, from
org.apache.flink.api.java.functions.RichMapFunction to
org.apache.flink.api.common.functions.RichMapFunction )
2) changed the run script to get the flink job manager from
conf/.yarn-properties instead of conf/.yarn-jobmanager
Good job!
Leonidas Fegaras

On 10/08/2014 06:15 AM, Robert Metzger wrote:

> Hi guys,
>
> I've created a first release candidate for the 0.7-incubating major release:
>
> http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
>
>
> This thread is not an official release vote thread, rather a discussion
> thread to collect all open issues that we want to address until we create
> the voting release candidate.
>
>
> Best,
> Robert

[Henry Saputra]

Are taking VOTE on this or you just trying to do dry run?

On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]> wrote:

> Hi guys,
>
> I've created a first release candidate for the 0.7-incubating major release:
>
> http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
>
>
> This thread is not an official release vote thread, rather a discussion
> thread to collect all open issues that we want to address until we create
> the voting release candidate.
>
>
> Best,
> Robert

[Robert Metzger]

No, its just a dry run and I want to do some tests on our cluster first.

Last time, I created a lot of release candidates and vote threads. This
time, I wanted to be smart and do the testing first ;)



On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <[hidden email]>
wrote:

> Are taking VOTE on this or you just trying to do dry run?
>
> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]>
> wrote:
> > Hi guys,
> >
> > I've created a first release candidate for the 0.7-incubating major
> release:
> >
> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> >
> >
> > This thread is not an official release vote thread, rather a discussion
> > thread to collect all open issues that we want to address until we create
> > the voting release candidate.
> >
> >
> > Best,
> > Robert
>

[Robert Metzger]

Thank you Leonidas for helping with the test.

I've successfully used the maven archetypes and pulled 0.7-incubating from
the staged repository, so that seems to work.
I found and fixed a bug with the YARN deployment and tested it on Hadoop
2.4.0 using Amazon EMR (8 nodes).
I also tested Flink on a 24 nodes cluster with K-Means and different
flavors of Wordcount... All good (except for this issue:
https://issues.apache.org/jira/browse/FLINK-1152 )

Has the Scala API been updated to reflect the newly integrated POJO support?

Could somebody who's using Windows try it there?

We should check if the web-client shows the optimizer plans of the examples
properly and if they all run without data.




On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <[hidden email]> wrote:

> No, its just a dry run and I want to do some tests on our cluster first.
>
> Last time, I created a lot of release candidates and vote threads. This
> time, I wanted to be smart and do the testing first ;)
>
>
>
> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <[hidden email]>
> wrote:
>
>> Are taking VOTE on this or you just trying to do dry run?
>>
>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]>
>> wrote:
>> > Hi guys,
>> >
>> > I've created a first release candidate for the 0.7-incubating major
>> release:
>> >
>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
>> >
>> >
>> > This thread is not an official release vote thread, rather a discussion
>> > thread to collect all open issues that we want to address until we
>> create
>> > the voting release candidate.
>> >
>> >
>> > Best,
>> > Robert
>>
>
>

[Aljoscha Krettek-2]

Yes, the Scala API should be up to date. The latest master also has
all the Operator IT Cases from Java ported to Java. They also contain
POJOs and nested key selectors.

On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <[hidden email]> wrote:

> Thank you Leonidas for helping with the test.
>
> I've successfully used the maven archetypes and pulled 0.7-incubating from
> the staged repository, so that seems to work.
> I found and fixed a bug with the YARN deployment and tested it on Hadoop
> 2.4.0 using Amazon EMR (8 nodes).
> I also tested Flink on a 24 nodes cluster with K-Means and different
> flavors of Wordcount... All good (except for this issue:
> https://issues.apache.org/jira/browse/FLINK-1152 )
>
> Has the Scala API been updated to reflect the newly integrated POJO support?
>
> Could somebody who's using Windows try it there?
>
> We should check if the web-client shows the optimizer plans of the examples
> properly and if they all run without data.
>
>
>
>
> On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <[hidden email]> wrote:
>
>> No, its just a dry run and I want to do some tests on our cluster first.
>>
>> Last time, I created a lot of release candidates and vote threads. This
>> time, I wanted to be smart and do the testing first ;)
>>
>>
>>
>> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <[hidden email]>
>> wrote:
>>
>>> Are taking VOTE on this or you just trying to do dry run?
>>>
>>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]>
>>> wrote:
>>> > Hi guys,
>>> >
>>> > I've created a first release candidate for the 0.7-incubating major
>>> release:
>>> >
>>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
>>> >
>>> >
>>> > This thread is not an official release vote thread, rather a discussion
>>> > thread to collect all open issues that we want to address until we
>>> create
>>> > the voting release candidate.
>>> >
>>> >
>>> > Best,
>>> > Robert
>>>
>>
>>

[Robert Metzger]

Ah damn .. I meant the "Scala API Documentation" ;)

I just found an issue: If I'm running an example from eclipse, the JVM
process won't terminate properly after the job has finished.
It seems that a "Destroy JVM" thread is started and the main() thread has
already gone away, but still, the JVM is not stopping.


On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Yes, the Scala API should be up to date. The latest master also has
> all the Operator IT Cases from Java ported to Java. They also contain
> POJOs and nested key selectors.
>
> On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <[hidden email]>
> wrote:
> > Thank you Leonidas for helping with the test.
> >
> > I've successfully used the maven archetypes and pulled 0.7-incubating
> from
> > the staged repository, so that seems to work.
> > I found and fixed a bug with the YARN deployment and tested it on Hadoop
> > 2.4.0 using Amazon EMR (8 nodes).
> > I also tested Flink on a 24 nodes cluster with K-Means and different
> > flavors of Wordcount... All good (except for this issue:
> > https://issues.apache.org/jira/browse/FLINK-1152 )
> >
> > Has the Scala API been updated to reflect the newly integrated POJO
> support?
> >
> > Could somebody who's using Windows try it there?
> >
> > We should check if the web-client shows the optimizer plans of the
> examples
> > properly and if they all run without data.
> >
> >
> >
> >
> > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <[hidden email]>
> wrote:
> >
> >> No, its just a dry run and I want to do some tests on our cluster first.
> >>
> >> Last time, I created a lot of release candidates and vote threads. This
> >> time, I wanted to be smart and do the testing first ;)
> >>
> >>
> >>
> >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <[hidden email]>
> >> wrote:
> >>
> >>> Are taking VOTE on this or you just trying to do dry run?
> >>>
> >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]>
> >>> wrote:
> >>> > Hi guys,
> >>> >
> >>> > I've created a first release candidate for the 0.7-incubating major
> >>> release:
> >>> >
> >>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> >>> >
> >>> >
> >>> > This thread is not an official release vote thread, rather a
> discussion
> >>> > thread to collect all open issues that we want to address until we
> >>> create
> >>> > the voting release candidate.
> >>> >
> >>> >
> >>> > Best,
> >>> > Robert
> >>>
> >>
> >>
>

[Stephan Ewen]

Which threads are active when the shutdown hangs?
Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]>:

> Ah damn .. I meant the "Scala API Documentation" ;)
>
> I just found an issue: If I'm running an example from eclipse, the JVM
> process won't terminate properly after the job has finished.
> It seems that a "Destroy JVM" thread is started and the main() thread has
> already gone away, but still, the JVM is not stopping.
>
>
> On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > Yes, the Scala API should be up to date. The latest master also has
> > all the Operator IT Cases from Java ported to Java. They also contain
> > POJOs and nested key selectors.
> >
> > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <[hidden email]>
> > wrote:
> > > Thank you Leonidas for helping with the test.
> > >
> > > I've successfully used the maven archetypes and pulled 0.7-incubating
> > from
> > > the staged repository, so that seems to work.
> > > I found and fixed a bug with the YARN deployment and tested it on
> Hadoop
> > > 2.4.0 using Amazon EMR (8 nodes).
> > > I also tested Flink on a 24 nodes cluster with K-Means and different
> > > flavors of Wordcount... All good (except for this issue:
> > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > >
> > > Has the Scala API been updated to reflect the newly integrated POJO
> > support?
> > >
> > > Could somebody who's using Windows try it there?
> > >
> > > We should check if the web-client shows the optimizer plans of the
> > examples
> > > properly and if they all run without data.
> > >
> > >
> > >
> > >
> > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <[hidden email]>
> > wrote:
> > >
> > >> No, its just a dry run and I want to do some tests on our cluster
> first.
> > >>
> > >> Last time, I created a lot of release candidates and vote threads.
> This
> > >> time, I wanted to be smart and do the testing first ;)
> > >>
> > >>
> > >>
> > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> [hidden email]>
> > >> wrote:
> > >>
> > >>> Are taking VOTE on this or you just trying to do dry run?
> > >>>
> > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <[hidden email]>
> > >>> wrote:
> > >>> > Hi guys,
> > >>> >
> > >>> > I've created a first release candidate for the 0.7-incubating major
> > >>> release:
> > >>> >
> > >>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > >>> >
> > >>> >
> > >>> > This thread is not an official release vote thread, rather a
> > discussion
> > >>> > thread to collect all open issues that we want to address until we
> > >>> create
> > >>> > the voting release candidate.
> > >>> >
> > >>> >
> > >>> > Best,
> > >>> > Robert
> > >>>
> > >>
> > >>
> >
>

[Robert Metzger]

WordCount [Java Application]
org.apache.flink.examples.java.wordcount.WordCount at localhost:38214
(Suspended)
Daemon System Thread [Signal Dispatcher] (Suspended)
Daemon System Thread [Finalizer] (Suspended)
Daemon System Thread [Reference Handler] (Suspended)
Thread [Timer-1] (Suspended)
waiting for: TaskQueue  (id=193)
Object.wait(long) line: not available [native method]
TimerThread.mainLoop() line: 552
TimerThread.run() line: 505
Thread [Timer-3] (Suspended)
waiting for: TaskQueue  (id=192)
Object.wait(long) line: not available [native method]
TimerThread.mainLoop() line: 552
TimerThread.run() line: 505
Thread [DestroyJavaVM] (Suspended)
/opt/java/bin/java (Oct 11, 2014, 12:02:52 PM)



On Fri, Oct 10, 2014 at 7:31 PM, Stephan Ewen <[hidden email]> wrote:

> Which threads are active when the shutdown hangs?
> Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]>:
>
> > Ah damn .. I meant the "Scala API Documentation" ;)
> >
> > I just found an issue: If I'm running an example from eclipse, the JVM
> > process won't terminate properly after the job has finished.
> > It seems that a "Destroy JVM" thread is started and the main() thread has
> > already gone away, but still, the JVM is not stopping.
> >
> >
> > On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> > > Yes, the Scala API should be up to date. The latest master also has
> > > all the Operator IT Cases from Java ported to Java. They also contain
> > > POJOs and nested key selectors.
> > >
> > > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <[hidden email]>
> > > wrote:
> > > > Thank you Leonidas for helping with the test.
> > > >
> > > > I've successfully used the maven archetypes and pulled 0.7-incubating
> > > from
> > > > the staged repository, so that seems to work.
> > > > I found and fixed a bug with the YARN deployment and tested it on
> > Hadoop
> > > > 2.4.0 using Amazon EMR (8 nodes).
> > > > I also tested Flink on a 24 nodes cluster with K-Means and different
> > > > flavors of Wordcount... All good (except for this issue:
> > > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > > >
> > > > Has the Scala API been updated to reflect the newly integrated POJO
> > > support?
> > > >
> > > > Could somebody who's using Windows try it there?
> > > >
> > > > We should check if the web-client shows the optimizer plans of the
> > > examples
> > > > properly and if they all run without data.
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <[hidden email]>
> > > wrote:
> > > >
> > > >> No, its just a dry run and I want to do some tests on our cluster
> > first.
> > > >>
> > > >> Last time, I created a lot of release candidates and vote threads.
> > This
> > > >> time, I wanted to be smart and do the testing first ;)
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> > [hidden email]>
> > > >> wrote:
> > > >>
> > > >>> Are taking VOTE on this or you just trying to do dry run?
> > > >>>
> > > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <
> [hidden email]>
> > > >>> wrote:
> > > >>> > Hi guys,
> > > >>> >
> > > >>> > I've created a first release candidate for the 0.7-incubating
> major
> > > >>> release:
> > > >>> >
> > > >>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > > >>> >
> > > >>> >
> > > >>> > This thread is not an official release vote thread, rather a
> > > discussion
> > > >>> > thread to collect all open issues that we want to address until
> we
> > > >>> create
> > > >>> > the voting release candidate.
> > > >>> >
> > > >>> >
> > > >>> > Best,
> > > >>> > Robert
> > > >>>
> > > >>
> > > >>
> > >
> >
>

[Stephan Ewen]

Is it reproducible?

Seems that some timers do not get canceled...

On Sat, Oct 11, 2014 at 12:07 PM, Robert Metzger <[hidden email]>
wrote:

> WordCount [Java Application]
> org.apache.flink.examples.java.wordcount.WordCount at localhost:38214
> (Suspended)
> Daemon System Thread [Signal Dispatcher] (Suspended)
> Daemon System Thread [Finalizer] (Suspended)
> Daemon System Thread [Reference Handler] (Suspended)
> Thread [Timer-1] (Suspended)
> waiting for: TaskQueue  (id=193)
> Object.wait(long) line: not available [native method]
> TimerThread.mainLoop() line: 552
> TimerThread.run() line: 505
> Thread [Timer-3] (Suspended)
> waiting for: TaskQueue  (id=192)
> Object.wait(long) line: not available [native method]
> TimerThread.mainLoop() line: 552
> TimerThread.run() line: 505
> Thread [DestroyJavaVM] (Suspended)
> /opt/java/bin/java (Oct 11, 2014, 12:02:52 PM)
>
>
>
> On Fri, Oct 10, 2014 at 7:31 PM, Stephan Ewen <[hidden email]> wrote:
>
> > Which threads are active when the shutdown hangs?
> > Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]>:
> >
> > > Ah damn .. I meant the "Scala API Documentation" ;)
> > >
> > > I just found an issue: If I'm running an example from eclipse, the JVM
> > > process won't terminate properly after the job has finished.
> > > It seems that a "Destroy JVM" thread is started and the main() thread
> has
> > > already gone away, but still, the JVM is not stopping.
> > >
> > >
> > > On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <[hidden email]
> >
> > > wrote:
> > >
> > > > Yes, the Scala API should be up to date. The latest master also has
> > > > all the Operator IT Cases from Java ported to Java. They also contain
> > > > POJOs and nested key selectors.
> > > >
> > > > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <[hidden email]
> >
> > > > wrote:
> > > > > Thank you Leonidas for helping with the test.
> > > > >
> > > > > I've successfully used the maven archetypes and pulled
> 0.7-incubating
> > > > from
> > > > > the staged repository, so that seems to work.
> > > > > I found and fixed a bug with the YARN deployment and tested it on
> > > Hadoop
> > > > > 2.4.0 using Amazon EMR (8 nodes).
> > > > > I also tested Flink on a 24 nodes cluster with K-Means and
> different
> > > > > flavors of Wordcount... All good (except for this issue:
> > > > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > > > >
> > > > > Has the Scala API been updated to reflect the newly integrated POJO
> > > > support?
> > > > >
> > > > > Could somebody who's using Windows try it there?
> > > > >
> > > > > We should check if the web-client shows the optimizer plans of the
> > > > examples
> > > > > properly and if they all run without data.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <
> [hidden email]>
> > > > wrote:
> > > > >
> > > > >> No, its just a dry run and I want to do some tests on our cluster
> > > first.
> > > > >>
> > > > >> Last time, I created a lot of release candidates and vote threads.
> > > This
> > > > >> time, I wanted to be smart and do the testing first ;)
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> > > [hidden email]>
> > > > >> wrote:
> > > > >>
> > > > >>> Are taking VOTE on this or you just trying to do dry run?
> > > > >>>
> > > > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <
> > [hidden email]>
> > > > >>> wrote:
> > > > >>> > Hi guys,
> > > > >>> >
> > > > >>> > I've created a first release candidate for the 0.7-incubating
> > major
> > > > >>> release:
> > > > >>> >
> > > > >>> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > > > >>> >
> > > > >>> >
> > > > >>> > This thread is not an official release vote thread, rather a
> > > > discussion
> > > > >>> > thread to collect all open issues that we want to address until
> > we
> > > > >>> create
> > > > >>> > the voting release candidate.
> > > > >>> >
> > > > >>> >
> > > > >>> > Best,
> > > > >>> > Robert
> > > > >>>
> > > > >>
> > > > >>
> > > >
> > >
> >
>

[Robert Metzger]

Yes, on different computers.

Another issue that came in my mind regarding the release: is there any
documentation on the collection-based execution? I think we should have at
least a small paragraph on it.

On Sat, Oct 11, 2014 at 1:15 PM, Stephan Ewen <[hidden email]> wrote:

> Is it reproducible?
>
> Seems that some timers do not get canceled...
>
> On Sat, Oct 11, 2014 at 12:07 PM, Robert Metzger <[hidden email]>
> wrote:
>
> > WordCount [Java Application]
> > org.apache.flink.examples.java.wordcount.WordCount at localhost:38214
> > (Suspended)
> > Daemon System Thread [Signal Dispatcher] (Suspended)
> > Daemon System Thread [Finalizer] (Suspended)
> > Daemon System Thread [Reference Handler] (Suspended)
> > Thread [Timer-1] (Suspended)
> > waiting for: TaskQueue  (id=193)
> > Object.wait(long) line: not available [native method]
> > TimerThread.mainLoop() line: 552
> > TimerThread.run() line: 505
> > Thread [Timer-3] (Suspended)
> > waiting for: TaskQueue  (id=192)
> > Object.wait(long) line: not available [native method]
> > TimerThread.mainLoop() line: 552
> > TimerThread.run() line: 505
> > Thread [DestroyJavaVM] (Suspended)
> > /opt/java/bin/java (Oct 11, 2014, 12:02:52 PM)
> >
> >
> >
> > On Fri, Oct 10, 2014 at 7:31 PM, Stephan Ewen <[hidden email]> wrote:
> >
> > > Which threads are active when the shutdown hangs?
> > > Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]>:
> > >
> > > > Ah damn .. I meant the "Scala API Documentation" ;)
> > > >
> > > > I just found an issue: If I'm running an example from eclipse, the
> JVM
> > > > process won't terminate properly after the job has finished.
> > > > It seems that a "Destroy JVM" thread is started and the main() thread
> > has
> > > > already gone away, but still, the JVM is not stopping.
> > > >
> > > >
> > > > On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Yes, the Scala API should be up to date. The latest master also has
> > > > > all the Operator IT Cases from Java ported to Java. They also
> contain
> > > > > POJOs and nested key selectors.
> > > > >
> > > > > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <
> [hidden email]
> > >
> > > > > wrote:
> > > > > > Thank you Leonidas for helping with the test.
> > > > > >
> > > > > > I've successfully used the maven archetypes and pulled
> > 0.7-incubating
> > > > > from
> > > > > > the staged repository, so that seems to work.
> > > > > > I found and fixed a bug with the YARN deployment and tested it on
> > > > Hadoop
> > > > > > 2.4.0 using Amazon EMR (8 nodes).
> > > > > > I also tested Flink on a 24 nodes cluster with K-Means and
> > different
> > > > > > flavors of Wordcount... All good (except for this issue:
> > > > > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > > > > >
> > > > > > Has the Scala API been updated to reflect the newly integrated
> POJO
> > > > > support?
> > > > > >
> > > > > > Could somebody who's using Windows try it there?
> > > > > >
> > > > > > We should check if the web-client shows the optimizer plans of
> the
> > > > > examples
> > > > > > properly and if they all run without data.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <
> > [hidden email]>
> > > > > wrote:
> > > > > >
> > > > > >> No, its just a dry run and I want to do some tests on our
> cluster
> > > > first.
> > > > > >>
> > > > > >> Last time, I created a lot of release candidates and vote
> threads.
> > > > This
> > > > > >> time, I wanted to be smart and do the testing first ;)
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> > > > [hidden email]>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Are taking VOTE on this or you just trying to do dry run?
> > > > > >>>
> > > > > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <
> > > [hidden email]>
> > > > > >>> wrote:
> > > > > >>> > Hi guys,
> > > > > >>> >
> > > > > >>> > I've created a first release candidate for the 0.7-incubating
> > > major
> > > > > >>> release:
> > > > > >>> >
> > > > > >>> >
> http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > This thread is not an official release vote thread, rather a
> > > > > discussion
> > > > > >>> > thread to collect all open issues that we want to address
> until
> > > we
> > > > > >>> create
> > > > > >>> > the voting release candidate.
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > Best,
> > > > > >>> > Robert
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
>

[Fabian Hueske]

Checked all binary packages (except the YARN one) on Windows. Started local
mode and executed a random example.
All worked fine.

2014-10-11 19:21 GMT+02:00 Robert Metzger <[hidden email]>:

> Yes, on different computers.
>
> Another issue that came in my mind regarding the release: is there any
> documentation on the collection-based execution? I think we should have at
> least a small paragraph on it.
>
> On Sat, Oct 11, 2014 at 1:15 PM, Stephan Ewen <[hidden email]> wrote:
>
> > Is it reproducible?
> >
> > Seems that some timers do not get canceled...
> >
> > On Sat, Oct 11, 2014 at 12:07 PM, Robert Metzger <[hidden email]>
> > wrote:
> >
> > > WordCount [Java Application]
> > > org.apache.flink.examples.java.wordcount.WordCount at localhost:38214
> > > (Suspended)
> > > Daemon System Thread [Signal Dispatcher] (Suspended)
> > > Daemon System Thread [Finalizer] (Suspended)
> > > Daemon System Thread [Reference Handler] (Suspended)
> > > Thread [Timer-1] (Suspended)
> > > waiting for: TaskQueue  (id=193)
> > > Object.wait(long) line: not available [native method]
> > > TimerThread.mainLoop() line: 552
> > > TimerThread.run() line: 505
> > > Thread [Timer-3] (Suspended)
> > > waiting for: TaskQueue  (id=192)
> > > Object.wait(long) line: not available [native method]
> > > TimerThread.mainLoop() line: 552
> > > TimerThread.run() line: 505
> > > Thread [DestroyJavaVM] (Suspended)
> > > /opt/java/bin/java (Oct 11, 2014, 12:02:52 PM)
> > >
> > >
> > >
> > > On Fri, Oct 10, 2014 at 7:31 PM, Stephan Ewen <[hidden email]>
> wrote:
> > >
> > > > Which threads are active when the shutdown hangs?
> > > > Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]>:
> > > >
> > > > > Ah damn .. I meant the "Scala API Documentation" ;)
> > > > >
> > > > > I just found an issue: If I'm running an example from eclipse, the
> > JVM
> > > > > process won't terminate properly after the job has finished.
> > > > > It seems that a "Destroy JVM" thread is started and the main()
> thread
> > > has
> > > > > already gone away, but still, the JVM is not stopping.
> > > > >
> > > > >
> > > > > On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > > > Yes, the Scala API should be up to date. The latest master also
> has
> > > > > > all the Operator IT Cases from Java ported to Java. They also
> > contain
> > > > > > POJOs and nested key selectors.
> > > > > >
> > > > > > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > > > Thank you Leonidas for helping with the test.
> > > > > > >
> > > > > > > I've successfully used the maven archetypes and pulled
> > > 0.7-incubating
> > > > > > from
> > > > > > > the staged repository, so that seems to work.
> > > > > > > I found and fixed a bug with the YARN deployment and tested it
> on
> > > > > Hadoop
> > > > > > > 2.4.0 using Amazon EMR (8 nodes).
> > > > > > > I also tested Flink on a 24 nodes cluster with K-Means and
> > > different
> > > > > > > flavors of Wordcount... All good (except for this issue:
> > > > > > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > > > > > >
> > > > > > > Has the Scala API been updated to reflect the newly integrated
> > POJO
> > > > > > support?
> > > > > > >
> > > > > > > Could somebody who's using Windows try it there?
> > > > > > >
> > > > > > > We should check if the web-client shows the optimizer plans of
> > the
> > > > > > examples
> > > > > > > properly and if they all run without data.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <
> > > [hidden email]>
> > > > > > wrote:
> > > > > > >
> > > > > > >> No, its just a dry run and I want to do some tests on our
> > cluster
> > > > > first.
> > > > > > >>
> > > > > > >> Last time, I created a lot of release candidates and vote
> > threads.
> > > > > This
> > > > > > >> time, I wanted to be smart and do the testing first ;)
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> > > > > [hidden email]>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Are taking VOTE on this or you just trying to do dry run?
> > > > > > >>>
> > > > > > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <
> > > > [hidden email]>
> > > > > > >>> wrote:
> > > > > > >>> > Hi guys,
> > > > > > >>> >
> > > > > > >>> > I've created a first release candidate for the
> 0.7-incubating
> > > > major
> > > > > > >>> release:
> > > > > > >>> >
> > > > > > >>> >
> > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > This thread is not an official release vote thread, rather
> a
> > > > > > discussion
> > > > > > >>> > thread to collect all open issues that we want to address
> > until
> > > > we
> > > > > > >>> create
> > > > > > >>> > the voting release candidate.
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > Best,
> > > > > > >>> > Robert
> > > > > > >>>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Robert Metzger]

Thank you.

The issue I've mentioned before (with the JVM not stopping) has been
resolved:
https://git-wip-us.apache.org/repos/asf?p=incubator-flink.git;a=commit;h=17e9b5b2

I'll now create the artifacts for the vote

On Mon, Oct 13, 2014 at 5:56 PM, Fabian Hueske <[hidden email]> wrote:

> Checked all binary packages (except the YARN one) on Windows. Started local
> mode and executed a random example.
> All worked fine.
>
> 2014-10-11 19:21 GMT+02:00 Robert Metzger <[hidden email]>:
>
> > Yes, on different computers.
> >
> > Another issue that came in my mind regarding the release: is there any
> > documentation on the collection-based execution? I think we should have
> at
> > least a small paragraph on it.
> >
> > On Sat, Oct 11, 2014 at 1:15 PM, Stephan Ewen <[hidden email]> wrote:
> >
> > > Is it reproducible?
> > >
> > > Seems that some timers do not get canceled...
> > >
> > > On Sat, Oct 11, 2014 at 12:07 PM, Robert Metzger <[hidden email]>
> > > wrote:
> > >
> > > > WordCount [Java Application]
> > > > org.apache.flink.examples.java.wordcount.WordCount at localhost:38214
> > > > (Suspended)
> > > > Daemon System Thread [Signal Dispatcher] (Suspended)
> > > > Daemon System Thread [Finalizer] (Suspended)
> > > > Daemon System Thread [Reference Handler] (Suspended)
> > > > Thread [Timer-1] (Suspended)
> > > > waiting for: TaskQueue  (id=193)
> > > > Object.wait(long) line: not available [native method]
> > > > TimerThread.mainLoop() line: 552
> > > > TimerThread.run() line: 505
> > > > Thread [Timer-3] (Suspended)
> > > > waiting for: TaskQueue  (id=192)
> > > > Object.wait(long) line: not available [native method]
> > > > TimerThread.mainLoop() line: 552
> > > > TimerThread.run() line: 505
> > > > Thread [DestroyJavaVM] (Suspended)
> > > > /opt/java/bin/java (Oct 11, 2014, 12:02:52 PM)
> > > >
> > > >
> > > >
> > > > On Fri, Oct 10, 2014 at 7:31 PM, Stephan Ewen <[hidden email]>
> > wrote:
> > > >
> > > > > Which threads are active when the shutdown hangs?
> > > > > Am 10.10.2014 19:25 schrieb "Robert Metzger" <[hidden email]
> >:
> > > > >
> > > > > > Ah damn .. I meant the "Scala API Documentation" ;)
> > > > > >
> > > > > > I just found an issue: If I'm running an example from eclipse,
> the
> > > JVM
> > > > > > process won't terminate properly after the job has finished.
> > > > > > It seems that a "Destroy JVM" thread is started and the main()
> > thread
> > > > has
> > > > > > already gone away, but still, the JVM is not stopping.
> > > > > >
> > > > > >
> > > > > > On Fri, Oct 10, 2014 at 6:53 PM, Aljoscha Krettek <
> > > [hidden email]
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Yes, the Scala API should be up to date. The latest master also
> > has
> > > > > > > all the Operator IT Cases from Java ported to Java. They also
> > > contain
> > > > > > > POJOs and nested key selectors.
> > > > > > >
> > > > > > > On Fri, Oct 10, 2014 at 6:32 PM, Robert Metzger <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > > > Thank you Leonidas for helping with the test.
> > > > > > > >
> > > > > > > > I've successfully used the maven archetypes and pulled
> > > > 0.7-incubating
> > > > > > > from
> > > > > > > > the staged repository, so that seems to work.
> > > > > > > > I found and fixed a bug with the YARN deployment and tested
> it
> > on
> > > > > > Hadoop
> > > > > > > > 2.4.0 using Amazon EMR (8 nodes).
> > > > > > > > I also tested Flink on a 24 nodes cluster with K-Means and
> > > > different
> > > > > > > > flavors of Wordcount... All good (except for this issue:
> > > > > > > > https://issues.apache.org/jira/browse/FLINK-1152 )
> > > > > > > >
> > > > > > > > Has the Scala API been updated to reflect the newly
> integrated
> > > POJO
> > > > > > > support?
> > > > > > > >
> > > > > > > > Could somebody who's using Windows try it there?
> > > > > > > >
> > > > > > > > We should check if the web-client shows the optimizer plans
> of
> > > the
> > > > > > > examples
> > > > > > > > properly and if they all run without data.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Oct 8, 2014 at 8:50 PM, Robert Metzger <
> > > > [hidden email]>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > >> No, its just a dry run and I want to do some tests on our
> > > cluster
> > > > > > first.
> > > > > > > >>
> > > > > > > >> Last time, I created a lot of release candidates and vote
> > > threads.
> > > > > > This
> > > > > > > >> time, I wanted to be smart and do the testing first ;)
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Oct 8, 2014 at 8:46 PM, Henry Saputra <
> > > > > > [hidden email]>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Are taking VOTE on this or you just trying to do dry run?
> > > > > > > >>>
> > > > > > > >>> On Wed, Oct 8, 2014 at 4:15 AM, Robert Metzger <
> > > > > [hidden email]>
> > > > > > > >>> wrote:
> > > > > > > >>> > Hi guys,
> > > > > > > >>> >
> > > > > > > >>> > I've created a first release candidate for the
> > 0.7-incubating
> > > > > major
> > > > > > > >>> release:
> > > > > > > >>> >
> > > > > > > >>> >
> > > http://people.apache.org/~rmetzger/flink-0.7.0-incubating-rc0/
> > > > > > > >>> >
> > > > > > > >>> >
> > > > > > > >>> > This thread is not an official release vote thread,
> rather
> > a
> > > > > > > discussion
> > > > > > > >>> > thread to collect all open issues that we want to address
> > > until
> > > > > we
> > > > > > > >>> create
> > > > > > > >>> > the voting release candidate.
> > > > > > > >>> >
> > > > > > > >>> >
> > > > > > > >>> > Best,
> > > > > > > >>> > Robert
> > > > > > > >>>
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>