Scala API rewrite almost complete

[Aljoscha Krettek-2]

As already mentioned this is not possible because of type erasure. We
can only have one join variant that takes a Function2.

On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]> wrote:

> It would be nice to have a join variant that directly returns the value
> rathern than an option. Why not have both (they are wrapped as flatJoins
> anyway below, right?)
>
> On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <[hidden email]> wrote:
>
>> Sweet! I'm lovin' this :-)
>>
>> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>>
>> > Also, you can use CaseClasses directly as the type for CSV input. So
>> > instead of reading it as tuples and then having a mapper that maps to
>> > your case classes you can use:
>> >
>> > env.readCsv[Edge](...)
>> >
>> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <[hidden email]>
>> > wrote:
>> > > I added support for specifying keys by name for CaseClasses. Check out
>> > > the PageRank and TriangleEnumeration examples to see it in action.
>> > >
>> > > @Kostas: I think you could use them for the TPC-H examples.
>> > >
>> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <[hidden email]
>> >
>> > wrote:
>> > >> Yes, that would allow list comprehensions. It would be possible to
>> > >> have the Collection signature for join (and coGroup), i.e.:
>> > >>
>> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
>> > >>
>> > >> (T and O are the left and right input type, R is result type)
>> > >>
>> > >> Then you can return collections and still return an option, as in:
>> > >>
>> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...) Some(l) else
>> > None }
>> > >>
>> > >> Because there is an implicit conversion from Options to a Collection.
>> > >> This will always wrap the return value in a List with only one value.
>> > >> I'm not sure we want the overhead here. I'm also not sure whether we
>> > >> want the overhead of always having to use an Option even though the
>> > >> join always returns a value.
>> > >>
>> > >> What do you think?
>> > >>
>> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <[hidden email]>
>> > wrote:
>> > >>> Hmmm, tricky question...
>> > >>> How about the Option for Join as this is a tuple-wise operation and
>> the
>> > >>> Collection for Cogroup which is group-wise?
>> > >>> Could we in that case use list comprehensions in Cogroup functions?
>> > >>>
>> > >>> Or is that too much mixing?
>> > >>>
>> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>> > >>>
>> > >>>> I didn't look at the example either.
>> > >>>>
>> > >>>> Addings collections is easy, it's just that we can either have
>> > >>>> Collections or the Option, not both.
>> > >>>>
>> > >>>> For the coding style I followed this:
>> > >>>>
>> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>> ,
>> > >>>> which itself is based on this: http://docs.scala-lang.org/style/.
>> It
>> > >>>> is different from the Java Code Guidelines we have in place, yes.
>> > >>>>
>> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <[hidden email]
>> >
>> > >>>> wrote:
>> > >>>> > I haven't looked at the LineRank example in detail, but if you
>> > think that
>> > >>>> > it adds something new to the examples collection, we can certainly
>> > port
>> > >>>> it
>> > >>>> > also to Java.
>> > >>>> > I think the Option and Collector return types are sufficient right
>> > now
>> > >>>> but
>> > >>>> > if Collections are easy to add, go for it. ;-)
>> > >>>> >
>> > >>>> > Great that the Scala primitives are working! Also thanks for
>> adding
>> > >>>> > genSequence and adapting my examples.
>> > >>>> > Btw. does the codestyle not apply for Scala files or do we have a
>> > >>>> different
>> > >>>> > there?
>> > >>>> >
>> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <[hidden email]
>> >:
>> > >>>> >
>> > >>>> >> What about the LineRank example? We had that in Scala but never
>> > had a
>> > >>>> >> Java Example.
>> > >>>> >>
>> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
>> > [hidden email]>
>> > >>>> >> wrote:
>> > >>>> >> > Yes, I like that. For the ITCases I always just copied the Java
>> > >>>> ITCase.
>> > >>>> >> >
>> > >>>> >> > The only examples that are missing now are LinearRegression and
>> > the
>> > >>>> >> > relational stuff.
>> > >>>> >> >
>> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
>> > [hidden email]>
>> > >>>> >> wrote:
>> > >>>> >> >> I just removed the old CountEdgeDegrees example.
>> > >>>> >> >> That was a preprocessing step for the TriangleEnumeration, and
>> > is now
>> > >>>> >> part
>> > >>>> >> >> of the new TriangleEnumerationOpt example.
>> > >>>> >> >> So I guess, we don't need to port that one. As I said before,
>> > I'd
>> > >>>> >> prefer to
>> > >>>> >> >> keep Java and Scala examples in sync.
>> > >>>> >> >>
>> > >>>> >> >> Cheers, Fabian
>> > >>>> >> >>
>> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
>> > [hidden email]>:
>> > >>>> >> >>
>> > >>>> >> >>> I added the PageRank example, thanks again fabian. :D
>> > >>>> >> >>>
>> > >>>> >> >>> Regarding the other stuff:
>> > >>>> >> >>>  - There is a comment in DataSet.scala about including
>> > >>>> >> >>> org.apache.flink.api.scala._ because of the TypeInformation.
>> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
>> > >>>> >> >>>  - It is possible to use Scala Primitives in Array, I noticed
>> > it
>> > >>>> while
>> > >>>> >> >>> writing the tests, you probably had an older version of the
>> > code.
>> > >>>> >> >>>  - Yes, using List and other Interfaces is not possible, this
>> > is
>> > >>>> also
>> > >>>> >> >>> a restriction in the Java API.
>> > >>>> >> >>>
>> > >>>> >> >>> What do you think about the interface of join and coGroup?
>> > Right
>> > >>>> now,
>> > >>>> >> >>> you can either use a lambda that returns an Option or the
>> > lambda
>> > >>>> with
>> > >>>> >> >>> the Collector. Originally I wanted to have also have a lambda
>> > that
>> > >>>> >> >>> returns a Collection, but due to type erasure this has the
>> > same type
>> > >>>> >> >>> as the lambda with the Option so I couldn't use it. There is
>> an
>> > >>>> >> >>> implicit conversion from Option to a Collection, so I could
>> > change
>> > >>>> it
>> > >>>> >> >>> without breaking the examples we have now. What do you think?
>> > >>>> >> >>>
>> > >>>> >> >>> So far we have ported: WordCount, KMeans,
>> ConnectedComponents,
>> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
>> > >>>> TriangleEnumerationNaive/Opt,
>> > >>>> >> >>> PageRank
>> > >>>> >> >>>
>> > >>>> >> >>> These are the examples people called dibs on:
>> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port of
>> > >>>> LinearRegression
>> > >>>> >> >>> Example from Java)
>> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
>> > >>>> >> >>>
>> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
>> > >>>> >> >>>  - The relational Stuff
>> > >>>> >> >>>
>> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
>> > [hidden email]>
>> > >>>> >> wrote:
>> > >>>> >> >>> > +1 for removing RelationQuery
>> > >>>> >> >>> >
>> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek <
>> > >>>> >> [hidden email]>
>> > >>>> >> >>> > wrote:
>> > >>>> >> >>> >
>> > >>>> >> >>> >> By the way, what was called BatchGradientDescent in the
>> > Scala
>> > >>>> >> examples
>> > >>>> >> >>> >> should be replaced by a port of the LinearRegression
>> > Example from
>> > >>>> >> >>> >> Java. I had them as two separate examples earlier.
>> > >>>> >> >>> >>
>> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any thoughts
>> about
>> > >>>> removing
>> > >>>> >> >>> >> RelationalQuery?
>> > >>>> >> >>> >>
>> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha Krettek <
>> > >>>> >> [hidden email]
>> > >>>> >> >>> >
>> > >>>> >> >>> >> wrote:
>> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, thanks
>> Fabian.
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
>> > ConnectedComponents,
>> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
>> > >>>> >> TriangleEnumerationNaive/Opt
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> > These are the examples people called dibs on:
>> > >>>> >> >>> >> >  - PageRank (Fabian)
>> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
>> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
>> > >>>> >> >>> >> >  - The relational Stuff
>> > >>>> >> >>> >> >  - LinearRegression
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha Krettek <
>> > >>>> >> >>> [hidden email]>
>> > >>>> >> >>> >> wrote:
>> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list of
>> > >>>> ported/unported
>> > >>>> >> >>> >> >> examples in my mails. I'll rename the java example
>> > package to
>> > >>>> >> >>> examples
>> > >>>> >> >>> >> >> once the Scala API merge is done.
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> I think the termination criterion is fine as it is.
>> Just
>> > >>>> because
>> > >>>> >> >>> Scala
>> > >>>> >> >>> >> >> enables functional programming doesn't mean it's always
>> > the
>> > >>>> best
>> > >>>> >> >>> >> >> choice. :D
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
>> > ConnectedComponents,
>> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> These are the examples people called dibs on:
>> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
>> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
>> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
>> > >>>> >> >>> >> >>  - The relational Stuff
>> > >>>> >> >>> >> >>  - LinearRegression
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> Cheers,
>> > >>>> >> >>> >> >> Aljoscha
>> > >>>> >> >>> >> >>
>> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas Tzoumas <
>> > >>>> >> [hidden email]
>> > >>>> >> >>> >
>> > >>>> >> >>> >> wrote:
>> > >>>> >> >>> >> >>> Transitive closure here, I also added a termination
>> > criterion
>> > >>>> >> in the
>> > >>>> >> >>> >> Java
>> > >>>> >> >>> >> >>> version:
>> > >>>> >> >>> >>
>> > >>>> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
>> > >>>> >> >>> >> >>>
>> > >>>> >> >>> >> >>> Perhaps you can make the termination criterion in
>> Scala
>> > more
>> > >>>> >> >>> >> functional?
>> > >>>> >> >>> >> >>>
>> > >>>> >> >>> >> >>> I noticed that the examples package name is
>> > example.java but
>> > >>>> >> >>> >> examples.scala
>> > >>>> >> >>> >> >>>
>> > >>>> >> >>> >> >>> Kostas
>> > >>>> >> >>> >> >>>
>> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas Tzoumas <
>> > >>>> >> [hidden email]
>> > >>>> >> >>> >
>> > >>>> >> >>> >> wrote:
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation (was not
>> > on
>> > >>>> your
>> > >>>> >> >>> list).
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>> If nobody volunteers for the relational stuff I can
>> > take
>> > >>>> those
>> > >>>> >> as
>> > >>>> >> >>> >> well.
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery" from both
>> > Scala and
>> > >>>> >> Java?
>> > >>>> >> >>> It
>> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3. Does it add
>> > some
>> > >>>> >> teaching
>> > >>>> >> >>> >> value on
>> > >>>> >> >>> >> >>>> top of TPC-H Q3?
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>> Kostas
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha Krettek <
>> > >>>> >> >>> [hidden email]
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> >>>> wrote:
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
>> > >>>> ConnectedComponents,
>> > >>>> >> >>> >> >>>>> WebLogAnalysis
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> These are the examples people called dibs on:
>> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
>> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
>> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
>> > >>>> >> >>> >> >>>>>  - TransitiveClosure
>> > >>>> >> >>> >> >>>>>  - The relational Stuff
>> > >>>> >> >>> >> >>>>>  - LinearRegression
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> Cheers,
>> > >>>> >> >>> >> >>>>> Aljoscha
>> > >>>> >> >>> >> >>>>>
>> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas Tzoumas <
>> > >>>> >> >>> [hidden email]>
>> > >>>> >> >>> >> >>>>> wrote:
>> > >>>> >> >>> >> >>>>> > WebLog here:
>> > >>>> >> >>> >> >>>>> >
>> > >>>> >> >>> >> >>>>> >
>> > >>>> >> >>> >>
>> > >>>> >> >>>
>> > >>>> >>
>> > >>>>
>> >
>> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
>> > >>>> >> >>> >> >>>>> >
>> > >>>> >> >>> >> >>>>> > Do you need any more done?
>> > >>>> >> >>> >> >>>>> >
>> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha Krettek <
>> > >>>> >> >>> >> [hidden email]>
>> > >>>> >> >>> >> >>>>> > wrote:
>> > >>>> >> >>> >> >>>>> >
>> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example from
>> Vasia.
>> > >>>> >> >>> >> >>>>> >>
>> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
>> > >>>> >> >>> >> >>>>> >>
>> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian Hueske <
>> > >>>> >> >>> [hidden email]
>> > >>>> >> >>> >> >
>> > >>>> >> >>> >> >>>>> >> wrote:
>> > >>>> >> >>> >> >>>>> >> > Alright, will do.
>> > >>>> >> >>> >> >>>>> >> > Thanks!
>> > >>>> >> >>> >> >>>>> >> >
>> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha Krettek <
>> > >>>> >> >>> >> [hidden email]>:
>> > >>>> >> >>> >> >>>>> >> >
>> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
>> > KMeans.scala. I'm
>> > >>>> >> storing
>> > >>>> >> >>> >> the
>> > >>>> >> >>> >> >>>>> >> >> data
>> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays and then
>> > >>>> converting
>> > >>>> >> it to
>> > >>>> >> >>> >> the
>> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo to make
>> it
>> > >>>> >> consistent
>> > >>>> >> >>> >> with the
>> > >>>> >> >>> >> >>>>> >> >> Java
>> > >>>> >> >>> >> >>>>> >> >> API.
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There is no need
>> > for a
>> > >>>> >> >>> keyword,
>> > >>>> >> >>> >> you
>> > >>>> >> >>> >> >>>>> >> >> can
>> > >>>> >> >>> >> >>>>> >> >> just write:
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1) { (le,
>> re)
>> > =>
>> > >>>> new
>> > >>>> >> >>> >> MyResult(le,
>> > >>>> >> >>> >> >>>>> >> >> re)
>> > >>>> >> >>> >> >>>>> >> }
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, Fabian Hueske
>> <
>> > >>>> >> >>> >> [hidden email]>
>> > >>>> >> >>> >> >>>>> >> wrote:
>> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I also found
>> an
>> > >>>> >> >>> inconsistency
>> > >>>> >> >>> >> with
>> > >>>> >> >>> >> >>>>> >> >> > the
>> > >>>> >> >>> >> >>>>> >> Java
>> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> > ds1.join(ds2).where(...).equalTo(...)
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> > ds1.join(d2).where(...).isEqualTo(...)
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to equalTo(),
>> > IMO.
>> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?) lacks the
>> > with()
>> > >>>> >> method
>> > >>>> >> >>> >> because
>> > >>>> >> >>> >> >>>>> >> "with"
>> > >>>> >> >>> >> >>>>> >> >> is
>> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
>> something
>> > >>>> similar
>> > >>>> >> for
>> > >>>> >> >>> >> Scala
>> > >>>> >> >>> >> >>>>> >> >> > or go
>> > >>>> >> >>> >> >>>>> >> >> with
>> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan Ewen <
>> > >>>> >> [hidden email]
>> > >>>> >> >>> >:
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][] would work
>> as
>> > well.
>> > >>>> >> That
>> > >>>> >> >>> is a
>> > >>>> >> >>> >> >>>>> >> >> >> generic
>> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
>> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be stored as Java
>> > or
>> > >>>> Scala
>> > >>>> >> >>> Tuples,
>> > >>>> >> >>> >> >>>>> >> >> >> with a
>> > >>>> >> >>> >> >>>>> >> >> generic
>> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between the two.
>> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM, Fabian
>> > Hueske
>> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> wrote:
>> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same problem...
>> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing them,
>> but
>> > >>>> using
>> > >>>> >> the
>> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
>> > >>>> >> >>> >> >>>>> >> won't
>> > >>>> >> >>> >> >>>>> >> >> >> work
>> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
>> FileInputFormat.
>> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the Strings
>> > >>>> manually...
>> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00 Aljoscha
>> Krettek
>> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
>> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
>> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we should just
>> > change
>> > >>>> >> all
>> > >>>> >> >>> the
>> > >>>> >> >>> >> >>>>> >> >> >> > > example
>> > >>>> >> >>> >> >>>>> >> input
>> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV input
>> > formats in
>> > >>>> all
>> > >>>> >> the
>> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
>> > >>>> >> >>> >> >>>>> >> What
>> > >>>> >> >>> >> >>>>> >> >> do
>> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
>> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
>> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
>> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46 AM,
>> Aljoscha
>> > >>>> Krettek
>> > >>>> >> <
>> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
>> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that the data
>> > types are
>> > >>>> >> >>> >> incompatible.
>> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
>> > >>>> >> >>> >> >>>>> >> >> afraid
>> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you proposed:
>> move
>> > the
>> > >>>> >> data to
>> > >>>> >> >>> a
>> > >>>> >> >>> >> >>>>> >> >> >> > > > static
>> > >>>> >> >>> >> >>>>> >> field
>> > >>>> >> >>> >> >>>>> >> >> and
>> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
>> > getDefaultEdgeDataSet()
>> > >>>> >> method in
>> > >>>> >> >>> >> Scala.
>> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
>> > >>>> >> >>> >> >>>>> >> >> not
>> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would duplicate the
>> > data
>> > >>>> and
>> > >>>> >> >>> make it
>> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
>> > >>>> >> >>> >> >>>>> >> for
>> > >>>> >> >>> >> >>>>> >> >> it
>> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the Java and
>> Scala
>> > >>>> >> versions.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think? This will
>> > probably
>> > >>>> >> occur
>> > >>>> >> >>> in
>> > >>>> >> >>> >> all
>> > >>>> >> >>> >> >>>>> >> >> >> > > > the
>> > >>>> >> >>> >> >>>>> >> >> >> examples.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
>> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
>> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04 PM,
>> > Vasiliki
>> > >>>> >> Kalavri
>> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]> wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
>> Components
>> > >>>> >> example,
>> > >>>> >> >>> but
>> > >>>> >> >>> >> I am
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
>> > >>>> >> >>> >> >>>>> >> sure
>> > >>>> >> >>> >> >>>>> >> >> >> how
>> > >>>> >> >>> >> >>>>> >> >> >> > to
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input data from
>> > >>>> >> java-examples.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the ConnectedComponentsData
>> class,
>> > the
>> > >>>> >> vertices
>> > >>>> >> >>> >> and
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
>> > >>>> >> >>> >> >>>>> >> data
>> > >>>> >> >>> >> >>>>> >> >> >> are
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
>> > >>>> >> getDefaultVertexDataSet()
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> and getDefaultEdgeDataSet(), which
>> > take
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
>> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
>> > >>>> >> >>> as
>> > >>>> >> >>> >> >>>>> >> parameter.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide public static
>> > fields
>> > >>>> >> (like
>> > >>>> >> >>> in
>> > >>>> >> >>> >> the
>> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this introduces a
>> > conversion
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
>> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
>> > >>>> >> >>> Scala
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
>> > >>>> >> >>> >> >>>>> >> >> from
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to scala.Long and I
>> > guess
>> > >>>> this
>> > >>>> >> is
>> > >>>> >> >>> an
>> > >>>> >> >>> >> >>>>> >> unnecessary
>> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course, to copy
>> the
>> > >>>> example
>> > >>>> >> >>> data
>> > >>>> >> >>> >> in
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
>> > >>>> >> >>> >> >>>>> >> Scala
>> > >>>> >> >>> >> >>>>> >> >> >> > > example.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something here?
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52, Aljoscha
>> > >>>> Krettek <
>> > >>>> >> >>> >> >>>>> >> [hidden email]
>> > >>>> >> >>> >> >>>>> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my repo:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >>
>> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working WordCount
>> > example.
>> > >>>> >> It's
>> > >>>> >> >>> >> pretty
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
>> > >>>> >> >>> >> >>>>> >> >> copy
>> > >>>> >> >>> >> >>>>> >> >> >> of
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some fixups
>> > for the
>> > >>>> >> syntax
>> > >>>> >> >>> and
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
>> > >>>> >> >>> >> >>>>> >> >> >> > functions.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I added the
>> > >>>> >> java-examples
>> > >>>> >> >>> >> as a
>> > >>>> >> >>> >> >>>>> >> >> dependency
>> > >>>> >> >>> >> >>>>> >> >> >> for
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did this to
>> > reuse
>> > >>>> the
>> > >>>> >> >>> example
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
>> > >>>> >> >>> >> >>>>> >> data.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program you can
>> do
>> > a
>> > >>>> pull
>> > >>>> >> >>> request
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
>> > >>>> >> >>> >> >>>>> >> my
>> > >>>> >> >>> >> >>>>> >> >> repo
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the examples.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at 12:19 PM,
>> > Hermann
>> > >>>> >> Gábor <
>> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
>> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for me!
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at 11:44 AM,
>> > Márton
>> > >>>> >> >>> Balassi <
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent for me :)
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at 11:15 AM,
>> > Kostas
>> > >>>> >> >>> Tzoumas <
>> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for WebLogAnalysis.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with Scala
>> > consists of
>> > >>>> >> going
>> > >>>> >> >>> >> through
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
>> > >>>> >> >>> >> >>>>> >> >> tutorial
>> > >>>> >> >>> >> >>>>> >> >> >> so
>> > >>>> >> >>> >> >>>>> >> >> >> > > this
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test both for
>> > me and
>> > >>>> >> the
>> > >>>> >> >>> new
>> > >>>> >> >>> >> API
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014 at 9:09
>> PM,
>> > >>>> Vasiliki
>> > >>>> >> >>> >> Kalavri <
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other people
>> > >>>> implement
>> > >>>> >> the
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected Components and
>> > Kmeans for
>> > >>>> >> me :)
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014 21:03,
>> > Fabian
>> > >>>> >> Hueske <
>> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
>> > TriangleEnumeration and
>> > >>>> >> >>> PageRank.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the examples
>> > >>>> similar
>> > >>>> >> to
>> > >>>> >> >>> the
>> > >>>> >> >>> >> Java
>> > >>>> >> >>> >> >>>>> >> >> examples:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running out-of-the-box
>> > without
>> > >>>> >> >>> parameters
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for external
>> > data
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a similar code
>> > structure
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56 GMT+02:00
>> > >>>> Aljoscha
>> > >>>> >> >>> >> Krettek <
>> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then people can
>> > >>>> reserve
>> > >>>> >> their
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
>> > >>>> >> >>> >> >>>>> >> >> >> examples
>> > >>>> >> >>> >> >>>>> >> >> >> > > here.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4, 2014 at
>> > 8:55 PM,
>> > >>>> >> Fabian
>> > >>>> >> >>> >> Hueske
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think having
>> examples
>> > >>>> >> implemented
>> > >>>> >> >>> by
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
>> > >>>> >> >>> >> >>>>> >> >> >> people
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in the past.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with two or
>> > three
>> > >>>> >> examples.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be helpful if
>> > you'd
>> > >>>> >> port a
>> > >>>> >> >>> >> simple
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
>> > >>>> >> >>> >> >>>>> >> >> one
>> > >>>> >> >>> >> >>>>> >> >> >> > such
>> > >>>> >> >>> >> >>>>> >> >> >> > > as
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04 18:47
>> > GMT+02:00
>> > >>>> >> Aljoscha
>> > >>>> >> >>> >> Krettek
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a working
>> > rewrite of
>> > >>>> the
>> > >>>> >> >>> Scala
>> > >>>> >> >>> >> API
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>>> >> >>> >>
>> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping that I'll
>> > only
>> > >>>> have
>> > >>>> >> to
>> > >>>> >> >>> >> write
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
>> > >>>> >> >>> >> >>>>> >> tests
>> > >>>> >> >>> >> >>>>> >> >> and
>> > >>>> >> >>> >> >>>>> >> >> >> > > port
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do you
>> think
>> > it
>> > >>>> makes
>> > >>>> >> >>> sense
>> > >>>> >> >>> >> to
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
>> > >>>> >> >>> >> >>>>> >> other
>> > >>>> >> >>> >> >>>>> >> >> >> > people
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so that
>> > someone
>> > >>>> else
>> > >>>> >> uses
>> > >>>> >> >>> >> it and
>> > >>>> >> >>> >> >>>>> >> maybe
>> > >>>> >> >>> >> >>>>> >> >> >> > notices
>> > >>>> >> >>> >> >>>>> >> >> >> > > some
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>>> >> >>> >> >>>>> >> >>
>> > >>>> >> >>> >> >>>>> >>
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>>
>> > >>>> >> >>> >> >>>
>> > >>>> >> >>> >>
>> > >>>> >> >>>
>> > >>>> >>
>> > >>>>
>> >
>>

[Stephan Ewen]

I think it seems weird that normal joins need to go through option.

The option variant is to allow filters in the join function. Wouldn't a
collector variant allow you to do the same, and would be function3 ? I know
that option reads more functionally...



Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]>:

>
> As already mentioned this is not possible because of type erasure. We
> can only have one join variant that takes a Function2.
>
> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]> wrote:
> > It would be nice to have a join variant that directly returns the value
> > rathern than an option. Why not have both (they are wrapped as flatJoins
> > anyway below, right?)
> >
> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <[hidden email]>

wrote:

> >
> >> Sweet! I'm lovin' this :-)
> >>
> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> >>
> >> > Also, you can use CaseClasses directly as the type for CSV input. So
> >> > instead of reading it as tuples and then having a mapper that maps to
> >> > your case classes you can use:
> >> >
> >> > env.readCsv[Edge](...)
> >> >
> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <

[hidden email]>
> >> > wrote:
> >> > > I added support for specifying keys by name for CaseClasses. Check
out
> >> > > the PageRank and TriangleEnumeration examples to see it in action.
> >> > >
> >> > > @Kostas: I think you could use them for the TPC-H examples.
> >> > >
> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
[hidden email]

> >> >
> >> > wrote:
> >> > >> Yes, that would allow list comprehensions. It would be possible to
> >> > >> have the Collection signature for join (and coGroup), i.e.:
> >> > >>
> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> >> > >>
> >> > >> (T and O are the left and right input type, R is result type)
> >> > >>
> >> > >> Then you can return collections and still return an option, as in:
> >> > >>
> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...) Some(l)

else
> >> > None }
> >> > >>
> >> > >> Because there is an implicit conversion from Options to a
Collection.
> >> > >> This will always wrap the return value in a List with only one
value.
> >> > >> I'm not sure we want the overhead here. I'm also not sure whether
we
> >> > >> want the overhead of always having to use an Option even though
the
> >> > >> join always returns a value.
> >> > >>
> >> > >> What do you think?
> >> > >>
> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
[hidden email]>
> >> > wrote:
> >> > >>> Hmmm, tricky question...
> >> > >>> How about the Option for Join as this is a tuple-wise operation
and
> >> the
> >> > >>> Collection for Cogroup which is group-wise?
> >> > >>> Could we in that case use list comprehensions in Cogroup
functions?

> >> > >>>
> >> > >>> Or is that too much mixing?
> >> > >>>
> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <[hidden email]
>:
> >> > >>>
> >> > >>>> I didn't look at the example either.
> >> > >>>>
> >> > >>>> Addings collections is easy, it's just that we can either have
> >> > >>>> Collections or the Option, not both.
> >> > >>>>
> >> > >>>> For the coding style I followed this:
> >> > >>>>
> >> >

https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> >> ,
> >> > >>>> which itself is based on this: http://docs.scala-lang.org/style/
.
> >> It
> >> > >>>> is different from the Java Code Guidelines we have in place,
yes.
> >> > >>>>
> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
[hidden email]
> >> >
> >> > >>>> wrote:
> >> > >>>> > I haven't looked at the LineRank example in detail, but if you
> >> > think that
> >> > >>>> > it adds something new to the examples collection, we can
certainly
> >> > port
> >> > >>>> it
> >> > >>>> > also to Java.
> >> > >>>> > I think the Option and Collector return types are sufficient
right
> >> > now
> >> > >>>> but
> >> > >>>> > if Collections are easy to add, go for it. ;-)
> >> > >>>> >
> >> > >>>> > Great that the Scala primitives are working! Also thanks for
> >> adding
> >> > >>>> > genSequence and adapting my examples.
> >> > >>>> > Btw. does the codestyle not apply for Scala files or do we
have a
> >> > >>>> different
> >> > >>>> > there?
> >> > >>>> >
> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
[hidden email]
> >> >:
> >> > >>>> >
> >> > >>>> >> What about the LineRank example? We had that in Scala but
never
> >> > had a
> >> > >>>> >> Java Example.
> >> > >>>> >>
> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
> >> > [hidden email]>
> >> > >>>> >> wrote:
> >> > >>>> >> > Yes, I like that. For the ITCases I always just copied the
Java
> >> > >>>> ITCase.
> >> > >>>> >> >
> >> > >>>> >> > The only examples that are missing now are
LinearRegression and
> >> > the
> >> > >>>> >> > relational stuff.
> >> > >>>> >> >
> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
> >> > [hidden email]>
> >> > >>>> >> wrote:
> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
> >> > >>>> >> >> That was a preprocessing step for the
TriangleEnumeration, and
> >> > is now
> >> > >>>> >> part
> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> >> > >>>> >> >> So I guess, we don't need to port that one. As I said
before,

> >> > I'd
> >> > >>>> >> prefer to
> >> > >>>> >> >> keep Java and Scala examples in sync.
> >> > >>>> >> >>
> >> > >>>> >> >> Cheers, Fabian
> >> > >>>> >> >>
> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> >> > [hidden email]>:
> >> > >>>> >> >>
> >> > >>>> >> >>> I added the PageRank example, thanks again fabian. :D
> >> > >>>> >> >>>
> >> > >>>> >> >>> Regarding the other stuff:
> >> > >>>> >> >>>  - There is a comment in DataSet.scala about including
> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the

TypeInformation.
> >> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
> >> > >>>> >> >>>  - It is possible to use Scala Primitives in Array, I
noticed
> >> > it
> >> > >>>> while
> >> > >>>> >> >>> writing the tests, you probably had an older version of
the
> >> > code.
> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not possible,
this
> >> > is
> >> > >>>> also
> >> > >>>> >> >>> a restriction in the Java API.
> >> > >>>> >> >>>
> >> > >>>> >> >>> What do you think about the interface of join and
coGroup?
> >> > Right
> >> > >>>> now,
> >> > >>>> >> >>> you can either use a lambda that returns an Option or the
> >> > lambda
> >> > >>>> with
> >> > >>>> >> >>> the Collector. Originally I wanted to have also have a
lambda
> >> > that
> >> > >>>> >> >>> returns a Collection, but due to type erasure this has
the
> >> > same type
> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
There is
> >> an
> >> > >>>> >> >>> implicit conversion from Option to a Collection, so I
could
> >> > change
> >> > >>>> it
> >> > >>>> >> >>> without breaking the examples we have now. What do you
think?

> >> > >>>> >> >>>
> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> >> ConnectedComponents,
> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> >> > >>>> TriangleEnumerationNaive/Opt,
> >> > >>>> >> >>> PageRank
> >> > >>>> >> >>>
> >> > >>>> >> >>> These are the examples people called dibs on:
> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port of
> >> > >>>> LinearRegression
> >> > >>>> >> >>> Example from Java)
> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> >> > >>>> >> >>>
> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> >> > >>>> >> >>>  - The relational Stuff
> >> > >>>> >> >>>
> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
> >> > [hidden email]>
> >> > >>>> >> wrote:
> >> > >>>> >> >>> > +1 for removing RelationQuery
> >> > >>>> >> >>> >
> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek <
> >> > >>>> >> [hidden email]>
> >> > >>>> >> >>> > wrote:
> >> > >>>> >> >>> >
> >> > >>>> >> >>> >> By the way, what was called BatchGradientDescent in

the

> >> > Scala
> >> > >>>> >> examples
> >> > >>>> >> >>> >> should be replaced by a port of the LinearRegression
> >> > Example from
> >> > >>>> >> >>> >> Java. I had them as two separate examples earlier.
> >> > >>>> >> >>> >>
> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any thoughts
> >> about
> >> > >>>> removing
> >> > >>>> >> >>> >> RelationalQuery?
> >> > >>>> >> >>> >>
> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha Krettek <
> >> > >>>> >> [hidden email]
> >> > >>>> >> >>> >
> >> > >>>> >> >>> >> wrote:
> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, thanks
> >> Fabian.
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
> >> > ConnectedComponents,
> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> >> > >>>> >> TriangleEnumerationNaive/Opt
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> > These are the examples people called dibs on:
> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
> >> > >>>> >> >>> >> >  - The relational Stuff
> >> > >>>> >> >>> >> >  - LinearRegression
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha Krettek <
> >> > >>>> >> >>> [hidden email]>
> >> > >>>> >> >>> >> wrote:
> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list of
> >> > >>>> ported/unported
> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java example
> >> > package to
> >> > >>>> >> >>> examples
> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> I think the termination criterion is fine as it is.
> >> Just
> >> > >>>> because
> >> > >>>> >> >>> Scala
> >> > >>>> >> >>> >> >> enables functional programming doesn't mean it's

always

> >> > the
> >> > >>>> best
> >> > >>>> >> >>> >> >> choice. :D
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
> >> > ConnectedComponents,
> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
> >> > >>>> >> >>> >> >>  - The relational Stuff
> >> > >>>> >> >>> >> >>  - LinearRegression
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> Cheers,
> >> > >>>> >> >>> >> >> Aljoscha
> >> > >>>> >> >>> >> >>
> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas Tzoumas <
> >> > >>>> >> [hidden email]
> >> > >>>> >> >>> >
> >> > >>>> >> >>> >> wrote:
> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a

termination
> >> > criterion
> >> > >>>> >> in the
> >> > >>>> >> >>> >> Java
> >> > >>>> >> >>> >> >>> version:
> >> > >>>> >> >>> >>
> >> > >>>>
https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example

> >> > >>>> >> >>> >> >>>
> >> > >>>> >> >>> >> >>> Perhaps you can make the termination criterion in
> >> Scala
> >> > more
> >> > >>>> >> >>> >> functional?
> >> > >>>> >> >>> >> >>>
> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
> >> > example.java but
> >> > >>>> >> >>> >> examples.scala
> >> > >>>> >> >>> >> >>>
> >> > >>>> >> >>> >> >>> Kostas
> >> > >>>> >> >>> >> >>>
> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas Tzoumas <
> >> > >>>> >> [hidden email]
> >> > >>>> >> >>> >
> >> > >>>> >> >>> >> wrote:
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation

(was not
> >> > on
> >> > >>>> your
> >> > >>>> >> >>> list).
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational stuff I
can
> >> > take
> >> > >>>> those
> >> > >>>> >> as
> >> > >>>> >> >>> >> well.
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery" from
both
> >> > Scala and
> >> > >>>> >> Java?
> >> > >>>> >> >>> It
> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3. Does it
add
> >> > some
> >> > >>>> >> teaching
> >> > >>>> >> >>> >> value on
> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>> Kostas
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha Krettek
<

> >> > >>>> >> >>> [hidden email]
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> >>>> wrote:
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
> >> > >>>> ConnectedComponents,
> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs on:
> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> Cheers,
> >> > >>>> >> >>> >> >>>>> Aljoscha
> >> > >>>> >> >>> >> >>>>>
> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas Tzoumas <
> >> > >>>> >> >>> [hidden email]>
> >> > >>>> >> >>> >> >>>>> wrote:
> >> > >>>> >> >>> >> >>>>> > WebLog here:
> >> > >>>> >> >>> >> >>>>> >
> >> > >>>> >> >>> >> >>>>> >
> >> > >>>> >> >>> >>
> >> > >>>> >> >>>
> >> > >>>> >>
> >> > >>>>
> >> >
> >>

https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> >> > >>>> >> >>> >> >>>>> >
> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> >> > >>>> >> >>> >> >>>>> >
> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
Krettek <
> >> > >>>> >> >>> >> [hidden email]>
> >> > >>>> >> >>> >> >>>>> > wrote:
> >> > >>>> >> >>> >> >>>>> >
> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example from
> >> Vasia.
> >> > >>>> >> >>> >> >>>>> >>
> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> >> > >>>> >> >>> >> >>>>> >>
> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
Hueske <
> >> > >>>> >> >>> [hidden email]
> >> > >>>> >> >>> >> >
> >> > >>>> >> >>> >> >>>>> >> wrote:
> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> >> > >>>> >> >>> >> >>>>> >> >
> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
Krettek <

> >> > >>>> >> >>> >> [hidden email]>:
> >> > >>>> >> >>> >> >>>>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
> >> > KMeans.scala. I'm
> >> > >>>> >> storing
> >> > >>>> >> >>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> data
> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays and

then
> >> > >>>> converting
> >> > >>>> >> it to
> >> > >>>> >> >>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo to
make
> >> it
> >> > >>>> >> consistent
> >> > >>>> >> >>> >> with the
> >> > >>>> >> >>> >> >>>>> >> >> Java
> >> > >>>> >> >>> >> >>>>> >> >> API.
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There is no
need
> >> > for a
> >> > >>>> >> >>> keyword,
> >> > >>>> >> >>> >> you
> >> > >>>> >> >>> >> >>>>> >> >> can
> >> > >>>> >> >>> >> >>>>> >> >> just write:
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1) {
(le,
> >> re)
> >> > =>
> >> > >>>> new
> >> > >>>> >> >>> >> MyResult(le,
> >> > >>>> >> >>> >> >>>>> >> >> re)
> >> > >>>> >> >>> >> >>>>> >> }
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, Fabian
Hueske
> >> <
> >> > >>>> >> >>> >> [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> wrote:
> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I also
found

> >> an
> >> > >>>> >> >>> inconsistency
> >> > >>>> >> >>> >> with
> >> > >>>> >> >>> >> >>>>> >> >> > the
> >> > >>>> >> >>> >> >>>>> >> Java
> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(ds2).where(...).equalTo(...)
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(d2).where(...).isEqualTo(...)
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to

equalTo(),
> >> > IMO.
> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?) lacks
the

> >> > with()
> >> > >>>> >> method
> >> > >>>> >> >>> >> because
> >> > >>>> >> >>> >> >>>>> >> "with"
> >> > >>>> >> >>> >> >>>>> >> >> is
> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
> >> something
> >> > >>>> similar
> >> > >>>> >> for
> >> > >>>> >> >>> >> Scala
> >> > >>>> >> >>> >> >>>>> >> >> > or go
> >> > >>>> >> >>> >> >>>>> >> >> with
> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan Ewen

<
> >> > >>>> >> [hidden email]
> >> > >>>> >> >>> >:
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][] would
work
> >> as
> >> > well.
> >> > >>>> >> That
> >> > >>>> >> >>> is a
> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be stored as
Java
> >> > or
> >> > >>>> Scala
> >> > >>>> >> >>> Tuples,
> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> >> > >>>> >> >>> >> >>>>> >> >> generic
> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between the
two.
> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM, Fabian
> >> > Hueske
> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same problem...
> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing
them,

> >> but
> >> > >>>> using
> >> > >>>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> >> > >>>> >> >>> >> >>>>> >> won't
> >> > >>>> >> >>> >> >>>>> >> >> >> work
> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
> >> FileInputFormat.
> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the Strings
> >> > >>>> manually...
> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00 Aljoscha
> >> Krettek
> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we should

just

> >> > change
> >> > >>>> >> all
> >> > >>>> >> >>> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> >> > >>>> >> >>> >> >>>>> >> input
> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV input
> >> > formats in
> >> > >>>> all
> >> > >>>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> >> > >>>> >> >>> >> >>>>> >> What
> >> > >>>> >> >>> >> >>>>> >> >> do
> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46 AM,
> >> Aljoscha
> >> > >>>> Krettek
> >> > >>>> >> <
> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that the

data

> >> > types are
> >> > >>>> >> >>> >> incompatible.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> >> > >>>> >> >>> >> >>>>> >> >> afraid
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you proposed:
> >> move
> >> > the
> >> > >>>> >> data to
> >> > >>>> >> >>> a
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> >> > >>>> >> >>> >> >>>>> >> field
> >> > >>>> >> >>> >> >>>>> >> >> and
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> >> > getDefaultEdgeDataSet()
> >> > >>>> >> method in
> >> > >>>> >> >>> >> Scala.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> >> > >>>> >> >>> >> >>>>> >> >> not
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would

duplicate the

> >> > data
> >> > >>>> and
> >> > >>>> >> >>> make it
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> >> > >>>> >> >>> >> >>>>> >> for
> >> > >>>> >> >>> >> >>>>> >> >> it
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the Java and
> >> Scala
> >> > >>>> >> versions.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think? This

will

> >> > probably
> >> > >>>> >> occur
> >> > >>>> >> >>> in
> >> > >>>> >> >>> >> all
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04 PM,
> >> > Vasiliki
> >> > >>>> >> Kalavri
> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]>

wrote:

> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
> >> Components
> >> > >>>> >> example,
> >> > >>>> >> >>> but
> >> > >>>> >> >>> >> I am
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> >> > >>>> >> >>> >> >>>>> >> sure
> >> > >>>> >> >>> >> >>>>> >> >> >> how
> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input data

from

> >> > >>>> >> java-examples.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the ConnectedComponentsData
> >> class,
> >> > the
> >> > >>>> >> vertices
> >> > >>>> >> >>> >> and
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> >> > >>>> >> >>> >> >>>>> >> data
> >> > >>>> >> >>> >> >>>>> >> >> >> are
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
> >> > >>>> >> getDefaultVertexDataSet()
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and getDefaultEdgeDataSet(),

which
> >> > take
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> >> > >>>> >> >>> as
> >> > >>>> >> >>> >> >>>>> >> parameter.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide public
static

> >> > fields
> >> > >>>> >> (like
> >> > >>>> >> >>> in
> >> > >>>> >> >>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this introduces a
> >> > conversion
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> >> > >>>> >> >>> Scala
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> >> > >>>> >> >>> >> >>>>> >> >> from
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to scala.Long

and I
> >> > guess
> >> > >>>> this
> >> > >>>> >> is
> >> > >>>> >> >>> an
> >> > >>>> >> >>> >> >>>>> >> unnecessary
> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course, to
copy

> >> the
> >> > >>>> example
> >> > >>>> >> >>> data
> >> > >>>> >> >>> >> in
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> >> > >>>> >> >>> >> >>>>> >> Scala
> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something here?
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,

Aljoscha

> >> > >>>> Krettek <
> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my repo:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >>
> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working

WordCount
> >> > example.
> >> > >>>> >> It's
> >> > >>>> >> >>> >> pretty
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> >> > >>>> >> >>> >> >>>>> >> >> copy
> >> > >>>> >> >>> >> >>>>> >> >> >> of
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some
fixups
> >> > for the
> >> > >>>> >> syntax
> >> > >>>> >> >>> and
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
added the
> >> > >>>> >> java-examples
> >> > >>>> >> >>> >> as a
> >> > >>>> >> >>> >> >>>>> >> >> dependency
> >> > >>>> >> >>> >> >>>>> >> >> >> for
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did this
to
> >> > reuse
> >> > >>>> the
> >> > >>>> >> >>> example
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> >> > >>>> >> >>> >> >>>>> >> data.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program you
can
> >> do
> >> > a
> >> > >>>> pull
> >> > >>>> >> >>> request
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> >> > >>>> >> >>> >> >>>>> >> my
> >> > >>>> >> >>> >> >>>>> >> >> repo
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
examples.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at 12:19
PM,

> >> > Hermann
> >> > >>>> >> Gábor <
> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for me!
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at 11:44

AM,
> >> > Márton
> >> > >>>> >> >>> Balassi <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent for me
:)
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
11:15 AM,

> >> > Kostas
> >> > >>>> >> >>> Tzoumas <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for WebLogAnalysis.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with Scala
> >> > consists of
> >> > >>>> >> going
> >> > >>>> >> >>> >> through
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> >> > >>>> >> >>> >> >>>>> >> >> >> so
> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test

both for
> >> > me and
> >> > >>>> >> the
> >> > >>>> >> >>> new
> >> > >>>> >> >>> >> API
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014 at
9:09
> >> PM,
> >> > >>>> Vasiliki
> >> > >>>> >> >>> >> Kalavri <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other
people

> >> > >>>> implement
> >> > >>>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected Components and
> >> > Kmeans for
> >> > >>>> >> me :)
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014

21:03,

> >> > Fabian
> >> > >>>> >> Hueske <
> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> >> > TriangleEnumeration and
> >> > >>>> >> >>> PageRank.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the

examples
> >> > >>>> similar
> >> > >>>> >> to
> >> > >>>> >> >>> the
> >> > >>>> >> >>> >> Java
> >> > >>>> >> >>> >> >>>>> >> >> examples:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
out-of-the-box
> >> > without
> >> > >>>> >> >>> parameters
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
external
> >> > data
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a similar
code
> >> > structure
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
GMT+02:00
> >> > >>>> Aljoscha
> >> > >>>> >> >>> >> Krettek <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
people can
> >> > >>>> reserve
> >> > >>>> >> their
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4, 2014
at

> >> > 8:55 PM,
> >> > >>>> >> Fabian
> >> > >>>> >> >>> >> Hueske
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think having
> >> examples
> >> > >>>> >> implemented
> >> > >>>> >> >>> by
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
> >> > >>>> >> >>> >> >>>>> >> >> >> people
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in the

past.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with two
or
> >> > three
> >> > >>>> >> examples.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
helpful if

> >> > you'd
> >> > >>>> >> port a
> >> > >>>> >> >>> >> simple
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> >> > >>>> >> >>> >> >>>>> >> >> one
> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04 18:47
> >> > GMT+02:00
> >> > >>>> >> Aljoscha
> >> > >>>> >> >>> >> Krettek
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a working
> >> > rewrite of
> >> > >>>> the
> >> > >>>> >> >>> Scala
> >> > >>>> >> >>> >> API
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>>> >> >>> >>
> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping that

I'll

> >> > only
> >> > >>>> have
> >> > >>>> >> to
> >> > >>>> >> >>> >> write
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> >> > >>>> >> >>> >> >>>>> >> tests
> >> > >>>> >> >>> >> >>>>> >> >> and
> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do you
> >> think
> >> > it
> >> > >>>> makes
> >> > >>>> >> >>> sense
> >> > >>>> >> >>> >> to
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> >> > >>>> >> >>> >> >>>>> >> other
> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so that
> >> > someone
> >> > >>>> else
> >> > >>>> >> uses
> >> > >>>> >> >>> >> it and
> >> > >>>> >> >>> >> >>>>> >> maybe
> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>>> >> >>> >> >>>>> >>
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>>
> >> > >>>> >> >>> >> >>>
> >> > >>>> >> >>> >>
> >> > >>>> >> >>>
> >> > >>>> >>
> >> > >>>>
> >> >
> >>

[Aljoscha Krettek-2]

Yes, there is already a Collector version, you can do:

left.join(right).where("foo").equalTo("bar") {
  (left, right, out: Collector[Page]) =>
    if (...) out.collect(...)
}

I wasn't sure on what our Function2 variant should be. That's why I
asked. There are some cases where you want to have the option (pun
intended) of either returning something from a join or not. But you
could also use a filter after the join. It's a preference thing, I
suppose.

On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]> wrote:

> I think it seems weird that normal joins need to go through option.
>
> The option variant is to allow filters in the join function. Wouldn't a
> collector variant allow you to do the same, and would be function3 ? I know
> that option reads more functionally...
>
>
>
> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]>:
>>
>> As already mentioned this is not possible because of type erasure. We
>> can only have one join variant that takes a Function2.
>>
>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]> wrote:
>> > It would be nice to have a join variant that directly returns the value
>> > rathern than an option. Why not have both (they are wrapped as flatJoins
>> > anyway below, right?)
>> >
>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <[hidden email]>
> wrote:
>> >
>> >> Sweet! I'm lovin' this :-)
>> >>
>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>> >>
>> >> > Also, you can use CaseClasses directly as the type for CSV input. So
>> >> > instead of reading it as tuples and then having a mapper that maps to
>> >> > your case classes you can use:
>> >> >
>> >> > env.readCsv[Edge](...)
>> >> >
>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> [hidden email]>
>> >> > wrote:
>> >> > > I added support for specifying keys by name for CaseClasses. Check
> out
>> >> > > the PageRank and TriangleEnumeration examples to see it in action.
>> >> > >
>> >> > > @Kostas: I think you could use them for the TPC-H examples.
>> >> > >
>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> [hidden email]
>> >> >
>> >> > wrote:
>> >> > >> Yes, that would allow list comprehensions. It would be possible to
>> >> > >> have the Collection signature for join (and coGroup), i.e.:
>> >> > >>
>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
>> >> > >>
>> >> > >> (T and O are the left and right input type, R is result type)
>> >> > >>
>> >> > >> Then you can return collections and still return an option, as in:
>> >> > >>
>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...) Some(l)
> else
>> >> > None }
>> >> > >>
>> >> > >> Because there is an implicit conversion from Options to a
> Collection.
>> >> > >> This will always wrap the return value in a List with only one
> value.
>> >> > >> I'm not sure we want the overhead here. I'm also not sure whether
> we
>> >> > >> want the overhead of always having to use an Option even though
> the
>> >> > >> join always returns a value.
>> >> > >>
>> >> > >> What do you think?
>> >> > >>
>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> [hidden email]>
>> >> > wrote:
>> >> > >>> Hmmm, tricky question...
>> >> > >>> How about the Option for Join as this is a tuple-wise operation
> and
>> >> the
>> >> > >>> Collection for Cogroup which is group-wise?
>> >> > >>> Could we in that case use list comprehensions in Cogroup
> functions?
>> >> > >>>
>> >> > >>> Or is that too much mixing?
>> >> > >>>
>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <[hidden email]
>>:
>> >> > >>>
>> >> > >>>> I didn't look at the example either.
>> >> > >>>>
>> >> > >>>> Addings collections is easy, it's just that we can either have
>> >> > >>>> Collections or the Option, not both.
>> >> > >>>>
>> >> > >>>> For the coding style I followed this:
>> >> > >>>>
>> >> >
> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>> >> ,
>> >> > >>>> which itself is based on this: http://docs.scala-lang.org/style/
> .
>> >> It
>> >> > >>>> is different from the Java Code Guidelines we have in place,
> yes.
>> >> > >>>>
>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> [hidden email]
>> >> >
>> >> > >>>> wrote:
>> >> > >>>> > I haven't looked at the LineRank example in detail, but if you
>> >> > think that
>> >> > >>>> > it adds something new to the examples collection, we can
> certainly
>> >> > port
>> >> > >>>> it
>> >> > >>>> > also to Java.
>> >> > >>>> > I think the Option and Collector return types are sufficient
> right
>> >> > now
>> >> > >>>> but
>> >> > >>>> > if Collections are easy to add, go for it. ;-)
>> >> > >>>> >
>> >> > >>>> > Great that the Scala primitives are working! Also thanks for
>> >> adding
>> >> > >>>> > genSequence and adapting my examples.
>> >> > >>>> > Btw. does the codestyle not apply for Scala files or do we
> have a
>> >> > >>>> different
>> >> > >>>> > there?
>> >> > >>>> >
>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> [hidden email]
>> >> >:
>> >> > >>>> >
>> >> > >>>> >> What about the LineRank example? We had that in Scala but
> never
>> >> > had a
>> >> > >>>> >> Java Example.
>> >> > >>>> >>
>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
>> >> > [hidden email]>
>> >> > >>>> >> wrote:
>> >> > >>>> >> > Yes, I like that. For the ITCases I always just copied the
> Java
>> >> > >>>> ITCase.
>> >> > >>>> >> >
>> >> > >>>> >> > The only examples that are missing now are
> LinearRegression and
>> >> > the
>> >> > >>>> >> > relational stuff.
>> >> > >>>> >> >
>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
>> >> > [hidden email]>
>> >> > >>>> >> wrote:
>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
>> >> > >>>> >> >> That was a preprocessing step for the
> TriangleEnumeration, and
>> >> > is now
>> >> > >>>> >> part
>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
>> >> > >>>> >> >> So I guess, we don't need to port that one. As I said
> before,
>> >> > I'd
>> >> > >>>> >> prefer to
>> >> > >>>> >> >> keep Java and Scala examples in sync.
>> >> > >>>> >> >>
>> >> > >>>> >> >> Cheers, Fabian
>> >> > >>>> >> >>
>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
>> >> > [hidden email]>:
>> >> > >>>> >> >>
>> >> > >>>> >> >>> I added the PageRank example, thanks again fabian. :D
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> Regarding the other stuff:
>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about including
>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> TypeInformation.
>> >> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in Array, I
> noticed
>> >> > it
>> >> > >>>> while
>> >> > >>>> >> >>> writing the tests, you probably had an older version of
> the
>> >> > code.
>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not possible,
> this
>> >> > is
>> >> > >>>> also
>> >> > >>>> >> >>> a restriction in the Java API.
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> What do you think about the interface of join and
> coGroup?
>> >> > Right
>> >> > >>>> now,
>> >> > >>>> >> >>> you can either use a lambda that returns an Option or the
>> >> > lambda
>> >> > >>>> with
>> >> > >>>> >> >>> the Collector. Originally I wanted to have also have a
> lambda
>> >> > that
>> >> > >>>> >> >>> returns a Collection, but due to type erasure this has
> the
>> >> > same type
>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
> There is
>> >> an
>> >> > >>>> >> >>> implicit conversion from Option to a Collection, so I
> could
>> >> > change
>> >> > >>>> it
>> >> > >>>> >> >>> without breaking the examples we have now. What do you
> think?
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
>> >> ConnectedComponents,
>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
>> >> > >>>> TriangleEnumerationNaive/Opt,
>> >> > >>>> >> >>> PageRank
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> These are the examples people called dibs on:
>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port of
>> >> > >>>> LinearRegression
>> >> > >>>> >> >>> Example from Java)
>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
>> >> > >>>> >> >>>  - The relational Stuff
>> >> > >>>> >> >>>
>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
>> >> > [hidden email]>
>> >> > >>>> >> wrote:
>> >> > >>>> >> >>> > +1 for removing RelationQuery
>> >> > >>>> >> >>> >
>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek <
>> >> > >>>> >> [hidden email]>
>> >> > >>>> >> >>> > wrote:
>> >> > >>>> >> >>> >
>> >> > >>>> >> >>> >> By the way, what was called BatchGradientDescent in
> the
>> >> > Scala
>> >> > >>>> >> examples
>> >> > >>>> >> >>> >> should be replaced by a port of the LinearRegression
>> >> > Example from
>> >> > >>>> >> >>> >> Java. I had them as two separate examples earlier.
>> >> > >>>> >> >>> >>
>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any thoughts
>> >> about
>> >> > >>>> removing
>> >> > >>>> >> >>> >> RelationalQuery?
>> >> > >>>> >> >>> >>
>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha Krettek <
>> >> > >>>> >> [hidden email]
>> >> > >>>> >> >>> >
>> >> > >>>> >> >>> >> wrote:
>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, thanks
>> >> Fabian.
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
>> >> > ConnectedComponents,
>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
>> >> > >>>> >> TriangleEnumerationNaive/Opt
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> > These are the examples people called dibs on:
>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
>> >> > >>>> >> >>> >> >  - The relational Stuff
>> >> > >>>> >> >>> >> >  - LinearRegression
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha Krettek <
>> >> > >>>> >> >>> [hidden email]>
>> >> > >>>> >> >>> >> wrote:
>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list of
>> >> > >>>> ported/unported
>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java example
>> >> > package to
>> >> > >>>> >> >>> examples
>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> I think the termination criterion is fine as it is.
>> >> Just
>> >> > >>>> because
>> >> > >>>> >> >>> Scala
>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean it's
> always
>> >> > the
>> >> > >>>> best
>> >> > >>>> >> >>> >> >> choice. :D
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
>> >> > ConnectedComponents,
>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
>> >> > >>>> >> >>> >> >>  - The relational Stuff
>> >> > >>>> >> >>> >> >>  - LinearRegression
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> Cheers,
>> >> > >>>> >> >>> >> >> Aljoscha
>> >> > >>>> >> >>> >> >>
>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas Tzoumas <
>> >> > >>>> >> [hidden email]
>> >> > >>>> >> >>> >
>> >> > >>>> >> >>> >> wrote:
>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
> termination
>> >> > criterion
>> >> > >>>> >> in the
>> >> > >>>> >> >>> >> Java
>> >> > >>>> >> >>> >> >>> version:
>> >> > >>>> >> >>> >>
>> >> > >>>>
> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
>> >> > >>>> >> >>> >> >>>
>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination criterion in
>> >> Scala
>> >> > more
>> >> > >>>> >> >>> >> functional?
>> >> > >>>> >> >>> >> >>>
>> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
>> >> > example.java but
>> >> > >>>> >> >>> >> examples.scala
>> >> > >>>> >> >>> >> >>>
>> >> > >>>> >> >>> >> >>> Kostas
>> >> > >>>> >> >>> >> >>>
>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas Tzoumas <
>> >> > >>>> >> [hidden email]
>> >> > >>>> >> >>> >
>> >> > >>>> >> >>> >> wrote:
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation
> (was not
>> >> > on
>> >> > >>>> your
>> >> > >>>> >> >>> list).
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational stuff I
> can
>> >> > take
>> >> > >>>> those
>> >> > >>>> >> as
>> >> > >>>> >> >>> >> well.
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery" from
> both
>> >> > Scala and
>> >> > >>>> >> Java?
>> >> > >>>> >> >>> It
>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3. Does it
> add
>> >> > some
>> >> > >>>> >> teaching
>> >> > >>>> >> >>> >> value on
>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>> Kostas
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha Krettek
> <
>> >> > >>>> >> >>> [hidden email]
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> >>>> wrote:
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
>> >> > >>>> ConnectedComponents,
>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs on:
>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> Cheers,
>> >> > >>>> >> >>> >> >>>>> Aljoscha
>> >> > >>>> >> >>> >> >>>>>
>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas Tzoumas <
>> >> > >>>> >> >>> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> wrote:
>> >> > >>>> >> >>> >> >>>>> > WebLog here:
>> >> > >>>> >> >>> >> >>>>> >
>> >> > >>>> >> >>> >> >>>>> >
>> >> > >>>> >> >>> >>
>> >> > >>>> >> >>>
>> >> > >>>> >>
>> >> > >>>>
>> >> >
>> >>
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
>> >> > >>>> >> >>> >> >>>>> >
>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
>> >> > >>>> >> >>> >> >>>>> >
>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
> Krettek <
>> >> > >>>> >> >>> >> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> > wrote:
>> >> > >>>> >> >>> >> >>>>> >
>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example from
>> >> Vasia.
>> >> > >>>> >> >>> >> >>>>> >>
>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
>> >> > >>>> >> >>> >> >>>>> >>
>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
> Hueske <
>> >> > >>>> >> >>> [hidden email]
>> >> > >>>> >> >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
>> >> > >>>> >> >>> >> >>>>> >> >
>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
> Krettek <
>> >> > >>>> >> >>> >> [hidden email]>:
>> >> > >>>> >> >>> >> >>>>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
>> >> > KMeans.scala. I'm
>> >> > >>>> >> storing
>> >> > >>>> >> >>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> data
>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays and
> then
>> >> > >>>> converting
>> >> > >>>> >> it to
>> >> > >>>> >> >>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo to
> make
>> >> it
>> >> > >>>> >> consistent
>> >> > >>>> >> >>> >> with the
>> >> > >>>> >> >>> >> >>>>> >> >> Java
>> >> > >>>> >> >>> >> >>>>> >> >> API.
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There is no
> need
>> >> > for a
>> >> > >>>> >> >>> keyword,
>> >> > >>>> >> >>> >> you
>> >> > >>>> >> >>> >> >>>>> >> >> can
>> >> > >>>> >> >>> >> >>>>> >> >> just write:
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1) {
> (le,
>> >> re)
>> >> > =>
>> >> > >>>> new
>> >> > >>>> >> >>> >> MyResult(le,
>> >> > >>>> >> >>> >> >>>>> >> >> re)
>> >> > >>>> >> >>> >> >>>>> >> }
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, Fabian
> Hueske
>> >> <
>> >> > >>>> >> >>> >> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I also
> found
>> >> an
>> >> > >>>> >> >>> inconsistency
>> >> > >>>> >> >>> >> with
>> >> > >>>> >> >>> >> >>>>> >> >> > the
>> >> > >>>> >> >>> >> >>>>> >> Java
>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(ds2).where(...).equalTo(...)
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(d2).where(...).isEqualTo(...)
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
> equalTo(),
>> >> > IMO.
>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?) lacks
> the
>> >> > with()
>> >> > >>>> >> method
>> >> > >>>> >> >>> >> because
>> >> > >>>> >> >>> >> >>>>> >> "with"
>> >> > >>>> >> >>> >> >>>>> >> >> is
>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
>> >> something
>> >> > >>>> similar
>> >> > >>>> >> for
>> >> > >>>> >> >>> >> Scala
>> >> > >>>> >> >>> >> >>>>> >> >> > or go
>> >> > >>>> >> >>> >> >>>>> >> >> with
>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan Ewen
> <
>> >> > >>>> >> [hidden email]
>> >> > >>>> >> >>> >:
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][] would
> work
>> >> as
>> >> > well.
>> >> > >>>> >> That
>> >> > >>>> >> >>> is a
>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be stored as
> Java
>> >> > or
>> >> > >>>> Scala
>> >> > >>>> >> >>> Tuples,
>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
>> >> > >>>> >> >>> >> >>>>> >> >> generic
>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between the
> two.
>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM, Fabian
>> >> > Hueske
>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same problem...
>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing
> them,
>> >> but
>> >> > >>>> using
>> >> > >>>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
>> >> > >>>> >> >>> >> >>>>> >> won't
>> >> > >>>> >> >>> >> >>>>> >> >> >> work
>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
>> >> FileInputFormat.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the Strings
>> >> > >>>> manually...
>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00 Aljoscha
>> >> Krettek
>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we should
> just
>> >> > change
>> >> > >>>> >> all
>> >> > >>>> >> >>> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
>> >> > >>>> >> >>> >> >>>>> >> input
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV input
>> >> > formats in
>> >> > >>>> all
>> >> > >>>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
>> >> > >>>> >> >>> >> >>>>> >> What
>> >> > >>>> >> >>> >> >>>>> >> >> do
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46 AM,
>> >> Aljoscha
>> >> > >>>> Krettek
>> >> > >>>> >> <
>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that the
> data
>> >> > types are
>> >> > >>>> >> >>> >> incompatible.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
>> >> > >>>> >> >>> >> >>>>> >> >> afraid
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you proposed:
>> >> move
>> >> > the
>> >> > >>>> >> data to
>> >> > >>>> >> >>> a
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
>> >> > >>>> >> >>> >> >>>>> >> field
>> >> > >>>> >> >>> >> >>>>> >> >> and
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
>> >> > getDefaultEdgeDataSet()
>> >> > >>>> >> method in
>> >> > >>>> >> >>> >> Scala.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
>> >> > >>>> >> >>> >> >>>>> >> >> not
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
> duplicate the
>> >> > data
>> >> > >>>> and
>> >> > >>>> >> >>> make it
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
>> >> > >>>> >> >>> >> >>>>> >> for
>> >> > >>>> >> >>> >> >>>>> >> >> it
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the Java and
>> >> Scala
>> >> > >>>> >> versions.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think? This
> will
>> >> > probably
>> >> > >>>> >> occur
>> >> > >>>> >> >>> in
>> >> > >>>> >> >>> >> all
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04 PM,
>> >> > Vasiliki
>> >> > >>>> >> Kalavri
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]>
> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
>> >> Components
>> >> > >>>> >> example,
>> >> > >>>> >> >>> but
>> >> > >>>> >> >>> >> I am
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
>> >> > >>>> >> >>> >> >>>>> >> sure
>> >> > >>>> >> >>> >> >>>>> >> >> >> how
>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input data
> from
>> >> > >>>> >> java-examples.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the ConnectedComponentsData
>> >> class,
>> >> > the
>> >> > >>>> >> vertices
>> >> > >>>> >> >>> >> and
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
>> >> > >>>> >> >>> >> >>>>> >> data
>> >> > >>>> >> >>> >> >>>>> >> >> >> are
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
>> >> > >>>> >> getDefaultVertexDataSet()
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and getDefaultEdgeDataSet(),
> which
>> >> > take
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
>> >> > >>>> >> >>> as
>> >> > >>>> >> >>> >> >>>>> >> parameter.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide public
> static
>> >> > fields
>> >> > >>>> >> (like
>> >> > >>>> >> >>> in
>> >> > >>>> >> >>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this introduces a
>> >> > conversion
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
>> >> > >>>> >> >>> Scala
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
>> >> > >>>> >> >>> >> >>>>> >> >> from
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to scala.Long
> and I
>> >> > guess
>> >> > >>>> this
>> >> > >>>> >> is
>> >> > >>>> >> >>> an
>> >> > >>>> >> >>> >> >>>>> >> unnecessary
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course, to
> copy
>> >> the
>> >> > >>>> example
>> >> > >>>> >> >>> data
>> >> > >>>> >> >>> >> in
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
>> >> > >>>> >> >>> >> >>>>> >> Scala
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something here?
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,
> Aljoscha
>> >> > >>>> Krettek <
>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
>> >> > >>>> >> >>> >> >>>>> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my repo:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >>
>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
> WordCount
>> >> > example.
>> >> > >>>> >> It's
>> >> > >>>> >> >>> >> pretty
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
>> >> > >>>> >> >>> >> >>>>> >> >> copy
>> >> > >>>> >> >>> >> >>>>> >> >> >> of
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some
> fixups
>> >> > for the
>> >> > >>>> >> syntax
>> >> > >>>> >> >>> and
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
> added the
>> >> > >>>> >> java-examples
>> >> > >>>> >> >>> >> as a
>> >> > >>>> >> >>> >> >>>>> >> >> dependency
>> >> > >>>> >> >>> >> >>>>> >> >> >> for
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did this
> to
>> >> > reuse
>> >> > >>>> the
>> >> > >>>> >> >>> example
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
>> >> > >>>> >> >>> >> >>>>> >> data.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program you
> can
>> >> do
>> >> > a
>> >> > >>>> pull
>> >> > >>>> >> >>> request
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
>> >> > >>>> >> >>> >> >>>>> >> my
>> >> > >>>> >> >>> >> >>>>> >> >> repo
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
> examples.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at 12:19
> PM,
>> >> > Hermann
>> >> > >>>> >> Gábor <
>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for me!
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at 11:44
> AM,
>> >> > Márton
>> >> > >>>> >> >>> Balassi <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent for me
> :)
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
> 11:15 AM,
>> >> > Kostas
>> >> > >>>> >> >>> Tzoumas <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for WebLogAnalysis.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with Scala
>> >> > consists of
>> >> > >>>> >> going
>> >> > >>>> >> >>> >> through
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
>> >> > >>>> >> >>> >> >>>>> >> >> >> so
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test
> both for
>> >> > me and
>> >> > >>>> >> the
>> >> > >>>> >> >>> new
>> >> > >>>> >> >>> >> API
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014 at
> 9:09
>> >> PM,
>> >> > >>>> Vasiliki
>> >> > >>>> >> >>> >> Kalavri <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other
> people
>> >> > >>>> implement
>> >> > >>>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected Components and
>> >> > Kmeans for
>> >> > >>>> >> me :)
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014
> 21:03,
>> >> > Fabian
>> >> > >>>> >> Hueske <
>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
>> >> > TriangleEnumeration and
>> >> > >>>> >> >>> PageRank.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the
> examples
>> >> > >>>> similar
>> >> > >>>> >> to
>> >> > >>>> >> >>> the
>> >> > >>>> >> >>> >> Java
>> >> > >>>> >> >>> >> >>>>> >> >> examples:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> out-of-the-box
>> >> > without
>> >> > >>>> >> >>> parameters
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
> external
>> >> > data
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a similar
> code
>> >> > structure
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
> GMT+02:00
>> >> > >>>> Aljoscha
>> >> > >>>> >> >>> >> Krettek <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
> people can
>> >> > >>>> reserve
>> >> > >>>> >> their
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4, 2014
> at
>> >> > 8:55 PM,
>> >> > >>>> >> Fabian
>> >> > >>>> >> >>> >> Hueske
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think having
>> >> examples
>> >> > >>>> >> implemented
>> >> > >>>> >> >>> by
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
>> >> > >>>> >> >>> >> >>>>> >> >> >> people
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in the
> past.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with two
> or
>> >> > three
>> >> > >>>> >> examples.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
> helpful if
>> >> > you'd
>> >> > >>>> >> port a
>> >> > >>>> >> >>> >> simple
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
>> >> > >>>> >> >>> >> >>>>> >> >> one
>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04 18:47
>> >> > GMT+02:00
>> >> > >>>> >> Aljoscha
>> >> > >>>> >> >>> >> Krettek
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a working
>> >> > rewrite of
>> >> > >>>> the
>> >> > >>>> >> >>> Scala
>> >> > >>>> >> >>> >> API
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> >> > >>>> >> >>> >>
>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping that
> I'll
>> >> > only
>> >> > >>>> have
>> >> > >>>> >> to
>> >> > >>>> >> >>> >> write
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
>> >> > >>>> >> >>> >> >>>>> >> tests
>> >> > >>>> >> >>> >> >>>>> >> >> and
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do you
>> >> think
>> >> > it
>> >> > >>>> makes
>> >> > >>>> >> >>> sense
>> >> > >>>> >> >>> >> to
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
>> >> > >>>> >> >>> >> >>>>> >> other
>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so that
>> >> > someone
>> >> > >>>> else
>> >> > >>>> >> uses
>> >> > >>>> >> >>> >> it and
>> >> > >>>> >> >>> >> >>>>> >> maybe
>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> >> > >>>> >> >>> >> >>>>> >> >>
>> >> > >>>> >> >>> >> >>>>> >>
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>>
>> >> > >>>> >> >>> >> >>>
>> >> > >>>> >> >>> >>
>> >> > >>>> >> >>>
>> >> > >>>> >>
>> >> > >>>>
>> >> >
>> >>

[Aljoscha Krettek-2]

So, should I change join and coGroup to have a simple return value, no
Option or Collection? Also what's happening with the relational
examples and the LinearRegression examples? I'd like to make a pull
request before this weekend.

I also added a test that checks whether the Scala API has the same
methods as the Java API (ScalaAPICompletenessTest).

On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <[hidden email]> wrote:

> Yes, there is already a Collector version, you can do:
>
> left.join(right).where("foo").equalTo("bar") {
>   (left, right, out: Collector[Page]) =>
>     if (...) out.collect(...)
> }
>
> I wasn't sure on what our Function2 variant should be. That's why I
> asked. There are some cases where you want to have the option (pun
> intended) of either returning something from a join or not. But you
> could also use a filter after the join. It's a preference thing, I
> suppose.
>
> On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]> wrote:
>> I think it seems weird that normal joins need to go through option.
>>
>> The option variant is to allow filters in the join function. Wouldn't a
>> collector variant allow you to do the same, and would be function3 ? I know
>> that option reads more functionally...
>>
>>
>>
>> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]>:
>>>
>>> As already mentioned this is not possible because of type erasure. We
>>> can only have one join variant that takes a Function2.
>>>
>>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]> wrote:
>>> > It would be nice to have a join variant that directly returns the value
>>> > rathern than an option. Why not have both (they are wrapped as flatJoins
>>> > anyway below, right?)
>>> >
>>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <[hidden email]>
>> wrote:
>>> >
>>> >> Sweet! I'm lovin' this :-)
>>> >>
>>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]>:
>>> >>
>>> >> > Also, you can use CaseClasses directly as the type for CSV input. So
>>> >> > instead of reading it as tuples and then having a mapper that maps to
>>> >> > your case classes you can use:
>>> >> >
>>> >> > env.readCsv[Edge](...)
>>> >> >
>>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
>> [hidden email]>
>>> >> > wrote:
>>> >> > > I added support for specifying keys by name for CaseClasses. Check
>> out
>>> >> > > the PageRank and TriangleEnumeration examples to see it in action.
>>> >> > >
>>> >> > > @Kostas: I think you could use them for the TPC-H examples.
>>> >> > >
>>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
>> [hidden email]
>>> >> >
>>> >> > wrote:
>>> >> > >> Yes, that would allow list comprehensions. It would be possible to
>>> >> > >> have the Collection signature for join (and coGroup), i.e.:
>>> >> > >>
>>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
>>> >> > >>
>>> >> > >> (T and O are the left and right input type, R is result type)
>>> >> > >>
>>> >> > >> Then you can return collections and still return an option, as in:
>>> >> > >>
>>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...) Some(l)
>> else
>>> >> > None }
>>> >> > >>
>>> >> > >> Because there is an implicit conversion from Options to a
>> Collection.
>>> >> > >> This will always wrap the return value in a List with only one
>> value.
>>> >> > >> I'm not sure we want the overhead here. I'm also not sure whether
>> we
>>> >> > >> want the overhead of always having to use an Option even though
>> the
>>> >> > >> join always returns a value.
>>> >> > >>
>>> >> > >> What do you think?
>>> >> > >>
>>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
>> [hidden email]>
>>> >> > wrote:
>>> >> > >>> Hmmm, tricky question...
>>> >> > >>> How about the Option for Join as this is a tuple-wise operation
>> and
>>> >> the
>>> >> > >>> Collection for Cogroup which is group-wise?
>>> >> > >>> Could we in that case use list comprehensions in Cogroup
>> functions?
>>> >> > >>>
>>> >> > >>> Or is that too much mixing?
>>> >> > >>>
>>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <[hidden email]
>>>:
>>> >> > >>>
>>> >> > >>>> I didn't look at the example either.
>>> >> > >>>>
>>> >> > >>>> Addings collections is easy, it's just that we can either have
>>> >> > >>>> Collections or the Option, not both.
>>> >> > >>>>
>>> >> > >>>> For the coding style I followed this:
>>> >> > >>>>
>>> >> >
>> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>>> >> ,
>>> >> > >>>> which itself is based on this: http://docs.scala-lang.org/style/
>> .
>>> >> It
>>> >> > >>>> is different from the Java Code Guidelines we have in place,
>> yes.
>>> >> > >>>>
>>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
>> [hidden email]
>>> >> >
>>> >> > >>>> wrote:
>>> >> > >>>> > I haven't looked at the LineRank example in detail, but if you
>>> >> > think that
>>> >> > >>>> > it adds something new to the examples collection, we can
>> certainly
>>> >> > port
>>> >> > >>>> it
>>> >> > >>>> > also to Java.
>>> >> > >>>> > I think the Option and Collector return types are sufficient
>> right
>>> >> > now
>>> >> > >>>> but
>>> >> > >>>> > if Collections are easy to add, go for it. ;-)
>>> >> > >>>> >
>>> >> > >>>> > Great that the Scala primitives are working! Also thanks for
>>> >> adding
>>> >> > >>>> > genSequence and adapting my examples.
>>> >> > >>>> > Btw. does the codestyle not apply for Scala files or do we
>> have a
>>> >> > >>>> different
>>> >> > >>>> > there?
>>> >> > >>>> >
>>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
>> [hidden email]
>>> >> >:
>>> >> > >>>> >
>>> >> > >>>> >> What about the LineRank example? We had that in Scala but
>> never
>>> >> > had a
>>> >> > >>>> >> Java Example.
>>> >> > >>>> >>
>>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
>>> >> > [hidden email]>
>>> >> > >>>> >> wrote:
>>> >> > >>>> >> > Yes, I like that. For the ITCases I always just copied the
>> Java
>>> >> > >>>> ITCase.
>>> >> > >>>> >> >
>>> >> > >>>> >> > The only examples that are missing now are
>> LinearRegression and
>>> >> > the
>>> >> > >>>> >> > relational stuff.
>>> >> > >>>> >> >
>>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
>>> >> > [hidden email]>
>>> >> > >>>> >> wrote:
>>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
>>> >> > >>>> >> >> That was a preprocessing step for the
>> TriangleEnumeration, and
>>> >> > is now
>>> >> > >>>> >> part
>>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
>>> >> > >>>> >> >> So I guess, we don't need to port that one. As I said
>> before,
>>> >> > I'd
>>> >> > >>>> >> prefer to
>>> >> > >>>> >> >> keep Java and Scala examples in sync.
>>> >> > >>>> >> >>
>>> >> > >>>> >> >> Cheers, Fabian
>>> >> > >>>> >> >>
>>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
>>> >> > [hidden email]>:
>>> >> > >>>> >> >>
>>> >> > >>>> >> >>> I added the PageRank example, thanks again fabian. :D
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> Regarding the other stuff:
>>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about including
>>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
>> TypeInformation.
>>> >> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
>>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in Array, I
>> noticed
>>> >> > it
>>> >> > >>>> while
>>> >> > >>>> >> >>> writing the tests, you probably had an older version of
>> the
>>> >> > code.
>>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not possible,
>> this
>>> >> > is
>>> >> > >>>> also
>>> >> > >>>> >> >>> a restriction in the Java API.
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> What do you think about the interface of join and
>> coGroup?
>>> >> > Right
>>> >> > >>>> now,
>>> >> > >>>> >> >>> you can either use a lambda that returns an Option or the
>>> >> > lambda
>>> >> > >>>> with
>>> >> > >>>> >> >>> the Collector. Originally I wanted to have also have a
>> lambda
>>> >> > that
>>> >> > >>>> >> >>> returns a Collection, but due to type erasure this has
>> the
>>> >> > same type
>>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
>> There is
>>> >> an
>>> >> > >>>> >> >>> implicit conversion from Option to a Collection, so I
>> could
>>> >> > change
>>> >> > >>>> it
>>> >> > >>>> >> >>> without breaking the examples we have now. What do you
>> think?
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
>>> >> ConnectedComponents,
>>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
>>> >> > >>>> TriangleEnumerationNaive/Opt,
>>> >> > >>>> >> >>> PageRank
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> These are the examples people called dibs on:
>>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port of
>>> >> > >>>> LinearRegression
>>> >> > >>>> >> >>> Example from Java)
>>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
>>> >> > >>>> >> >>>  - The relational Stuff
>>> >> > >>>> >> >>>
>>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
>>> >> > [hidden email]>
>>> >> > >>>> >> wrote:
>>> >> > >>>> >> >>> > +1 for removing RelationQuery
>>> >> > >>>> >> >>> >
>>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek <
>>> >> > >>>> >> [hidden email]>
>>> >> > >>>> >> >>> > wrote:
>>> >> > >>>> >> >>> >
>>> >> > >>>> >> >>> >> By the way, what was called BatchGradientDescent in
>> the
>>> >> > Scala
>>> >> > >>>> >> examples
>>> >> > >>>> >> >>> >> should be replaced by a port of the LinearRegression
>>> >> > Example from
>>> >> > >>>> >> >>> >> Java. I had them as two separate examples earlier.
>>> >> > >>>> >> >>> >>
>>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any thoughts
>>> >> about
>>> >> > >>>> removing
>>> >> > >>>> >> >>> >> RelationalQuery?
>>> >> > >>>> >> >>> >>
>>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha Krettek <
>>> >> > >>>> >> [hidden email]
>>> >> > >>>> >> >>> >
>>> >> > >>>> >> >>> >> wrote:
>>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, thanks
>>> >> Fabian.
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
>>> >> > ConnectedComponents,
>>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
>>> >> > >>>> >> TriangleEnumerationNaive/Opt
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> > These are the examples people called dibs on:
>>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
>>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
>>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
>>> >> > >>>> >> >>> >> >  - The relational Stuff
>>> >> > >>>> >> >>> >> >  - LinearRegression
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha Krettek <
>>> >> > >>>> >> >>> [hidden email]>
>>> >> > >>>> >> >>> >> wrote:
>>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list of
>>> >> > >>>> ported/unported
>>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java example
>>> >> > package to
>>> >> > >>>> >> >>> examples
>>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> I think the termination criterion is fine as it is.
>>> >> Just
>>> >> > >>>> because
>>> >> > >>>> >> >>> Scala
>>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean it's
>> always
>>> >> > the
>>> >> > >>>> best
>>> >> > >>>> >> >>> >> >> choice. :D
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
>>> >> > ConnectedComponents,
>>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
>>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
>>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
>>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
>>> >> > >>>> >> >>> >> >>  - The relational Stuff
>>> >> > >>>> >> >>> >> >>  - LinearRegression
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> Cheers,
>>> >> > >>>> >> >>> >> >> Aljoscha
>>> >> > >>>> >> >>> >> >>
>>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas Tzoumas <
>>> >> > >>>> >> [hidden email]
>>> >> > >>>> >> >>> >
>>> >> > >>>> >> >>> >> wrote:
>>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
>> termination
>>> >> > criterion
>>> >> > >>>> >> in the
>>> >> > >>>> >> >>> >> Java
>>> >> > >>>> >> >>> >> >>> version:
>>> >> > >>>> >> >>> >>
>>> >> > >>>>
>> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
>>> >> > >>>> >> >>> >> >>>
>>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination criterion in
>>> >> Scala
>>> >> > more
>>> >> > >>>> >> >>> >> functional?
>>> >> > >>>> >> >>> >> >>>
>>> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
>>> >> > example.java but
>>> >> > >>>> >> >>> >> examples.scala
>>> >> > >>>> >> >>> >> >>>
>>> >> > >>>> >> >>> >> >>> Kostas
>>> >> > >>>> >> >>> >> >>>
>>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas Tzoumas <
>>> >> > >>>> >> [hidden email]
>>> >> > >>>> >> >>> >
>>> >> > >>>> >> >>> >> wrote:
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation
>> (was not
>>> >> > on
>>> >> > >>>> your
>>> >> > >>>> >> >>> list).
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational stuff I
>> can
>>> >> > take
>>> >> > >>>> those
>>> >> > >>>> >> as
>>> >> > >>>> >> >>> >> well.
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery" from
>> both
>>> >> > Scala and
>>> >> > >>>> >> Java?
>>> >> > >>>> >> >>> It
>>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3. Does it
>> add
>>> >> > some
>>> >> > >>>> >> teaching
>>> >> > >>>> >> >>> >> value on
>>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>> Kostas
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha Krettek
>> <
>>> >> > >>>> >> >>> [hidden email]
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> >>>> wrote:
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
>>> >> > >>>> ConnectedComponents,
>>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs on:
>>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
>>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
>>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
>>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
>>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
>>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> Cheers,
>>> >> > >>>> >> >>> >> >>>>> Aljoscha
>>> >> > >>>> >> >>> >> >>>>>
>>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas Tzoumas <
>>> >> > >>>> >> >>> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> wrote:
>>> >> > >>>> >> >>> >> >>>>> > WebLog here:
>>> >> > >>>> >> >>> >> >>>>> >
>>> >> > >>>> >> >>> >> >>>>> >
>>> >> > >>>> >> >>> >>
>>> >> > >>>> >> >>>
>>> >> > >>>> >>
>>> >> > >>>>
>>> >> >
>>> >>
>> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
>>> >> > >>>> >> >>> >> >>>>> >
>>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
>>> >> > >>>> >> >>> >> >>>>> >
>>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
>> Krettek <
>>> >> > >>>> >> >>> >> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> > wrote:
>>> >> > >>>> >> >>> >> >>>>> >
>>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example from
>>> >> Vasia.
>>> >> > >>>> >> >>> >> >>>>> >>
>>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
>>> >> > >>>> >> >>> >> >>>>> >>
>>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
>> Hueske <
>>> >> > >>>> >> >>> [hidden email]
>>> >> > >>>> >> >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
>>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
>>> >> > >>>> >> >>> >> >>>>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
>> Krettek <
>>> >> > >>>> >> >>> >> [hidden email]>:
>>> >> > >>>> >> >>> >> >>>>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
>>> >> > KMeans.scala. I'm
>>> >> > >>>> >> storing
>>> >> > >>>> >> >>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> data
>>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays and
>> then
>>> >> > >>>> converting
>>> >> > >>>> >> it to
>>> >> > >>>> >> >>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo to
>> make
>>> >> it
>>> >> > >>>> >> consistent
>>> >> > >>>> >> >>> >> with the
>>> >> > >>>> >> >>> >> >>>>> >> >> Java
>>> >> > >>>> >> >>> >> >>>>> >> >> API.
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There is no
>> need
>>> >> > for a
>>> >> > >>>> >> >>> keyword,
>>> >> > >>>> >> >>> >> you
>>> >> > >>>> >> >>> >> >>>>> >> >> can
>>> >> > >>>> >> >>> >> >>>>> >> >> just write:
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1) {
>> (le,
>>> >> re)
>>> >> > =>
>>> >> > >>>> new
>>> >> > >>>> >> >>> >> MyResult(le,
>>> >> > >>>> >> >>> >> >>>>> >> >> re)
>>> >> > >>>> >> >>> >> >>>>> >> }
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, Fabian
>> Hueske
>>> >> <
>>> >> > >>>> >> >>> >> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I also
>> found
>>> >> an
>>> >> > >>>> >> >>> inconsistency
>>> >> > >>>> >> >>> >> with
>>> >> > >>>> >> >>> >> >>>>> >> >> > the
>>> >> > >>>> >> >>> >> >>>>> >> Java
>>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(ds2).where(...).equalTo(...)
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(d2).where(...).isEqualTo(...)
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
>> equalTo(),
>>> >> > IMO.
>>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?) lacks
>> the
>>> >> > with()
>>> >> > >>>> >> method
>>> >> > >>>> >> >>> >> because
>>> >> > >>>> >> >>> >> >>>>> >> "with"
>>> >> > >>>> >> >>> >> >>>>> >> >> is
>>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
>>> >> something
>>> >> > >>>> similar
>>> >> > >>>> >> for
>>> >> > >>>> >> >>> >> Scala
>>> >> > >>>> >> >>> >> >>>>> >> >> > or go
>>> >> > >>>> >> >>> >> >>>>> >> >> with
>>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan Ewen
>> <
>>> >> > >>>> >> [hidden email]
>>> >> > >>>> >> >>> >:
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][] would
>> work
>>> >> as
>>> >> > well.
>>> >> > >>>> >> That
>>> >> > >>>> >> >>> is a
>>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
>>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
>>> >> > >>>> >> >>> >> >>>>> >> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be stored as
>> Java
>>> >> > or
>>> >> > >>>> Scala
>>> >> > >>>> >> >>> Tuples,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
>>> >> > >>>> >> >>> >> >>>>> >> >> generic
>>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between the
>> two.
>>> >> > >>>> >> >>> >> >>>>> >> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM, Fabian
>>> >> > Hueske
>>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same problem...
>>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing
>> them,
>>> >> but
>>> >> > >>>> using
>>> >> > >>>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
>>> >> > >>>> >> >>> >> >>>>> >> won't
>>> >> > >>>> >> >>> >> >>>>> >> >> >> work
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
>>> >> FileInputFormat.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the Strings
>>> >> > >>>> manually...
>>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00 Aljoscha
>>> >> Krettek
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we should
>> just
>>> >> > change
>>> >> > >>>> >> all
>>> >> > >>>> >> >>> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
>>> >> > >>>> >> >>> >> >>>>> >> input
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV input
>>> >> > formats in
>>> >> > >>>> all
>>> >> > >>>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
>>> >> > >>>> >> >>> >> >>>>> >> What
>>> >> > >>>> >> >>> >> >>>>> >> >> do
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46 AM,
>>> >> Aljoscha
>>> >> > >>>> Krettek
>>> >> > >>>> >> <
>>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that the
>> data
>>> >> > types are
>>> >> > >>>> >> >>> >> incompatible.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
>>> >> > >>>> >> >>> >> >>>>> >> >> afraid
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you proposed:
>>> >> move
>>> >> > the
>>> >> > >>>> >> data to
>>> >> > >>>> >> >>> a
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
>>> >> > >>>> >> >>> >> >>>>> >> field
>>> >> > >>>> >> >>> >> >>>>> >> >> and
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
>>> >> > getDefaultEdgeDataSet()
>>> >> > >>>> >> method in
>>> >> > >>>> >> >>> >> Scala.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
>>> >> > >>>> >> >>> >> >>>>> >> >> not
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
>> duplicate the
>>> >> > data
>>> >> > >>>> and
>>> >> > >>>> >> >>> make it
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
>>> >> > >>>> >> >>> >> >>>>> >> for
>>> >> > >>>> >> >>> >> >>>>> >> >> it
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the Java and
>>> >> Scala
>>> >> > >>>> >> versions.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think? This
>> will
>>> >> > probably
>>> >> > >>>> >> occur
>>> >> > >>>> >> >>> in
>>> >> > >>>> >> >>> >> all
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04 PM,
>>> >> > Vasiliki
>>> >> > >>>> >> Kalavri
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]>
>> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
>>> >> Components
>>> >> > >>>> >> example,
>>> >> > >>>> >> >>> but
>>> >> > >>>> >> >>> >> I am
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
>>> >> > >>>> >> >>> >> >>>>> >> sure
>>> >> > >>>> >> >>> >> >>>>> >> >> >> how
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input data
>> from
>>> >> > >>>> >> java-examples.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the ConnectedComponentsData
>>> >> class,
>>> >> > the
>>> >> > >>>> >> vertices
>>> >> > >>>> >> >>> >> and
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
>>> >> > >>>> >> >>> >> >>>>> >> data
>>> >> > >>>> >> >>> >> >>>>> >> >> >> are
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
>>> >> > >>>> >> getDefaultVertexDataSet()
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and getDefaultEdgeDataSet(),
>> which
>>> >> > take
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
>>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
>>> >> > >>>> >> >>> as
>>> >> > >>>> >> >>> >> >>>>> >> parameter.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide public
>> static
>>> >> > fields
>>> >> > >>>> >> (like
>>> >> > >>>> >> >>> in
>>> >> > >>>> >> >>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this introduces a
>>> >> > conversion
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
>>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
>>> >> > >>>> >> >>> Scala
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
>>> >> > >>>> >> >>> >> >>>>> >> >> from
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to scala.Long
>> and I
>>> >> > guess
>>> >> > >>>> this
>>> >> > >>>> >> is
>>> >> > >>>> >> >>> an
>>> >> > >>>> >> >>> >> >>>>> >> unnecessary
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course, to
>> copy
>>> >> the
>>> >> > >>>> example
>>> >> > >>>> >> >>> data
>>> >> > >>>> >> >>> >> in
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
>>> >> > >>>> >> >>> >> >>>>> >> Scala
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something here?
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,
>> Aljoscha
>>> >> > >>>> Krettek <
>>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
>>> >> > >>>> >> >>> >> >>>>> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my repo:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >>
>>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
>> WordCount
>>> >> > example.
>>> >> > >>>> >> It's
>>> >> > >>>> >> >>> >> pretty
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
>>> >> > >>>> >> >>> >> >>>>> >> >> copy
>>> >> > >>>> >> >>> >> >>>>> >> >> >> of
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some
>> fixups
>>> >> > for the
>>> >> > >>>> >> syntax
>>> >> > >>>> >> >>> and
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
>> added the
>>> >> > >>>> >> java-examples
>>> >> > >>>> >> >>> >> as a
>>> >> > >>>> >> >>> >> >>>>> >> >> dependency
>>> >> > >>>> >> >>> >> >>>>> >> >> >> for
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did this
>> to
>>> >> > reuse
>>> >> > >>>> the
>>> >> > >>>> >> >>> example
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
>>> >> > >>>> >> >>> >> >>>>> >> data.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program you
>> can
>>> >> do
>>> >> > a
>>> >> > >>>> pull
>>> >> > >>>> >> >>> request
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
>>> >> > >>>> >> >>> >> >>>>> >> my
>>> >> > >>>> >> >>> >> >>>>> >> >> repo
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
>> examples.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at 12:19
>> PM,
>>> >> > Hermann
>>> >> > >>>> >> Gábor <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for me!
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at 11:44
>> AM,
>>> >> > Márton
>>> >> > >>>> >> >>> Balassi <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent for me
>> :)
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
>> 11:15 AM,
>>> >> > Kostas
>>> >> > >>>> >> >>> Tzoumas <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for WebLogAnalysis.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with Scala
>>> >> > consists of
>>> >> > >>>> >> going
>>> >> > >>>> >> >>> >> through
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
>>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
>>> >> > >>>> >> >>> >> >>>>> >> >> >> so
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test
>> both for
>>> >> > me and
>>> >> > >>>> >> the
>>> >> > >>>> >> >>> new
>>> >> > >>>> >> >>> >> API
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014 at
>> 9:09
>>> >> PM,
>>> >> > >>>> Vasiliki
>>> >> > >>>> >> >>> >> Kalavri <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other
>> people
>>> >> > >>>> implement
>>> >> > >>>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected Components and
>>> >> > Kmeans for
>>> >> > >>>> >> me :)
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014
>> 21:03,
>>> >> > Fabian
>>> >> > >>>> >> Hueske <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
>>> >> > TriangleEnumeration and
>>> >> > >>>> >> >>> PageRank.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the
>> examples
>>> >> > >>>> similar
>>> >> > >>>> >> to
>>> >> > >>>> >> >>> the
>>> >> > >>>> >> >>> >> Java
>>> >> > >>>> >> >>> >> >>>>> >> >> examples:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
>> out-of-the-box
>>> >> > without
>>> >> > >>>> >> >>> parameters
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
>> external
>>> >> > data
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a similar
>> code
>>> >> > structure
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
>> GMT+02:00
>>> >> > >>>> Aljoscha
>>> >> > >>>> >> >>> >> Krettek <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
>> people can
>>> >> > >>>> reserve
>>> >> > >>>> >> their
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
>>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4, 2014
>> at
>>> >> > 8:55 PM,
>>> >> > >>>> >> Fabian
>>> >> > >>>> >> >>> >> Hueske
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think having
>>> >> examples
>>> >> > >>>> >> implemented
>>> >> > >>>> >> >>> by
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
>>> >> > >>>> >> >>> >> >>>>> >> >> >> people
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in the
>> past.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with two
>> or
>>> >> > three
>>> >> > >>>> >> examples.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
>> helpful if
>>> >> > you'd
>>> >> > >>>> >> port a
>>> >> > >>>> >> >>> >> simple
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
>>> >> > >>>> >> >>> >> >>>>> >> >> one
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04 18:47
>>> >> > GMT+02:00
>>> >> > >>>> >> Aljoscha
>>> >> > >>>> >> >>> >> Krettek
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a working
>>> >> > rewrite of
>>> >> > >>>> the
>>> >> > >>>> >> >>> Scala
>>> >> > >>>> >> >>> >> API
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >>
>>> >> > >>>> >> >>> >>
>>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping that
>> I'll
>>> >> > only
>>> >> > >>>> have
>>> >> > >>>> >> to
>>> >> > >>>> >> >>> >> write
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
>>> >> > >>>> >> >>> >> >>>>> >> tests
>>> >> > >>>> >> >>> >> >>>>> >> >> and
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do you
>>> >> think
>>> >> > it
>>> >> > >>>> makes
>>> >> > >>>> >> >>> sense
>>> >> > >>>> >> >>> >> to
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
>>> >> > >>>> >> >>> >> >>>>> >> other
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so that
>>> >> > someone
>>> >> > >>>> else
>>> >> > >>>> >> uses
>>> >> > >>>> >> >>> >> it and
>>> >> > >>>> >> >>> >> >>>>> >> maybe
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>>> >> > >>>> >> >>> >> >>>>> >> >> >>
>>> >> > >>>> >> >>> >> >>>>> >> >>
>>> >> > >>>> >> >>> >> >>>>> >>
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>>
>>> >> > >>>> >> >>> >> >>>
>>> >> > >>>> >> >>> >>
>>> >> > >>>> >> >>>
>>> >> > >>>> >>
>>> >> > >>>>
>>> >> >
>>> >>

[Stephan Ewen]

I suppose that having the option between simple return type, and a
collector is the easiest to understand.
 Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <[hidden email]>:

> So, should I change join and coGroup to have a simple return value, no
> Option or Collection? Also what's happening with the relational
> examples and the LinearRegression examples? I'd like to make a pull
> request before this weekend.
>
> I also added a test that checks whether the Scala API has the same
> methods as the Java API (ScalaAPICompletenessTest).
>
> On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <[hidden email]>
> wrote:
> > Yes, there is already a Collector version, you can do:
> >
> > left.join(right).where("foo").equalTo("bar") {
> >   (left, right, out: Collector[Page]) =>
> >     if (...) out.collect(...)
> > }
> >
> > I wasn't sure on what our Function2 variant should be. That's why I
> > asked. There are some cases where you want to have the option (pun
> > intended) of either returning something from a join or not. But you
> > could also use a filter after the join. It's a preference thing, I
> > suppose.
> >
> > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]> wrote:
> >> I think it seems weird that normal joins need to go through option.
> >>
> >> The option variant is to allow filters in the join function. Wouldn't a
> >> collector variant allow you to do the same, and would be function3 ? I
> know
> >> that option reads more functionally...
> >>
> >>
> >>
> >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]>:
> >>>
> >>> As already mentioned this is not possible because of type erasure. We
> >>> can only have one join variant that takes a Function2.
> >>>
> >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]>
> wrote:
> >>> > It would be nice to have a join variant that directly returns the
> value
> >>> > rathern than an option. Why not have both (they are wrapped as
> flatJoins
> >>> > anyway below, right?)
> >>> >
> >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <[hidden email]>
> >> wrote:
> >>> >
> >>> >> Sweet! I'm lovin' this :-)
> >>> >>
> >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]>:
> >>> >>
> >>> >> > Also, you can use CaseClasses directly as the type for CSV input.
> So
> >>> >> > instead of reading it as tuples and then having a mapper that
> maps to
> >>> >> > your case classes you can use:
> >>> >> >
> >>> >> > env.readCsv[Edge](...)
> >>> >> >
> >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> >> [hidden email]>
> >>> >> > wrote:
> >>> >> > > I added support for specifying keys by name for CaseClasses.
> Check
> >> out
> >>> >> > > the PageRank and TriangleEnumeration examples to see it in
> action.
> >>> >> > >
> >>> >> > > @Kostas: I think you could use them for the TPC-H examples.
> >>> >> > >
> >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> >> [hidden email]
> >>> >> >
> >>> >> > wrote:
> >>> >> > >> Yes, that would allow list comprehensions. It would be
> possible to
> >>> >> > >> have the Collection signature for join (and coGroup), i.e.:
> >>> >> > >>
> >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> >>> >> > >>
> >>> >> > >> (T and O are the left and right input type, R is result type)
> >>> >> > >>
> >>> >> > >> Then you can return collections and still return an option, as
> in:
> >>> >> > >>
> >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...) Some(l)
> >> else
> >>> >> > None }
> >>> >> > >>
> >>> >> > >> Because there is an implicit conversion from Options to a
> >> Collection.
> >>> >> > >> This will always wrap the return value in a List with only one
> >> value.
> >>> >> > >> I'm not sure we want the overhead here. I'm also not sure
> whether
> >> we
> >>> >> > >> want the overhead of always having to use an Option even though
> >> the
> >>> >> > >> join always returns a value.
> >>> >> > >>
> >>> >> > >> What do you think?
> >>> >> > >>
> >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> >> [hidden email]>
> >>> >> > wrote:
> >>> >> > >>> Hmmm, tricky question...
> >>> >> > >>> How about the Option for Join as this is a tuple-wise
> operation
> >> and
> >>> >> the
> >>> >> > >>> Collection for Cogroup which is group-wise?
> >>> >> > >>> Could we in that case use list comprehensions in Cogroup
> >> functions?
> >>> >> > >>>
> >>> >> > >>> Or is that too much mixing?
> >>> >> > >>>
> >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> [hidden email]
> >>>:
> >>> >> > >>>
> >>> >> > >>>> I didn't look at the example either.
> >>> >> > >>>>
> >>> >> > >>>> Addings collections is easy, it's just that we can either
> have
> >>> >> > >>>> Collections or the Option, not both.
> >>> >> > >>>>
> >>> >> > >>>> For the coding style I followed this:
> >>> >> > >>>>
> >>> >> >
> >>
> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> >>> >> ,
> >>> >> > >>>> which itself is based on this:
> http://docs.scala-lang.org/style/
> >> .
> >>> >> It
> >>> >> > >>>> is different from the Java Code Guidelines we have in place,
> >> yes.
> >>> >> > >>>>
> >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> >> [hidden email]
> >>> >> >
> >>> >> > >>>> wrote:
> >>> >> > >>>> > I haven't looked at the LineRank example in detail, but if
> you
> >>> >> > think that
> >>> >> > >>>> > it adds something new to the examples collection, we can
> >> certainly
> >>> >> > port
> >>> >> > >>>> it
> >>> >> > >>>> > also to Java.
> >>> >> > >>>> > I think the Option and Collector return types are
> sufficient
> >> right
> >>> >> > now
> >>> >> > >>>> but
> >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> >>> >> > >>>> >
> >>> >> > >>>> > Great that the Scala primitives are working! Also thanks
> for
> >>> >> adding
> >>> >> > >>>> > genSequence and adapting my examples.
> >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or do we
> >> have a
> >>> >> > >>>> different
> >>> >> > >>>> > there?
> >>> >> > >>>> >
> >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> >> [hidden email]
> >>> >> >:
> >>> >> > >>>> >
> >>> >> > >>>> >> What about the LineRank example? We had that in Scala but
> >> never
> >>> >> > had a
> >>> >> > >>>> >> Java Example.
> >>> >> > >>>> >>
> >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
> >>> >> > [hidden email]>
> >>> >> > >>>> >> wrote:
> >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just copied
> the
> >> Java
> >>> >> > >>>> ITCase.
> >>> >> > >>>> >> >
> >>> >> > >>>> >> > The only examples that are missing now are
> >> LinearRegression and
> >>> >> > the
> >>> >> > >>>> >> > relational stuff.
> >>> >> > >>>> >> >
> >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
> >>> >> > [hidden email]>
> >>> >> > >>>> >> wrote:
> >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
> >>> >> > >>>> >> >> That was a preprocessing step for the
> >> TriangleEnumeration, and
> >>> >> > is now
> >>> >> > >>>> >> part
> >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I said
> >> before,
> >>> >> > I'd
> >>> >> > >>>> >> prefer to
> >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> >>> >> > >>>> >> >>
> >>> >> > >>>> >> >> Cheers, Fabian
> >>> >> > >>>> >> >>
> >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> >>> >> > [hidden email]>:
> >>> >> > >>>> >> >>
> >>> >> > >>>> >> >>> I added the PageRank example, thanks again fabian. :D
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> Regarding the other stuff:
> >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about including
> >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> >> TypeInformation.
> >>> >> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
> >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in Array, I
> >> noticed
> >>> >> > it
> >>> >> > >>>> while
> >>> >> > >>>> >> >>> writing the tests, you probably had an older version
> of
> >> the
> >>> >> > code.
> >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not
> possible,
> >> this
> >>> >> > is
> >>> >> > >>>> also
> >>> >> > >>>> >> >>> a restriction in the Java API.
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> What do you think about the interface of join and
> >> coGroup?
> >>> >> > Right
> >>> >> > >>>> now,
> >>> >> > >>>> >> >>> you can either use a lambda that returns an Option or
> the
> >>> >> > lambda
> >>> >> > >>>> with
> >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also have a
> >> lambda
> >>> >> > that
> >>> >> > >>>> >> >>> returns a Collection, but due to type erasure this has
> >> the
> >>> >> > same type
> >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
> >> There is
> >>> >> an
> >>> >> > >>>> >> >>> implicit conversion from Option to a Collection, so I
> >> could
> >>> >> > change
> >>> >> > >>>> it
> >>> >> > >>>> >> >>> without breaking the examples we have now. What do you
> >> think?
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> >>> >> ConnectedComponents,
> >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> >>> >> > >>>> TriangleEnumerationNaive/Opt,
> >>> >> > >>>> >> >>> PageRank
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> These are the examples people called dibs on:
> >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port of
> >>> >> > >>>> LinearRegression
> >>> >> > >>>> >> >>> Example from Java)
> >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> >>> >> > >>>> >> >>>  - The relational Stuff
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
> >>> >> > [hidden email]>
> >>> >> > >>>> >> wrote:
> >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> >>> >> > >>>> >> >>> >
> >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek <
> >>> >> > >>>> >> [hidden email]>
> >>> >> > >>>> >> >>> > wrote:
> >>> >> > >>>> >> >>> >
> >>> >> > >>>> >> >>> >> By the way, what was called BatchGradientDescent in
> >> the
> >>> >> > Scala
> >>> >> > >>>> >> examples
> >>> >> > >>>> >> >>> >> should be replaced by a port of the
> LinearRegression
> >>> >> > Example from
> >>> >> > >>>> >> >>> >> Java. I had them as two separate examples earlier.
> >>> >> > >>>> >> >>> >>
> >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any
> thoughts
> >>> >> about
> >>> >> > >>>> removing
> >>> >> > >>>> >> >>> >> RelationalQuery?
> >>> >> > >>>> >> >>> >>
> >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha Krettek
> <
> >>> >> > >>>> >> [hidden email]
> >>> >> > >>>> >> >>> >
> >>> >> > >>>> >> >>> >> wrote:
> >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, thanks
> >>> >> Fabian.
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
> >>> >> > ConnectedComponents,
> >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> > These are the examples people called dibs on:
> >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
> >>> >> > >>>> >> >>> >> >  - The relational Stuff
> >>> >> > >>>> >> >>> >> >  - LinearRegression
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha
> Krettek <
> >>> >> > >>>> >> >>> [hidden email]>
> >>> >> > >>>> >> >>> >> wrote:
> >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list of
> >>> >> > >>>> ported/unported
> >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java
> example
> >>> >> > package to
> >>> >> > >>>> >> >>> examples
> >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine as it
> is.
> >>> >> Just
> >>> >> > >>>> because
> >>> >> > >>>> >> >>> Scala
> >>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean it's
> >> always
> >>> >> > the
> >>> >> > >>>> best
> >>> >> > >>>> >> >>> >> >> choice. :D
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
> >>> >> > ConnectedComponents,
> >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
> >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
> >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
> >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> >>> >> > >>>> >> >>> >> >>  - LinearRegression
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> Cheers,
> >>> >> > >>>> >> >>> >> >> Aljoscha
> >>> >> > >>>> >> >>> >> >>
> >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas Tzoumas
> <
> >>> >> > >>>> >> [hidden email]
> >>> >> > >>>> >> >>> >
> >>> >> > >>>> >> >>> >> wrote:
> >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
> >> termination
> >>> >> > criterion
> >>> >> > >>>> >> in the
> >>> >> > >>>> >> >>> >> Java
> >>> >> > >>>> >> >>> >> >>> version:
> >>> >> > >>>> >> >>> >>
> >>> >> > >>>>
> >> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> >>> >> > >>>> >> >>> >> >>>
> >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination criterion
> in
> >>> >> Scala
> >>> >> > more
> >>> >> > >>>> >> >>> >> functional?
> >>> >> > >>>> >> >>> >> >>>
> >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
> >>> >> > example.java but
> >>> >> > >>>> >> >>> >> examples.scala
> >>> >> > >>>> >> >>> >> >>>
> >>> >> > >>>> >> >>> >> >>> Kostas
> >>> >> > >>>> >> >>> >> >>>
> >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas Tzoumas
> <
> >>> >> > >>>> >> [hidden email]
> >>> >> > >>>> >> >>> >
> >>> >> > >>>> >> >>> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation
> >> (was not
> >>> >> > on
> >>> >> > >>>> your
> >>> >> > >>>> >> >>> list).
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational stuff
> I
> >> can
> >>> >> > take
> >>> >> > >>>> those
> >>> >> > >>>> >> as
> >>> >> > >>>> >> >>> >> well.
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery" from
> >> both
> >>> >> > Scala and
> >>> >> > >>>> >> Java?
> >>> >> > >>>> >> >>> It
> >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3. Does
> it
> >> add
> >>> >> > some
> >>> >> > >>>> >> teaching
> >>> >> > >>>> >> >>> >> value on
> >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>> Kostas
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha
> Krettek
> >> <
> >>> >> > >>>> >> >>> [hidden email]
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> >>>> wrote:
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
> >>> >> > >>>> ConnectedComponents,
> >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs on:
> >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
> >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
> >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> Cheers,
> >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> >>> >> > >>>> >> >>> >> >>>>>
> >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas
> Tzoumas <
> >>> >> > >>>> >> >>> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> wrote:
> >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> >>> >> > >>>> >> >>> >> >>>>> >
> >>> >> > >>>> >> >>> >> >>>>> >
> >>> >> > >>>> >> >>> >>
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >>
> >>> >> > >>>>
> >>> >> >
> >>> >>
> >>
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> >>> >> > >>>> >> >>> >> >>>>> >
> >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> >>> >> > >>>> >> >>> >> >>>>> >
> >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
> >> Krettek <
> >>> >> > >>>> >> >>> >> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >
> >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example
> from
> >>> >> Vasia.
> >>> >> > >>>> >> >>> >> >>>>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> >>> >> > >>>> >> >>> >> >>>>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
> >> Hueske <
> >>> >> > >>>> >> >>> [hidden email]
> >>> >> > >>>> >> >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> >>> >> > >>>> >> >>> >> >>>>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
> >> Krettek <
> >>> >> > >>>> >> >>> >> [hidden email]>:
> >>> >> > >>>> >> >>> >> >>>>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
> >>> >> > KMeans.scala. I'm
> >>> >> > >>>> >> storing
> >>> >> > >>>> >> >>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> data
> >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays and
> >> then
> >>> >> > >>>> converting
> >>> >> > >>>> >> it to
> >>> >> > >>>> >> >>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo to
> >> make
> >>> >> it
> >>> >> > >>>> >> consistent
> >>> >> > >>>> >> >>> >> with the
> >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There is
> no
> >> need
> >>> >> > for a
> >>> >> > >>>> >> >>> keyword,
> >>> >> > >>>> >> >>> >> you
> >>> >> > >>>> >> >>> >> >>>>> >> >> can
> >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1) {
> >> (le,
> >>> >> re)
> >>> >> > =>
> >>> >> > >>>> new
> >>> >> > >>>> >> >>> >> MyResult(le,
> >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> >>> >> > >>>> >> >>> >> >>>>> >> }
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, Fabian
> >> Hueske
> >>> >> <
> >>> >> > >>>> >> >>> >> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I also
> >> found
> >>> >> an
> >>> >> > >>>> >> >>> inconsistency
> >>> >> > >>>> >> >>> >> with
> >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> >>> >> > >>>> >> >>> >> >>>>> >> Java
> >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> > ds1.join(ds2).where(...).equalTo(...)
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> ds1.join(d2).where(...).isEqualTo(...)
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
> >> equalTo(),
> >>> >> > IMO.
> >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?)
> lacks
> >> the
> >>> >> > with()
> >>> >> > >>>> >> method
> >>> >> > >>>> >> >>> >> because
> >>> >> > >>>> >> >>> >> >>>>> >> "with"
> >>> >> > >>>> >> >>> >> >>>>> >> >> is
> >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
> >>> >> something
> >>> >> > >>>> similar
> >>> >> > >>>> >> for
> >>> >> > >>>> >> >>> >> Scala
> >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> >>> >> > >>>> >> >>> >> >>>>> >> >> with
> >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan
> Ewen
> >> <
> >>> >> > >>>> >> [hidden email]
> >>> >> > >>>> >> >>> >:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][] would
> >> work
> >>> >> as
> >>> >> > well.
> >>> >> > >>>> >> That
> >>> >> > >>>> >> >>> is a
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be stored
> as
> >> Java
> >>> >> > or
> >>> >> > >>>> Scala
> >>> >> > >>>> >> >>> Tuples,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between
> the
> >> two.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM,
> Fabian
> >>> >> > Hueske
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
> problem...
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing
> >> them,
> >>> >> but
> >>> >> > >>>> using
> >>> >> > >>>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> >>> >> > >>>> >> >>> >> >>>>> >> won't
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
> >>> >> FileInputFormat.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the
> Strings
> >>> >> > >>>> manually...
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00
> Aljoscha
> >>> >> Krettek
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we
> should
> >> just
> >>> >> > change
> >>> >> > >>>> >> all
> >>> >> > >>>> >> >>> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> >>> >> > >>>> >> >>> >> >>>>> >> input
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV
> input
> >>> >> > formats in
> >>> >> > >>>> all
> >>> >> > >>>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> >>> >> > >>>> >> >>> >> >>>>> >> What
> >>> >> > >>>> >> >>> >> >>>>> >> >> do
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46 AM,
> >>> >> Aljoscha
> >>> >> > >>>> Krettek
> >>> >> > >>>> >> <
> >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that the
> >> data
> >>> >> > types are
> >>> >> > >>>> >> >>> >> incompatible.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you
> proposed:
> >>> >> move
> >>> >> > the
> >>> >> > >>>> >> data to
> >>> >> > >>>> >> >>> a
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> >>> >> > >>>> >> >>> >> >>>>> >> field
> >>> >> > >>>> >> >>> >> >>>>> >> >> and
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> >>> >> > getDefaultEdgeDataSet()
> >>> >> > >>>> >> method in
> >>> >> > >>>> >> >>> >> Scala.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> >>> >> > >>>> >> >>> >> >>>>> >> >> not
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
> >> duplicate the
> >>> >> > data
> >>> >> > >>>> and
> >>> >> > >>>> >> >>> make it
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> >>> >> > >>>> >> >>> >> >>>>> >> for
> >>> >> > >>>> >> >>> >> >>>>> >> >> it
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the Java
> and
> >>> >> Scala
> >>> >> > >>>> >> versions.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think? This
> >> will
> >>> >> > probably
> >>> >> > >>>> >> occur
> >>> >> > >>>> >> >>> in
> >>> >> > >>>> >> >>> >> all
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04
> PM,
> >>> >> > Vasiliki
> >>> >> > >>>> >> Kalavri
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]>
> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
> >>> >> Components
> >>> >> > >>>> >> example,
> >>> >> > >>>> >> >>> but
> >>> >> > >>>> >> >>> >> I am
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> >>> >> > >>>> >> >>> >> >>>>> >> sure
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input data
> >> from
> >>> >> > >>>> >> java-examples.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> ConnectedComponentsData
> >>> >> class,
> >>> >> > the
> >>> >> > >>>> >> vertices
> >>> >> > >>>> >> >>> >> and
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> >>> >> > >>>> >> >>> >> >>>>> >> data
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
> >>> >> > >>>> >> getDefaultVertexDataSet()
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and getDefaultEdgeDataSet(),
> >> which
> >>> >> > take
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> >>> >> > >>>> >> >>> as
> >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide public
> >> static
> >>> >> > fields
> >>> >> > >>>> >> (like
> >>> >> > >>>> >> >>> in
> >>> >> > >>>> >> >>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this introduces a
> >>> >> > conversion
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> >>> >> > >>>> >> >>> Scala
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> >>> >> > >>>> >> >>> >> >>>>> >> >> from
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to scala.Long
> >> and I
> >>> >> > guess
> >>> >> > >>>> this
> >>> >> > >>>> >> is
> >>> >> > >>>> >> >>> an
> >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course, to
> >> copy
> >>> >> the
> >>> >> > >>>> example
> >>> >> > >>>> >> >>> data
> >>> >> > >>>> >> >>> >> in
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> >>> >> > >>>> >> >>> >> >>>>> >> Scala
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something here?
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,
> >> Aljoscha
> >>> >> > >>>> Krettek <
> >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my repo:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >>
> >>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
> >> WordCount
> >>> >> > example.
> >>> >> > >>>> >> It's
> >>> >> > >>>> >> >>> >> pretty
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some
> >> fixups
> >>> >> > for the
> >>> >> > >>>> >> syntax
> >>> >> > >>>> >> >>> and
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
> >> added the
> >>> >> > >>>> >> java-examples
> >>> >> > >>>> >> >>> >> as a
> >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did
> this
> >> to
> >>> >> > reuse
> >>> >> > >>>> the
> >>> >> > >>>> >> >>> example
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> >>> >> > >>>> >> >>> >> >>>>> >> data.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program
> you
> >> can
> >>> >> do
> >>> >> > a
> >>> >> > >>>> pull
> >>> >> > >>>> >> >>> request
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> >>> >> > >>>> >> >>> >> >>>>> >> my
> >>> >> > >>>> >> >>> >> >>>>> >> >> repo
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
> >> examples.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at 12:19
> >> PM,
> >>> >> > Hermann
> >>> >> > >>>> >> Gábor <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for me!
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at
> 11:44
> >> AM,
> >>> >> > Márton
> >>> >> > >>>> >> >>> Balassi <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent for
> me
> >> :)
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
> >> 11:15 AM,
> >>> >> > Kostas
> >>> >> > >>>> >> >>> Tzoumas <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
> WebLogAnalysis.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with
> Scala
> >>> >> > consists of
> >>> >> > >>>> >> going
> >>> >> > >>>> >> >>> >> through
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test
> >> both for
> >>> >> > me and
> >>> >> > >>>> >> the
> >>> >> > >>>> >> >>> new
> >>> >> > >>>> >> >>> >> API
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014 at
> >> 9:09
> >>> >> PM,
> >>> >> > >>>> Vasiliki
> >>> >> > >>>> >> >>> >> Kalavri <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other
> >> people
> >>> >> > >>>> implement
> >>> >> > >>>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected Components
> and
> >>> >> > Kmeans for
> >>> >> > >>>> >> me :)
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014
> >> 21:03,
> >>> >> > Fabian
> >>> >> > >>>> >> Hueske <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> >>> >> > TriangleEnumeration and
> >>> >> > >>>> >> >>> PageRank.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the
> >> examples
> >>> >> > >>>> similar
> >>> >> > >>>> >> to
> >>> >> > >>>> >> >>> the
> >>> >> > >>>> >> >>> >> Java
> >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> >> out-of-the-box
> >>> >> > without
> >>> >> > >>>> >> >>> parameters
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
> >> external
> >>> >> > data
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a similar
> >> code
> >>> >> > structure
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
> >> GMT+02:00
> >>> >> > >>>> Aljoscha
> >>> >> > >>>> >> >>> >> Krettek <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
> >> people can
> >>> >> > >>>> reserve
> >>> >> > >>>> >> their
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4,
> 2014
> >> at
> >>> >> > 8:55 PM,
> >>> >> > >>>> >> Fabian
> >>> >> > >>>> >> >>> >> Hueske
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think having
> >>> >> examples
> >>> >> > >>>> >> implemented
> >>> >> > >>>> >> >>> by
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in the
> >> past.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with
> two
> >> or
> >>> >> > three
> >>> >> > >>>> >> examples.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
> >> helpful if
> >>> >> > you'd
> >>> >> > >>>> >> port a
> >>> >> > >>>> >> >>> >> simple
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> >>> >> > >>>> >> >>> >> >>>>> >> >> one
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04
> 18:47
> >>> >> > GMT+02:00
> >>> >> > >>>> >> Aljoscha
> >>> >> > >>>> >> >>> >> Krettek
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a
> working
> >>> >> > rewrite of
> >>> >> > >>>> the
> >>> >> > >>>> >> >>> Scala
> >>> >> > >>>> >> >>> >> API
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >>> >> > >>>> >> >>> >>
> >>> >> > https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping
> that
> >> I'll
> >>> >> > only
> >>> >> > >>>> have
> >>> >> > >>>> >> to
> >>> >> > >>>> >> >>> >> write
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> >>> >> > >>>> >> >>> >> >>>>> >> tests
> >>> >> > >>>> >> >>> >> >>>>> >> >> and
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do
> you
> >>> >> think
> >>> >> > it
> >>> >> > >>>> makes
> >>> >> > >>>> >> >>> sense
> >>> >> > >>>> >> >>> >> to
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> >>> >> > >>>> >> >>> >> >>>>> >> other
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so
> that
> >>> >> > someone
> >>> >> > >>>> else
> >>> >> > >>>> >> uses
> >>> >> > >>>> >> >>> >> it and
> >>> >> > >>>> >> >>> >> >>>>> >> maybe
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >> >>
> >>> >> > >>>> >> >>> >> >>>>> >>
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>>
> >>> >> > >>>> >> >>> >> >>>
> >>> >> > >>>> >> >>> >>
> >>> >> > >>>> >> >>>
> >>> >> > >>>> >>
> >>> >> > >>>>
> >>> >> >
> >>> >>
>

[Márton Balassi]

Sorry for being a bit silent after already bidding on LR. The pull request
is coming soon.

On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]> wrote:

> I suppose that having the option between simple return type, and a
> collector is the easiest to understand.
>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <[hidden email]>:
>
> > So, should I change join and coGroup to have a simple return value, no
> > Option or Collection? Also what's happening with the relational
> > examples and the LinearRegression examples? I'd like to make a pull
> > request before this weekend.
> >
> > I also added a test that checks whether the Scala API has the same
> > methods as the Java API (ScalaAPICompletenessTest).
> >
> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <[hidden email]>
> > wrote:
> > > Yes, there is already a Collector version, you can do:
> > >
> > > left.join(right).where("foo").equalTo("bar") {
> > >   (left, right, out: Collector[Page]) =>
> > >     if (...) out.collect(...)
> > > }
> > >
> > > I wasn't sure on what our Function2 variant should be. That's why I
> > > asked. There are some cases where you want to have the option (pun
> > > intended) of either returning something from a join or not. But you
> > > could also use a filter after the join. It's a preference thing, I
> > > suppose.
> > >
> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]>
> wrote:
> > >> I think it seems weird that normal joins need to go through option.
> > >>
> > >> The option variant is to allow filters in the join function. Wouldn't
> a
> > >> collector variant allow you to do the same, and would be function3 ? I
> > know
> > >> that option reads more functionally...
> > >>
> > >>
> > >>
> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]>:
> > >>>
> > >>> As already mentioned this is not possible because of type erasure. We
> > >>> can only have one join variant that takes a Function2.
> > >>>
> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]>
> > wrote:
> > >>> > It would be nice to have a join variant that directly returns the
> > value
> > >>> > rathern than an option. Why not have both (they are wrapped as
> > flatJoins
> > >>> > anyway below, right?)
> > >>> >
> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
> [hidden email]>
> > >> wrote:
> > >>> >
> > >>> >> Sweet! I'm lovin' this :-)
> > >>> >>
> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]
> >:
> > >>> >>
> > >>> >> > Also, you can use CaseClasses directly as the type for CSV
> input.
> > So
> > >>> >> > instead of reading it as tuples and then having a mapper that
> > maps to
> > >>> >> > your case classes you can use:
> > >>> >> >
> > >>> >> > env.readCsv[Edge](...)
> > >>> >> >
> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> > >> [hidden email]>
> > >>> >> > wrote:
> > >>> >> > > I added support for specifying keys by name for CaseClasses.
> > Check
> > >> out
> > >>> >> > > the PageRank and TriangleEnumeration examples to see it in
> > action.
> > >>> >> > >
> > >>> >> > > @Kostas: I think you could use them for the TPC-H examples.
> > >>> >> > >
> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> > >> [hidden email]
> > >>> >> >
> > >>> >> > wrote:
> > >>> >> > >> Yes, that would allow list comprehensions. It would be
> > possible to
> > >>> >> > >> have the Collection signature for join (and coGroup), i.e.:
> > >>> >> > >>
> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> > >>> >> > >>
> > >>> >> > >> (T and O are the left and right input type, R is result type)
> > >>> >> > >>
> > >>> >> > >> Then you can return collections and still return an option,
> as
> > in:
> > >>> >> > >>
> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...)
> Some(l)
> > >> else
> > >>> >> > None }
> > >>> >> > >>
> > >>> >> > >> Because there is an implicit conversion from Options to a
> > >> Collection.
> > >>> >> > >> This will always wrap the return value in a List with only
> one
> > >> value.
> > >>> >> > >> I'm not sure we want the overhead here. I'm also not sure
> > whether
> > >> we
> > >>> >> > >> want the overhead of always having to use an Option even
> though
> > >> the
> > >>> >> > >> join always returns a value.
> > >>> >> > >>
> > >>> >> > >> What do you think?
> > >>> >> > >>
> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> > >> [hidden email]>
> > >>> >> > wrote:
> > >>> >> > >>> Hmmm, tricky question...
> > >>> >> > >>> How about the Option for Join as this is a tuple-wise
> > operation
> > >> and
> > >>> >> the
> > >>> >> > >>> Collection for Cogroup which is group-wise?
> > >>> >> > >>> Could we in that case use list comprehensions in Cogroup
> > >> functions?
> > >>> >> > >>>
> > >>> >> > >>> Or is that too much mixing?
> > >>> >> > >>>
> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> > [hidden email]
> > >>>:
> > >>> >> > >>>
> > >>> >> > >>>> I didn't look at the example either.
> > >>> >> > >>>>
> > >>> >> > >>>> Addings collections is easy, it's just that we can either
> > have
> > >>> >> > >>>> Collections or the Option, not both.
> > >>> >> > >>>>
> > >>> >> > >>>> For the coding style I followed this:
> > >>> >> > >>>>
> > >>> >> >
> > >>
> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> > >>> >> ,
> > >>> >> > >>>> which itself is based on this:
> > http://docs.scala-lang.org/style/
> > >> .
> > >>> >> It
> > >>> >> > >>>> is different from the Java Code Guidelines we have in
> place,
> > >> yes.
> > >>> >> > >>>>
> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> > >> [hidden email]
> > >>> >> >
> > >>> >> > >>>> wrote:
> > >>> >> > >>>> > I haven't looked at the LineRank example in detail, but
> if
> > you
> > >>> >> > think that
> > >>> >> > >>>> > it adds something new to the examples collection, we can
> > >> certainly
> > >>> >> > port
> > >>> >> > >>>> it
> > >>> >> > >>>> > also to Java.
> > >>> >> > >>>> > I think the Option and Collector return types are
> > sufficient
> > >> right
> > >>> >> > now
> > >>> >> > >>>> but
> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> > >>> >> > >>>> >
> > >>> >> > >>>> > Great that the Scala primitives are working! Also thanks
> > for
> > >>> >> adding
> > >>> >> > >>>> > genSequence and adapting my examples.
> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or do
> we
> > >> have a
> > >>> >> > >>>> different
> > >>> >> > >>>> > there?
> > >>> >> > >>>> >
> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> > >> [hidden email]
> > >>> >> >:
> > >>> >> > >>>> >
> > >>> >> > >>>> >> What about the LineRank example? We had that in Scala
> but
> > >> never
> > >>> >> > had a
> > >>> >> > >>>> >> Java Example.
> > >>> >> > >>>> >>
> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
> > >>> >> > [hidden email]>
> > >>> >> > >>>> >> wrote:
> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just copied
> > the
> > >> Java
> > >>> >> > >>>> ITCase.
> > >>> >> > >>>> >> >
> > >>> >> > >>>> >> > The only examples that are missing now are
> > >> LinearRegression and
> > >>> >> > the
> > >>> >> > >>>> >> > relational stuff.
> > >>> >> > >>>> >> >
> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
> > >>> >> > [hidden email]>
> > >>> >> > >>>> >> wrote:
> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
> > >>> >> > >>>> >> >> That was a preprocessing step for the
> > >> TriangleEnumeration, and
> > >>> >> > is now
> > >>> >> > >>>> >> part
> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I said
> > >> before,
> > >>> >> > I'd
> > >>> >> > >>>> >> prefer to
> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> > >>> >> > >>>> >> >>
> > >>> >> > >>>> >> >> Cheers, Fabian
> > >>> >> > >>>> >> >>
> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> > >>> >> > [hidden email]>:
> > >>> >> > >>>> >> >>
> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again fabian.
> :D
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> Regarding the other stuff:
> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about
> including
> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> > >> TypeInformation.
> > >>> >> > >>>> >> >>>  - I added generateSequence to ExecutionEnvironment.
> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in Array,
> I
> > >> noticed
> > >>> >> > it
> > >>> >> > >>>> while
> > >>> >> > >>>> >> >>> writing the tests, you probably had an older version
> > of
> > >> the
> > >>> >> > code.
> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not
> > possible,
> > >> this
> > >>> >> > is
> > >>> >> > >>>> also
> > >>> >> > >>>> >> >>> a restriction in the Java API.
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> What do you think about the interface of join and
> > >> coGroup?
> > >>> >> > Right
> > >>> >> > >>>> now,
> > >>> >> > >>>> >> >>> you can either use a lambda that returns an Option
> or
> > the
> > >>> >> > lambda
> > >>> >> > >>>> with
> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also
> have a
> > >> lambda
> > >>> >> > that
> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure this
> has
> > >> the
> > >>> >> > same type
> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
> > >> There is
> > >>> >> an
> > >>> >> > >>>> >> >>> implicit conversion from Option to a Collection, so
> I
> > >> could
> > >>> >> > change
> > >>> >> > >>>> it
> > >>> >> > >>>> >> >>> without breaking the examples we have now. What do
> you
> > >> think?
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> > >>> >> ConnectedComponents,
> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
> > >>> >> > >>>> >> >>> PageRank
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> These are the examples people called dibs on:
> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port
> of
> > >>> >> > >>>> LinearRegression
> > >>> >> > >>>> >> >>> Example from Java)
> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> > >>> >> > >>>> >> >>>  - The relational Stuff
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
> > >>> >> > [hidden email]>
> > >>> >> > >>>> >> wrote:
> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> > >>> >> > >>>> >> >>> >
> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha Krettek
> <
> > >>> >> > >>>> >> [hidden email]>
> > >>> >> > >>>> >> >>> > wrote:
> > >>> >> > >>>> >> >>> >
> > >>> >> > >>>> >> >>> >> By the way, what was called BatchGradientDescent
> in
> > >> the
> > >>> >> > Scala
> > >>> >> > >>>> >> examples
> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
> > LinearRegression
> > >>> >> > Example from
> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples
> earlier.
> > >>> >> > >>>> >> >>> >>
> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any
> > thoughts
> > >>> >> about
> > >>> >> > >>>> removing
> > >>> >> > >>>> >> >>> >> RelationalQuery?
> > >>> >> > >>>> >> >>> >>
> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha
> Krettek
> > <
> > >>> >> > >>>> >> [hidden email]
> > >>> >> > >>>> >> >>> >
> > >>> >> > >>>> >> >>> >> wrote:
> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples,
> thanks
> > >>> >> Fabian.
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
> > >>> >> > ConnectedComponents,
> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> > These are the examples people called dibs on:
> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
> > >>> >> > >>>> >> >>> >> >  - LinearRegression
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha
> > Krettek <
> > >>> >> > >>>> >> >>> [hidden email]>
> > >>> >> > >>>> >> >>> >> wrote:
> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list
> of
> > >>> >> > >>>> ported/unported
> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java
> > example
> > >>> >> > package to
> > >>> >> > >>>> >> >>> examples
> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine as
> it
> > is.
> > >>> >> Just
> > >>> >> > >>>> because
> > >>> >> > >>>> >> >>> Scala
> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean
> it's
> > >> always
> > >>> >> > the
> > >>> >> > >>>> best
> > >>> >> > >>>> >> >>> >> >> choice. :D
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
> > >>> >> > ConnectedComponents,
> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> Cheers,
> > >>> >> > >>>> >> >>> >> >> Aljoscha
> > >>> >> > >>>> >> >>> >> >>
> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas
> Tzoumas
> > <
> > >>> >> > >>>> >> [hidden email]
> > >>> >> > >>>> >> >>> >
> > >>> >> > >>>> >> >>> >> wrote:
> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
> > >> termination
> > >>> >> > criterion
> > >>> >> > >>>> >> in the
> > >>> >> > >>>> >> >>> >> Java
> > >>> >> > >>>> >> >>> >> >>> version:
> > >>> >> > >>>> >> >>> >>
> > >>> >> > >>>>
> > >> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> > >>> >> > >>>> >> >>> >> >>>
> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
> criterion
> > in
> > >>> >> Scala
> > >>> >> > more
> > >>> >> > >>>> >> >>> >> functional?
> > >>> >> > >>>> >> >>> >> >>>
> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
> > >>> >> > example.java but
> > >>> >> > >>>> >> >>> >> examples.scala
> > >>> >> > >>>> >> >>> >> >>>
> > >>> >> > >>>> >> >>> >> >>> Kostas
> > >>> >> > >>>> >> >>> >> >>>
> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas
> Tzoumas
> > <
> > >>> >> > >>>> >> [hidden email]
> > >>> >> > >>>> >> >>> >
> > >>> >> > >>>> >> >>> >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and PiEstimation
> > >> (was not
> > >>> >> > on
> > >>> >> > >>>> your
> > >>> >> > >>>> >> >>> list).
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational
> stuff
> > I
> > >> can
> > >>> >> > take
> > >>> >> > >>>> those
> > >>> >> > >>>> >> as
> > >>> >> > >>>> >> >>> >> well.
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery"
> from
> > >> both
> > >>> >> > Scala and
> > >>> >> > >>>> >> Java?
> > >>> >> > >>>> >> >>> It
> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3.
> Does
> > it
> > >> add
> > >>> >> > some
> > >>> >> > >>>> >> teaching
> > >>> >> > >>>> >> >>> >> value on
> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>> Kostas
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha
> > Krettek
> > >> <
> > >>> >> > >>>> >> >>> [hidden email]
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> >>>> wrote:
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
> > >>> >> > >>>> ConnectedComponents,
> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs
> on:
> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank (Fabian)
> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> > >>> >> > >>>> >> >>> >> >>>>>
> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas
> > Tzoumas <
> > >>> >> > >>>> >> >>> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> > >>> >> > >>>> >> >>> >> >>>>> >
> > >>> >> > >>>> >> >>> >> >>>>> >
> > >>> >> > >>>> >> >>> >>
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >>
> > >>> >> > >>>>
> > >>> >> >
> > >>> >>
> > >>
> >
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> > >>> >> > >>>> >> >>> >> >>>>> >
> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> > >>> >> > >>>> >> >>> >> >>>>> >
> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
> > >> Krettek <
> > >>> >> > >>>> >> >>> >> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >
> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example
> > from
> > >>> >> Vasia.
> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
> > >> Hueske <
> > >>> >> > >>>> >> >>> [hidden email]
> > >>> >> > >>>> >> >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
> > >> Krettek <
> > >>> >> > >>>> >> >>> >> [hidden email]>:
> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
> > >>> >> > KMeans.scala. I'm
> > >>> >> > >>>> >> storing
> > >>> >> > >>>> >> >>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays
> and
> > >> then
> > >>> >> > >>>> converting
> > >>> >> > >>>> >> it to
> > >>> >> > >>>> >> >>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to equalTo
> to
> > >> make
> > >>> >> it
> > >>> >> > >>>> >> consistent
> > >>> >> > >>>> >> >>> >> with the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There
> is
> > no
> > >> need
> > >>> >> > for a
> > >>> >> > >>>> >> >>> keyword,
> > >>> >> > >>>> >> >>> >> you
> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> left.join(right).where(0).equalTo(1)
> {
> > >> (le,
> > >>> >> re)
> > >>> >> > =>
> > >>> >> > >>>> new
> > >>> >> > >>>> >> >>> >> MyResult(le,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> > >>> >> > >>>> >> >>> >> >>>>> >> }
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM,
> Fabian
> > >> Hueske
> > >>> >> <
> > >>> >> > >>>> >> >>> >> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I
> also
> > >> found
> > >>> >> an
> > >>> >> > >>>> >> >>> inconsistency
> > >>> >> > >>>> >> >>> >> with
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> > >>> >> > >>>> >> >>> >> >>>>> >> Java
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> ds1.join(ds2).where(...).equalTo(...)
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this is:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > ds1.join(d2).where(...).isEqualTo(...)
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
> > >> equalTo(),
> > >>> >> > IMO.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?)
> > lacks
> > >> the
> > >>> >> > with()
> > >>> >> > >>>> >> method
> > >>> >> > >>>> >> >>> >> because
> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be offer
> > >>> >> something
> > >>> >> > >>>> similar
> > >>> >> > >>>> >> for
> > >>> >> > >>>> >> >>> >> Scala
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan
> > Ewen
> > >> <
> > >>> >> > >>>> >> [hidden email]
> > >>> >> > >>>> >> >>> >:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][]
> would
> > >> work
> > >>> >> as
> > >>> >> > well.
> > >>> >> > >>>> >> That
> > >>> >> > >>>> >> >>> is a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be
> stored
> > as
> > >> Java
> > >>> >> > or
> > >>> >> > >>>> Scala
> > >>> >> > >>>> >> >>> Tuples,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between
> > the
> > >> two.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM,
> > Fabian
> > >>> >> > Hueske
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
> > problem...
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and parsing
> > >> them,
> > >>> >> but
> > >>> >> > >>>> using
> > >>> >> > >>>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> > >>> >> > >>>> >> >>> >> >>>>> >> won't
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
> > >>> >> FileInputFormat.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the
> > Strings
> > >>> >> > >>>> manually...
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00
> > Aljoscha
> > >>> >> Krettek
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we
> > should
> > >> just
> > >>> >> > change
> > >>> >> > >>>> >> all
> > >>> >> > >>>> >> >>> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> > >>> >> > >>>> >> >>> >> >>>>> >> input
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV
> > input
> > >>> >> > formats in
> > >>> >> > >>>> all
> > >>> >> > >>>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> > >>> >> > >>>> >> >>> >> >>>>> >> What
> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46
> AM,
> > >>> >> Aljoscha
> > >>> >> > >>>> Krettek
> > >>> >> > >>>> >> <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that
> the
> > >> data
> > >>> >> > types are
> > >>> >> > >>>> >> >>> >> incompatible.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you
> > proposed:
> > >>> >> move
> > >>> >> > the
> > >>> >> > >>>> >> data to
> > >>> >> > >>>> >> >>> a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> > >>> >> > >>>> >> >>> >> >>>>> >> field
> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> > >>> >> > getDefaultEdgeDataSet()
> > >>> >> > >>>> >> method in
> > >>> >> > >>>> >> >>> >> Scala.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
> > >> duplicate the
> > >>> >> > data
> > >>> >> > >>>> and
> > >>> >> > >>>> >> >>> make it
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> > >>> >> > >>>> >> >>> >> >>>>> >> for
> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the
> Java
> > and
> > >>> >> Scala
> > >>> >> > >>>> >> versions.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think?
> This
> > >> will
> > >>> >> > probably
> > >>> >> > >>>> >> occur
> > >>> >> > >>>> >> >>> in
> > >>> >> > >>>> >> >>> >> all
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at 10:04
> > PM,
> > >>> >> > Vasiliki
> > >>> >> > >>>> >> Kalavri
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]>
> > >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the Connected
> > >>> >> Components
> > >>> >> > >>>> >> example,
> > >>> >> > >>>> >> >>> but
> > >>> >> > >>>> >> >>> >> I am
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> > >>> >> > >>>> >> >>> >> >>>>> >> sure
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input
> data
> > >> from
> > >>> >> > >>>> >> java-examples.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> > ConnectedComponentsData
> > >>> >> class,
> > >>> >> > the
> > >>> >> > >>>> >> vertices
> > >>> >> > >>>> >> >>> >> and
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> > >>> >> > >>>> >> >>> >> >>>>> >> data
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
> > >>> >> > >>>> >> getDefaultVertexDataSet()
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
> getDefaultEdgeDataSet(),
> > >> which
> > >>> >> > take
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> > >>> >> > >>>> >> >>> as
> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide
> public
> > >> static
> > >>> >> > fields
> > >>> >> > >>>> >> (like
> > >>> >> > >>>> >> >>> in
> > >>> >> > >>>> >> >>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
> introduces a
> > >>> >> > conversion
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> > >>> >> > >>>> >> >>> Scala
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
> scala.Long
> > >> and I
> > >>> >> > guess
> > >>> >> > >>>> this
> > >>> >> > >>>> >> is
> > >>> >> > >>>> >> >>> an
> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of course,
> to
> > >> copy
> > >>> >> the
> > >>> >> > >>>> example
> > >>> >> > >>>> >> >>> data
> > >>> >> > >>>> >> >>> >> in
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something
> here?
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,
> > >> Aljoscha
> > >>> >> > >>>> Krettek <
> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my
> repo:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >>
> > >>> >> >
> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
> > >> WordCount
> > >>> >> > example.
> > >>> >> > >>>> >> It's
> > >>> >> > >>>> >> >>> >> pretty
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with some
> > >> fixups
> > >>> >> > for the
> > >>> >> > >>>> >> syntax
> > >>> >> > >>>> >> >>> and
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
> > >> added the
> > >>> >> > >>>> >> java-examples
> > >>> >> > >>>> >> >>> >> as a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did
> > this
> > >> to
> > >>> >> > reuse
> > >>> >> > >>>> the
> > >>> >> > >>>> >> >>> example
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> > >>> >> > >>>> >> >>> >> >>>>> >> data.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program
> > you
> > >> can
> > >>> >> do
> > >>> >> > a
> > >>> >> > >>>> pull
> > >>> >> > >>>> >> >>> request
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> > >>> >> > >>>> >> >>> >> >>>>> >> my
> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
> > >> examples.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at
> 12:19
> > >> PM,
> > >>> >> > Hermann
> > >>> >> > >>>> >> Gábor <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for
> me!
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at
> > 11:44
> > >> AM,
> > >>> >> > Márton
> > >>> >> > >>>> >> >>> Balassi <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent
> for
> > me
> > >> :)
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
> > >> 11:15 AM,
> > >>> >> > Kostas
> > >>> >> > >>>> >> >>> Tzoumas <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
> > WebLogAnalysis.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with
> > Scala
> > >>> >> > consists of
> > >>> >> > >>>> >> going
> > >>> >> > >>>> >> >>> >> through
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress test
> > >> both for
> > >>> >> > me and
> > >>> >> > >>>> >> the
> > >>> >> > >>>> >> >>> new
> > >>> >> > >>>> >> >>> >> API
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014
> at
> > >> 9:09
> > >>> >> PM,
> > >>> >> > >>>> Vasiliki
> > >>> >> > >>>> >> >>> >> Kalavri <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having other
> > >> people
> > >>> >> > >>>> implement
> > >>> >> > >>>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected
> Components
> > and
> > >>> >> > Kmeans for
> > >>> >> > >>>> >> me :)
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September 2014
> > >> 21:03,
> > >>> >> > Fabian
> > >>> >> > >>>> >> Hueske <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> > >>> >> > TriangleEnumeration and
> > >>> >> > >>>> >> >>> PageRank.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do the
> > >> examples
> > >>> >> > >>>> similar
> > >>> >> > >>>> >> to
> > >>> >> > >>>> >> >>> the
> > >>> >> > >>>> >> >>> >> Java
> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> > >> out-of-the-box
> > >>> >> > without
> > >>> >> > >>>> >> >>> parameters
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
> > >> external
> > >>> >> > data
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a
> similar
> > >> code
> > >>> >> > structure
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
> > >> GMT+02:00
> > >>> >> > >>>> Aljoscha
> > >>> >> > >>>> >> >>> >> Krettek <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
> > >> people can
> > >>> >> > >>>> reserve
> > >>> >> > >>>> >> their
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4,
> > 2014
> > >> at
> > >>> >> > 8:55 PM,
> > >>> >> > >>>> >> Fabian
> > >>> >> > >>>> >> >>> >> Hueske
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think
> having
> > >>> >> examples
> > >>> >> > >>>> >> implemented
> > >>> >> > >>>> >> >>> by
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in
> the
> > >> past.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help with
> > two
> > >> or
> > >>> >> > three
> > >>> >> > >>>> >> examples.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
> > >> helpful if
> > >>> >> > you'd
> > >>> >> > >>>> >> port a
> > >>> >> > >>>> >> >>> >> simple
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> > >>> >> > >>>> >> >>> >> >>>>> >> >> one
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04
> > 18:47
> > >>> >> > GMT+02:00
> > >>> >> > >>>> >> Aljoscha
> > >>> >> > >>>> >> >>> >> Krettek
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a
> > working
> > >>> >> > rewrite of
> > >>> >> > >>>> the
> > >>> >> > >>>> >> >>> Scala
> > >>> >> > >>>> >> >>> >> API
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >>> >> > >>>> >> >>> >>
> > >>> >> >
> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping
> > that
> > >> I'll
> > >>> >> > only
> > >>> >> > >>>> have
> > >>> >> > >>>> >> to
> > >>> >> > >>>> >> >>> >> write
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> tests
> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples. Do
> > you
> > >>> >> think
> > >>> >> > it
> > >>> >> > >>>> makes
> > >>> >> > >>>> >> >>> sense
> > >>> >> > >>>> >> >>> >> to
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> > >>> >> > >>>> >> >>> >> >>>>> >> other
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples, so
> > that
> > >>> >> > someone
> > >>> >> > >>>> else
> > >>> >> > >>>> >> uses
> > >>> >> > >>>> >> >>> >> it and
> > >>> >> > >>>> >> >>> >> >>>>> >> maybe
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>>
> > >>> >> > >>>> >> >>> >> >>>
> > >>> >> > >>>> >> >>> >>
> > >>> >> > >>>> >> >>>
> > >>> >> > >>>> >>
> > >>> >> > >>>>
> > >>> >> >
> > >>> >>
> >
>

[Márton Balassi]

Pull request issued. One minor naming concern:

As of today the scala examples are located at
the org.apache.flink.examples.scala package, while the java ones in
the org.apache.flink.example.java. I suggest using only one convention for
this either example or examples.

Cheers,

Marton

On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <[hidden email]>
wrote:

> Sorry for being a bit silent after already bidding on LR. The pull request
> is coming soon.
>
> On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]> wrote:
>
>> I suppose that having the option between simple return type, and a
>> collector is the easiest to understand.
>>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <[hidden email]>:
>>
>> > So, should I change join and coGroup to have a simple return value, no
>> > Option or Collection? Also what's happening with the relational
>> > examples and the LinearRegression examples? I'd like to make a pull
>> > request before this weekend.
>> >
>> > I also added a test that checks whether the Scala API has the same
>> > methods as the Java API (ScalaAPICompletenessTest).
>> >
>> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <[hidden email]>
>> > wrote:
>> > > Yes, there is already a Collector version, you can do:
>> > >
>> > > left.join(right).where("foo").equalTo("bar") {
>> > >   (left, right, out: Collector[Page]) =>
>> > >     if (...) out.collect(...)
>> > > }
>> > >
>> > > I wasn't sure on what our Function2 variant should be. That's why I
>> > > asked. There are some cases where you want to have the option (pun
>> > > intended) of either returning something from a join or not. But you
>> > > could also use a filter after the join. It's a preference thing, I
>> > > suppose.
>> > >
>> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]>
>> wrote:
>> > >> I think it seems weird that normal joins need to go through option.
>> > >>
>> > >> The option variant is to allow filters in the join function.
>> Wouldn't a
>> > >> collector variant allow you to do the same, and would be function3 ?
>> I
>> > know
>> > >> that option reads more functionally...
>> > >>
>> > >>
>> > >>
>> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <[hidden email]
>> >:
>> > >>>
>> > >>> As already mentioned this is not possible because of type erasure.
>> We
>> > >>> can only have one join variant that takes a Function2.
>> > >>>
>> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]>
>> > wrote:
>> > >>> > It would be nice to have a join variant that directly returns the
>> > value
>> > >>> > rathern than an option. Why not have both (they are wrapped as
>> > flatJoins
>> > >>> > anyway below, right?)
>> > >>> >
>> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
>> [hidden email]>
>> > >> wrote:
>> > >>> >
>> > >>> >> Sweet! I'm lovin' this :-)
>> > >>> >>
>> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <[hidden email]
>> >:
>> > >>> >>
>> > >>> >> > Also, you can use CaseClasses directly as the type for CSV
>> input.
>> > So
>> > >>> >> > instead of reading it as tuples and then having a mapper that
>> > maps to
>> > >>> >> > your case classes you can use:
>> > >>> >> >
>> > >>> >> > env.readCsv[Edge](...)
>> > >>> >> >
>> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
>> > >> [hidden email]>
>> > >>> >> > wrote:
>> > >>> >> > > I added support for specifying keys by name for CaseClasses.
>> > Check
>> > >> out
>> > >>> >> > > the PageRank and TriangleEnumeration examples to see it in
>> > action.
>> > >>> >> > >
>> > >>> >> > > @Kostas: I think you could use them for the TPC-H examples.
>> > >>> >> > >
>> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
>> > >> [hidden email]
>> > >>> >> >
>> > >>> >> > wrote:
>> > >>> >> > >> Yes, that would allow list comprehensions. It would be
>> > possible to
>> > >>> >> > >> have the Collection signature for join (and coGroup), i.e.:
>> > >>> >> > >>
>> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
>> > >>> >> > >>
>> > >>> >> > >> (T and O are the left and right input type, R is result
>> type)
>> > >>> >> > >>
>> > >>> >> > >> Then you can return collections and still return an option,
>> as
>> > in:
>> > >>> >> > >>
>> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...)
>> Some(l)
>> > >> else
>> > >>> >> > None }
>> > >>> >> > >>
>> > >>> >> > >> Because there is an implicit conversion from Options to a
>> > >> Collection.
>> > >>> >> > >> This will always wrap the return value in a List with only
>> one
>> > >> value.
>> > >>> >> > >> I'm not sure we want the overhead here. I'm also not sure
>> > whether
>> > >> we
>> > >>> >> > >> want the overhead of always having to use an Option even
>> though
>> > >> the
>> > >>> >> > >> join always returns a value.
>> > >>> >> > >>
>> > >>> >> > >> What do you think?
>> > >>> >> > >>
>> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
>> > >> [hidden email]>
>> > >>> >> > wrote:
>> > >>> >> > >>> Hmmm, tricky question...
>> > >>> >> > >>> How about the Option for Join as this is a tuple-wise
>> > operation
>> > >> and
>> > >>> >> the
>> > >>> >> > >>> Collection for Cogroup which is group-wise?
>> > >>> >> > >>> Could we in that case use list comprehensions in Cogroup
>> > >> functions?
>> > >>> >> > >>>
>> > >>> >> > >>> Or is that too much mixing?
>> > >>> >> > >>>
>> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
>> > [hidden email]
>> > >>>:
>> > >>> >> > >>>
>> > >>> >> > >>>> I didn't look at the example either.
>> > >>> >> > >>>>
>> > >>> >> > >>>> Addings collections is easy, it's just that we can either
>> > have
>> > >>> >> > >>>> Collections or the Option, not both.
>> > >>> >> > >>>>
>> > >>> >> > >>>> For the coding style I followed this:
>> > >>> >> > >>>>
>> > >>> >> >
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>> > >>> >> ,
>> > >>> >> > >>>> which itself is based on this:
>> > http://docs.scala-lang.org/style/
>> > >> .
>> > >>> >> It
>> > >>> >> > >>>> is different from the Java Code Guidelines we have in
>> place,
>> > >> yes.
>> > >>> >> > >>>>
>> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
>> > >> [hidden email]
>> > >>> >> >
>> > >>> >> > >>>> wrote:
>> > >>> >> > >>>> > I haven't looked at the LineRank example in detail, but
>> if
>> > you
>> > >>> >> > think that
>> > >>> >> > >>>> > it adds something new to the examples collection, we can
>> > >> certainly
>> > >>> >> > port
>> > >>> >> > >>>> it
>> > >>> >> > >>>> > also to Java.
>> > >>> >> > >>>> > I think the Option and Collector return types are
>> > sufficient
>> > >> right
>> > >>> >> > now
>> > >>> >> > >>>> but
>> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
>> > >>> >> > >>>> >
>> > >>> >> > >>>> > Great that the Scala primitives are working! Also thanks
>> > for
>> > >>> >> adding
>> > >>> >> > >>>> > genSequence and adapting my examples.
>> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or do
>> we
>> > >> have a
>> > >>> >> > >>>> different
>> > >>> >> > >>>> > there?
>> > >>> >> > >>>> >
>> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
>> > >> [hidden email]
>> > >>> >> >:
>> > >>> >> > >>>> >
>> > >>> >> > >>>> >> What about the LineRank example? We had that in Scala
>> but
>> > >> never
>> > >>> >> > had a
>> > >>> >> > >>>> >> Java Example.
>> > >>> >> > >>>> >>
>> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
>> > >>> >> > [hidden email]>
>> > >>> >> > >>>> >> wrote:
>> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just
>> copied
>> > the
>> > >> Java
>> > >>> >> > >>>> ITCase.
>> > >>> >> > >>>> >> >
>> > >>> >> > >>>> >> > The only examples that are missing now are
>> > >> LinearRegression and
>> > >>> >> > the
>> > >>> >> > >>>> >> > relational stuff.
>> > >>> >> > >>>> >> >
>> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
>> > >>> >> > [hidden email]>
>> > >>> >> > >>>> >> wrote:
>> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
>> > >>> >> > >>>> >> >> That was a preprocessing step for the
>> > >> TriangleEnumeration, and
>> > >>> >> > is now
>> > >>> >> > >>>> >> part
>> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
>> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I
>> said
>> > >> before,
>> > >>> >> > I'd
>> > >>> >> > >>>> >> prefer to
>> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
>> > >>> >> > >>>> >> >>
>> > >>> >> > >>>> >> >> Cheers, Fabian
>> > >>> >> > >>>> >> >>
>> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
>> > >>> >> > [hidden email]>:
>> > >>> >> > >>>> >> >>
>> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again fabian.
>> :D
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> Regarding the other stuff:
>> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about
>> including
>> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
>> > >> TypeInformation.
>> > >>> >> > >>>> >> >>>  - I added generateSequence to
>> ExecutionEnvironment.
>> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in
>> Array, I
>> > >> noticed
>> > >>> >> > it
>> > >>> >> > >>>> while
>> > >>> >> > >>>> >> >>> writing the tests, you probably had an older
>> version
>> > of
>> > >> the
>> > >>> >> > code.
>> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not
>> > possible,
>> > >> this
>> > >>> >> > is
>> > >>> >> > >>>> also
>> > >>> >> > >>>> >> >>> a restriction in the Java API.
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> What do you think about the interface of join and
>> > >> coGroup?
>> > >>> >> > Right
>> > >>> >> > >>>> now,
>> > >>> >> > >>>> >> >>> you can either use a lambda that returns an Option
>> or
>> > the
>> > >>> >> > lambda
>> > >>> >> > >>>> with
>> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also
>> have a
>> > >> lambda
>> > >>> >> > that
>> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure this
>> has
>> > >> the
>> > >>> >> > same type
>> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use it.
>> > >> There is
>> > >>> >> an
>> > >>> >> > >>>> >> >>> implicit conversion from Option to a Collection,
>> so I
>> > >> could
>> > >>> >> > change
>> > >>> >> > >>>> it
>> > >>> >> > >>>> >> >>> without breaking the examples we have now. What do
>> you
>> > >> think?
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
>> > >>> >> ConnectedComponents,
>> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
>> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
>> > >>> >> > >>>> >> >>> PageRank
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> These are the examples people called dibs on:
>> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a port
>> of
>> > >>> >> > >>>> LinearRegression
>> > >>> >> > >>>> >> >>> Example from Java)
>> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
>> > >>> >> > >>>> >> >>>  - The relational Stuff
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
>> > >>> >> > [hidden email]>
>> > >>> >> > >>>> >> wrote:
>> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
>> > >>> >> > >>>> >> >>> >
>> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
>> Krettek <
>> > >>> >> > >>>> >> [hidden email]>
>> > >>> >> > >>>> >> >>> > wrote:
>> > >>> >> > >>>> >> >>> >
>> > >>> >> > >>>> >> >>> >> By the way, what was called
>> BatchGradientDescent in
>> > >> the
>> > >>> >> > Scala
>> > >>> >> > >>>> >> examples
>> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
>> > LinearRegression
>> > >>> >> > Example from
>> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples
>> earlier.
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any
>> > thoughts
>> > >>> >> about
>> > >>> >> > >>>> removing
>> > >>> >> > >>>> >> >>> >> RelationalQuery?
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha
>> Krettek
>> > <
>> > >>> >> > >>>> >> [hidden email]
>> > >>> >> > >>>> >> >>> >
>> > >>> >> > >>>> >> >>> >> wrote:
>> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples,
>> thanks
>> > >>> >> Fabian.
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
>> > >>> >> > ConnectedComponents,
>> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
>> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> > These are the examples people called dibs on:
>> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
>> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
>> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
>> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
>> > >>> >> > >>>> >> >>> >> >  - LinearRegression
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha
>> > Krettek <
>> > >>> >> > >>>> >> >>> [hidden email]>
>> > >>> >> > >>>> >> >>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running list
>> of
>> > >>> >> > >>>> ported/unported
>> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java
>> > example
>> > >>> >> > package to
>> > >>> >> > >>>> >> >>> examples
>> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine as
>> it
>> > is.
>> > >>> >> Just
>> > >>> >> > >>>> because
>> > >>> >> > >>>> >> >>> Scala
>> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean
>> it's
>> > >> always
>> > >>> >> > the
>> > >>> >> > >>>> best
>> > >>> >> > >>>> >> >>> >> >> choice. :D
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
>> > >>> >> > ConnectedComponents,
>> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> These are the examples people called dibs on:
>> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
>> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
>> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
>> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
>> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> Cheers,
>> > >>> >> > >>>> >> >>> >> >> Aljoscha
>> > >>> >> > >>>> >> >>> >> >>
>> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas
>> Tzoumas
>> > <
>> > >>> >> > >>>> >> [hidden email]
>> > >>> >> > >>>> >> >>> >
>> > >>> >> > >>>> >> >>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
>> > >> termination
>> > >>> >> > criterion
>> > >>> >> > >>>> >> in the
>> > >>> >> > >>>> >> >>> >> Java
>> > >>> >> > >>>> >> >>> >> >>> version:
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> > >>>>
>> > >> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
>> > >>> >> > >>>> >> >>> >> >>>
>> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
>> criterion
>> > in
>> > >>> >> Scala
>> > >>> >> > more
>> > >>> >> > >>>> >> >>> >> functional?
>> > >>> >> > >>>> >> >>> >> >>>
>> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name is
>> > >>> >> > example.java but
>> > >>> >> > >>>> >> >>> >> examples.scala
>> > >>> >> > >>>> >> >>> >> >>>
>> > >>> >> > >>>> >> >>> >> >>> Kostas
>> > >>> >> > >>>> >> >>> >> >>>
>> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas
>> Tzoumas
>> > <
>> > >>> >> > >>>> >> [hidden email]
>> > >>> >> > >>>> >> >>> >
>> > >>> >> > >>>> >> >>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
>> PiEstimation
>> > >> (was not
>> > >>> >> > on
>> > >>> >> > >>>> your
>> > >>> >> > >>>> >> >>> list).
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational
>> stuff
>> > I
>> > >> can
>> > >>> >> > take
>> > >>> >> > >>>> those
>> > >>> >> > >>>> >> as
>> > >>> >> > >>>> >> >>> >> well.
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery"
>> from
>> > >> both
>> > >>> >> > Scala and
>> > >>> >> > >>>> >> Java?
>> > >>> >> > >>>> >> >>> It
>> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3.
>> Does
>> > it
>> > >> add
>> > >>> >> > some
>> > >>> >> > >>>> >> teaching
>> > >>> >> > >>>> >> >>> >> value on
>> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>> Kostas
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha
>> > Krettek
>> > >> <
>> > >>> >> > >>>> >> >>> [hidden email]
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an ITCase.
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, KMeans,
>> > >>> >> > >>>> ConnectedComponents,
>> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called dibs
>> on:
>> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
>> (Fabian)
>> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
>> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not mistaken):
>> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
>> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
>> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
>> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
>> > >>> >> > >>>> >> >>> >> >>>>>
>> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas
>> > Tzoumas <
>> > >>> >> > >>>> >> >>> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
>> > >>> >> > >>>> >> >>> >> >>>>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >>
>> > >>> >> > >>>>
>> > >>> >> >
>> > >>> >>
>> > >>
>> >
>> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
>> > >>> >> > >>>> >> >>> >> >>>>> >
>> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
>> > >>> >> > >>>> >> >>> >> >>>>> >
>> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, Aljoscha
>> > >> Krettek <
>> > >>> >> > >>>> >> >>> >> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents Example
>> > from
>> > >>> >> Vasia.
>> > >>> >> > >>>> >> >>> >> >>>>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
>> > >>> >> > >>>> >> >>> >> >>>>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, Fabian
>> > >> Hueske <
>> > >>> >> > >>>> >> >>> [hidden email]
>> > >>> >> > >>>> >> >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
>> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
>> > >>> >> > >>>> >> >>> >> >>>>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
>> > >> Krettek <
>> > >>> >> > >>>> >> >>> >> [hidden email]>:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
>> > >>> >> > KMeans.scala. I'm
>> > >>> >> > >>>> >> storing
>> > >>> >> > >>>> >> >>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays
>> and
>> > >> then
>> > >>> >> > >>>> converting
>> > >>> >> > >>>> >> it to
>> > >>> >> > >>>> >> >>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
>> equalTo to
>> > >> make
>> > >>> >> it
>> > >>> >> > >>>> >> consistent
>> > >>> >> > >>>> >> >>> >> with the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). There
>> is
>> > no
>> > >> need
>> > >>> >> > for a
>> > >>> >> > >>>> >> >>> keyword,
>> > >>> >> > >>>> >> >>> >> you
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> left.join(right).where(0).equalTo(1) {
>> > >> (le,
>> > >>> >> re)
>> > >>> >> > =>
>> > >>> >> > >>>> new
>> > >>> >> > >>>> >> >>> >> MyResult(le,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
>> > >>> >> > >>>> >> >>> >> >>>>> >> }
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM,
>> Fabian
>> > >> Hueske
>> > >>> >> <
>> > >>> >> > >>>> >> >>> >> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I
>> also
>> > >> found
>> > >>> >> an
>> > >>> >> > >>>> >> >>> inconsistency
>> > >>> >> > >>>> >> >>> >> with
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
>> > >>> >> > >>>> >> >>> >> >>>>> >> Java
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> ds1.join(ds2).where(...).equalTo(...)
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this
>> is:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > ds1.join(d2).where(...).isEqualTo(...)
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
>> > >> equalTo(),
>> > >>> >> > IMO.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?)
>> > lacks
>> > >> the
>> > >>> >> > with()
>> > >>> >> > >>>> >> method
>> > >>> >> > >>>> >> >>> >> because
>> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be
>> offer
>> > >>> >> something
>> > >>> >> > >>>> similar
>> > >>> >> > >>>> >> for
>> > >>> >> > >>>> >> >>> >> Scala
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 Stephan
>> > Ewen
>> > >> <
>> > >>> >> > >>>> >> [hidden email]
>> > >>> >> > >>>> >> >>> >:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][]
>> would
>> > >> work
>> > >>> >> as
>> > >>> >> > well.
>> > >>> >> > >>>> >> That
>> > >>> >> > >>>> >> >>> is a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be
>> stored
>> > as
>> > >> Java
>> > >>> >> > or
>> > >>> >> > >>>> Scala
>> > >>> >> > >>>> >> >>> Tuples,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert between
>> > the
>> > >> two.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 AM,
>> > Fabian
>> > >>> >> > Hueske
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
>> > problem...
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and
>> parsing
>> > >> them,
>> > >>> >> but
>> > >>> >> > >>>> using
>> > >>> >> > >>>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
>> > >>> >> > >>>> >> >>> >> >>>>> >> won't
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
>> > >>> >> FileInputFormat.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the
>> > Strings
>> > >>> >> > >>>> manually...
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00
>> > Aljoscha
>> > >>> >> Krettek
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we
>> > should
>> > >> just
>> > >>> >> > change
>> > >>> >> > >>>> >> all
>> > >>> >> > >>>> >> >>> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
>> > >>> >> > >>>> >> >>> >> >>>>> >> input
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV
>> > input
>> > >>> >> > formats in
>> > >>> >> > >>>> all
>> > >>> >> > >>>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
>> > >>> >> > >>>> >> >>> >> >>>>> >> What
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46
>> AM,
>> > >>> >> Aljoscha
>> > >>> >> > >>>> Krettek
>> > >>> >> > >>>> >> <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that
>> the
>> > >> data
>> > >>> >> > types are
>> > >>> >> > >>>> >> >>> >> incompatible.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you
>> > proposed:
>> > >>> >> move
>> > >>> >> > the
>> > >>> >> > >>>> >> data to
>> > >>> >> > >>>> >> >>> a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
>> > >>> >> > >>>> >> >>> >> >>>>> >> field
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
>> > >>> >> > getDefaultEdgeDataSet()
>> > >>> >> > >>>> >> method in
>> > >>> >> > >>>> >> >>> >> Scala.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
>> > >> duplicate the
>> > >>> >> > data
>> > >>> >> > >>>> and
>> > >>> >> > >>>> >> >>> make it
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
>> > >>> >> > >>>> >> >>> >> >>>>> >> for
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the
>> Java
>> > and
>> > >>> >> Scala
>> > >>> >> > >>>> >> versions.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think?
>> This
>> > >> will
>> > >>> >> > probably
>> > >>> >> > >>>> >> occur
>> > >>> >> > >>>> >> >>> in
>> > >>> >> > >>>> >> >>> >> all
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at
>> 10:04
>> > PM,
>> > >>> >> > Vasiliki
>> > >>> >> > >>>> >> Kalavri
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <[hidden email]
>> >
>> > >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
>> Connected
>> > >>> >> Components
>> > >>> >> > >>>> >> example,
>> > >>> >> > >>>> >> >>> but
>> > >>> >> > >>>> >> >>> >> I am
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
>> > >>> >> > >>>> >> >>> >> >>>>> >> sure
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input
>> data
>> > >> from
>> > >>> >> > >>>> >> java-examples.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
>> > ConnectedComponentsData
>> > >>> >> class,
>> > >>> >> > the
>> > >>> >> > >>>> >> vertices
>> > >>> >> > >>>> >> >>> >> and
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
>> > >>> >> > >>>> >> >>> >> >>>>> >> data
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
>> > >>> >> > >>>> >> getDefaultVertexDataSet()
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
>> getDefaultEdgeDataSet(),
>> > >> which
>> > >>> >> > take
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
>> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
>> > >>> >> > >>>> >> >>> as
>> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide
>> public
>> > >> static
>> > >>> >> > fields
>> > >>> >> > >>>> >> (like
>> > >>> >> > >>>> >> >>> in
>> > >>> >> > >>>> >> >>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
>> introduces a
>> > >>> >> > conversion
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
>> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
>> > >>> >> > >>>> >> >>> Scala
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
>> scala.Long
>> > >> and I
>> > >>> >> > guess
>> > >>> >> > >>>> this
>> > >>> >> > >>>> >> is
>> > >>> >> > >>>> >> >>> an
>> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
>> course, to
>> > >> copy
>> > >>> >> the
>> > >>> >> > >>>> example
>> > >>> >> > >>>> >> >>> data
>> > >>> >> > >>>> >> >>> >> in
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something
>> here?
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 15:52,
>> > >> Aljoscha
>> > >>> >> > >>>> Krettek <
>> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my
>> repo:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> >
>> https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
>> > >> WordCount
>> > >>> >> > example.
>> > >>> >> > >>>> >> It's
>> > >>> >> > >>>> >> >>> >> pretty
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with
>> some
>> > >> fixups
>> > >>> >> > for the
>> > >>> >> > >>>> >> syntax
>> > >>> >> > >>>> >> >>> and
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice that I
>> > >> added the
>> > >>> >> > >>>> >> java-examples
>> > >>> >> > >>>> >> >>> >> as a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I did
>> > this
>> > >> to
>> > >>> >> > reuse
>> > >>> >> > >>>> the
>> > >>> >> > >>>> >> >>> example
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
>> > >>> >> > >>>> >> >>> >> >>>>> >> data.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a program
>> > you
>> > >> can
>> > >>> >> do
>> > >>> >> > a
>> > >>> >> > >>>> pull
>> > >>> >> > >>>> >> >>> request
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
>> > >>> >> > >>>> >> >>> >> >>>>> >> my
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
>> > >> examples.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at
>> 12:19
>> > >> PM,
>> > >>> >> > Hermann
>> > >>> >> > >>>> >> Gábor <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees for
>> me!
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 at
>> > 11:44
>> > >> AM,
>> > >>> >> > Márton
>> > >>> >> > >>>> >> >>> Balassi <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent
>> for
>> > me
>> > >> :)
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014 at
>> > >> 11:15 AM,
>> > >>> >> > Kostas
>> > >>> >> > >>>> >> >>> Tzoumas <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
>> > WebLogAnalysis.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with
>> > Scala
>> > >>> >> > consists of
>> > >>> >> > >>>> >> going
>> > >>> >> > >>>> >> >>> >> through
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress
>> test
>> > >> both for
>> > >>> >> > me and
>> > >>> >> > >>>> >> the
>> > >>> >> > >>>> >> >>> new
>> > >>> >> > >>>> >> >>> >> API
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, 2014
>> at
>> > >> 9:09
>> > >>> >> PM,
>> > >>> >> > >>>> Vasiliki
>> > >>> >> > >>>> >> >>> >> Kalavri <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having
>> other
>> > >> people
>> > >>> >> > >>>> implement
>> > >>> >> > >>>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected
>> Components
>> > and
>> > >>> >> > Kmeans for
>> > >>> >> > >>>> >> me :)
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September
>> 2014
>> > >> 21:03,
>> > >>> >> > Fabian
>> > >>> >> > >>>> >> Hueske <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
>> > >>> >> > TriangleEnumeration and
>> > >>> >> > >>>> >> >>> PageRank.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do
>> the
>> > >> examples
>> > >>> >> > >>>> similar
>> > >>> >> > >>>> >> to
>> > >>> >> > >>>> >> >>> the
>> > >>> >> > >>>> >> >>> >> Java
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
>> > >> out-of-the-box
>> > >>> >> > without
>> > >>> >> > >>>> >> >>> parameters
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters for
>> > >> external
>> > >>> >> > data
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a
>> similar
>> > >> code
>> > >>> >> > structure
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 20:56
>> > >> GMT+02:00
>> > >>> >> > >>>> Aljoscha
>> > >>> >> > >>>> >> >>> >> Krettek <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, then
>> > >> people can
>> > >>> >> > >>>> reserve
>> > >>> >> > >>>> >> their
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep 4,
>> > 2014
>> > >> at
>> > >>> >> > 8:55 PM,
>> > >>> >> > >>>> >> Fabian
>> > >>> >> > >>>> >> >>> >> Hueske
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think
>> having
>> > >>> >> examples
>> > >>> >> > >>>> >> implemented
>> > >>> >> > >>>> >> >>> by
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable in
>> the
>> > >> past.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help
>> with
>> > two
>> > >> or
>> > >>> >> > three
>> > >>> >> > >>>> >> examples.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might be
>> > >> helpful if
>> > >>> >> > you'd
>> > >>> >> > >>>> >> port a
>> > >>> >> > >>>> >> >>> >> simple
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> one
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04
>> > 18:47
>> > >>> >> > GMT+02:00
>> > >>> >> > >>>> >> Aljoscha
>> > >>> >> > >>>> >> >>> >> Krettek
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a
>> > working
>> > >>> >> > rewrite of
>> > >>> >> > >>>> the
>> > >>> >> > >>>> >> >>> Scala
>> > >>> >> > >>>> >> >>> >> API
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> >
>> https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm hoping
>> > that
>> > >> I'll
>> > >>> >> > only
>> > >>> >> > >>>> have
>> > >>> >> > >>>> >> to
>> > >>> >> > >>>> >> >>> >> write
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> tests
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples.
>> Do
>> > you
>> > >>> >> think
>> > >>> >> > it
>> > >>> >> > >>>> makes
>> > >>> >> > >>>> >> >>> sense
>> > >>> >> > >>>> >> >>> >> to
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
>> > >>> >> > >>>> >> >>> >> >>>>> >> other
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples,
>> so
>> > that
>> > >>> >> > someone
>> > >>> >> > >>>> else
>> > >>> >> > >>>> >> uses
>> > >>> >> > >>>> >> >>> >> it and
>> > >>> >> > >>>> >> >>> >> >>>>> >> maybe
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the API?
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > >>> >> > >>>> >> >>> >> >>>>> >>
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>>
>> > >>> >> > >>>> >> >>> >> >>>
>> > >>> >> > >>>> >> >>> >>
>> > >>> >> > >>>> >> >>>
>> > >>> >> > >>>> >>
>> > >>> >> > >>>>
>> > >>> >> >
>> > >>> >>
>> >
>>
>
>

[Kostas Tzoumas-2]

Good catch, I suggest to use examples

On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi <[hidden email]>
wrote:

> Pull request issued. One minor naming concern:
>
> As of today the scala examples are located at
> the org.apache.flink.examples.scala package, while the java ones in
> the org.apache.flink.example.java. I suggest using only one convention for
> this either example or examples.
>
> Cheers,
>
> Marton
>
> On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <[hidden email]>
> wrote:
>
> > Sorry for being a bit silent after already bidding on LR. The pull
> request
> > is coming soon.
> >
> > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]> wrote:
> >
> >> I suppose that having the option between simple return type, and a
> >> collector is the easiest to understand.
> >>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <[hidden email]>:
> >>
> >> > So, should I change join and coGroup to have a simple return value, no
> >> > Option or Collection? Also what's happening with the relational
> >> > examples and the LinearRegression examples? I'd like to make a pull
> >> > request before this weekend.
> >> >
> >> > I also added a test that checks whether the Scala API has the same
> >> > methods as the Java API (ScalaAPICompletenessTest).
> >> >
> >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <
> [hidden email]>
> >> > wrote:
> >> > > Yes, there is already a Collector version, you can do:
> >> > >
> >> > > left.join(right).where("foo").equalTo("bar") {
> >> > >   (left, right, out: Collector[Page]) =>
> >> > >     if (...) out.collect(...)
> >> > > }
> >> > >
> >> > > I wasn't sure on what our Function2 variant should be. That's why I
> >> > > asked. There are some cases where you want to have the option (pun
> >> > > intended) of either returning something from a join or not. But you
> >> > > could also use a filter after the join. It's a preference thing, I
> >> > > suppose.
> >> > >
> >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]>
> >> wrote:
> >> > >> I think it seems weird that normal joins need to go through option.
> >> > >>
> >> > >> The option variant is to allow filters in the join function.
> >> Wouldn't a
> >> > >> collector variant allow you to do the same, and would be function3
> ?
> >> I
> >> > know
> >> > >> that option reads more functionally...
> >> > >>
> >> > >>
> >> > >>
> >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <
> [hidden email]
> >> >:
> >> > >>>
> >> > >>> As already mentioned this is not possible because of type erasure.
> >> We
> >> > >>> can only have one join variant that takes a Function2.
> >> > >>>
> >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <[hidden email]>
> >> > wrote:
> >> > >>> > It would be nice to have a join variant that directly returns
> the
> >> > value
> >> > >>> > rathern than an option. Why not have both (they are wrapped as
> >> > flatJoins
> >> > >>> > anyway below, right?)
> >> > >>> >
> >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
> >> [hidden email]>
> >> > >> wrote:
> >> > >>> >
> >> > >>> >> Sweet! I'm lovin' this :-)
> >> > >>> >>
> >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <
> [hidden email]
> >> >:
> >> > >>> >>
> >> > >>> >> > Also, you can use CaseClasses directly as the type for CSV
> >> input.
> >> > So
> >> > >>> >> > instead of reading it as tuples and then having a mapper that
> >> > maps to
> >> > >>> >> > your case classes you can use:
> >> > >>> >> >
> >> > >>> >> > env.readCsv[Edge](...)
> >> > >>> >> >
> >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> >> > >> [hidden email]>
> >> > >>> >> > wrote:
> >> > >>> >> > > I added support for specifying keys by name for
> CaseClasses.
> >> > Check
> >> > >> out
> >> > >>> >> > > the PageRank and TriangleEnumeration examples to see it in
> >> > action.
> >> > >>> >> > >
> >> > >>> >> > > @Kostas: I think you could use them for the TPC-H examples.
> >> > >>> >> > >
> >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> >> > >> [hidden email]
> >> > >>> >> >
> >> > >>> >> > wrote:
> >> > >>> >> > >> Yes, that would allow list comprehensions. It would be
> >> > possible to
> >> > >>> >> > >> have the Collection signature for join (and coGroup),
> i.e.:
> >> > >>> >> > >>
> >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> >> > >>> >> > >>
> >> > >>> >> > >> (T and O are the left and right input type, R is result
> >> type)
> >> > >>> >> > >>
> >> > >>> >> > >> Then you can return collections and still return an
> option,
> >> as
> >> > in:
> >> > >>> >> > >>
> >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...)
> >> Some(l)
> >> > >> else
> >> > >>> >> > None }
> >> > >>> >> > >>
> >> > >>> >> > >> Because there is an implicit conversion from Options to a
> >> > >> Collection.
> >> > >>> >> > >> This will always wrap the return value in a List with only
> >> one
> >> > >> value.
> >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not sure
> >> > whether
> >> > >> we
> >> > >>> >> > >> want the overhead of always having to use an Option even
> >> though
> >> > >> the
> >> > >>> >> > >> join always returns a value.
> >> > >>> >> > >>
> >> > >>> >> > >> What do you think?
> >> > >>> >> > >>
> >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> >> > >> [hidden email]>
> >> > >>> >> > wrote:
> >> > >>> >> > >>> Hmmm, tricky question...
> >> > >>> >> > >>> How about the Option for Join as this is a tuple-wise
> >> > operation
> >> > >> and
> >> > >>> >> the
> >> > >>> >> > >>> Collection for Cogroup which is group-wise?
> >> > >>> >> > >>> Could we in that case use list comprehensions in Cogroup
> >> > >> functions?
> >> > >>> >> > >>>
> >> > >>> >> > >>> Or is that too much mixing?
> >> > >>> >> > >>>
> >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> >> > [hidden email]
> >> > >>>:
> >> > >>> >> > >>>
> >> > >>> >> > >>>> I didn't look at the example either.
> >> > >>> >> > >>>>
> >> > >>> >> > >>>> Addings collections is easy, it's just that we can
> either
> >> > have
> >> > >>> >> > >>>> Collections or the Option, not both.
> >> > >>> >> > >>>>
> >> > >>> >> > >>>> For the coding style I followed this:
> >> > >>> >> > >>>>
> >> > >>> >> >
> >> > >>
> >> >
> >>
> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> >> > >>> >> ,
> >> > >>> >> > >>>> which itself is based on this:
> >> > http://docs.scala-lang.org/style/
> >> > >> .
> >> > >>> >> It
> >> > >>> >> > >>>> is different from the Java Code Guidelines we have in
> >> place,
> >> > >> yes.
> >> > >>> >> > >>>>
> >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> >> > >> [hidden email]
> >> > >>> >> >
> >> > >>> >> > >>>> wrote:
> >> > >>> >> > >>>> > I haven't looked at the LineRank example in detail,
> but
> >> if
> >> > you
> >> > >>> >> > think that
> >> > >>> >> > >>>> > it adds something new to the examples collection, we
> can
> >> > >> certainly
> >> > >>> >> > port
> >> > >>> >> > >>>> it
> >> > >>> >> > >>>> > also to Java.
> >> > >>> >> > >>>> > I think the Option and Collector return types are
> >> > sufficient
> >> > >> right
> >> > >>> >> > now
> >> > >>> >> > >>>> but
> >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> >> > >>> >> > >>>> >
> >> > >>> >> > >>>> > Great that the Scala primitives are working! Also
> thanks
> >> > for
> >> > >>> >> adding
> >> > >>> >> > >>>> > genSequence and adapting my examples.
> >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or
> do
> >> we
> >> > >> have a
> >> > >>> >> > >>>> different
> >> > >>> >> > >>>> > there?
> >> > >>> >> > >>>> >
> >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> >> > >> [hidden email]
> >> > >>> >> >:
> >> > >>> >> > >>>> >
> >> > >>> >> > >>>> >> What about the LineRank example? We had that in Scala
> >> but
> >> > >> never
> >> > >>> >> > had a
> >> > >>> >> > >>>> >> Java Example.
> >> > >>> >> > >>>> >>
> >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
> >> > >>> >> > [hidden email]>
> >> > >>> >> > >>>> >> wrote:
> >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just
> >> copied
> >> > the
> >> > >> Java
> >> > >>> >> > >>>> ITCase.
> >> > >>> >> > >>>> >> >
> >> > >>> >> > >>>> >> > The only examples that are missing now are
> >> > >> LinearRegression and
> >> > >>> >> > the
> >> > >>> >> > >>>> >> > relational stuff.
> >> > >>> >> > >>>> >> >
> >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
> >> > >>> >> > [hidden email]>
> >> > >>> >> > >>>> >> wrote:
> >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
> >> > >>> >> > >>>> >> >> That was a preprocessing step for the
> >> > >> TriangleEnumeration, and
> >> > >>> >> > is now
> >> > >>> >> > >>>> >> part
> >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I
> >> said
> >> > >> before,
> >> > >>> >> > I'd
> >> > >>> >> > >>>> >> prefer to
> >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> >> > >>> >> > >>>> >> >>
> >> > >>> >> > >>>> >> >> Cheers, Fabian
> >> > >>> >> > >>>> >> >>
> >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> >> > >>> >> > [hidden email]>:
> >> > >>> >> > >>>> >> >>
> >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again
> fabian.
> >> :D
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> Regarding the other stuff:
> >> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about
> >> including
> >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> >> > >> TypeInformation.
> >> > >>> >> > >>>> >> >>>  - I added generateSequence to
> >> ExecutionEnvironment.
> >> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in
> >> Array, I
> >> > >> noticed
> >> > >>> >> > it
> >> > >>> >> > >>>> while
> >> > >>> >> > >>>> >> >>> writing the tests, you probably had an older
> >> version
> >> > of
> >> > >> the
> >> > >>> >> > code.
> >> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not
> >> > possible,
> >> > >> this
> >> > >>> >> > is
> >> > >>> >> > >>>> also
> >> > >>> >> > >>>> >> >>> a restriction in the Java API.
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> What do you think about the interface of join and
> >> > >> coGroup?
> >> > >>> >> > Right
> >> > >>> >> > >>>> now,
> >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an
> Option
> >> or
> >> > the
> >> > >>> >> > lambda
> >> > >>> >> > >>>> with
> >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also
> >> have a
> >> > >> lambda
> >> > >>> >> > that
> >> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure
> this
> >> has
> >> > >> the
> >> > >>> >> > same type
> >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use
> it.
> >> > >> There is
> >> > >>> >> an
> >> > >>> >> > >>>> >> >>> implicit conversion from Option to a Collection,
> >> so I
> >> > >> could
> >> > >>> >> > change
> >> > >>> >> > >>>> it
> >> > >>> >> > >>>> >> >>> without breaking the examples we have now. What
> do
> >> you
> >> > >> think?
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> >> > >>> >> ConnectedComponents,
> >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> >> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
> >> > >>> >> > >>>> >> >>> PageRank
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> These are the examples people called dibs on:
> >> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a
> port
> >> of
> >> > >>> >> > >>>> LinearRegression
> >> > >>> >> > >>>> >> >>> Example from Java)
> >> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> >> > >>> >> > >>>> >> >>>  - The relational Stuff
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
> >> > >>> >> > [hidden email]>
> >> > >>> >> > >>>> >> wrote:
> >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> >> > >>> >> > >>>> >> >>> >
> >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
> >> Krettek <
> >> > >>> >> > >>>> >> [hidden email]>
> >> > >>> >> > >>>> >> >>> > wrote:
> >> > >>> >> > >>>> >> >>> >
> >> > >>> >> > >>>> >> >>> >> By the way, what was called
> >> BatchGradientDescent in
> >> > >> the
> >> > >>> >> > Scala
> >> > >>> >> > >>>> >> examples
> >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
> >> > LinearRegression
> >> > >>> >> > Example from
> >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples
> >> earlier.
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any
> >> > thoughts
> >> > >>> >> about
> >> > >>> >> > >>>> removing
> >> > >>> >> > >>>> >> >>> >> RelationalQuery?
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha
> >> Krettek
> >> > <
> >> > >>> >> > >>>> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >
> >> > >>> >> > >>>> >> >>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples,
> >> thanks
> >> > >>> >> Fabian.
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
> >> > >>> >> > ConnectedComponents,
> >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> > These are the examples people called dibs
> on:
> >> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> >> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> >> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
> >> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
> >> > >>> >> > >>>> >> >>> >> >  - LinearRegression
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha
> >> > Krettek <
> >> > >>> >> > >>>> >> >>> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running
> list
> >> of
> >> > >>> >> > >>>> ported/unported
> >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the java
> >> > example
> >> > >>> >> > package to
> >> > >>> >> > >>>> >> >>> examples
> >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine
> as
> >> it
> >> > is.
> >> > >>> >> Just
> >> > >>> >> > >>>> because
> >> > >>> >> > >>>> >> >>> Scala
> >> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't mean
> >> it's
> >> > >> always
> >> > >>> >> > the
> >> > >>> >> > >>>> best
> >> > >>> >> > >>>> >> >>> >> >> choice. :D
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
> >> > >>> >> > ConnectedComponents,
> >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> These are the examples people called dibs
> on:
> >> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank (Fabian)
> >> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> >> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken):
> >> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> >> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> Cheers,
> >> > >>> >> > >>>> >> >>> >> >> Aljoscha
> >> > >>> >> > >>>> >> >>> >> >>
> >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas
> >> Tzoumas
> >> > <
> >> > >>> >> > >>>> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >
> >> > >>> >> > >>>> >> >>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
> >> > >> termination
> >> > >>> >> > criterion
> >> > >>> >> > >>>> >> in the
> >> > >>> >> > >>>> >> >>> >> Java
> >> > >>> >> > >>>> >> >>> >> >>> version:
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> > >>>>
> >> > >> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> >> > >>> >> > >>>> >> >>> >> >>>
> >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
> >> criterion
> >> > in
> >> > >>> >> Scala
> >> > >>> >> > more
> >> > >>> >> > >>>> >> >>> >> functional?
> >> > >>> >> > >>>> >> >>> >> >>>
> >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name
> is
> >> > >>> >> > example.java but
> >> > >>> >> > >>>> >> >>> >> examples.scala
> >> > >>> >> > >>>> >> >>> >> >>>
> >> > >>> >> > >>>> >> >>> >> >>> Kostas
> >> > >>> >> > >>>> >> >>> >> >>>
> >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas
> >> Tzoumas
> >> > <
> >> > >>> >> > >>>> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >
> >> > >>> >> > >>>> >> >>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
> >> PiEstimation
> >> > >> (was not
> >> > >>> >> > on
> >> > >>> >> > >>>> your
> >> > >>> >> > >>>> >> >>> list).
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational
> >> stuff
> >> > I
> >> > >> can
> >> > >>> >> > take
> >> > >>> >> > >>>> those
> >> > >>> >> > >>>> >> as
> >> > >>> >> > >>>> >> >>> >> well.
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery"
> >> from
> >> > >> both
> >> > >>> >> > Scala and
> >> > >>> >> > >>>> >> Java?
> >> > >>> >> > >>>> >> >>> It
> >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3.
> >> Does
> >> > it
> >> > >> add
> >> > >>> >> > some
> >> > >>> >> > >>>> >> teaching
> >> > >>> >> > >>>> >> >>> >> value on
> >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>> Kostas
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljoscha
> >> > Krettek
> >> > >> <
> >> > >>> >> > >>>> >> >>> [hidden email]
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an
> ITCase.
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount,
> KMeans,
> >> > >>> >> > >>>> ConnectedComponents,
> >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called
> dibs
> >> on:
> >> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
> >> (Fabian)
> >> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> >> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not
> mistaken):
> >> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> >> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> >> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
> >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> >> > >>> >> > >>>> >> >>> >> >>>>>
> >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas
> >> > Tzoumas <
> >> > >>> >> > >>>> >> >>> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> >> > >>> >> > >>>> >> >>> >> >>>>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >>
> >> > >>> >> > >>>>
> >> > >>> >> >
> >> > >>> >>
> >> > >>
> >> >
> >>
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> >> > >>> >> > >>>> >> >>> >> >>>>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> >> > >>> >> > >>>> >> >>> >> >>>>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM,
> Aljoscha
> >> > >> Krettek <
> >> > >>> >> > >>>> >> >>> >> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents
> Example
> >> > from
> >> > >>> >> Vasia.
> >> > >>> >> > >>>> >> >>> >> >>>>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> >> > >>> >> > >>>> >> >>> >> >>>>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM,
> Fabian
> >> > >> Hueske <
> >> > >>> >> > >>>> >> >>> [hidden email]
> >> > >>> >> > >>>> >> >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljoscha
> >> > >> Krettek <
> >> > >>> >> > >>>> >> >>> >> [hidden email]>:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. :D
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java and
> >> > >>> >> > KMeans.scala. I'm
> >> > >>> >> > >>>> >> storing
> >> > >>> >> > >>>> >> >>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arrays
> >> and
> >> > >> then
> >> > >>> >> > >>>> converting
> >> > >>> >> > >>>> >> it to
> >> > >>> >> > >>>> >> >>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
> >> equalTo to
> >> > >> make
> >> > >>> >> it
> >> > >>> >> > >>>> >> consistent
> >> > >>> >> > >>>> >> >>> >> with the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup).
> There
> >> is
> >> > no
> >> > >> need
> >> > >>> >> > for a
> >> > >>> >> > >>>> >> >>> keyword,
> >> > >>> >> > >>>> >> >>> >> you
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> left.join(right).where(0).equalTo(1) {
> >> > >> (le,
> >> > >>> >> re)
> >> > >>> >> > =>
> >> > >>> >> > >>>> new
> >> > >>> >> > >>>> >> >>> >> MyResult(le,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> }
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM,
> >> Fabian
> >> > >> Hueske
> >> > >>> >> <
> >> > >>> >> > >>>> >> >>> >> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, I
> >> also
> >> > >> found
> >> > >>> >> an
> >> > >>> >> > >>>> >> >>> inconsistency
> >> > >>> >> > >>>> >> >>> >> with
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> Java
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> ds1.join(ds2).where(...).equalTo(...)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala this
> >> is:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > ds1.join(d2).where(...).isEqualTo(...)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed to
> >> > >> equalTo(),
> >> > >>> >> > IMO.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup?)
> >> > lacks
> >> > >> the
> >> > >>> >> > with()
> >> > >>> >> > >>>> >> method
> >> > >>> >> > >>>> >> >>> >> because
> >> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be
> >> offer
> >> > >>> >> something
> >> > >>> >> > >>>> similar
> >> > >>> >> > >>>> >> for
> >> > >>> >> > >>>> >> >>> >> Scala
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00
> Stephan
> >> > Ewen
> >> > >> <
> >> > >>> >> > >>>> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][]
> >> would
> >> > >> work
> >> > >>> >> as
> >> > >>> >> > well.
> >> > >>> >> > >>>> >> That
> >> > >>> >> > >>>> >> >>> is a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be
> >> stored
> >> > as
> >> > >> Java
> >> > >>> >> > or
> >> > >>> >> > >>>> Scala
> >> > >>> >> > >>>> >> >>> Tuples,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert
> between
> >> > the
> >> > >> two.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55
> AM,
> >> > Fabian
> >> > >>> >> > Hueske
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
> >> > problem...
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and
> >> parsing
> >> > >> them,
> >> > >>> >> but
> >> > >>> >> > >>>> using
> >> > >>> >> > >>>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> >> > >>> >> > >>>> >> >>> >> >>>>> >> won't
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
> >> > >>> >> FileInputFormat.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse the
> >> > Strings
> >> > >>> >> > >>>> manually...
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00
> >> > Aljoscha
> >> > >>> >> Krettek
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe we
> >> > should
> >> > >> just
> >> > >>> >> > change
> >> > >>> >> > >>>> >> all
> >> > >>> >> > >>>> >> >>> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> >> > >>> >> > >>>> >> >>> >> >>>>> >> input
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use CSV
> >> > input
> >> > >>> >> > formats in
> >> > >>> >> > >>>> all
> >> > >>> >> > >>>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> What
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:46
> >> AM,
> >> > >>> >> Aljoscha
> >> > >>> >> > >>>> Krettek
> >> > >>> >> > >>>> >> <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate that
> >> the
> >> > >> data
> >> > >>> >> > types are
> >> > >>> >> > >>>> >> >>> >> incompatible.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you
> >> > proposed:
> >> > >>> >> move
> >> > >>> >> > the
> >> > >>> >> > >>>> >> data to
> >> > >>> >> > >>>> >> >>> a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> >> > >>> >> > >>>> >> >>> >> >>>>> >> field
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> >> > >>> >> > getDefaultEdgeDataSet()
> >> > >>> >> > >>>> >> method in
> >> > >>> >> > >>>> >> >>> >> Scala.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
> >> > >> duplicate the
> >> > >>> >> > data
> >> > >>> >> > >>>> and
> >> > >>> >> > >>>> >> >>> make it
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> >> > >>> >> > >>>> >> >>> >> >>>>> >> for
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in the
> >> Java
> >> > and
> >> > >>> >> Scala
> >> > >>> >> > >>>> >> versions.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others think?
> >> This
> >> > >> will
> >> > >>> >> > probably
> >> > >>> >> > >>>> >> occur
> >> > >>> >> > >>>> >> >>> in
> >> > >>> >> > >>>> >> >>> >> all
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at
> >> 10:04
> >> > PM,
> >> > >>> >> > Vasiliki
> >> > >>> >> > >>>> >> Kalavri
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <
> [hidden email]
> >> >
> >> > >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
> >> Connected
> >> > >>> >> Components
> >> > >>> >> > >>>> >> example,
> >> > >>> >> > >>>> >> >>> but
> >> > >>> >> > >>>> >> >>> >> I am
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> >> > >>> >> > >>>> >> >>> >> >>>>> >> sure
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example input
> >> data
> >> > >> from
> >> > >>> >> > >>>> >> java-examples.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> >> > ConnectedComponentsData
> >> > >>> >> class,
> >> > >>> >> > the
> >> > >>> >> > >>>> >> vertices
> >> > >>> >> > >>>> >> >>> >> and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> >> > >>> >> > >>>> >> >>> >> >>>>> >> data
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the methods
> >> > >>> >> > >>>> >> getDefaultVertexDataSet()
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
> >> getDefaultEdgeDataSet(),
> >> > >> which
> >> > >>> >> > take
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> >> > >>> >> > >>>> >> >>> as
> >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide
> >> public
> >> > >> static
> >> > >>> >> > fields
> >> > >>> >> > >>>> >> (like
> >> > >>> >> > >>>> >> >>> in
> >> > >>> >> > >>>> >> >>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
> >> introduces a
> >> > >>> >> > conversion
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> >> > >>> >> > >>>> >> >>> Scala
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
> >> scala.Long
> >> > >> and I
> >> > >>> >> > guess
> >> > >>> >> > >>>> this
> >> > >>> >> > >>>> >> is
> >> > >>> >> > >>>> >> >>> an
> >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
> >> course, to
> >> > >> copy
> >> > >>> >> the
> >> > >>> >> > >>>> example
> >> > >>> >> > >>>> >> >>> data
> >> > >>> >> > >>>> >> >>> >> in
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something
> >> here?
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014
> 15:52,
> >> > >> Aljoscha
> >> > >>> >> > >>>> Krettek <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my
> >> repo:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> >
> >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a working
> >> > >> WordCount
> >> > >>> >> > example.
> >> > >>> >> > >>>> >> It's
> >> > >>> >> > >>>> >> >>> >> pretty
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with
> >> some
> >> > >> fixups
> >> > >>> >> > for the
> >> > >>> >> > >>>> >> syntax
> >> > >>> >> > >>>> >> >>> and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice
> that I
> >> > >> added the
> >> > >>> >> > >>>> >> java-examples
> >> > >>> >> > >>>> >> >>> >> as a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I
> did
> >> > this
> >> > >> to
> >> > >>> >> > reuse
> >> > >>> >> > >>>> the
> >> > >>> >> > >>>> >> >>> example
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> >> > >>> >> > >>>> >> >>> >> >>>>> >> data.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a
> program
> >> > you
> >> > >> can
> >> > >>> >> do
> >> > >>> >> > a
> >> > >>> >> > >>>> pull
> >> > >>> >> > >>>> >> >>> request
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> >> > >>> >> > >>>> >> >>> >> >>>>> >> my
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect the
> >> > >> examples.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 at
> >> 12:19
> >> > >> PM,
> >> > >>> >> > Hermann
> >> > >>> >> > >>>> >> Gábor <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees
> for
> >> me!
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014
> at
> >> > 11:44
> >> > >> AM,
> >> > >>> >> > Márton
> >> > >>> >> > >>>> >> >>> Balassi <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> [hidden email]
> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDescent
> >> for
> >> > me
> >> > >> :)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 2014
> at
> >> > >> 11:15 AM,
> >> > >>> >> > Kostas
> >> > >>> >> > >>>> >> >>> Tzoumas <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
> >> > WebLogAnalysis.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience with
> >> > Scala
> >> > >>> >> > consists of
> >> > >>> >> > >>>> >> going
> >> > >>> >> > >>>> >> >>> >> through
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress
> >> test
> >> > >> both for
> >> > >>> >> > me and
> >> > >>> >> > >>>> >> the
> >> > >>> >> > >>>> >> >>> new
> >> > >>> >> > >>>> >> >>> >> API
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4,
> 2014
> >> at
> >> > >> 9:09
> >> > >>> >> PM,
> >> > >>> >> > >>>> Vasiliki
> >> > >>> >> > >>>> >> >>> >> Kalavri <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having
> >> other
> >> > >> people
> >> > >>> >> > >>>> implement
> >> > >>> >> > >>>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected
> >> Components
> >> > and
> >> > >>> >> > Kmeans for
> >> > >>> >> > >>>> >> me :)
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September
> >> 2014
> >> > >> 21:03,
> >> > >>> >> > Fabian
> >> > >>> >> > >>>> >> Hueske <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> >> > >>> >> > TriangleEnumeration and
> >> > >>> >> > >>>> >> >>> PageRank.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also do
> >> the
> >> > >> examples
> >> > >>> >> > >>>> similar
> >> > >>> >> > >>>> >> to
> >> > >>> >> > >>>> >> >>> the
> >> > >>> >> > >>>> >> >>> >> Java
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> >> > >> out-of-the-box
> >> > >>> >> > without
> >> > >>> >> > >>>> >> >>> parameters
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters
> for
> >> > >> external
> >> > >>> >> > data
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a
> >> similar
> >> > >> code
> >> > >>> >> > structure
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04
> 20:56
> >> > >> GMT+02:00
> >> > >>> >> > >>>> Aljoscha
> >> > >>> >> > >>>> >> >>> >> Krettek <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do,
> then
> >> > >> people can
> >> > >>> >> > >>>> reserve
> >> > >>> >> > >>>> >> their
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Sep
> 4,
> >> > 2014
> >> > >> at
> >> > >>> >> > 8:55 PM,
> >> > >>> >> > >>>> >> Fabian
> >> > >>> >> > >>>> >> >>> >> Hueske
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think
> >> having
> >> > >>> >> examples
> >> > >>> >> > >>>> >> implemented
> >> > >>> >> > >>>> >> >>> by
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > different
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable
> in
> >> the
> >> > >> past.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help
> >> with
> >> > two
> >> > >> or
> >> > >>> >> > three
> >> > >>> >> > >>>> >> examples.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might
> be
> >> > >> helpful if
> >> > >>> >> > you'd
> >> > >>> >> > >>>> >> port a
> >> > >>> >> > >>>> >> >>> >> simple
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> one
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-04
> >> > 18:47
> >> > >>> >> > GMT+02:00
> >> > >>> >> > >>>> >> Aljoscha
> >> > >>> >> > >>>> >> >>> >> Krettek
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have a
> >> > working
> >> > >>> >> > rewrite of
> >> > >>> >> > >>>> the
> >> > >>> >> > >>>> >> >>> Scala
> >> > >>> >> > >>>> >> >>> >> API
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> >
> >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm
> hoping
> >> > that
> >> > >> I'll
> >> > >>> >> > only
> >> > >>> >> > >>>> have
> >> > >>> >> > >>>> >> to
> >> > >>> >> > >>>> >> >>> >> write
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> tests
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples.
> >> Do
> >> > you
> >> > >>> >> think
> >> > >>> >> > it
> >> > >>> >> > >>>> makes
> >> > >>> >> > >>>> >> >>> sense
> >> > >>> >> > >>>> >> >>> >> to
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> >> > >>> >> > >>>> >> >>> >> >>>>> >> other
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> examples,
> >> so
> >> > that
> >> > >>> >> > someone
> >> > >>> >> > >>>> else
> >> > >>> >> > >>>> >> uses
> >> > >>> >> > >>>> >> >>> >> it and
> >> > >>> >> > >>>> >> >>> >> >>>>> >> maybe
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the
> API?
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljoscha
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> >> > >>> >> > >>>> >> >>> >> >>>>> >>
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>>
> >> > >>> >> > >>>> >> >>> >> >>>
> >> > >>> >> > >>>> >> >>> >>
> >> > >>> >> > >>>> >> >>>
> >> > >>> >> > >>>> >>
> >> > >>> >> > >>>>
> >> > >>> >> >
> >> > >>> >>
> >> >
> >>
> >
> >
>

[Márton Balassi]

Answer posted to "Example packages naming convention" thread as the issue
diverged from this topic.

On Sun, Sep 14, 2014 at 11:14 AM, Kostas Tzoumas <[hidden email]>
wrote:

> Good catch, I suggest to use examples
>
> On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi <[hidden email]>
> wrote:
>
> > Pull request issued. One minor naming concern:
> >
> > As of today the scala examples are located at
> > the org.apache.flink.examples.scala package, while the java ones in
> > the org.apache.flink.example.java. I suggest using only one convention
> for
> > this either example or examples.
> >
> > Cheers,
> >
> > Marton
> >
> > On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <
> [hidden email]>
> > wrote:
> >
> > > Sorry for being a bit silent after already bidding on LR. The pull
> > request
> > > is coming soon.
> > >
> > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]>
> wrote:
> > >
> > >> I suppose that having the option between simple return type, and a
> > >> collector is the easiest to understand.
> > >>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <[hidden email]
> >:
> > >>
> > >> > So, should I change join and coGroup to have a simple return value,
> no
> > >> > Option or Collection? Also what's happening with the relational
> > >> > examples and the LinearRegression examples? I'd like to make a pull
> > >> > request before this weekend.
> > >> >
> > >> > I also added a test that checks whether the Scala API has the same
> > >> > methods as the Java API (ScalaAPICompletenessTest).
> > >> >
> > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <
> > [hidden email]>
> > >> > wrote:
> > >> > > Yes, there is already a Collector version, you can do:
> > >> > >
> > >> > > left.join(right).where("foo").equalTo("bar") {
> > >> > >   (left, right, out: Collector[Page]) =>
> > >> > >     if (...) out.collect(...)
> > >> > > }
> > >> > >
> > >> > > I wasn't sure on what our Function2 variant should be. That's why
> I
> > >> > > asked. There are some cases where you want to have the option (pun
> > >> > > intended) of either returning something from a join or not. But
> you
> > >> > > could also use a filter after the join. It's a preference thing, I
> > >> > > suppose.
> > >> > >
> > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]>
> > >> wrote:
> > >> > >> I think it seems weird that normal joins need to go through
> option.
> > >> > >>
> > >> > >> The option variant is to allow filters in the join function.
> > >> Wouldn't a
> > >> > >> collector variant allow you to do the same, and would be
> function3
> > ?
> > >> I
> > >> > know
> > >> > >> that option reads more functionally...
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <
> > [hidden email]
> > >> >:
> > >> > >>>
> > >> > >>> As already mentioned this is not possible because of type
> erasure.
> > >> We
> > >> > >>> can only have one join variant that takes a Function2.
> > >> > >>>
> > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <
> [hidden email]>
> > >> > wrote:
> > >> > >>> > It would be nice to have a join variant that directly returns
> > the
> > >> > value
> > >> > >>> > rathern than an option. Why not have both (they are wrapped as
> > >> > flatJoins
> > >> > >>> > anyway below, right?)
> > >> > >>> >
> > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
> > >> [hidden email]>
> > >> > >> wrote:
> > >> > >>> >
> > >> > >>> >> Sweet! I'm lovin' this :-)
> > >> > >>> >>
> > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <
> > [hidden email]
> > >> >:
> > >> > >>> >>
> > >> > >>> >> > Also, you can use CaseClasses directly as the type for CSV
> > >> input.
> > >> > So
> > >> > >>> >> > instead of reading it as tuples and then having a mapper
> that
> > >> > maps to
> > >> > >>> >> > your case classes you can use:
> > >> > >>> >> >
> > >> > >>> >> > env.readCsv[Edge](...)
> > >> > >>> >> >
> > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> > >> > >> [hidden email]>
> > >> > >>> >> > wrote:
> > >> > >>> >> > > I added support for specifying keys by name for
> > CaseClasses.
> > >> > Check
> > >> > >> out
> > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see it
> in
> > >> > action.
> > >> > >>> >> > >
> > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H
> examples.
> > >> > >>> >> > >
> > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> > >> > >> [hidden email]
> > >> > >>> >> >
> > >> > >>> >> > wrote:
> > >> > >>> >> > >> Yes, that would allow list comprehensions. It would be
> > >> > possible to
> > >> > >>> >> > >> have the Collection signature for join (and coGroup),
> > i.e.:
> > >> > >>> >> > >>
> > >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> > >> > >>> >> > >>
> > >> > >>> >> > >> (T and O are the left and right input type, R is result
> > >> type)
> > >> > >>> >> > >>
> > >> > >>> >> > >> Then you can return collections and still return an
> > option,
> > >> as
> > >> > in:
> > >> > >>> >> > >>
> > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...)
> > >> Some(l)
> > >> > >> else
> > >> > >>> >> > None }
> > >> > >>> >> > >>
> > >> > >>> >> > >> Because there is an implicit conversion from Options to
> a
> > >> > >> Collection.
> > >> > >>> >> > >> This will always wrap the return value in a List with
> only
> > >> one
> > >> > >> value.
> > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not
> sure
> > >> > whether
> > >> > >> we
> > >> > >>> >> > >> want the overhead of always having to use an Option even
> > >> though
> > >> > >> the
> > >> > >>> >> > >> join always returns a value.
> > >> > >>> >> > >>
> > >> > >>> >> > >> What do you think?
> > >> > >>> >> > >>
> > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> > >> > >> [hidden email]>
> > >> > >>> >> > wrote:
> > >> > >>> >> > >>> Hmmm, tricky question...
> > >> > >>> >> > >>> How about the Option for Join as this is a tuple-wise
> > >> > operation
> > >> > >> and
> > >> > >>> >> the
> > >> > >>> >> > >>> Collection for Cogroup which is group-wise?
> > >> > >>> >> > >>> Could we in that case use list comprehensions in
> Cogroup
> > >> > >> functions?
> > >> > >>> >> > >>>
> > >> > >>> >> > >>> Or is that too much mixing?
> > >> > >>> >> > >>>
> > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> > >> > [hidden email]
> > >> > >>>:
> > >> > >>> >> > >>>
> > >> > >>> >> > >>>> I didn't look at the example either.
> > >> > >>> >> > >>>>
> > >> > >>> >> > >>>> Addings collections is easy, it's just that we can
> > either
> > >> > have
> > >> > >>> >> > >>>> Collections or the Option, not both.
> > >> > >>> >> > >>>>
> > >> > >>> >> > >>>> For the coding style I followed this:
> > >> > >>> >> > >>>>
> > >> > >>> >> >
> > >> > >>
> > >> >
> > >>
> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> > >> > >>> >> ,
> > >> > >>> >> > >>>> which itself is based on this:
> > >> > http://docs.scala-lang.org/style/
> > >> > >> .
> > >> > >>> >> It
> > >> > >>> >> > >>>> is different from the Java Code Guidelines we have in
> > >> place,
> > >> > >> yes.
> > >> > >>> >> > >>>>
> > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> > >> > >> [hidden email]
> > >> > >>> >> >
> > >> > >>> >> > >>>> wrote:
> > >> > >>> >> > >>>> > I haven't looked at the LineRank example in detail,
> > but
> > >> if
> > >> > you
> > >> > >>> >> > think that
> > >> > >>> >> > >>>> > it adds something new to the examples collection, we
> > can
> > >> > >> certainly
> > >> > >>> >> > port
> > >> > >>> >> > >>>> it
> > >> > >>> >> > >>>> > also to Java.
> > >> > >>> >> > >>>> > I think the Option and Collector return types are
> > >> > sufficient
> > >> > >> right
> > >> > >>> >> > now
> > >> > >>> >> > >>>> but
> > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> > >> > >>> >> > >>>> >
> > >> > >>> >> > >>>> > Great that the Scala primitives are working! Also
> > thanks
> > >> > for
> > >> > >>> >> adding
> > >> > >>> >> > >>>> > genSequence and adapting my examples.
> > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or
> > do
> > >> we
> > >> > >> have a
> > >> > >>> >> > >>>> different
> > >> > >>> >> > >>>> > there?
> > >> > >>> >> > >>>> >
> > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> > >> > >> [hidden email]
> > >> > >>> >> >:
> > >> > >>> >> > >>>> >
> > >> > >>> >> > >>>> >> What about the LineRank example? We had that in
> Scala
> > >> but
> > >> > >> never
> > >> > >>> >> > had a
> > >> > >>> >> > >>>> >> Java Example.
> > >> > >>> >> > >>>> >>
> > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek <
> > >> > >>> >> > [hidden email]>
> > >> > >>> >> > >>>> >> wrote:
> > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just
> > >> copied
> > >> > the
> > >> > >> Java
> > >> > >>> >> > >>>> ITCase.
> > >> > >>> >> > >>>> >> >
> > >> > >>> >> > >>>> >> > The only examples that are missing now are
> > >> > >> LinearRegression and
> > >> > >>> >> > the
> > >> > >>> >> > >>>> >> > relational stuff.
> > >> > >>> >> > >>>> >> >
> > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske <
> > >> > >>> >> > [hidden email]>
> > >> > >>> >> > >>>> >> wrote:
> > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example.
> > >> > >>> >> > >>>> >> >> That was a preprocessing step for the
> > >> > >> TriangleEnumeration, and
> > >> > >>> >> > is now
> > >> > >>> >> > >>>> >> part
> > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I
> > >> said
> > >> > >> before,
> > >> > >>> >> > I'd
> > >> > >>> >> > >>>> >> prefer to
> > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> > >> > >>> >> > >>>> >> >>
> > >> > >>> >> > >>>> >> >> Cheers, Fabian
> > >> > >>> >> > >>>> >> >>
> > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> > >> > >>> >> > [hidden email]>:
> > >> > >>> >> > >>>> >> >>
> > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again
> > fabian.
> > >> :D
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> Regarding the other stuff:
> > >> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about
> > >> including
> > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> > >> > >> TypeInformation.
> > >> > >>> >> > >>>> >> >>>  - I added generateSequence to
> > >> ExecutionEnvironment.
> > >> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in
> > >> Array, I
> > >> > >> noticed
> > >> > >>> >> > it
> > >> > >>> >> > >>>> while
> > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an older
> > >> version
> > >> > of
> > >> > >> the
> > >> > >>> >> > code.
> > >> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is not
> > >> > possible,
> > >> > >> this
> > >> > >>> >> > is
> > >> > >>> >> > >>>> also
> > >> > >>> >> > >>>> >> >>> a restriction in the Java API.
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> What do you think about the interface of join
> and
> > >> > >> coGroup?
> > >> > >>> >> > Right
> > >> > >>> >> > >>>> now,
> > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an
> > Option
> > >> or
> > >> > the
> > >> > >>> >> > lambda
> > >> > >>> >> > >>>> with
> > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also
> > >> have a
> > >> > >> lambda
> > >> > >>> >> > that
> > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure
> > this
> > >> has
> > >> > >> the
> > >> > >>> >> > same type
> > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use
> > it.
> > >> > >> There is
> > >> > >>> >> an
> > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a
> Collection,
> > >> so I
> > >> > >> could
> > >> > >>> >> > change
> > >> > >>> >> > >>>> it
> > >> > >>> >> > >>>> >> >>> without breaking the examples we have now. What
> > do
> > >> you
> > >> > >> think?
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> > >> > >>> >> ConnectedComponents,
> > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
> > >> > >>> >> > >>>> >> >>> PageRank
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> These are the examples people called dibs on:
> > >> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a
> > port
> > >> of
> > >> > >>> >> > >>>> LinearRegression
> > >> > >>> >> > >>>> >> >>> Example from Java)
> > >> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> > >> > >>> >> > >>>> >> >>>  - The relational Stuff
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen <
> > >> > >>> >> > [hidden email]>
> > >> > >>> >> > >>>> >> wrote:
> > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> > >> > >>> >> > >>>> >> >>> >
> > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
> > >> Krettek <
> > >> > >>> >> > >>>> >> [hidden email]>
> > >> > >>> >> > >>>> >> >>> > wrote:
> > >> > >>> >> > >>>> >> >>> >
> > >> > >>> >> > >>>> >> >>> >> By the way, what was called
> > >> BatchGradientDescent in
> > >> > >> the
> > >> > >>> >> > Scala
> > >> > >>> >> > >>>> >> examples
> > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
> > >> > LinearRegression
> > >> > >>> >> > Example from
> > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples
> > >> earlier.
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any
> > >> > thoughts
> > >> > >>> >> about
> > >> > >>> >> > >>>> removing
> > >> > >>> >> > >>>> >> >>> >> RelationalQuery?
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha
> > >> Krettek
> > >> > <
> > >> > >>> >> > >>>> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >
> > >> > >>> >> > >>>> >> >>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples,
> > >> thanks
> > >> > >>> >> Fabian.
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans,
> > >> > >>> >> > ConnectedComponents,
> > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> > These are the examples people called dibs
> > on:
> > >> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> > >> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> > >> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken):
> > >> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
> > >> > >>> >> > >>>> >> >>> >> >  - LinearRegression
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha
> > >> > Krettek <
> > >> > >>> >> > >>>> >> >>> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running
> > list
> > >> of
> > >> > >>> >> > >>>> ported/unported
> > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the
> java
> > >> > example
> > >> > >>> >> > package to
> > >> > >>> >> > >>>> >> >>> examples
> > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine
> > as
> > >> it
> > >> > is.
> > >> > >>> >> Just
> > >> > >>> >> > >>>> because
> > >> > >>> >> > >>>> >> >>> Scala
> > >> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't
> mean
> > >> it's
> > >> > >> always
> > >> > >>> >> > the
> > >> > >>> >> > >>>> best
> > >> > >>> >> > >>>> >> >>> >> >> choice. :D
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans,
> > >> > >>> >> > ConnectedComponents,
> > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called dibs
> > on:
> > >> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank
> (Fabian)
> > >> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> > >> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not
> mistaken):
> > >> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> > >> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> Cheers,
> > >> > >>> >> > >>>> >> >>> >> >> Aljoscha
> > >> > >>> >> > >>>> >> >>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas
> > >> Tzoumas
> > >> > <
> > >> > >>> >> > >>>> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >
> > >> > >>> >> > >>>> >> >>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a
> > >> > >> termination
> > >> > >>> >> > criterion
> > >> > >>> >> > >>>> >> in the
> > >> > >>> >> > >>>> >> >>> >> Java
> > >> > >>> >> > >>>> >> >>> >> >>> version:
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> > >>>>
> > >> > >>
> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> > >> > >>> >> > >>>> >> >>> >> >>>
> > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
> > >> criterion
> > >> > in
> > >> > >>> >> Scala
> > >> > >>> >> > more
> > >> > >>> >> > >>>> >> >>> >> functional?
> > >> > >>> >> > >>>> >> >>> >> >>>
> > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name
> > is
> > >> > >>> >> > example.java but
> > >> > >>> >> > >>>> >> >>> >> examples.scala
> > >> > >>> >> > >>>> >> >>> >> >>>
> > >> > >>> >> > >>>> >> >>> >> >>> Kostas
> > >> > >>> >> > >>>> >> >>> >> >>>
> > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas
> > >> Tzoumas
> > >> > <
> > >> > >>> >> > >>>> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >
> > >> > >>> >> > >>>> >> >>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
> > >> PiEstimation
> > >> > >> (was not
> > >> > >>> >> > on
> > >> > >>> >> > >>>> your
> > >> > >>> >> > >>>> >> >>> list).
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational
> > >> stuff
> > >> > I
> > >> > >> can
> > >> > >>> >> > take
> > >> > >>> >> > >>>> those
> > >> > >>> >> > >>>> >> as
> > >> > >>> >> > >>>> >> >>> >> well.
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the
> "RelationalQuery"
> > >> from
> > >> > >> both
> > >> > >>> >> > Scala and
> > >> > >>> >> > >>>> >> Java?
> > >> > >>> >> > >>>> >> >>> It
> > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H
> Q3.
> > >> Does
> > >> > it
> > >> > >> add
> > >> > >>> >> > some
> > >> > >>> >> > >>>> >> teaching
> > >> > >>> >> > >>>> >> >>> >> value on
> > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>> Kostas
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM,
> Aljoscha
> > >> > Krettek
> > >> > >> <
> > >> > >>> >> > >>>> >> >>> [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an
> > ITCase.
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount,
> > KMeans,
> > >> > >>> >> > >>>> ConnectedComponents,
> > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called
> > dibs
> > >> on:
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
> > >> (Fabian)
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not
> > mistaken):
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> > >> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
> > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> > >> > >>> >> > >>>> >> >>> >> >>>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas
> > >> > Tzoumas <
> > >> > >>> >> > >>>> >> >>> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >>
> > >> > >>> >> > >>>>
> > >> > >>> >> >
> > >> > >>> >>
> > >> > >>
> > >> >
> > >>
> >
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM,
> > Aljoscha
> > >> > >> Krettek <
> > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents
> > Example
> > >> > from
> > >> > >>> >> Vasia.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM,
> > Fabian
> > >> > >> Hueske <
> > >> > >>> >> > >>>> >> >>> [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00
> Aljoscha
> > >> > >> Krettek <
> > >> > >>> >> > >>>> >> >>> >> [hidden email]>:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision.
> :D
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java
> and
> > >> > >>> >> > KMeans.scala. I'm
> > >> > >>> >> > >>>> >> storing
> > >> > >>> >> > >>>> >> >>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object
> arrays
> > >> and
> > >> > >> then
> > >> > >>> >> > >>>> converting
> > >> > >>> >> > >>>> >> it to
> > >> > >>> >> > >>>> >> >>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
> > >> equalTo to
> > >> > >> make
> > >> > >>> >> it
> > >> > >>> >> > >>>> >> consistent
> > >> > >>> >> > >>>> >> >>> >> with the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup).
> > There
> > >> is
> > >> > no
> > >> > >> need
> > >> > >>> >> > for a
> > >> > >>> >> > >>>> >> >>> keyword,
> > >> > >>> >> > >>>> >> >>> >> you
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> left.join(right).where(0).equalTo(1) {
> > >> > >> (le,
> > >> > >>> >> re)
> > >> > >>> >> > =>
> > >> > >>> >> > >>>> new
> > >> > >>> >> > >>>> >> >>> >> MyResult(le,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> }
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM,
> > >> Fabian
> > >> > >> Hueske
> > >> > >>> >> <
> > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue,
> I
> > >> also
> > >> > >> found
> > >> > >>> >> an
> > >> > >>> >> > >>>> >> >>> inconsistency
> > >> > >>> >> > >>>> >> >>> >> with
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> ds1.join(ds2).where(...).equalTo(...)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala
> this
> > >> is:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > ds1.join(d2).where(...).isEqualTo(...)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed
> to
> > >> > >> equalTo(),
> > >> > >>> >> > IMO.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and
> coGroup?)
> > >> > lacks
> > >> > >> the
> > >> > >>> >> > with()
> > >> > >>> >> > >>>> >> method
> > >> > >>> >> > >>>> >> >>> >> because
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be
> > >> offer
> > >> > >>> >> something
> > >> > >>> >> > >>>> similar
> > >> > >>> >> > >>>> >> for
> > >> > >>> >> > >>>> >> >>> >> Scala
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)?
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00
> > Stephan
> > >> > Ewen
> > >> > >> <
> > >> > >>> >> > >>>> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings,
> Object[][]
> > >> would
> > >> > >> work
> > >> > >>> >> as
> > >> > >>> >> > well.
> > >> > >>> >> > >>>> >> That
> > >> > >>> >> > >>>> >> >>> is a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be
> > >> stored
> > >> > as
> > >> > >> Java
> > >> > >>> >> > or
> > >> > >>> >> > >>>> Scala
> > >> > >>> >> > >>>> >> >>> Tuples,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert
> > between
> > >> > the
> > >> > >> two.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55
> > AM,
> > >> > Fabian
> > >> > >>> >> > Hueske
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
> > >> > problem...
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and
> > >> parsing
> > >> > >> them,
> > >> > >>> >> but
> > >> > >>> >> > >>>> using
> > >> > >>> >> > >>>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a
> > >> > >>> >> FileInputFormat.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse
> the
> > >> > Strings
> > >> > >>> >> > >>>> manually...
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00
> > >> > Aljoscha
> > >> > >>> >> Krettek
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe
> we
> > >> > should
> > >> > >> just
> > >> > >>> >> > change
> > >> > >>> >> > >>>> >> all
> > >> > >>> >> > >>>> >> >>> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> input
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use
> CSV
> > >> > input
> > >> > >>> >> > formats in
> > >> > >>> >> > >>>> all
> > >> > >>> >> > >>>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> What
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at
> 7:46
> > >> AM,
> > >> > >>> >> Aljoscha
> > >> > >>> >> > >>>> Krettek
> > >> > >>> >> > >>>> >> <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate
> that
> > >> the
> > >> > >> data
> > >> > >>> >> > types are
> > >> > >>> >> > >>>> >> >>> >> incompatible.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you
> > >> > proposed:
> > >> > >>> >> move
> > >> > >>> >> > the
> > >> > >>> >> > >>>> >> data to
> > >> > >>> >> > >>>> >> >>> a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> field
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> > >> > >>> >> > getDefaultEdgeDataSet()
> > >> > >>> >> > >>>> >> method in
> > >> > >>> >> > >>>> >> >>> >> Scala.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would
> > >> > >> duplicate the
> > >> > >>> >> > data
> > >> > >>> >> > >>>> and
> > >> > >>> >> > >>>> >> >>> make it
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> for
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in
> the
> > >> Java
> > >> > and
> > >> > >>> >> Scala
> > >> > >>> >> > >>>> >> versions.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others
> think?
> > >> This
> > >> > >> will
> > >> > >>> >> > probably
> > >> > >>> >> > >>>> >> occur
> > >> > >>> >> > >>>> >> >>> in
> > >> > >>> >> > >>>> >> >>> >> all
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at
> > >> 10:04
> > >> > PM,
> > >> > >>> >> > Vasiliki
> > >> > >>> >> > >>>> >> Kalavri
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <
> > [hidden email]
> > >> >
> > >> > >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
> > >> Connected
> > >> > >>> >> Components
> > >> > >>> >> > >>>> >> example,
> > >> > >>> >> > >>>> >> >>> but
> > >> > >>> >> > >>>> >> >>> >> I am
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example
> input
> > >> data
> > >> > >> from
> > >> > >>> >> > >>>> >> java-examples.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> > >> > ConnectedComponentsData
> > >> > >>> >> class,
> > >> > >>> >> > the
> > >> > >>> >> > >>>> >> vertices
> > >> > >>> >> > >>>> >> >>> >> and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> data
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the
> methods
> > >> > >>> >> > >>>> >> getDefaultVertexDataSet()
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
> > >> getDefaultEdgeDataSet(),
> > >> > >> which
> > >> > >>> >> > take
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> > >> > >>> >> > >>>> >> >>> as
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide
> > >> public
> > >> > >> static
> > >> > >>> >> > fields
> > >> > >>> >> > >>>> >> (like
> > >> > >>> >> > >>>> >> >>> in
> > >> > >>> >> > >>>> >> >>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
> > >> introduces a
> > >> > >>> >> > conversion
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> > >> > >>> >> > >>>> >> >>> Scala
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
> > >> scala.Long
> > >> > >> and I
> > >> > >>> >> > guess
> > >> > >>> >> > >>>> this
> > >> > >>> >> > >>>> >> is
> > >> > >>> >> > >>>> >> >>> an
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
> > >> course, to
> > >> > >> copy
> > >> > >>> >> the
> > >> > >>> >> > >>>> example
> > >> > >>> >> > >>>> >> >>> data
> > >> > >>> >> > >>>> >> >>> >> in
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something
> > >> here?
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014
> > 15:52,
> > >> > >> Aljoscha
> > >> > >>> >> > >>>> Krettek <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my
> > >> repo:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> >
> > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a
> working
> > >> > >> WordCount
> > >> > >>> >> > example.
> > >> > >>> >> > >>>> >> It's
> > >> > >>> >> > >>>> >> >>> >> pretty
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with
> > >> some
> > >> > >> fixups
> > >> > >>> >> > for the
> > >> > >>> >> > >>>> >> syntax
> > >> > >>> >> > >>>> >> >>> and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice
> > that I
> > >> > >> added the
> > >> > >>> >> > >>>> >> java-examples
> > >> > >>> >> > >>>> >> >>> >> as a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I
> > did
> > >> > this
> > >> > >> to
> > >> > >>> >> > reuse
> > >> > >>> >> > >>>> the
> > >> > >>> >> > >>>> >> >>> example
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> data.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a
> > program
> > >> > you
> > >> > >> can
> > >> > >>> >> do
> > >> > >>> >> > a
> > >> > >>> >> > >>>> pull
> > >> > >>> >> > >>>> >> >>> request
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> my
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect
> the
> > >> > >> examples.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014
> at
> > >> 12:19
> > >> > >> PM,
> > >> > >>> >> > Hermann
> > >> > >>> >> > >>>> >> Gábor <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees
> > for
> > >> me!
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014
> > at
> > >> > 11:44
> > >> > >> AM,
> > >> > >>> >> > Márton
> > >> > >>> >> > >>>> >> >>> Balassi <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > [hidden email]
> > >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> BatchGradientDescent
> > >> for
> > >> > me
> > >> > >> :)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5,
> 2014
> > at
> > >> > >> 11:15 AM,
> > >> > >>> >> > Kostas
> > >> > >>> >> > >>>> >> >>> Tzoumas <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
> > >> > WebLogAnalysis.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience
> with
> > >> > Scala
> > >> > >>> >> > consists of
> > >> > >>> >> > >>>> >> going
> > >> > >>> >> > >>>> >> >>> >> through
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress
> > >> test
> > >> > >> both for
> > >> > >>> >> > me and
> > >> > >>> >> > >>>> >> the
> > >> > >>> >> > >>>> >> >>> new
> > >> > >>> >> > >>>> >> >>> >> API
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4,
> > 2014
> > >> at
> > >> > >> 9:09
> > >> > >>> >> PM,
> > >> > >>> >> > >>>> Vasiliki
> > >> > >>> >> > >>>> >> >>> >> Kalavri <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having
> > >> other
> > >> > >> people
> > >> > >>> >> > >>>> implement
> > >> > >>> >> > >>>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected
> > >> Components
> > >> > and
> > >> > >>> >> > Kmeans for
> > >> > >>> >> > >>>> >> me :)
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September
> > >> 2014
> > >> > >> 21:03,
> > >> > >>> >> > Fabian
> > >> > >>> >> > >>>> >> Hueske <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> > >> > >>> >> > TriangleEnumeration and
> > >> > >>> >> > >>>> >> >>> PageRank.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also
> do
> > >> the
> > >> > >> examples
> > >> > >>> >> > >>>> similar
> > >> > >>> >> > >>>> >> to
> > >> > >>> >> > >>>> >> >>> the
> > >> > >>> >> > >>>> >> >>> >> Java
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> > >> > >> out-of-the-box
> > >> > >>> >> > without
> > >> > >>> >> > >>>> >> >>> parameters
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters
> > for
> > >> > >> external
> > >> > >>> >> > data
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a
> > >> similar
> > >> > >> code
> > >> > >>> >> > structure
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04
> > 20:56
> > >> > >> GMT+02:00
> > >> > >>> >> > >>>> Aljoscha
> > >> > >>> >> > >>>> >> >>> >> Krettek <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do,
> > then
> > >> > >> people can
> > >> > >>> >> > >>>> reserve
> > >> > >>> >> > >>>> >> their
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu,
> Sep
> > 4,
> > >> > 2014
> > >> > >> at
> > >> > >>> >> > 8:55 PM,
> > >> > >>> >> > >>>> >> Fabian
> > >> > >>> >> > >>>> >> >>> >> Hueske
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think
> > >> having
> > >> > >>> >> examples
> > >> > >>> >> > >>>> >> implemented
> > >> > >>> >> > >>>> >> >>> by
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> different
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable
> > in
> > >> the
> > >> > >> past.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help
> > >> with
> > >> > two
> > >> > >> or
> > >> > >>> >> > three
> > >> > >>> >> > >>>> >> examples.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might
> > be
> > >> > >> helpful if
> > >> > >>> >> > you'd
> > >> > >>> >> > >>>> >> port a
> > >> > >>> >> > >>>> >> >>> >> simple
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> one
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> 2014-09-04
> > >> > 18:47
> > >> > >>> >> > GMT+02:00
> > >> > >>> >> > >>>> >> Aljoscha
> > >> > >>> >> > >>>> >> >>> >> Krettek
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have
> a
> > >> > working
> > >> > >>> >> > rewrite of
> > >> > >>> >> > >>>> the
> > >> > >>> >> > >>>> >> >>> Scala
> > >> > >>> >> > >>>> >> >>> >> API
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> >
> > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm
> > hoping
> > >> > that
> > >> > >> I'll
> > >> > >>> >> > only
> > >> > >>> >> > >>>> have
> > >> > >>> >> > >>>> >> to
> > >> > >>> >> > >>>> >> >>> >> write
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> tests
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> examples.
> > >> Do
> > >> > you
> > >> > >>> >> think
> > >> > >>> >> > it
> > >> > >>> >> > >>>> makes
> > >> > >>> >> > >>>> >> >>> sense
> > >> > >>> >> > >>>> >> >>> >> to
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> other
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> examples,
> > >> so
> > >> > that
> > >> > >>> >> > someone
> > >> > >>> >> > >>>> else
> > >> > >>> >> > >>>> >> uses
> > >> > >>> >> > >>>> >> >>> >> it and
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> maybe
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the
> > API?
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers,
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> Aljoscha
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>>
> > >> > >>> >> > >>>> >> >>> >> >>>
> > >> > >>> >> > >>>> >> >>> >>
> > >> > >>> >> > >>>> >> >>>
> > >> > >>> >> > >>>> >>
> > >> > >>> >> > >>>>
> > >> > >>> >> >
> > >> > >>> >>
> > >> >
> > >>
> > >
> > >
> >
>

[Robert Metzger]

I've worked on a quite outdated version of Flink for a while now and
rebased my code to the latest master on Friday.
Back at home, I wanted to continue my work and found that it is very
difficult to properly set up the latest eclipse for Flink.

What I've done so far:
- Downloaded Eclipse Luna SR1 (for linux)
- Installed Scala IDE 4.0 nightly
- add the -clean flag to the eclipse.ini, because eclipse is not starting
anymore due to a bug
- installed some missing m2e connectors
- Set the Scala compiler to 2.10 (thats a new feature in Scala IDE) (See:
http://scala-ide.org/blog/scala-installations.html)

Now, all the tests in flink-scala/src/test are mostly about this:
could not find implicit value for evidence parameter of type
org.apache.flink.api.common.typeinfo.TypeInformation[(Array[String],)];
/flink-scala/src/test/scala/org/apache/flink/api/scala/runtime

It seems that the flink-tests project is calling Scala examples for testing
purposes. So if the flink-scala project is not building, its blocking
flink-tests from building.

Any ideas?

If you are wondering why I'm not switching to IntelliJ: It is currently not
possible to deactivate a maven build profile that is activated by default,
so I'm not able to properly work on the "flink-yarn" project with IntelliJ
(the respective issue has been closed at IntelliJ, so I guess the next
release will contain the fix).
In addition, I think its good if at least a few core developers make sure
that Flink is also "developable" using Eclipse.




On Sun, Sep 14, 2014 at 12:48 PM, Márton Balassi <[hidden email]>
wrote:

> Answer posted to "Example packages naming convention" thread as the issue
> diverged from this topic.
>
> On Sun, Sep 14, 2014 at 11:14 AM, Kostas Tzoumas <[hidden email]>
> wrote:
>
> > Good catch, I suggest to use examples
> >
> > On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi <
> [hidden email]>
> > wrote:
> >
> > > Pull request issued. One minor naming concern:
> > >
> > > As of today the scala examples are located at
> > > the org.apache.flink.examples.scala package, while the java ones in
> > > the org.apache.flink.example.java. I suggest using only one convention
> > for
> > > this either example or examples.
> > >
> > > Cheers,
> > >
> > > Marton
> > >
> > > On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <
> > [hidden email]>
> > > wrote:
> > >
> > > > Sorry for being a bit silent after already bidding on LR. The pull
> > > request
> > > > is coming soon.
> > > >
> > > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]>
> > wrote:
> > > >
> > > >> I suppose that having the option between simple return type, and a
> > > >> collector is the easiest to understand.
> > > >>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <
> [hidden email]
> > >:
> > > >>
> > > >> > So, should I change join and coGroup to have a simple return
> value,
> > no
> > > >> > Option or Collection? Also what's happening with the relational
> > > >> > examples and the LinearRegression examples? I'd like to make a
> pull
> > > >> > request before this weekend.
> > > >> >
> > > >> > I also added a test that checks whether the Scala API has the same
> > > >> > methods as the Java API (ScalaAPICompletenessTest).
> > > >> >
> > > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <
> > > [hidden email]>
> > > >> > wrote:
> > > >> > > Yes, there is already a Collector version, you can do:
> > > >> > >
> > > >> > > left.join(right).where("foo").equalTo("bar") {
> > > >> > >   (left, right, out: Collector[Page]) =>
> > > >> > >     if (...) out.collect(...)
> > > >> > > }
> > > >> > >
> > > >> > > I wasn't sure on what our Function2 variant should be. That's
> why
> > I
> > > >> > > asked. There are some cases where you want to have the option
> (pun
> > > >> > > intended) of either returning something from a join or not. But
> > you
> > > >> > > could also use a filter after the join. It's a preference
> thing, I
> > > >> > > suppose.
> > > >> > >
> > > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <[hidden email]
> >
> > > >> wrote:
> > > >> > >> I think it seems weird that normal joins need to go through
> > option.
> > > >> > >>
> > > >> > >> The option variant is to allow filters in the join function.
> > > >> Wouldn't a
> > > >> > >> collector variant allow you to do the same, and would be
> > function3
> > > ?
> > > >> I
> > > >> > know
> > > >> > >> that option reads more functionally...
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <
> > > [hidden email]
> > > >> >:
> > > >> > >>>
> > > >> > >>> As already mentioned this is not possible because of type
> > erasure.
> > > >> We
> > > >> > >>> can only have one join variant that takes a Function2.
> > > >> > >>>
> > > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <
> > [hidden email]>
> > > >> > wrote:
> > > >> > >>> > It would be nice to have a join variant that directly
> returns
> > > the
> > > >> > value
> > > >> > >>> > rathern than an option. Why not have both (they are wrapped
> as
> > > >> > flatJoins
> > > >> > >>> > anyway below, right?)
> > > >> > >>> >
> > > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
> > > >> [hidden email]>
> > > >> > >> wrote:
> > > >> > >>> >
> > > >> > >>> >> Sweet! I'm lovin' this :-)
> > > >> > >>> >>
> > > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <
> > > [hidden email]
> > > >> >:
> > > >> > >>> >>
> > > >> > >>> >> > Also, you can use CaseClasses directly as the type for
> CSV
> > > >> input.
> > > >> > So
> > > >> > >>> >> > instead of reading it as tuples and then having a mapper
> > that
> > > >> > maps to
> > > >> > >>> >> > your case classes you can use:
> > > >> > >>> >> >
> > > >> > >>> >> > env.readCsv[Edge](...)
> > > >> > >>> >> >
> > > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> > > >> > >> [hidden email]>
> > > >> > >>> >> > wrote:
> > > >> > >>> >> > > I added support for specifying keys by name for
> > > CaseClasses.
> > > >> > Check
> > > >> > >> out
> > > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see it
> > in
> > > >> > action.
> > > >> > >>> >> > >
> > > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H
> > examples.
> > > >> > >>> >> > >
> > > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> > > >> > >> [hidden email]
> > > >> > >>> >> >
> > > >> > >>> >> > wrote:
> > > >> > >>> >> > >> Yes, that would allow list comprehensions. It would be
> > > >> > possible to
> > > >> > >>> >> > >> have the Collection signature for join (and coGroup),
> > > i.e.:
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> (T and O are the left and right input type, R is
> result
> > > >> type)
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> Then you can return collections and still return an
> > > option,
> > > >> as
> > > >> > in:
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r > ...)
> > > >> Some(l)
> > > >> > >> else
> > > >> > >>> >> > None }
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> Because there is an implicit conversion from Options
> to
> > a
> > > >> > >> Collection.
> > > >> > >>> >> > >> This will always wrap the return value in a List with
> > only
> > > >> one
> > > >> > >> value.
> > > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not
> > sure
> > > >> > whether
> > > >> > >> we
> > > >> > >>> >> > >> want the overhead of always having to use an Option
> even
> > > >> though
> > > >> > >> the
> > > >> > >>> >> > >> join always returns a value.
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> What do you think?
> > > >> > >>> >> > >>
> > > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> > > >> > >> [hidden email]>
> > > >> > >>> >> > wrote:
> > > >> > >>> >> > >>> Hmmm, tricky question...
> > > >> > >>> >> > >>> How about the Option for Join as this is a tuple-wise
> > > >> > operation
> > > >> > >> and
> > > >> > >>> >> the
> > > >> > >>> >> > >>> Collection for Cogroup which is group-wise?
> > > >> > >>> >> > >>> Could we in that case use list comprehensions in
> > Cogroup
> > > >> > >> functions?
> > > >> > >>> >> > >>>
> > > >> > >>> >> > >>> Or is that too much mixing?
> > > >> > >>> >> > >>>
> > > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> > > >> > [hidden email]
> > > >> > >>>:
> > > >> > >>> >> > >>>
> > > >> > >>> >> > >>>> I didn't look at the example either.
> > > >> > >>> >> > >>>>
> > > >> > >>> >> > >>>> Addings collections is easy, it's just that we can
> > > either
> > > >> > have
> > > >> > >>> >> > >>>> Collections or the Option, not both.
> > > >> > >>> >> > >>>>
> > > >> > >>> >> > >>>> For the coding style I followed this:
> > > >> > >>> >> > >>>>
> > > >> > >>> >> >
> > > >> > >>
> > > >> >
> > > >>
> > >
> https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> > > >> > >>> >> ,
> > > >> > >>> >> > >>>> which itself is based on this:
> > > >> > http://docs.scala-lang.org/style/
> > > >> > >> .
> > > >> > >>> >> It
> > > >> > >>> >> > >>>> is different from the Java Code Guidelines we have
> in
> > > >> place,
> > > >> > >> yes.
> > > >> > >>> >> > >>>>
> > > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> > > >> > >> [hidden email]
> > > >> > >>> >> >
> > > >> > >>> >> > >>>> wrote:
> > > >> > >>> >> > >>>> > I haven't looked at the LineRank example in
> detail,
> > > but
> > > >> if
> > > >> > you
> > > >> > >>> >> > think that
> > > >> > >>> >> > >>>> > it adds something new to the examples collection,
> we
> > > can
> > > >> > >> certainly
> > > >> > >>> >> > port
> > > >> > >>> >> > >>>> it
> > > >> > >>> >> > >>>> > also to Java.
> > > >> > >>> >> > >>>> > I think the Option and Collector return types are
> > > >> > sufficient
> > > >> > >> right
> > > >> > >>> >> > now
> > > >> > >>> >> > >>>> but
> > > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> > > >> > >>> >> > >>>> >
> > > >> > >>> >> > >>>> > Great that the Scala primitives are working! Also
> > > thanks
> > > >> > for
> > > >> > >>> >> adding
> > > >> > >>> >> > >>>> > genSequence and adapting my examples.
> > > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files
> or
> > > do
> > > >> we
> > > >> > >> have a
> > > >> > >>> >> > >>>> different
> > > >> > >>> >> > >>>> > there?
> > > >> > >>> >> > >>>> >
> > > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> > > >> > >> [hidden email]
> > > >> > >>> >> >:
> > > >> > >>> >> > >>>> >
> > > >> > >>> >> > >>>> >> What about the LineRank example? We had that in
> > Scala
> > > >> but
> > > >> > >> never
> > > >> > >>> >> > had a
> > > >> > >>> >> > >>>> >> Java Example.
> > > >> > >>> >> > >>>> >>
> > > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha
> Krettek <
> > > >> > >>> >> > [hidden email]>
> > > >> > >>> >> > >>>> >> wrote:
> > > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just
> > > >> copied
> > > >> > the
> > > >> > >> Java
> > > >> > >>> >> > >>>> ITCase.
> > > >> > >>> >> > >>>> >> >
> > > >> > >>> >> > >>>> >> > The only examples that are missing now are
> > > >> > >> LinearRegression and
> > > >> > >>> >> > the
> > > >> > >>> >> > >>>> >> > relational stuff.
> > > >> > >>> >> > >>>> >> >
> > > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske
> <
> > > >> > >>> >> > [hidden email]>
> > > >> > >>> >> > >>>> >> wrote:
> > > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees
> example.
> > > >> > >>> >> > >>>> >> >> That was a preprocessing step for the
> > > >> > >> TriangleEnumeration, and
> > > >> > >>> >> > is now
> > > >> > >>> >> > >>>> >> part
> > > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> > > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one.
> As I
> > > >> said
> > > >> > >> before,
> > > >> > >>> >> > I'd
> > > >> > >>> >> > >>>> >> prefer to
> > > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> > > >> > >>> >> > >>>> >> >>
> > > >> > >>> >> > >>>> >> >> Cheers, Fabian
> > > >> > >>> >> > >>>> >> >>
> > > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek <
> > > >> > >>> >> > [hidden email]>:
> > > >> > >>> >> > >>>> >> >>
> > > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again
> > > fabian.
> > > >> :D
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> Regarding the other stuff:
> > > >> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala about
> > > >> including
> > > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> > > >> > >> TypeInformation.
> > > >> > >>> >> > >>>> >> >>>  - I added generateSequence to
> > > >> ExecutionEnvironment.
> > > >> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives in
> > > >> Array, I
> > > >> > >> noticed
> > > >> > >>> >> > it
> > > >> > >>> >> > >>>> while
> > > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an older
> > > >> version
> > > >> > of
> > > >> > >> the
> > > >> > >>> >> > code.
> > > >> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is
> not
> > > >> > possible,
> > > >> > >> this
> > > >> > >>> >> > is
> > > >> > >>> >> > >>>> also
> > > >> > >>> >> > >>>> >> >>> a restriction in the Java API.
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> What do you think about the interface of join
> > and
> > > >> > >> coGroup?
> > > >> > >>> >> > Right
> > > >> > >>> >> > >>>> now,
> > > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an
> > > Option
> > > >> or
> > > >> > the
> > > >> > >>> >> > lambda
> > > >> > >>> >> > >>>> with
> > > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have
> also
> > > >> have a
> > > >> > >> lambda
> > > >> > >>> >> > that
> > > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure
> > > this
> > > >> has
> > > >> > >> the
> > > >> > >>> >> > same type
> > > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't
> use
> > > it.
> > > >> > >> There is
> > > >> > >>> >> an
> > > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a
> > Collection,
> > > >> so I
> > > >> > >> could
> > > >> > >>> >> > change
> > > >> > >>> >> > >>>> it
> > > >> > >>> >> > >>>> >> >>> without breaking the examples we have now.
> What
> > > do
> > > >> you
> > > >> > >> think?
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> > > >> > >>> >> ConnectedComponents,
> > > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> > > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
> > > >> > >>> >> > >>>> >> >>> PageRank
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> These are the examples people called dibs on:
> > > >> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should be a
> > > port
> > > >> of
> > > >> > >>> >> > >>>> LinearRegression
> > > >> > >>> >> > >>>> >> >>> Example from Java)
> > > >> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> > > >> > >>> >> > >>>> >> >>>  - The relational Stuff
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan
> Ewen <
> > > >> > >>> >> > [hidden email]>
> > > >> > >>> >> > >>>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> > > >> > >>> >> > >>>> >> >>> >
> > > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
> > > >> Krettek <
> > > >> > >>> >> > >>>> >> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> > wrote:
> > > >> > >>> >> > >>>> >> >>> >
> > > >> > >>> >> > >>>> >> >>> >> By the way, what was called
> > > >> BatchGradientDescent in
> > > >> > >> the
> > > >> > >>> >> > Scala
> > > >> > >>> >> > >>>> >> examples
> > > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
> > > >> > LinearRegression
> > > >> > >>> >> > Example from
> > > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples
> > > >> earlier.
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3.
> Any
> > > >> > thoughts
> > > >> > >>> >> about
> > > >> > >>> >> > >>>> removing
> > > >> > >>> >> > >>>> >> >>> >> RelationalQuery?
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha
> > > >> Krettek
> > > >> > <
> > > >> > >>> >> > >>>> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >
> > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration
> Examples,
> > > >> thanks
> > > >> > >>> >> Fabian.
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount,
> KMeans,
> > > >> > >>> >> > ConnectedComponents,
> > > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive,
> > > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> > These are the examples people called
> dibs
> > > on:
> > > >> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> > > >> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> > > >> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not
> mistaken):
> > > >> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
> > > >> > >>> >> > >>>> >> >>> >> >  - LinearRegression
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM,
> Aljoscha
> > > >> > Krettek <
> > > >> > >>> >> > >>>> >> >>> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running
> > > list
> > > >> of
> > > >> > >>> >> > >>>> ported/unported
> > > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the
> > java
> > > >> > example
> > > >> > >>> >> > package to
> > > >> > >>> >> > >>>> >> >>> examples
> > > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is
> fine
> > > as
> > > >> it
> > > >> > is.
> > > >> > >>> >> Just
> > > >> > >>> >> > >>>> because
> > > >> > >>> >> > >>>> >> >>> Scala
> > > >> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't
> > mean
> > > >> it's
> > > >> > >> always
> > > >> > >>> >> > the
> > > >> > >>> >> > >>>> best
> > > >> > >>> >> > >>>> >> >>> >> >> choice. :D
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount,
> KMeans,
> > > >> > >>> >> > ConnectedComponents,
> > > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called
> dibs
> > > on:
> > > >> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank
> > (Fabian)
> > > >> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> > > >> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not
> > mistaken):
> > > >> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> > > >> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >> Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas
> > > >> Tzoumas
> > > >> > <
> > > >> > >>> >> > >>>> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >
> > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added
> a
> > > >> > >> termination
> > > >> > >>> >> > criterion
> > > >> > >>> >> > >>>> >> in the
> > > >> > >>> >> > >>>> >> >>> >> Java
> > > >> > >>> >> > >>>> >> >>> >> >>> version:
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> > >>>>
> > > >> > >>
> > https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> > > >> > >>> >> > >>>> >> >>> >> >>>
> > > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
> > > >> criterion
> > > >> > in
> > > >> > >>> >> Scala
> > > >> > >>> >> > more
> > > >> > >>> >> > >>>> >> >>> >> functional?
> > > >> > >>> >> > >>>> >> >>> >> >>>
> > > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package
> name
> > > is
> > > >> > >>> >> > example.java but
> > > >> > >>> >> > >>>> >> >>> >> examples.scala
> > > >> > >>> >> > >>>> >> >>> >> >>>
> > > >> > >>> >> > >>>> >> >>> >> >>> Kostas
> > > >> > >>> >> > >>>> >> >>> >> >>>
> > > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas
> > > >> Tzoumas
> > > >> > <
> > > >> > >>> >> > >>>> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >
> > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
> > > >> PiEstimation
> > > >> > >> (was not
> > > >> > >>> >> > on
> > > >> > >>> >> > >>>> your
> > > >> > >>> >> > >>>> >> >>> list).
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the
> relational
> > > >> stuff
> > > >> > I
> > > >> > >> can
> > > >> > >>> >> > take
> > > >> > >>> >> > >>>> those
> > > >> > >>> >> > >>>> >> as
> > > >> > >>> >> > >>>> >> >>> >> well.
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the
> > "RelationalQuery"
> > > >> from
> > > >> > >> both
> > > >> > >>> >> > Scala and
> > > >> > >>> >> > >>>> >> Java?
> > > >> > >>> >> > >>>> >> >>> It
> > > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H
> > Q3.
> > > >> Does
> > > >> > it
> > > >> > >> add
> > > >> > >>> >> > some
> > > >> > >>> >> > >>>> >> teaching
> > > >> > >>> >> > >>>> >> >>> >> value on
> > > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>> Kostas
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM,
> > Aljoscha
> > > >> > Krettek
> > > >> > >> <
> > > >> > >>> >> > >>>> >> >>> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an
> > > ITCase.
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount,
> > > KMeans,
> > > >> > >>> >> > >>>> ConnectedComponents,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called
> > > dibs
> > > >> on:
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
> > > >> (Fabian)
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not
> > > mistaken):
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> > > >> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM,
> Kostas
> > > >> > Tzoumas <
> > > >> > >>> >> > >>>> >> >>> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >>
> > > >> > >>> >> > >>>>
> > > >> > >>> >> >
> > > >> > >>> >>
> > > >> > >>
> > > >> >
> > > >>
> > >
> >
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM,
> > > Aljoscha
> > > >> > >> Krettek <
> > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents
> > > Example
> > > >> > from
> > > >> > >>> >> Vasia.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM,
> > > Fabian
> > > >> > >> Hueske <
> > > >> > >>> >> > >>>> >> >>> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00
> > Aljoscha
> > > >> > >> Krettek <
> > > >> > >>> >> > >>>> >> >>> >> [hidden email]>:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision.
> > :D
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java
> > and
> > > >> > >>> >> > KMeans.scala. I'm
> > > >> > >>> >> > >>>> >> storing
> > > >> > >>> >> > >>>> >> >>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object
> > arrays
> > > >> and
> > > >> > >> then
> > > >> > >>> >> > >>>> converting
> > > >> > >>> >> > >>>> >> it to
> > > >> > >>> >> > >>>> >> >>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala
> objects.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
> > > >> equalTo to
> > > >> > >> make
> > > >> > >>> >> it
> > > >> > >>> >> > >>>> >> consistent
> > > >> > >>> >> > >>>> >> >>> >> with the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup).
> > > There
> > > >> is
> > > >> > no
> > > >> > >> need
> > > >> > >>> >> > for a
> > > >> > >>> >> > >>>> >> >>> keyword,
> > > >> > >>> >> > >>>> >> >>> >> you
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> left.join(right).where(0).equalTo(1) {
> > > >> > >> (le,
> > > >> > >>> >> re)
> > > >> > >>> >> > =>
> > > >> > >>> >> > >>>> new
> > > >> > >>> >> > >>>> >> >>> >> MyResult(le,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> }
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07
> PM,
> > > >> Fabian
> > > >> > >> Hueske
> > > >> > >>> >> <
> > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet
> issue,
> > I
> > > >> also
> > > >> > >> found
> > > >> > >>> >> an
> > > >> > >>> >> > >>>> >> >>> inconsistency
> > > >> > >>> >> > >>>> >> >>> >> with
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done
> as:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> ds1.join(ds2).where(...).equalTo(...)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala
> > this
> > > >> is:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > ds1.join(d2).where(...).isEqualTo(...)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be
> renamed
> > to
> > > >> > >> equalTo(),
> > > >> > >>> >> > IMO.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and
> > coGroup?)
> > > >> > lacks
> > > >> > >> the
> > > >> > >>> >> > with()
> > > >> > >>> >> > >>>> >> method
> > > >> > >>> >> > >>>> >> >>> >> because
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should
> be
> > > >> offer
> > > >> > >>> >> something
> > > >> > >>> >> > >>>> similar
> > > >> > >>> >> > >>>> >> for
> > > >> > >>> >> > >>>> >> >>> >> Scala
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left,
> right)?
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00
> > > Stephan
> > > >> > Ewen
> > > >> > >> <
> > > >> > >>> >> > >>>> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings,
> > Object[][]
> > > >> would
> > > >> > >> work
> > > >> > >>> >> as
> > > >> > >>> >> > well.
> > > >> > >>> >> > >>>> >> That
> > > >> > >>> >> > >>>> >> >>> is a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could
> be
> > > >> stored
> > > >> > as
> > > >> > >> Java
> > > >> > >>> >> > or
> > > >> > >>> >> > >>>> Scala
> > > >> > >>> >> > >>>> >> >>> Tuples,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert
> > > between
> > > >> > the
> > > >> > >> two.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at
> 10:55
> > > AM,
> > > >> > Fabian
> > > >> > >>> >> > Hueske
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same
> > > >> > problem...
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and
> > > >> parsing
> > > >> > >> them,
> > > >> > >>> >> but
> > > >> > >>> >> > >>>> using
> > > >> > >>> >> > >>>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on
> a
> > > >> > >>> >> FileInputFormat.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse
> > the
> > > >> > Strings
> > > >> > >>> >> > >>>> manually...
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35
> GMT+02:00
> > > >> > Aljoscha
> > > >> > >>> >> Krettek
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought.
> Maybe
> > we
> > > >> > should
> > > >> > >> just
> > > >> > >>> >> > change
> > > >> > >>> >> > >>>> >> all
> > > >> > >>> >> > >>>> >> >>> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> input
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use
> > CSV
> > > >> > input
> > > >> > >>> >> > formats in
> > > >> > >>> >> > >>>> all
> > > >> > >>> >> > >>>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> What
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at
> > 7:46
> > > >> AM,
> > > >> > >>> >> Aljoscha
> > > >> > >>> >> > >>>> Krettek
> > > >> > >>> >> > >>>> >> <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate
> > that
> > > >> the
> > > >> > >> data
> > > >> > >>> >> > types are
> > > >> > >>> >> > >>>> >> >>> >> incompatible.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what
> you
> > > >> > proposed:
> > > >> > >>> >> move
> > > >> > >>> >> > the
> > > >> > >>> >> > >>>> >> data to
> > > >> > >>> >> > >>>> >> >>> a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> field
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> > > >> > >>> >> > getDefaultEdgeDataSet()
> > > >> > >>> >> > >>>> >> method in
> > > >> > >>> >> > >>>> >> >>> >> Scala.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying
> would
> > > >> > >> duplicate the
> > > >> > >>> >> > data
> > > >> > >>> >> > >>>> and
> > > >> > >>> >> > >>>> >> >>> make it
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> for
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in
> > the
> > > >> Java
> > > >> > and
> > > >> > >>> >> Scala
> > > >> > >>> >> > >>>> >> versions.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others
> > think?
> > > >> This
> > > >> > >> will
> > > >> > >>> >> > probably
> > > >> > >>> >> > >>>> >> occur
> > > >> > >>> >> > >>>> >> >>> in
> > > >> > >>> >> > >>>> >> >>> >> all
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014
> at
> > > >> 10:04
> > > >> > PM,
> > > >> > >>> >> > Vasiliki
> > > >> > >>> >> > >>>> >> Kalavri
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <
> > > [hidden email]
> > > >> >
> > > >> > >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
> > > >> Connected
> > > >> > >>> >> Components
> > > >> > >>> >> > >>>> >> example,
> > > >> > >>> >> > >>>> >> >>> but
> > > >> > >>> >> > >>>> >> >>> >> I am
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example
> > input
> > > >> data
> > > >> > >> from
> > > >> > >>> >> > >>>> >> java-examples.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> > > >> > ConnectedComponentsData
> > > >> > >>> >> class,
> > > >> > >>> >> > the
> > > >> > >>> >> > >>>> >> vertices
> > > >> > >>> >> > >>>> >> >>> >> and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> data
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the
> > methods
> > > >> > >>> >> > >>>> >> getDefaultVertexDataSet()
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
> > > >> getDefaultEdgeDataSet(),
> > > >> > >> which
> > > >> > >>> >> > take
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> > > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> > > >> > >>> >> > >>>> >> >>> as
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to
> provide
> > > >> public
> > > >> > >> static
> > > >> > >>> >> > fields
> > > >> > >>> >> > >>>> >> (like
> > > >> > >>> >> > >>>> >> >>> in
> > > >> > >>> >> > >>>> >> >>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
> > > >> introduces a
> > > >> > >>> >> > conversion
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> > > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> > > >> > >>> >> > >>>> >> >>> Scala
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
> > > >> scala.Long
> > > >> > >> and I
> > > >> > >>> >> > guess
> > > >> > >>> >> > >>>> this
> > > >> > >>> >> > >>>> >> is
> > > >> > >>> >> > >>>> >> >>> an
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?).
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
> > > >> course, to
> > > >> > >> copy
> > > >> > >>> >> the
> > > >> > >>> >> > >>>> example
> > > >> > >>> >> > >>>> >> >>> data
> > > >> > >>> >> > >>>> >> >>> >> in
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing
> something
> > > >> here?
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014
> > > 15:52,
> > > >> > >> Aljoscha
> > > >> > >>> >> > >>>> Krettek <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated
> my
> > > >> repo:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> >
> > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a
> > working
> > > >> > >> WordCount
> > > >> > >>> >> > example.
> > > >> > >>> >> > >>>> >> It's
> > > >> > >>> >> > >>>> >> >>> >> pretty
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example
> with
> > > >> some
> > > >> > >> fixups
> > > >> > >>> >> > for the
> > > >> > >>> >> > >>>> >> syntax
> > > >> > >>> >> > >>>> >> >>> and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice
> > > that I
> > > >> > >> added the
> > > >> > >>> >> > >>>> >> java-examples
> > > >> > >>> >> > >>>> >> >>> >> as a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> scala-examples. I
> > > did
> > > >> > this
> > > >> > >> to
> > > >> > >>> >> > reuse
> > > >> > >>> >> > >>>> the
> > > >> > >>> >> > >>>> >> >>> example
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> data.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a
> > > program
> > > >> > you
> > > >> > >> can
> > > >> > >>> >> do
> > > >> > >>> >> > a
> > > >> > >>> >> > >>>> pull
> > > >> > >>> >> > >>>> >> >>> request
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> my
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect
> > the
> > > >> > >> examples.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014
> > at
> > > >> 12:19
> > > >> > >> PM,
> > > >> > >>> >> > Hermann
> > > >> > >>> >> > >>>> >> Gábor <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> ComputeEdgeDegrees
> > > for
> > > >> me!
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5,
> 2014
> > > at
> > > >> > 11:44
> > > >> > >> AM,
> > > >> > >>> >> > Márton
> > > >> > >>> >> > >>>> >> >>> Balassi <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > [hidden email]
> > > >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > BatchGradientDescent
> > > >> for
> > > >> > me
> > > >> > >> :)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5,
> > 2014
> > > at
> > > >> > >> 11:15 AM,
> > > >> > >>> >> > Kostas
> > > >> > >>> >> > >>>> >> >>> Tzoumas <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for
> > > >> > WebLogAnalysis.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience
> > with
> > > >> > Scala
> > > >> > >>> >> > consists of
> > > >> > >>> >> > >>>> >> going
> > > >> > >>> >> > >>>> >> >>> >> through
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good
> stress
> > > >> test
> > > >> > >> both for
> > > >> > >>> >> > me and
> > > >> > >>> >> > >>>> >> the
> > > >> > >>> >> > >>>> >> >>> new
> > > >> > >>> >> > >>>> >> >>> >> API
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4,
> > > 2014
> > > >> at
> > > >> > >> 9:09
> > > >> > >>> >> PM,
> > > >> > >>> >> > >>>> Vasiliki
> > > >> > >>> >> > >>>> >> >>> >> Kalavri <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for
> having
> > > >> other
> > > >> > >> people
> > > >> > >>> >> > >>>> implement
> > > >> > >>> >> > >>>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples!
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected
> > > >> Components
> > > >> > and
> > > >> > >>> >> > Kmeans for
> > > >> > >>> >> > >>>> >> me :)
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4
> September
> > > >> 2014
> > > >> > >> 21:03,
> > > >> > >>> >> > Fabian
> > > >> > >>> >> > >>>> >> Hueske <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> [hidden email]>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for
> > > >> > >>> >> > TriangleEnumeration and
> > > >> > >>> >> > >>>> >> >>> PageRank.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also
> > do
> > > >> the
> > > >> > >> examples
> > > >> > >>> >> > >>>> similar
> > > >> > >>> >> > >>>> >> to
> > > >> > >>> >> > >>>> >> >>> the
> > > >> > >>> >> > >>>> >> >>> >> Java
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running
> > > >> > >> out-of-the-box
> > > >> > >>> >> > without
> > > >> > >>> >> > >>>> >> >>> parameters
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > -
> parameters
> > > for
> > > >> > >> external
> > > >> > >>> >> > data
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a
> > > >> similar
> > > >> > >> code
> > > >> > >>> >> > structure
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04
> > > 20:56
> > > >> > >> GMT+02:00
> > > >> > >>> >> > >>>> Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> Krettek <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do,
> > > then
> > > >> > >> people can
> > > >> > >>> >> > >>>> reserve
> > > >> > >>> >> > >>>> >> their
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> favourite
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu,
> > Sep
> > > 4,
> > > >> > 2014
> > > >> > >> at
> > > >> > >>> >> > 8:55 PM,
> > > >> > >>> >> > >>>> >> Fabian
> > > >> > >>> >> > >>>> >> >>> >> Hueske
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> [hidden email]
> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I
> think
> > > >> having
> > > >> > >>> >> examples
> > > >> > >>> >> > >>>> >> implemented
> > > >> > >>> >> > >>>> >> >>> by
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > different
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> valuable
> > > in
> > > >> the
> > > >> > >> past.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd
> help
> > > >> with
> > > >> > two
> > > >> > >> or
> > > >> > >>> >> > three
> > > >> > >>> >> > >>>> >> examples.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It
> might
> > > be
> > > >> > >> helpful if
> > > >> > >>> >> > you'd
> > > >> > >>> >> > >>>> >> port a
> > > >> > >>> >> > >>>> >> >>> >> simple
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> one
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount.
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > 2014-09-04
> > > >> > 18:47
> > > >> > >>> >> > GMT+02:00
> > > >> > >>> >> > >>>> >> Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> Krettek
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > <
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> [hidden email]
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I
> have
> > a
> > > >> > working
> > > >> > >>> >> > rewrite of
> > > >> > >>> >> > >>>> the
> > > >> > >>> >> > >>>> >> >>> Scala
> > > >> > >>> >> > >>>> >> >>> >> API
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here:
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> >
> > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm
> > > hoping
> > > >> > that
> > > >> > >> I'll
> > > >> > >>> >> > only
> > > >> > >>> >> > >>>> have
> > > >> > >>> >> > >>>> >> to
> > > >> > >>> >> > >>>> >> >>> >> write
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> tests
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > examples.
> > > >> Do
> > > >> > you
> > > >> > >>> >> think
> > > >> > >>> >> > it
> > > >> > >>> >> > >>>> makes
> > > >> > >>> >> > >>>> >> >>> sense
> > > >> > >>> >> > >>>> >> >>> >> to
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> other
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > examples,
> > > >> so
> > > >> > that
> > > >> > >>> >> > someone
> > > >> > >>> >> > >>>> else
> > > >> > >>> >> > >>>> >> uses
> > > >> > >>> >> > >>>> >> >>> >> it and
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> maybe
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in
> the
> > > API?
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> Cheers,
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > Aljoscha
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > >> > >>> >> > >>>> >> >>> >> >>>
> > > >> > >>> >> > >>>> >> >>> >>
> > > >> > >>> >> > >>>> >> >>>
> > > >> > >>> >> > >>>> >>
> > > >> > >>> >> > >>>>
> > > >> > >>> >> >
> > > >> > >>> >>
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

[Henry Saputra]

Hi Robert,  I didn't have problem with IntelliJ idea last week, will try it
again with latest master.

User can deactivate unwanted maven profile via idea's maven project option
(not remember exact name of that feature)

On Sunday, September 28, 2014, Robert Metzger <[hidden email]> wrote:

> I've worked on a quite outdated version of Flink for a while now and
> rebased my code to the latest master on Friday.
> Back at home, I wanted to continue my work and found that it is very
> difficult to properly set up the latest eclipse for Flink.
>
> What I've done so far:
> - Downloaded Eclipse Luna SR1 (for linux)
> - Installed Scala IDE 4.0 nightly
> - add the -clean flag to the eclipse.ini, because eclipse is not starting
> anymore due to a bug
> - installed some missing m2e connectors
> - Set the Scala compiler to 2.10 (thats a new feature in Scala IDE) (See:
> http://scala-ide.org/blog/scala-installations.html)
>
> Now, all the tests in flink-scala/src/test are mostly about this:
> could not find implicit value for evidence parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[(Array[String],)];
> /flink-scala/src/test/scala/org/apache/flink/api/scala/runtime
>
> It seems that the flink-tests project is calling Scala examples for testing
> purposes. So if the flink-scala project is not building, its blocking
> flink-tests from building.
>
> Any ideas?
>
> If you are wondering why I'm not switching to IntelliJ: It is currently not
> possible to deactivate a maven build profile that is activated by default,
> so I'm not able to properly work on the "flink-yarn" project with IntelliJ
> (the respective issue has been closed at IntelliJ, so I guess the next
> release will contain the fix).
> In addition, I think its good if at least a few core developers make sure
> that Flink is also "developable" using Eclipse.
>
>
>
>
> On Sun, Sep 14, 2014 at 12:48 PM, Márton Balassi <[hidden email]
> <javascript:;>>
> wrote:
>
> > Answer posted to "Example packages naming convention" thread as the issue
> > diverged from this topic.
> >
> > On Sun, Sep 14, 2014 at 11:14 AM, Kostas Tzoumas <[hidden email]>
> > wrote:
> >
> > > Good catch, I suggest to use examples
> > >
> > > On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi <
> > [hidden email]>
> > > wrote:
> > >
> > > > Pull request issued. One minor naming concern:
> > > >
> > > > As of today the scala examples are located at
> > > > the org.apache.flink.examples.scala package, while the java ones in
> > > > the org.apache.flink.example.java. I suggest using only one
> convention
> > > for
> > > > this either example or examples.
> > > >
> > > > Cheers,
> > > >
> > > > Marton
> > > >
> > > > On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <
> > > [hidden email]>
> > > > wrote:
> > > >
> > > > > Sorry for being a bit silent after already bidding on LR. The pull
> > > > request
> > > > > is coming soon.
> > > > >
> > > > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > > >
> > > > >> I suppose that having the option between simple return type, and a
> > > > >> collector is the easiest to understand.
> > > > >>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <
> > [hidden email]
> > > >:
> > > > >>
> > > > >> > So, should I change join and coGroup to have a simple return
> > value,
> > > no
> > > > >> > Option or Collection? Also what's happening with the relational
> > > > >> > examples and the LinearRegression examples? I'd like to make a
> > pull
> > > > >> > request before this weekend.
> > > > >> >
> > > > >> > I also added a test that checks whether the Scala API has the
> same
> > > > >> > methods as the Java API (ScalaAPICompletenessTest).
> > > > >> >
> > > > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <
> > > > [hidden email]>
> > > > >> > wrote:
> > > > >> > > Yes, there is already a Collector version, you can do:
> > > > >> > >
> > > > >> > > left.join(right).where("foo").equalTo("bar") {
> > > > >> > >   (left, right, out: Collector[Page]) =>
> > > > >> > >     if (...) out.collect(...)
> > > > >> > > }
> > > > >> > >
> > > > >> > > I wasn't sure on what our Function2 variant should be. That's
> > why
> > > I
> > > > >> > > asked. There are some cases where you want to have the option
> > (pun
> > > > >> > > intended) of either returning something from a join or not.
> But
> > > you
> > > > >> > > could also use a filter after the join. It's a preference
> > thing, I
> > > > >> > > suppose.
> > > > >> > >
> > > > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <
> [hidden email]
> > >
> > > > >> wrote:
> > > > >> > >> I think it seems weird that normal joins need to go through
> > > option.
> > > > >> > >>
> > > > >> > >> The option variant is to allow filters in the join function.
> > > > >> Wouldn't a
> > > > >> > >> collector variant allow you to do the same, and would be
> > > function3
> > > > ?
> > > > >> I
> > > > >> > know
> > > > >> > >> that option reads more functionally...
> > > > >> > >>
> > > > >> > >>
> > > > >> > >>
> > > > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <
> > > > [hidden email]
> > > > >> >:
> > > > >> > >>>
> > > > >> > >>> As already mentioned this is not possible because of type
> > > erasure.
> > > > >> We
> > > > >> > >>> can only have one join variant that takes a Function2.
> > > > >> > >>>
> > > > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <
> > > [hidden email]>
> > > > >> > wrote:
> > > > >> > >>> > It would be nice to have a join variant that directly
> > returns
> > > > the
> > > > >> > value
> > > > >> > >>> > rathern than an option. Why not have both (they are
> wrapped
> > as
> > > > >> > flatJoins
> > > > >> > >>> > anyway below, right?)
> > > > >> > >>> >
> > > > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
> > > > >> [hidden email]>
> > > > >> > >> wrote:
> > > > >> > >>> >
> > > > >> > >>> >> Sweet! I'm lovin' this :-)
> > > > >> > >>> >>
> > > > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <
> > > > [hidden email]
> > > > >> >:
> > > > >> > >>> >>
> > > > >> > >>> >> > Also, you can use CaseClasses directly as the type for
> > CSV
> > > > >> input.
> > > > >> > So
> > > > >> > >>> >> > instead of reading it as tuples and then having a
> mapper
> > > that
> > > > >> > maps to
> > > > >> > >>> >> > your case classes you can use:
> > > > >> > >>> >> >
> > > > >> > >>> >> > env.readCsv[Edge](...)
> > > > >> > >>> >> >
> > > > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
> > > > >> > >> [hidden email]>
> > > > >> > >>> >> > wrote:
> > > > >> > >>> >> > > I added support for specifying keys by name for
> > > > CaseClasses.
> > > > >> > Check
> > > > >> > >> out
> > > > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see
> it
> > > in
> > > > >> > action.
> > > > >> > >>> >> > >
> > > > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H
> > > examples.
> > > > >> > >>> >> > >
> > > > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
> > > > >> > >> [hidden email]
> > > > >> > >>> >> >
> > > > >> > >>> >> > wrote:
> > > > >> > >>> >> > >> Yes, that would allow list comprehensions. It would
> be
> > > > >> > possible to
> > > > >> > >>> >> > >> have the Collection signature for join (and
> coGroup),
> > > > i.e.:
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> (T and O are the left and right input type, R is
> > result
> > > > >> type)
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> Then you can return collections and still return an
> > > > option,
> > > > >> as
> > > > >> > in:
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r >
> ...)
> > > > >> Some(l)
> > > > >> > >> else
> > > > >> > >>> >> > None }
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> Because there is an implicit conversion from Options
> > to
> > > a
> > > > >> > >> Collection.
> > > > >> > >>> >> > >> This will always wrap the return value in a List
> with
> > > only
> > > > >> one
> > > > >> > >> value.
> > > > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not
> > > sure
> > > > >> > whether
> > > > >> > >> we
> > > > >> > >>> >> > >> want the overhead of always having to use an Option
> > even
> > > > >> though
> > > > >> > >> the
> > > > >> > >>> >> > >> join always returns a value.
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> What do you think?
> > > > >> > >>> >> > >>
> > > > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
> > > > >> > >> [hidden email]>
> > > > >> > >>> >> > wrote:
> > > > >> > >>> >> > >>> Hmmm, tricky question...
> > > > >> > >>> >> > >>> How about the Option for Join as this is a
> tuple-wise
> > > > >> > operation
> > > > >> > >> and
> > > > >> > >>> >> the
> > > > >> > >>> >> > >>> Collection for Cogroup which is group-wise?
> > > > >> > >>> >> > >>> Could we in that case use list comprehensions in
> > > Cogroup
> > > > >> > >> functions?
> > > > >> > >>> >> > >>>
> > > > >> > >>> >> > >>> Or is that too much mixing?
> > > > >> > >>> >> > >>>
> > > > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
> > > > >> > [hidden email]
> > > > >> > >>>:
> > > > >> > >>> >> > >>>
> > > > >> > >>> >> > >>>> I didn't look at the example either.
> > > > >> > >>> >> > >>>>
> > > > >> > >>> >> > >>>> Addings collections is easy, it's just that we can
> > > > either
> > > > >> > have
> > > > >> > >>> >> > >>>> Collections or the Option, not both.
> > > > >> > >>> >> > >>>>
> > > > >> > >>> >> > >>>> For the coding style I followed this:
> > > > >> > >>> >> > >>>>
> > > > >> > >>> >> >
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
> > > > >> > >>> >> ,
> > > > >> > >>> >> > >>>> which itself is based on this:
> > > > >> > http://docs.scala-lang.org/style/
> > > > >> > >> .
> > > > >> > >>> >> It
> > > > >> > >>> >> > >>>> is different from the Java Code Guidelines we have
> > in
> > > > >> place,
> > > > >> > >> yes.
> > > > >> > >>> >> > >>>>
> > > > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
> > > > >> > >> [hidden email]
> > > > >> > >>> >> >
> > > > >> > >>> >> > >>>> wrote:
> > > > >> > >>> >> > >>>> > I haven't looked at the LineRank example in
> > detail,
> > > > but
> > > > >> if
> > > > >> > you
> > > > >> > >>> >> > think that
> > > > >> > >>> >> > >>>> > it adds something new to the examples
> collection,
> > we
> > > > can
> > > > >> > >> certainly
> > > > >> > >>> >> > port
> > > > >> > >>> >> > >>>> it
> > > > >> > >>> >> > >>>> > also to Java.
> > > > >> > >>> >> > >>>> > I think the Option and Collector return types
> are
> > > > >> > sufficient
> > > > >> > >> right
> > > > >> > >>> >> > now
> > > > >> > >>> >> > >>>> but
> > > > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
> > > > >> > >>> >> > >>>> >
> > > > >> > >>> >> > >>>> > Great that the Scala primitives are working!
> Also
> > > > thanks
> > > > >> > for
> > > > >> > >>> >> adding
> > > > >> > >>> >> > >>>> > genSequence and adapting my examples.
> > > > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala
> files
> > or
> > > > do
> > > > >> we
> > > > >> > >> have a
> > > > >> > >>> >> > >>>> different
> > > > >> > >>> >> > >>>> > there?
> > > > >> > >>> >> > >>>> >
> > > > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
> > > > >> > >> [hidden email]
> > > > >> > >>> >> >:
> > > > >> > >>> >> > >>>> >
> > > > >> > >>> >> > >>>> >> What about the LineRank example? We had that in
> > > Scala
> > > > >> but
> > > > >> > >> never
> > > > >> > >>> >> > had a
> > > > >> > >>> >> > >>>> >> Java Example.
> > > > >> > >>> >> > >>>> >>
> > > > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha
> > Krettek <
> > > > >> > >>> >> > [hidden email]>
> > > > >> > >>> >> > >>>> >> wrote:
> > > > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always
> just
> > > > >> copied
> > > > >> > the
> > > > >> > >> Java
> > > > >> > >>> >> > >>>> ITCase.
> > > > >> > >>> >> > >>>> >> >
> > > > >> > >>> >> > >>>> >> > The only examples that are missing now are
> > > > >> > >> LinearRegression and
> > > > >> > >>> >> > the
> > > > >> > >>> >> > >>>> >> > relational stuff.
> > > > >> > >>> >> > >>>> >> >
> > > > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian
> Hueske
> > <
> > > > >> > >>> >> > [hidden email]>
> > > > >> > >>> >> > >>>> >> wrote:
> > > > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees
> > example.
> > > > >> > >>> >> > >>>> >> >> That was a preprocessing step for the
> > > > >> > >> TriangleEnumeration, and
> > > > >> > >>> >> > is now
> > > > >> > >>> >> > >>>> >> part
> > > > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
> > > > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one.
> > As I
> > > > >> said
> > > > >> > >> before,
> > > > >> > >>> >> > I'd
> > > > >> > >>> >> > >>>> >> prefer to
> > > > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
> > > > >> > >>> >> > >>>> >> >>
> > > > >> > >>> >> > >>>> >> >> Cheers, Fabian
> > > > >> > >>> >> > >>>> >> >>
> > > > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek
> <
> > > > >> > >>> >> > [hidden email]>:
> > > > >> > >>> >> > >>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again
> > > > fabian.
> > > > >> :D
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> Regarding the other stuff:
> > > > >> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala
> about
> > > > >> including
> > > > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
> > > > >> > >> TypeInformation.
> > > > >> > >>> >> > >>>> >> >>>  - I added generateSequence to
> > > > >> ExecutionEnvironment.
> > > > >> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives
> in
> > > > >> Array, I
> > > > >> > >> noticed
> > > > >> > >>> >> > it
> > > > >> > >>> >> > >>>> while
> > > > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an
> older
> > > > >> version
> > > > >> > of
> > > > >> > >> the
> > > > >> > >>> >> > code.
> > > > >> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is
> > not
> > > > >> > possible,
> > > > >> > >> this
> > > > >> > >>> >> > is
> > > > >> > >>> >> > >>>> also
> > > > >> > >>> >> > >>>> >> >>> a restriction in the Java API.
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> What do you think about the interface of
> join
> > > and
> > > > >> > >> coGroup?
> > > > >> > >>> >> > Right
> > > > >> > >>> >> > >>>> now,
> > > > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an
> > > > Option
> > > > >> or
> > > > >> > the
> > > > >> > >>> >> > lambda
> > > > >> > >>> >> > >>>> with
> > > > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have
> > also
> > > > >> have a
> > > > >> > >> lambda
> > > > >> > >>> >> > that
> > > > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type
> erasure
> > > > this
> > > > >> has
> > > > >> > >> the
> > > > >> > >>> >> > same type
> > > > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't
> > use
> > > > it.
> > > > >> > >> There is
> > > > >> > >>> >> an
> > > > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a
> > > Collection,
> > > > >> so I
> > > > >> > >> could
> > > > >> > >>> >> > change
> > > > >> > >>> >> > >>>> it
> > > > >> > >>> >> > >>>> >> >>> without breaking the examples we have now.
> > What
> > > > do
> > > > >> you
> > > > >> > >> think?
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
> > > > >> > >>> >> ConnectedComponents,
> > > > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
> > > > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
> > > > >> > >>> >> > >>>> >> >>> PageRank
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> These are the examples people called dibs
> on:
> > > > >> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should
> be a
> > > > port
> > > > >> of
> > > > >> > >>> >> > >>>> LinearRegression
> > > > >> > >>> >> > >>>> >> >>> Example from Java)
> > > > >> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
> > > > >> > >>> >> > >>>> >> >>>  - The relational Stuff
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan
> > Ewen <
> > > > >> > >>> >> > [hidden email]>
> > > > >> > >>> >> > >>>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
> > > > >> > >>> >> > >>>> >> >>> >
> > > > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
> > > > >> Krettek <
> > > > >> > >>> >> > >>>> >> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> > wrote:
> > > > >> > >>> >> > >>>> >> >>> >
> > > > >> > >>> >> > >>>> >> >>> >> By the way, what was called
> > > > >> BatchGradientDescent in
> > > > >> > >> the
> > > > >> > >>> >> > Scala
> > > > >> > >>> >> > >>>> >> examples
> > > > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
> > > > >> > LinearRegression
> > > > >> > >>> >> > Example from
> > > > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate
> examples
> > > > >> earlier.
> > > > >> > >>> >> > >>>> >> >>> >>
> > > > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3.
> > Any
> > > > >> > thoughts
> > > > >> > >>> >> about
> > > > >> > >>> >> > >>>> removing
> > > > >> > >>> >> > >>>> >> >>> >> RelationalQuery?
> > > > >> > >>> >> > >>>> >> >>> >>
> > > > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM,
> Aljoscha
> > > > >> Krettek
> > > > >> > <
> > > > >> > >>> >> > >>>> >> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >
> > > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration
> > Examples,
> > > > >> thanks
> > > > >> > >>> >> Fabian.
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount,
> > KMeans,
> > > > >> > >>> >> > ConnectedComponents,
> > > > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis,
> TransitiveClosureNaive,
> > > > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> > These are the examples people called
> > dibs
> > > > on:
> > > > >> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
> > > > >> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
> > > > >> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not
> > mistaken):
> > > > >> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
> > > > >> > >>> >> > >>>> >> >>> >> >  - LinearRegression
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM,
> > Aljoscha
> > > > >> > Krettek <
> > > > >> > >>> >> > >>>> >> >>> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a
> running
> > > > list
> > > > >> of
> > > > >> > >>> >> > >>>> ported/unported
> > > > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the
> > > java
> > > > >> > example
> > > > >> > >>> >> > package to
> > > > >> > >>> >> > >>>> >> >>> examples
> > > > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is
> > fine
> > > > as
> > > > >> it
> > > > >> > is.
> > > > >> > >>> >> Just
> > > > >> > >>> >> > >>>> because
> > > > >> > >>> >> > >>>> >> >>> Scala
> > > > >> > >>> >> > >>>> >> >>> >> >> enables functional programming
> doesn't
> > > mean
> > > > >> it's
> > > > >> > >> always
> > > > >> > >>> >> > the
> > > > >> > >>> >> > >>>> best
> > > > >> > >>> >> > >>>> >> >>> >> >> choice. :D
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount,
> > KMeans,
> > > > >> > >>> >> > ConnectedComponents,
> > > > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis,
> TransitiveClosureNaive
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called
> > dibs
> > > > on:
> > > > >> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank
> > > (Fabian)
> > > > >> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
> > > > >> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not
> > > mistaken):
> > > > >> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
> > > > >> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> Cheers,
> > > > >> > >>> >> > >>>> >> >>> >> >> Aljoscha
> > > > >> > >>> >> > >>>> >> >>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM,
> Kostas
> > > > >> Tzoumas
> > > > >> > <
> > > > >> > >>> >> > >>>> >> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >
> > > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also
> added
> > a
> > > > >> > >> termination
> > > > >> > >>> >> > criterion
> > > > >> > >>> >> > >>>> >> in the
> > > > >> > >>> >> > >>>> >> >>> >> Java
> > > > >> > >>> >> > >>>> >> >>> >> >>> version:
> > > > >> > >>> >> > >>>> >> >>> >>
> > > > >> > >>> >> > >>>>
> > > > >> > >>
> > > https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
> > > > >> > >>> >> > >>>> >> >>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
> > > > >> criterion
> > > > >> > in
> > > > >> > >>> >> Scala
> > > > >> > >>> >> > more
> > > > >> > >>> >> > >>>> >> >>> >> functional?
> > > > >> > >>> >> > >>>> >> >>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package
> > name
> > > > is
> > > > >> > >>> >> > example.java but
> > > > >> > >>> >> > >>>> >> >>> >> examples.scala
> > > > >> > >>> >> > >>>> >> >>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>> Kostas
> > > > >> > >>> >> > >>>> >> >>> >> >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM,
> Kostas
> > > > >> Tzoumas
> > > > >> > <
> > > > >> > >>> >> > >>>> >> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >
> > > > >> > >>> >> > >>>> >> >>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
> > > > >> PiEstimation
> > > > >> > >> (was not
> > > > >> > >>> >> > on
> > > > >> > >>> >> > >>>> your
> > > > >> > >>> >> > >>>> >> >>> list).
> > > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the
> > relational
> > > > >> stuff
> > > > >> > I
> > > > >> > >> can
> > > > >> > >>> >> > take
> > > > >> > >>> >> > >>>> those
> > > > >> > >>> >> > >>>> >> as
> > > > >> > >>> >> > >>>> >> >>> >> well.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the
> > > "RelationalQuery"
> > > > >> from
> > > > >> > >> both
> > > > >> > >>> >> > Scala and
> > > > >> > >>> >> > >>>> >> Java?
> > > > >> > >>> >> > >>>> >> >>> It
> > > > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of
> TPC-H
> > > Q3.
> > > > >> Does
> > > > >> > it
> > > > >> > >> add
> > > > >> > >>> >> > some
> > > > >> > >>> >> > >>>> >> teaching
> > > > >> > >>> >> > >>>> >> >>> >> value on
> > > > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
> > > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>> Kostas
> > > > >> > >>> >> > >>>> >> >>> >> >>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM,
> > > Aljoscha
> > > > >> > Krettek
> > > > >> > >> <
> > > > >> > >>> >> > >>>> >> >>> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an
> > > > ITCase.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount,
> > > > KMeans,
> > > > >> > >>> >> > >>>> ConnectedComponents,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people
> called
> > > > dibs
> > > > >> on:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
> > > > >> (Fabian)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not
> > > > mistaken):
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM,
> > Kostas
> > > > >> > Tzoumas <
> > > > >> > >>> >> > >>>> >> >>> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > > >> > >>> >> > >>>> >> >>> >>
> > > > >> > >>> >> > >>>> >> >>>
> > > > >> > >>> >> > >>>> >>
> > > > >> > >>> >> > >>>>
> > > > >> > >>> >> >
> > > > >> > >>> >>
> > > > >> > >>
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM,
> > > > Aljoscha
> > > > >> > >> Krettek <
> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents
> > > > Example
> > > > >> > from
> > > > >> > >>> >> Vasia.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM,
> > > > Fabian
> > > > >> > >> Hueske <
> > > > >> > >>> >> > >>>> >> >>> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00
> > > Aljoscha
> > > > >> > >> Krettek <
> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive
> decision.
> > > :D
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at
> KMeansData.java
> > > and
> > > > >> > >>> >> > KMeans.scala. I'm
> > > > >> > >>> >> > >>>> >> storing
> > > > >> > >>> >> > >>>> >> >>> >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object
> > > arrays
> > > > >> and
> > > > >> > >> then
> > > > >> > >>> >> > >>>> converting
> > > > >> > >>> >> > >>>> >> it to
> > > > >> > >>> >> > >>>> >> >>> >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala
> > objects.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
> > > > >> equalTo to
> > > > >> > >> make
> > > > >> > >>> >> it
> > > > >> > >>> >> > >>>> >> consistent
> > > > >> > >>> >> > >>>> >> >>> >> with the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and
> coGroup).
> > > > There
> > > > >> is
> > > > >> > no
> > > > >> > >> need
> > > > >> > >>> >> > for a
> > > > >> > >>> >> > >>>> >> >>> keyword,
> > > > >> > >>> >> > >>>> >> >>> >> you
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> left.join(right).where(0).equalTo(1) {
> > > > >> > >> (le,
> > > > >> > >>> >> re)
> > > > >> > >>> >> > =>
> > > > >> > >>> >> > >>>> new
> > > > >> > >>> >> > >>>> >> >>> >> MyResult(le,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> }
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07
> > PM,
> > > > >> Fabian
> > > > >> > >> Hueske
> > > > >> > >>> >> <
> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet
> > issue,
> > > I
> > > > >> also
> > > > >> > >> found
> > > > >> > >>> >> an
> > > > >> > >>> >> > >>>> >> >>> inconsistency
> > > > >> > >>> >> > >>>> >> >>> >> with
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done
> > as:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> ds1.join(ds2).where(...).equalTo(...)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala
> > > this
> > > > >> is:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > ds1.join(d2).where(...).isEqualTo(...)
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be
> > renamed
> > > to
> > > > >> > >> equalTo(),
> > > > >> > >>> >> > IMO.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and
> > > coGroup?)
> > > > >> > lacks
> > > > >> > >> the
> > > > >> > >>> >> > with()
> > > > >> > >>> >> > >>>> >> method
> > > > >> > >>> >> > >>>> >> >>> >> because
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should
> > be
> > > > >> offer
> > > > >> > >>> >> something
> > > > >> > >>> >> > >>>> similar
> > > > >> > >>> >> > >>>> >> for
> > > > >> > >>> >> > >>>> >> >>> >> Scala
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left,
> > right)?
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00
> > > > Stephan
> > > > >> > Ewen
> > > > >> > >> <
> > > > >> > >>> >> > >>>> >> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings,
> > > Object[][]
> > > > >> would
> > > > >> > >> work
> > > > >> > >>> >> as
> > > > >> > >>> >> > well.
> > > > >> > >>> >> > >>>> >> That
> > > > >> > >>> >> > >>>> >> >>> is a
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a
> Tuple.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could
> > be
> > > > >> stored
> > > > >> > as
> > > > >> > >> Java
> > > > >> > >>> >> > or
> > > > >> > >>> >> > >>>> Scala
> > > > >> > >>> >> > >>>> >> >>> Tuples,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert
> > > > between
> > > > >> > the
> > > > >> > >> two.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at
> > 10:55
> > > > AM,
> > > > >> > Fabian
> > > > >> > >>> >> > Hueske
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the
> same
> > > > >> > problem...
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings
> and
> > > > >> parsing
> > > > >> > >> them,
> > > > >> > >>> >> but
> > > > >> > >>> >> > >>>> using
> > > > >> > >>> >> > >>>> >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based
> on
> > a
> > > > >> > >>> >> FileInputFormat.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to
> parse
> > > the
> > > > >> > Strings
> > > > >> > >>> >> > >>>> manually...
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35
> > GMT+02:00
> > > > >> > Aljoscha
> > > > >> > >>> >> Krettek
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought.
> > Maybe
> > > we
> > > > >> > should
> > > > >> > >> just
> > > > >> > >>> >> > change
> > > > >> > >>> >> > >>>> >> all
> > > > >> > >>> >> > >>>> >> >>> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> input
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and
> use
> > > CSV
> > > > >> > input
> > > > >> > >>> >> > formats in
> > > > >> > >>> >> > >>>> all
> > > > >> > >>> >> > >>>> >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> What
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014
> at
> > > 7:46
> > > > >> AM,
> > > > >> > >>> >> Aljoscha
> > > > >> > >>> >> > >>>> Krettek
> > > > >> > >>> >> > >>>> >> <
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's
> unfortunate
> > > that
> > > > >> the
> > > > >> > >> data
> > > > >> > >>> >> > types are
> > > > >> > >>> >> > >>>> >> >>> >> incompatible.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what
> > you
> > > > >> > proposed:
> > > > >> > >>> >> move
> > > > >> > >>> >> > the
> > > > >> > >>> >> > >>>> >> data to
> > > > >> > >>> >> > >>>> >> >>> a
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> field
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
> > > > >> > >>> >> > getDefaultEdgeDataSet()
> > > > >> > >>> >> > >>>> >> method in
> > > > >> > >>> >> > >>>> >> >>> >> Scala.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying
> > would
> > > > >> > >> duplicate the
> > > > >> > >>> >> > data
> > > > >> > >>> >> > >>>> and
> > > > >> > >>> >> > >>>> >> >>> make it
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> for
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync
> in
> > > the
> > > > >> Java
> > > > >> > and
> > > > >> > >>> >> Scala
> > > > >> > >>> >> > >>>> >> versions.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others
> > > think?
> > > > >> This
> > > > >> > >> will
> > > > >> > >>> >> > probably
> > > > >> > >>> >> > >>>> >> occur
> > > > >> > >>> >> > >>>> >> >>> in
> > > > >> > >>> >> > >>>> >> >>> >> all
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014
> > at
> > > > >> 10:04
> > > > >> > PM,
> > > > >> > >>> >> > Vasiliki
> > > > >> > >>> >> > >>>> >> Kalavri
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <
> > > > [hidden email]
> > > > >> >
> > > > >> > >> wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
> > > > >> Connected
> > > > >> > >>> >> Components
> > > > >> > >>> >> > >>>> >> example,
> > > > >> > >>> >> > >>>> >> >>> but
> > > > >> > >>> >> > >>>> >> >>> >> I am
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example
> > > input
> > > > >> data
> > > > >> > >> from
> > > > >> > >>> >> > >>>> >> java-examples.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
> > > > >> > ConnectedComponentsData
> > > > >> > >>> >> class,
> > > > >> > >>> >> > the
> > > > >> > >>> >> > >>>> >> vertices
> > > > >> > >>> >> > >>>> >> >>> >> and
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> data
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the
> > > methods
> > > > >> > >>> >> > >>>> >> getDefaultVertexDataSet()
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
> > > > >> getDefaultEdgeDataSet(),
> > > > >> > >> which
> > > > >> > >>> >> > take
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
> > > > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
> > > > >> > >>> >> > >>>> >> >>> as
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to
> > provide
> > > > >> public
> > > > >> > >> static
> > > > >> > >>> >> > fields
> > > > >> > >>> >> > >>>> >> (like
> > > > >> > >>> >> > >>>> >> >>> in
> > > > >> > >>> >> > >>>> >> >>> >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
> > > > >> introduces a
> > > > >> > >>> >> > conversion
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
> > > > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
> > > > >> > >>> >> > >>>> >> >>> Scala
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
> > > > >> scala.Long
> > > > >> > >> and I
> > > > >> > >>> >> > guess
> > > > >> > >>> >> > >>>> this
> > > > >> > >>> >> > >>>> >> is
> > > > >> > >>> >> > >>>> >> >>> an
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example
> (?).
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
> > > > >> course, to
> > > > >> > >> copy
> > > > >> > >>> >> the
> > > > >> > >>> >> > >>>> example
> > > > >> > >>> >> > >>>> >> >>> data
> > > > >> > >>> >> > >>>> >> >>> >> in
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing
> > something
> > > > >> here?
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September
> 2014
> > > > 15:52,
> > > > >> > >> Aljoscha
> > > > >> > >>> >> > >>>> Krettek <
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I
> updated
> > my
> > > > >> repo:
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
> > > > >> > >>> >> > >>>> >> >>> >>
> > > > >> > >>> >> >
> > > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a
> > > working
> > > > >> > >> WordCount
> > > > >> > >>> >> > example.
> > > > >> > >>> >> > >>>> >> It's
> > > > >> > >>> >> > >>>> >> >>> >> pretty
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example
> > with
> > > > >> some
> > > > >> > >> fixups
> > > > >> > >>> >> > for the
> > > > >> > >>> >> > >>>> >> syntax
> > > > >> > >>> >> > >>>> >> >>> and
> > > > >> > >>> >> > >>>> >> >>> >> >>

[Aljoscha Krettek-2]

Hi Robert,
this might be a problem with Eclipse not having a strict separation
between compiling the src/main and src/test code. The code that
generates the TypeInformation is a macro. Macros are only usable if
the code that uses them is compiled in a separate compilation step
from the compilation step that copiles the macro itself.

If Eclipse cannot deal with this we might have to add all the Scala
tests to the flink-tests package. There it would be ensured that they
are compiled in a separate step. Hopefully?

Cheers,
Aljoscha

On Sun, Sep 28, 2014 at 9:25 PM, Henry Saputra <[hidden email]> wrote:

> Hi Robert,  I didn't have problem with IntelliJ idea last week, will try it
> again with latest master.
>
> User can deactivate unwanted maven profile via idea's maven project option
> (not remember exact name of that feature)
>
> On Sunday, September 28, 2014, Robert Metzger <[hidden email]> wrote:
>
>> I've worked on a quite outdated version of Flink for a while now and
>> rebased my code to the latest master on Friday.
>> Back at home, I wanted to continue my work and found that it is very
>> difficult to properly set up the latest eclipse for Flink.
>>
>> What I've done so far:
>> - Downloaded Eclipse Luna SR1 (for linux)
>> - Installed Scala IDE 4.0 nightly
>> - add the -clean flag to the eclipse.ini, because eclipse is not starting
>> anymore due to a bug
>> - installed some missing m2e connectors
>> - Set the Scala compiler to 2.10 (thats a new feature in Scala IDE) (See:
>> http://scala-ide.org/blog/scala-installations.html)
>>
>> Now, all the tests in flink-scala/src/test are mostly about this:
>> could not find implicit value for evidence parameter of type
>> org.apache.flink.api.common.typeinfo.TypeInformation[(Array[String],)];
>> /flink-scala/src/test/scala/org/apache/flink/api/scala/runtime
>>
>> It seems that the flink-tests project is calling Scala examples for testing
>> purposes. So if the flink-scala project is not building, its blocking
>> flink-tests from building.
>>
>> Any ideas?
>>
>> If you are wondering why I'm not switching to IntelliJ: It is currently not
>> possible to deactivate a maven build profile that is activated by default,
>> so I'm not able to properly work on the "flink-yarn" project with IntelliJ
>> (the respective issue has been closed at IntelliJ, so I guess the next
>> release will contain the fix).
>> In addition, I think its good if at least a few core developers make sure
>> that Flink is also "developable" using Eclipse.
>>
>>
>>
>>
>> On Sun, Sep 14, 2014 at 12:48 PM, Márton Balassi <[hidden email]
>> <javascript:;>>
>> wrote:
>>
>> > Answer posted to "Example packages naming convention" thread as the issue
>> > diverged from this topic.
>> >
>> > On Sun, Sep 14, 2014 at 11:14 AM, Kostas Tzoumas <[hidden email]>
>> > wrote:
>> >
>> > > Good catch, I suggest to use examples
>> > >
>> > > On Sat, Sep 13, 2014 at 3:27 PM, Márton Balassi <
>> > [hidden email]>
>> > > wrote:
>> > >
>> > > > Pull request issued. One minor naming concern:
>> > > >
>> > > > As of today the scala examples are located at
>> > > > the org.apache.flink.examples.scala package, while the java ones in
>> > > > the org.apache.flink.example.java. I suggest using only one
>> convention
>> > > for
>> > > > this either example or examples.
>> > > >
>> > > > Cheers,
>> > > >
>> > > > Marton
>> > > >
>> > > > On Fri, Sep 12, 2014 at 9:17 PM, Márton Balassi <
>> > > [hidden email]>
>> > > > wrote:
>> > > >
>> > > > > Sorry for being a bit silent after already bidding on LR. The pull
>> > > > request
>> > > > > is coming soon.
>> > > > >
>> > > > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen <[hidden email]>
>> > > wrote:
>> > > > >
>> > > > >> I suppose that having the option between simple return type, and a
>> > > > >> collector is the easiest to understand.
>> > > > >>  Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" <
>> > [hidden email]
>> > > >:
>> > > > >>
>> > > > >> > So, should I change join and coGroup to have a simple return
>> > value,
>> > > no
>> > > > >> > Option or Collection? Also what's happening with the relational
>> > > > >> > examples and the LinearRegression examples? I'd like to make a
>> > pull
>> > > > >> > request before this weekend.
>> > > > >> >
>> > > > >> > I also added a test that checks whether the Scala API has the
>> same
>> > > > >> > methods as the Java API (ScalaAPICompletenessTest).
>> > > > >> >
>> > > > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek <
>> > > > [hidden email]>
>> > > > >> > wrote:
>> > > > >> > > Yes, there is already a Collector version, you can do:
>> > > > >> > >
>> > > > >> > > left.join(right).where("foo").equalTo("bar") {
>> > > > >> > >   (left, right, out: Collector[Page]) =>
>> > > > >> > >     if (...) out.collect(...)
>> > > > >> > > }
>> > > > >> > >
>> > > > >> > > I wasn't sure on what our Function2 variant should be. That's
>> > why
>> > > I
>> > > > >> > > asked. There are some cases where you want to have the option
>> > (pun
>> > > > >> > > intended) of either returning something from a join or not.
>> But
>> > > you
>> > > > >> > > could also use a filter after the join. It's a preference
>> > thing, I
>> > > > >> > > suppose.
>> > > > >> > >
>> > > > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen <
>> [hidden email]
>> > >
>> > > > >> wrote:
>> > > > >> > >> I think it seems weird that normal joins need to go through
>> > > option.
>> > > > >> > >>
>> > > > >> > >> The option variant is to allow filters in the join function.
>> > > > >> Wouldn't a
>> > > > >> > >> collector variant allow you to do the same, and would be
>> > > function3
>> > > > ?
>> > > > >> I
>> > > > >> > know
>> > > > >> > >> that option reads more functionally...
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >>
>> > > > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" <
>> > > > [hidden email]
>> > > > >> >:
>> > > > >> > >>>
>> > > > >> > >>> As already mentioned this is not possible because of type
>> > > erasure.
>> > > > >> We
>> > > > >> > >>> can only have one join variant that takes a Function2.
>> > > > >> > >>>
>> > > > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen <
>> > > [hidden email]>
>> > > > >> > wrote:
>> > > > >> > >>> > It would be nice to have a join variant that directly
>> > returns
>> > > > the
>> > > > >> > value
>> > > > >> > >>> > rathern than an option. Why not have both (they are
>> wrapped
>> > as
>> > > > >> > flatJoins
>> > > > >> > >>> > anyway below, right?)
>> > > > >> > >>> >
>> > > > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske <
>> > > > >> [hidden email]>
>> > > > >> > >> wrote:
>> > > > >> > >>> >
>> > > > >> > >>> >> Sweet! I'm lovin' this :-)
>> > > > >> > >>> >>
>> > > > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek <
>> > > > [hidden email]
>> > > > >> >:
>> > > > >> > >>> >>
>> > > > >> > >>> >> > Also, you can use CaseClasses directly as the type for
>> > CSV
>> > > > >> input.
>> > > > >> > So
>> > > > >> > >>> >> > instead of reading it as tuples and then having a
>> mapper
>> > > that
>> > > > >> > maps to
>> > > > >> > >>> >> > your case classes you can use:
>> > > > >> > >>> >> >
>> > > > >> > >>> >> > env.readCsv[Edge](...)
>> > > > >> > >>> >> >
>> > > > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek <
>> > > > >> > >> [hidden email]>
>> > > > >> > >>> >> > wrote:
>> > > > >> > >>> >> > > I added support for specifying keys by name for
>> > > > CaseClasses.
>> > > > >> > Check
>> > > > >> > >> out
>> > > > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see
>> it
>> > > in
>> > > > >> > action.
>> > > > >> > >>> >> > >
>> > > > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H
>> > > examples.
>> > > > >> > >>> >> > >
>> > > > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek <
>> > > > >> > >> [hidden email]
>> > > > >> > >>> >> >
>> > > > >> > >>> >> > wrote:
>> > > > >> > >>> >> > >> Yes, that would allow list comprehensions. It would
>> be
>> > > > >> > possible to
>> > > > >> > >>> >> > >> have the Collection signature for join (and
>> coGroup),
>> > > > i.e.:
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> apply[R]((T, O) => TraversableOnce[O]): DataSet[O]
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> (T and O are the left and right input type, R is
>> > result
>> > > > >> type)
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> Then you can return collections and still return an
>> > > > option,
>> > > > >> as
>> > > > >> > in:
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) => if (r >
>> ...)
>> > > > >> Some(l)
>> > > > >> > >> else
>> > > > >> > >>> >> > None }
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> Because there is an implicit conversion from Options
>> > to
>> > > a
>> > > > >> > >> Collection.
>> > > > >> > >>> >> > >> This will always wrap the return value in a List
>> with
>> > > only
>> > > > >> one
>> > > > >> > >> value.
>> > > > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not
>> > > sure
>> > > > >> > whether
>> > > > >> > >> we
>> > > > >> > >>> >> > >> want the overhead of always having to use an Option
>> > even
>> > > > >> though
>> > > > >> > >> the
>> > > > >> > >>> >> > >> join always returns a value.
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> What do you think?
>> > > > >> > >>> >> > >>
>> > > > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske <
>> > > > >> > >> [hidden email]>
>> > > > >> > >>> >> > wrote:
>> > > > >> > >>> >> > >>> Hmmm, tricky question...
>> > > > >> > >>> >> > >>> How about the Option for Join as this is a
>> tuple-wise
>> > > > >> > operation
>> > > > >> > >> and
>> > > > >> > >>> >> the
>> > > > >> > >>> >> > >>> Collection for Cogroup which is group-wise?
>> > > > >> > >>> >> > >>> Could we in that case use list comprehensions in
>> > > Cogroup
>> > > > >> > >> functions?
>> > > > >> > >>> >> > >>>
>> > > > >> > >>> >> > >>> Or is that too much mixing?
>> > > > >> > >>> >> > >>>
>> > > > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek <
>> > > > >> > [hidden email]
>> > > > >> > >>>:
>> > > > >> > >>> >> > >>>
>> > > > >> > >>> >> > >>>> I didn't look at the example either.
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>> >> > >>>> Addings collections is easy, it's just that we can
>> > > > either
>> > > > >> > have
>> > > > >> > >>> >> > >>>> Collections or the Option, not both.
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>> >> > >>>> For the coding style I followed this:
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>> >> >
>> > > > >> > >>
>> > > > >> >
>> > > > >>
>> > > >
>> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
>> > > > >> > >>> >> ,
>> > > > >> > >>> >> > >>>> which itself is based on this:
>> > > > >> > http://docs.scala-lang.org/style/
>> > > > >> > >> .
>> > > > >> > >>> >> It
>> > > > >> > >>> >> > >>>> is different from the Java Code Guidelines we have
>> > in
>> > > > >> place,
>> > > > >> > >> yes.
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske <
>> > > > >> > >> [hidden email]
>> > > > >> > >>> >> >
>> > > > >> > >>> >> > >>>> wrote:
>> > > > >> > >>> >> > >>>> > I haven't looked at the LineRank example in
>> > detail,
>> > > > but
>> > > > >> if
>> > > > >> > you
>> > > > >> > >>> >> > think that
>> > > > >> > >>> >> > >>>> > it adds something new to the examples
>> collection,
>> > we
>> > > > can
>> > > > >> > >> certainly
>> > > > >> > >>> >> > port
>> > > > >> > >>> >> > >>>> it
>> > > > >> > >>> >> > >>>> > also to Java.
>> > > > >> > >>> >> > >>>> > I think the Option and Collector return types
>> are
>> > > > >> > sufficient
>> > > > >> > >> right
>> > > > >> > >>> >> > now
>> > > > >> > >>> >> > >>>> but
>> > > > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-)
>> > > > >> > >>> >> > >>>> >
>> > > > >> > >>> >> > >>>> > Great that the Scala primitives are working!
>> Also
>> > > > thanks
>> > > > >> > for
>> > > > >> > >>> >> adding
>> > > > >> > >>> >> > >>>> > genSequence and adapting my examples.
>> > > > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala
>> files
>> > or
>> > > > do
>> > > > >> we
>> > > > >> > >> have a
>> > > > >> > >>> >> > >>>> different
>> > > > >> > >>> >> > >>>> > there?
>> > > > >> > >>> >> > >>>> >
>> > > > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek <
>> > > > >> > >> [hidden email]
>> > > > >> > >>> >> >:
>> > > > >> > >>> >> > >>>> >
>> > > > >> > >>> >> > >>>> >> What about the LineRank example? We had that in
>> > > Scala
>> > > > >> but
>> > > > >> > >> never
>> > > > >> > >>> >> > had a
>> > > > >> > >>> >> > >>>> >> Java Example.
>> > > > >> > >>> >> > >>>> >>
>> > > > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha
>> > Krettek <
>> > > > >> > >>> >> > [hidden email]>
>> > > > >> > >>> >> > >>>> >> wrote:
>> > > > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always
>> just
>> > > > >> copied
>> > > > >> > the
>> > > > >> > >> Java
>> > > > >> > >>> >> > >>>> ITCase.
>> > > > >> > >>> >> > >>>> >> >
>> > > > >> > >>> >> > >>>> >> > The only examples that are missing now are
>> > > > >> > >> LinearRegression and
>> > > > >> > >>> >> > the
>> > > > >> > >>> >> > >>>> >> > relational stuff.
>> > > > >> > >>> >> > >>>> >> >
>> > > > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian
>> Hueske
>> > <
>> > > > >> > >>> >> > [hidden email]>
>> > > > >> > >>> >> > >>>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees
>> > example.
>> > > > >> > >>> >> > >>>> >> >> That was a preprocessing step for the
>> > > > >> > >> TriangleEnumeration, and
>> > > > >> > >>> >> > is now
>> > > > >> > >>> >> > >>>> >> part
>> > > > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example.
>> > > > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one.
>> > As I
>> > > > >> said
>> > > > >> > >> before,
>> > > > >> > >>> >> > I'd
>> > > > >> > >>> >> > >>>> >> prefer to
>> > > > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync.
>> > > > >> > >>> >> > >>>> >> >>
>> > > > >> > >>> >> > >>>> >> >> Cheers, Fabian
>> > > > >> > >>> >> > >>>> >> >>
>> > > > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek
>> <
>> > > > >> > >>> >> > [hidden email]>:
>> > > > >> > >>> >> > >>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again
>> > > > fabian.
>> > > > >> :D
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> Regarding the other stuff:
>> > > > >> > >>> >> > >>>> >> >>>  - There is a comment in DataSet.scala
>> about
>> > > > >> including
>> > > > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the
>> > > > >> > >> TypeInformation.
>> > > > >> > >>> >> > >>>> >> >>>  - I added generateSequence to
>> > > > >> ExecutionEnvironment.
>> > > > >> > >>> >> > >>>> >> >>>  - It is possible to use Scala Primitives
>> in
>> > > > >> Array, I
>> > > > >> > >> noticed
>> > > > >> > >>> >> > it
>> > > > >> > >>> >> > >>>> while
>> > > > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an
>> older
>> > > > >> version
>> > > > >> > of
>> > > > >> > >> the
>> > > > >> > >>> >> > code.
>> > > > >> > >>> >> > >>>> >> >>>  - Yes, using List and other Interfaces is
>> > not
>> > > > >> > possible,
>> > > > >> > >> this
>> > > > >> > >>> >> > is
>> > > > >> > >>> >> > >>>> also
>> > > > >> > >>> >> > >>>> >> >>> a restriction in the Java API.
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> What do you think about the interface of
>> join
>> > > and
>> > > > >> > >> coGroup?
>> > > > >> > >>> >> > Right
>> > > > >> > >>> >> > >>>> now,
>> > > > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an
>> > > > Option
>> > > > >> or
>> > > > >> > the
>> > > > >> > >>> >> > lambda
>> > > > >> > >>> >> > >>>> with
>> > > > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have
>> > also
>> > > > >> have a
>> > > > >> > >> lambda
>> > > > >> > >>> >> > that
>> > > > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type
>> erasure
>> > > > this
>> > > > >> has
>> > > > >> > >> the
>> > > > >> > >>> >> > same type
>> > > > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't
>> > use
>> > > > it.
>> > > > >> > >> There is
>> > > > >> > >>> >> an
>> > > > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a
>> > > Collection,
>> > > > >> so I
>> > > > >> > >> could
>> > > > >> > >>> >> > change
>> > > > >> > >>> >> > >>>> it
>> > > > >> > >>> >> > >>>> >> >>> without breaking the examples we have now.
>> > What
>> > > > do
>> > > > >> you
>> > > > >> > >> think?
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans,
>> > > > >> > >>> >> ConnectedComponents,
>> > > > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive,
>> > > > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt,
>> > > > >> > >>> >> > >>>> >> >>> PageRank
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> These are the examples people called dibs
>> on:
>> > > > >> > >>> >> > >>>> >> >>>  - BatchGradientDescent (Márton) (Should
>> be a
>> > > > port
>> > > > >> of
>> > > > >> > >>> >> > >>>> LinearRegression
>> > > > >> > >>> >> > >>>> >> >>> Example from Java)
>> > > > >> > >>> >> > >>>> >> >>>  - ComputeEdgeDegrees (Hermann)
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken):
>> > > > >> > >>> >> > >>>> >> >>>  - The relational Stuff
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan
>> > Ewen <
>> > > > >> > >>> >> > [hidden email]>
>> > > > >> > >>> >> > >>>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery
>> > > > >> > >>> >> > >>>> >> >>> >
>> > > > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha
>> > > > >> Krettek <
>> > > > >> > >>> >> > >>>> >> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> > wrote:
>> > > > >> > >>> >> > >>>> >> >>> >
>> > > > >> > >>> >> > >>>> >> >>> >> By the way, what was called
>> > > > >> BatchGradientDescent in
>> > > > >> > >> the
>> > > > >> > >>> >> > Scala
>> > > > >> > >>> >> > >>>> >> examples
>> > > > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the
>> > > > >> > LinearRegression
>> > > > >> > >>> >> > Example from
>> > > > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate
>> examples
>> > > > >> earlier.
>> > > > >> > >>> >> > >>>> >> >>> >>
>> > > > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3.
>> > Any
>> > > > >> > thoughts
>> > > > >> > >>> >> about
>> > > > >> > >>> >> > >>>> removing
>> > > > >> > >>> >> > >>>> >> >>> >> RelationalQuery?
>> > > > >> > >>> >> > >>>> >> >>> >>
>> > > > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM,
>> Aljoscha
>> > > > >> Krettek
>> > > > >> > <
>> > > > >> > >>> >> > >>>> >> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >
>> > > > >> > >>> >> > >>>> >> >>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration
>> > Examples,
>> > > > >> thanks
>> > > > >> > >>> >> Fabian.
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount,
>> > KMeans,
>> > > > >> > >>> >> > ConnectedComponents,
>> > > > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis,
>> TransitiveClosureNaive,
>> > > > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> > These are the examples people called
>> > dibs
>> > > > on:
>> > > > >> > >>> >> > >>>> >> >>> >> >  - PageRank (Fabian)
>> > > > >> > >>> >> > >>>> >> >>> >> >  - BatchGradientDescent (Márton)
>> > > > >> > >>> >> > >>>> >> >>> >> >  - ComputeEdgeDegrees (Hermann)
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not
>> > mistaken):
>> > > > >> > >>> >> > >>>> >> >>> >> >  - The relational Stuff
>> > > > >> > >>> >> > >>>> >> >>> >> >  - LinearRegression
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM,
>> > Aljoscha
>> > > > >> > Krettek <
>> > > > >> > >>> >> > >>>> >> >>> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a
>> running
>> > > > list
>> > > > >> of
>> > > > >> > >>> >> > >>>> ported/unported
>> > > > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the
>> > > java
>> > > > >> > example
>> > > > >> > >>> >> > package to
>> > > > >> > >>> >> > >>>> >> >>> examples
>> > > > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done.
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is
>> > fine
>> > > > as
>> > > > >> it
>> > > > >> > is.
>> > > > >> > >>> >> Just
>> > > > >> > >>> >> > >>>> because
>> > > > >> > >>> >> > >>>> >> >>> Scala
>> > > > >> > >>> >> > >>>> >> >>> >> >> enables functional programming
>> doesn't
>> > > mean
>> > > > >> it's
>> > > > >> > >> always
>> > > > >> > >>> >> > the
>> > > > >> > >>> >> > >>>> best
>> > > > >> > >>> >> > >>>> >> >>> >> >> choice. :D
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount,
>> > KMeans,
>> > > > >> > >>> >> > ConnectedComponents,
>> > > > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis,
>> TransitiveClosureNaive
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called
>> > dibs
>> > > > on:
>> > > > >> > >>> >> > >>>> >> >>> >> >>  - TriangleEnumration and PageRank
>> > > (Fabian)
>> > > > >> > >>> >> > >>>> >> >>> >> >>  - BatchGradientDescent (Márton)
>> > > > >> > >>> >> > >>>> >> >>> >> >>  - ComputeEdgeDegrees (Hermann)
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not
>> > > mistaken):
>> > > > >> > >>> >> > >>>> >> >>> >> >>  - The relational Stuff
>> > > > >> > >>> >> > >>>> >> >>> >> >>  - LinearRegression
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> Cheers,
>> > > > >> > >>> >> > >>>> >> >>> >> >> Aljoscha
>> > > > >> > >>> >> > >>>> >> >>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM,
>> Kostas
>> > > > >> Tzoumas
>> > > > >> > <
>> > > > >> > >>> >> > >>>> >> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >
>> > > > >> > >>> >> > >>>> >> >>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also
>> added
>> > a
>> > > > >> > >> termination
>> > > > >> > >>> >> > criterion
>> > > > >> > >>> >> > >>>> >> in the
>> > > > >> > >>> >> > >>>> >> >>> >> Java
>> > > > >> > >>> >> > >>>> >> >>> >> >>> version:
>> > > > >> > >>> >> > >>>> >> >>> >>
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>
>> > > https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example
>> > > > >> > >>> >> > >>>> >> >>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination
>> > > > >> criterion
>> > > > >> > in
>> > > > >> > >>> >> Scala
>> > > > >> > >>> >> > more
>> > > > >> > >>> >> > >>>> >> >>> >> functional?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package
>> > name
>> > > > is
>> > > > >> > >>> >> > example.java but
>> > > > >> > >>> >> > >>>> >> >>> >> examples.scala
>> > > > >> > >>> >> > >>>> >> >>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>> Kostas
>> > > > >> > >>> >> > >>>> >> >>> >> >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM,
>> Kostas
>> > > > >> Tzoumas
>> > > > >> > <
>> > > > >> > >>> >> > >>>> >> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >
>> > > > >> > >>> >> > >>>> >> >>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and
>> > > > >> PiEstimation
>> > > > >> > >> (was not
>> > > > >> > >>> >> > on
>> > > > >> > >>> >> > >>>> your
>> > > > >> > >>> >> > >>>> >> >>> list).
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the
>> > relational
>> > > > >> stuff
>> > > > >> > I
>> > > > >> > >> can
>> > > > >> > >>> >> > take
>> > > > >> > >>> >> > >>>> those
>> > > > >> > >>> >> > >>>> >> as
>> > > > >> > >>> >> > >>>> >> >>> >> well.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the
>> > > "RelationalQuery"
>> > > > >> from
>> > > > >> > >> both
>> > > > >> > >>> >> > Scala and
>> > > > >> > >>> >> > >>>> >> Java?
>> > > > >> > >>> >> > >>>> >> >>> It
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of
>> TPC-H
>> > > Q3.
>> > > > >> Does
>> > > > >> > it
>> > > > >> > >> add
>> > > > >> > >>> >> > some
>> > > > >> > >>> >> > >>>> >> teaching
>> > > > >> > >>> >> > >>>> >> >>> >> value on
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> Kostas
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM,
>> > > Aljoscha
>> > > > >> > Krettek
>> > > > >> > >> <
>> > > > >> > >>> >> > >>>> >> >>> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an
>> > > > ITCase.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount,
>> > > > KMeans,
>> > > > >> > >>> >> > >>>> ConnectedComponents,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people
>> called
>> > > > dibs
>> > > > >> on:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TriangleEnumration and PageRank
>> > > > >> (Fabian)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - BatchGradientDescent (Márton)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - ComputeEdgeDegrees (Hermann)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not
>> > > > mistaken):
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - TransitiveClosure
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - The relational Stuff
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>  - LinearRegression
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM,
>> > Kostas
>> > > > >> > Tzoumas <
>> > > > >> > >>> >> > >>>> >> >>> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
>> > > > >> > >>> >> > >>>> >> >>> >>
>> > > > >> > >>> >> > >>>> >> >>>
>> > > > >> > >>> >> > >>>> >>
>> > > > >> > >>> >> > >>>>
>> > > > >> > >>> >> >
>> > > > >> > >>> >>
>> > > > >> > >>
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-scala
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM,
>> > > > Aljoscha
>> > > > >> > >> Krettek <
>> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents
>> > > > Example
>> > > > >> > from
>> > > > >> > >>> >> Vasia.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM,
>> > > > Fabian
>> > > > >> > >> Hueske <
>> > > > >> > >>> >> > >>>> >> >>> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks!
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00
>> > > Aljoscha
>> > > > >> > >> Krettek <
>> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive
>> decision.
>> > > :D
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at
>> KMeansData.java
>> > > and
>> > > > >> > >>> >> > KMeans.scala. I'm
>> > > > >> > >>> >> > >>>> >> storing
>> > > > >> > >>> >> > >>>> >> >>> >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object
>> > > arrays
>> > > > >> and
>> > > > >> > >> then
>> > > > >> > >>> >> > >>>> converting
>> > > > >> > >>> >> > >>>> >> it to
>> > > > >> > >>> >> > >>>> >> >>> >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala
>> > objects.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to
>> > > > >> equalTo to
>> > > > >> > >> make
>> > > > >> > >>> >> it
>> > > > >> > >>> >> > >>>> >> consistent
>> > > > >> > >>> >> > >>>> >> >>> >> with the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and
>> coGroup).
>> > > > There
>> > > > >> is
>> > > > >> > no
>> > > > >> > >> need
>> > > > >> > >>> >> > for a
>> > > > >> > >>> >> > >>>> >> >>> keyword,
>> > > > >> > >>> >> > >>>> >> >>> >> you
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> left.join(right).where(0).equalTo(1) {
>> > > > >> > >> (le,
>> > > > >> > >>> >> re)
>> > > > >> > >>> >> > =>
>> > > > >> > >>> >> > >>>> new
>> > > > >> > >>> >> > >>>> >> >>> >> MyResult(le,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> }
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07
>> > PM,
>> > > > >> Fabian
>> > > > >> > >> Hueske
>> > > > >> > >>> >> <
>> > > > >> > >>> >> > >>>> >> >>> >> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet
>> > issue,
>> > > I
>> > > > >> also
>> > > > >> > >> found
>> > > > >> > >>> >> an
>> > > > >> > >>> >> > >>>> >> >>> inconsistency
>> > > > >> > >>> >> > >>>> >> >>> >> with
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done
>> > as:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> ds1.join(ds2).where(...).equalTo(...)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala
>> > > this
>> > > > >> is:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > ds1.join(d2).where(...).isEqualTo(...)
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be
>> > renamed
>> > > to
>> > > > >> > >> equalTo(),
>> > > > >> > >>> >> > IMO.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and
>> > > coGroup?)
>> > > > >> > lacks
>> > > > >> > >> the
>> > > > >> > >>> >> > with()
>> > > > >> > >>> >> > >>>> >> method
>> > > > >> > >>> >> > >>>> >> >>> >> because
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with"
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should
>> > be
>> > > > >> offer
>> > > > >> > >>> >> something
>> > > > >> > >>> >> > >>>> similar
>> > > > >> > >>> >> > >>>> >> for
>> > > > >> > >>> >> > >>>> >> >>> >> Scala
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left,
>> > right)?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00
>> > > > Stephan
>> > > > >> > Ewen
>> > > > >> > >> <
>> > > > >> > >>> >> > >>>> >> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings,
>> > > Object[][]
>> > > > >> would
>> > > > >> > >> work
>> > > > >> > >>> >> as
>> > > > >> > >>> >> > well.
>> > > > >> > >>> >> > >>>> >> That
>> > > > >> > >>> >> > >>>> >> >>> is a
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a
>> Tuple.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could
>> > be
>> > > > >> stored
>> > > > >> > as
>> > > > >> > >> Java
>> > > > >> > >>> >> > or
>> > > > >> > >>> >> > >>>> Scala
>> > > > >> > >>> >> > >>>> >> >>> Tuples,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert
>> > > > between
>> > > > >> > the
>> > > > >> > >> two.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at
>> > 10:55
>> > > > AM,
>> > > > >> > Fabian
>> > > > >> > >>> >> > Hueske
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> <[hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the
>> same
>> > > > >> > problem...
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings
>> and
>> > > > >> parsing
>> > > > >> > >> them,
>> > > > >> > >>> >> but
>> > > > >> > >>> >> > >>>> using
>> > > > >> > >>> >> > >>>> >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based
>> on
>> > a
>> > > > >> > >>> >> FileInputFormat.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to
>> parse
>> > > the
>> > > > >> > Strings
>> > > > >> > >>> >> > >>>> manually...
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35
>> > GMT+02:00
>> > > > >> > Aljoscha
>> > > > >> > >>> >> Krettek
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > <[hidden email]>:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought.
>> > Maybe
>> > > we
>> > > > >> > should
>> > > > >> > >> just
>> > > > >> > >>> >> > change
>> > > > >> > >>> >> > >>>> >> all
>> > > > >> > >>> >> > >>>> >> >>> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> input
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and
>> use
>> > > CSV
>> > > > >> > input
>> > > > >> > >>> >> > formats in
>> > > > >> > >>> >> > >>>> all
>> > > > >> > >>> >> > >>>> >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> What
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014
>> at
>> > > 7:46
>> > > > >> AM,
>> > > > >> > >>> >> Aljoscha
>> > > > >> > >>> >> > >>>> Krettek
>> > > > >> > >>> >> > >>>> >> <
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> [hidden email]>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's
>> unfortunate
>> > > that
>> > > > >> the
>> > > > >> > >> data
>> > > > >> > >>> >> > types are
>> > > > >> > >>> >> > >>>> >> >>> >> incompatible.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what
>> > you
>> > > > >> > proposed:
>> > > > >> > >>> >> move
>> > > > >> > >>> >> > the
>> > > > >> > >>> >> > >>>> >> data to
>> > > > >> > >>> >> > >>>> >> >>> a
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> field
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the
>> > > > >> > >>> >> > getDefaultEdgeDataSet()
>> > > > >> > >>> >> > >>>> >> method in
>> > > > >> > >>> >> > >>>> >> >>> >> Scala.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying
>> > would
>> > > > >> > >> duplicate the
>> > > > >> > >>> >> > data
>> > > > >> > >>> >> > >>>> and
>> > > > >> > >>> >> > >>>> >> >>> make it
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> for
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync
>> in
>> > > the
>> > > > >> Java
>> > > > >> > and
>> > > > >> > >>> >> Scala
>> > > > >> > >>> >> > >>>> >> versions.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others
>> > > think?
>> > > > >> This
>> > > > >> > >> will
>> > > > >> > >>> >> > probably
>> > > > >> > >>> >> > >>>> >> occur
>> > > > >> > >>> >> > >>>> >> >>> in
>> > > > >> > >>> >> > >>>> >> >>> >> all
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014
>> > at
>> > > > >> 10:04
>> > > > >> > PM,
>> > > > >> > >>> >> > Vasiliki
>> > > > >> > >>> >> > >>>> >> Kalavri
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > <
>> > > > [hidden email]
>> > > > >> >
>> > > > >> > >> wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the
>> > > > >> Connected
>> > > > >> > >>> >> Components
>> > > > >> > >>> >> > >>>> >> example,
>> > > > >> > >>> >> > >>>> >> >>> but
>> > > > >> > >>> >> > >>>> >> >>> >> I am
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example
>> > > input
>> > > > >> data
>> > > > >> > >> from
>> > > > >> > >>> >> > >>>> >> java-examples.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the
>> > > > >> > ConnectedComponentsData
>> > > > >> > >>> >> class,
>> > > > >> > >>> >> > the
>> > > > >> > >>> >> > >>>> >> vertices
>> > > > >> > >>> >> > >>>> >> >>> >> and
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> data
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the
>> > > methods
>> > > > >> > >>> >> > >>>> >> getDefaultVertexDataSet()
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and
>> > > > >> getDefaultEdgeDataSet(),
>> > > > >> > >> which
>> > > > >> > >>> >> > take
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an
>> > > > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment
>> > > > >> > >>> >> > >>>> >> >>> as
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to
>> > provide
>> > > > >> public
>> > > > >> > >> static
>> > > > >> > >>> >> > fields
>> > > > >> > >>> >> > >>>> >> (like
>> > > > >> > >>> >> > >>>> >> >>> in
>> > > > >> > >>> >> > >>>> >> >>> >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this
>> > > > >> introduces a
>> > > > >> > >>> >> > conversion
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from
>> > > > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to
>> > > > >> > >>> >> > >>>> >> >>> Scala
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to
>> > > > >> scala.Long
>> > > > >> > >> and I
>> > > > >> > >>> >> > guess
>> > > > >> > >>> >> > >>>> this
>> > > > >> > >>> >> > >>>> >> is
>> > > > >> > >>> >> > >>>> >> >>> an
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example
>> (?).
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of
>> > > > >> course, to
>> > > > >> > >> copy
>> > > > >> > >>> >> the
>> > > > >> > >>> >> > >>>> example
>> > > > >> > >>> >> > >>>> >> >>> data
>> > > > >> > >>> >> > >>>> >> >>> >> in
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing
>> > something
>> > > > >> here?
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks!
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers,
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V.
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September
>> 2014
>> > > > 15:52,
>> > > > >> > >> Aljoscha
>> > > > >> > >>> >> > >>>> Krettek <
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> [hidden email]
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I
>> updated
>> > my
>> > > > >> repo:
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >>
>> > > > >> > >>> >> > >>>> >> >>> >>
>> > > > >> > >>> >> >
>> > > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>>
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a
>> > > working
>> > > > >> > >> WordCount
>> > > > >> > >>> >> > example.
>> > > > >> > >>> >> > >>>> >> It's
>> > > > >> > >>> >> > >>>> >> >>> >> pretty
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of
>> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example
>> > with
>> > > > >> some
>> > > > >> > >> fixups
>> > > > >> > >>> >> > for the
>> > > > >> > >>> >> > >>>> >> syntax
>> > > > >> > >>> >> > >>>> >> >>> and
>> > > > >> > >>> >> > >>>> >> >>> >> >>

[Stephan Ewen]

I think Aljoscha's suspicion is correct. Eclipse let's you also reference
test code from main code, so it seems not to separate code. An extra
scala-tests project is a good idea.

A workaround: call maven install on the Shell and close the scala projects
in Eclipse. The flink-tests project will then pick up the scala examples
from your m2 cache.