Fwd: [stratosphere-users] FlatJoin implementation

Posted by Robert Metzger on Jun 18, 2014; 11:30am
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/Fwd-stratosphere-users-FlatJoin-implementation-tp394.html

forwarding to the new mailing list ...

---------- Forwarded message ----------
From: Fabian Hueske <[hidden email]>
Date: Wed, Jun 18, 2014 at 11:20 AM
Subject: Re: [stratosphere-users] FlatJoin implementation
To: "[hidden email]" <
[hidden email]>


Hi Asterios,

this mailing list is no longer used.
All mails (user + dev) should go to the [hidden email]
mailing list.

To the topic: This feature has been requested by quite a few people. So I
think it makes sense to provide this interface (plus joinFilter). The same
applies to Cross which is less often used though...
A less confusing workaround could be to use join.project() an flatMap.

Cheers, Fabian


2014-06-18 11:10 GMT+02:00 Asterios Katsifodimos <[hidden email]>:

Hi,

>
> I've noticed that the join implementation supports a collector in the
> sense that it can become a "flat" join.
>
> The result is that we have to implement something like this that is kind
> of ugly:
> public static final class VertexComponentIDProjectorWithFilter extends
> JoinFunction<Tuple2<Long, Long>, Tuple2<Long, Long>, Tuple2<Long, Long>>{
>  @Override
> public void join(Tuple2<Long, Long> first, Tuple2<Long, Long> second,
> Collector<Tuple2<Long, Long>> out) throws Exception {
>  if(first.f1 < second.f1){
>  out.collect(new Tuple2<Long,Long>(first.f0, first.f1));
> }
>  else{
> out.collect(second);
>  }
> }
>
> @Override
> public Tuple2<Long, Long> join(Tuple2<Long, Long> first,
>  Tuple2<Long, Long> second) throws Exception {
> return null;
>  }
>  }
>
>
> A first comment on the above code is that the developer has to provide a
> null-returning, non-collector default join function. This makes code ugly
> and introduces a confusion: which of the two is going to be actually
> executed? Shouldn't there be a "Flatjoin" operator that would be
> "semantically correct"? Or would it complicate developer's life?
>
> Cheers,
> Asterios
>
> --
> You received this message because you are subscribed to the Google Groups
> "stratosphere-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> Visit this group at http://groups.google.com/group/stratosphere-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/stratosphere-users/7b905c5a-7ec6-4e9d-a284-5b6ae3b0b977%40googlegroups.com
> <https://groups.google.com/d/msgid/stratosphere-users/7b905c5a-7ec6-4e9d-a284-5b6ae3b0b977%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

 --
You received this message because you are subscribed to the Google Groups
"stratosphere-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [hidden email].
Visit this group at http://groups.google.com/group/stratosphere-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/stratosphere-users/CAAdrtT2t7iKUbDxswFND9VJKv0sF%3Da1C6s_0Whojo4Bmtv94Vg%40mail.gmail.com
<https://groups.google.com/d/msgid/stratosphere-users/CAAdrtT2t7iKUbDxswFND9VJKv0sF%3Da1C6s_0Whojo4Bmtv94Vg%40mail.gmail.com?utm_medium=email&utm_source=footer>
.

For more options, visit https://groups.google.com/d/optout.