http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/Apache-Tinkerpop-Geode-Integration-tp9252p9622.html
I am not very familiar with Gremlin, but I remember a brainstorming session
It looked like it would work well. We remember we could even model
> Hi Dr. Fabian,
>
> thanks a lot for your answer!
>
>
> On 15 December 2015 at 15:42, Fabian Hueske <
[hidden email]> wrote:
>
> > Hi Vasia,
> >
> > I agree, Gremlin definitely looks like an interesting API for Flink.
> > I'm not sure how it relates to Gelly. I guess Gelly would (initially) be
> > more tightly integrated with the DataSet API whereas Gremlin would be a
> > connector for other languages. Any ideas on this?
> >
>
> The idea is to provide a FlinkGraphComputer which will use Gelly's
> iterations to compile the Gremlin query language to Flink.
> In my previous email, I linked to our discussion over at the Tinkerpop
> mailing list, where you can find more details on this. By adding the
> FlinkGraphComputer, we basically get any graph query language that compiles
> to the Gremlin VM for free.
>
>
> >
> > Another question would be whether the connector should to into Flink or
> > Tinkerpop. For example, the Spark, Giraph, and Neo4J connectors are all
> > included in Tinkerpop.
> > This should be discussed with the Tinkerpop community.
> >
> >
> I'm copying from the Tinkerpop mailing list thread (link for full thread in
> my previous email):
>
>
> *In the past, TinkerPop use to be a "dumping ground" for all
> implementations, but we decided for TinkerPop3 that we would only have
> "reference implementations" so users can play, system providers can learn,
> and ultimately, system providers would provide TinkerPop support in their
> distribution. As such, we would like to have FlinkGraphComputer distributed
> with Flink. If that sounds like something your project would be comfortable
> with, I think we can provide a JIRA/PR for FlinkGraphComputer (as well as
> any necessary documentation). We can start with a JIRA ticket to get things
> going. Thoughts?*
>
>
> This is why I brought the conversation over here, so I hear the opinions
> of the Flink community on this :)
>
>
>
> > Best, Fabian
> >
>
>
> -Vasia.
>
>
>
> >
> >
> > 2015-12-14 18:33 GMT+01:00 Vasiliki Kalavri <
[hidden email]>:
> >
> > > Ping squirrels! Any thoughts/opinions on this?
> > >
> > > On 9 December 2015 at 20:40, Vasiliki Kalavri <
>
[hidden email]
> > >
> > > wrote:
> > >
> > > > Hello squirrels,
> > > >
> > > > I have been discussing with the Apache Tinkerpop [1] community
> > regarding
> > > > an integration with Flink/Gelly.
> > > > You can read our discussion in [2].
> > > >
> > > > Tinkerpop has a graph traversal machine called Gremlin, which
> supports
> > > > many high-level graph processing languages and runs on top of
> different
> > > > systems (e.g. Giraph, Spark, Graph DBs). You can read more in this
> > great
> > > > blog post [3].
> > > >
> > > > The idea is to provide a FlinkGraphComputer implementation, which
> will
> > > add
> > > > Gremlin support to Flink.
> > > >
> > > > I believe Tinkerpop is a great project and I would love to see an
> > > > integration with Gelly.
> > > > Before we move forward, I would like your input!
> > > > To me, it seems that this addition would nicely fit in flink-contrib,
> > > > where we also have connectors to other projects.
> > > > If you agree, I will go ahead and open a JIRA about it.
> > > >
> > > > Thank you!
> > > > -Vasia.
> > > >
> > > > [1]:
https://tinkerpop.incubator.apache.org/> > > > [2]:
> > > >
> > >
> >
>
https://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201511.mbox/%3CCANva_A390L7g169r8Sn+ej1-yfKBUdLnd4Td6ATwnP0uzA--gA@...%3E> > > > [3]:
> > > >
> > >
> >
>
http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine> > > >
> > > > On 25 November 2015 at 16:54, Vasiliki Kalavri <
> > >
[hidden email]>
> > > > wrote:
> > > >
> > > >> Hi James,
> > > >>
> > > >> I've just subscribed to the Tinkerpop dev mailing list. Could you
> > please
> > > >> send a reply to the thread, so then I can reply to it?
> > > >> I'm not sure how I can reply to the thread otherwise...
> > > >> I also saw that there is a grafos.ml project thread. I could also
> > > >> provide some input there :)
> > > >>
> > > >> Thanks!
> > > >> -Vasia.
> > > >>
> > > >> On 25 November 2015 at 15:09, James Thornton <
> >
[hidden email]>
> > > >> wrote:
> > > >>
> > > >>> Hi Vasia -
> > > >>>
> > > >>> Yes, a FlinkGraphComputer should be a straight-forward first step.
> > > Also,
> > > >>> on
> > > >>> the Apache Tinkerpop dev mailing list, Marko thought it might be
> cool
> > > if
> > > >>> there was a "Graph API" similar to the "Table API" -- hooking in
> > > Gremlin
> > > >>> to
> > > >>> Flink's fluent API would give Flink users a full graph query
> > language.
> > > >>>
> > > >>> Stephen Mallette is a TinkerPop core contributor, and he has
> already
> > > >>> started working on a FlinkGraphComputer. There is a Flink/Tinkerpop
> > > >>> thread
> > > >>> on the TinkerPop dev list -- it would be great to have you part of
> > the
> > > >>> conversation there too as we work on the integration:
> > > >>>
> > > >>>
>
http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/> > > >>>
> > > >>> Thanks, Vasia.
> > > >>>
> > > >>> - James
> > > >>>
> > > >>>
> > > >>> On Mon, Nov 23, 2015 at 10:28 AM, Vasiliki Kalavri <
> > > >>>
[hidden email]> wrote:
> > > >>>
> > > >>> > Hi James,
> > > >>> >
> > > >>> > thank you for your e-mail and your interest in Flink :)
> > > >>> >
> > > >>> > I've recently taken a _quick_ look into Apache TinkerPop and I
> > think
> > > >>> it'd
> > > >>> > be very interesting to integrate with Flink/Gelly.
> > > >>> > Are you thinking about something like a Flink GraphComputer,
> > similar
> > > to
> > > >>> > Giraph and Spark GraphComputer's?
> > > >>> > I believe such an integration should be straight-forward to
> > > implement.
> > > >>> You
> > > >>> > can start by looking into Flink iteration operators [1] and Gelly
> > > >>> iteration
> > > >>> > abstractions [2].
> > > >>> >
> > > >>> > Regarding Apache Geode, I'm not familiar with project, but I'll
> try
> > > to
> > > >>> take
> > > >>> > a look in the following days!
> > > >>> >
> > > >>> > Cheers,
> > > >>> > -Vasia.
> > > >>> >
> > > >>> >
> > > >>> > [1]:
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#iteration-operators> > > >>> > [2]:
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
https://ci.apache.org/projects/flink/flink-docs-master/libs/gelly_guide.html#iterative-graph-processing> > > >>> >
> > > >>> >
> > > >>> > On 20 November 2015 at 08:32, James Thornton <
> > >
[hidden email]
> > > >>> >
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Hi -
> > > >>> > >
> > > >>> > > This is James Thornton (espeed) from the Apache Tinkerpop
> > project (
> > > >>> > >
http://tinkerpop.incubator.apache.org/).
> > > >>> > >
> > > >>> > > The Flink iterators should pair well with Gremlin's Graph
> > Traversal
> > > >>> > Machine
> > > >>> > > (
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine> > > >>> > > )
> > > >>> > > -- it would be good to coordinate on creating an integration.
> > > >>> > >
> > > >>> > > Also, Apache Geode made a splash today on HN (
> > > >>> > >
https://news.ycombinator.com/item?id=10596859) -- connecting
> > Geode
> > > >>> and
> > > >>> > > Flink would be killer. Here's the Geode/Spark connector for
> > > >>> refefference:
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
https://github.com/apache/incubator-geode/tree/develop/gemfire-spark-connector> > > >>> > >
> > > >>> > > - James
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >
> > >
> >
>