Graph API Questions

[Aljoscha Krettek-2]

Hi,
I guess this is mostly directed at Vasia, since you are the main
developer behind the graph API right now. (Correct me if I'm wrong).

Could you share some of the difficulties that you encountered while
developing the API and some of the stuff that bothers you. I want to
work on making our core API better for developing specialised APIs.

Cheers,
Aljoscha

[Kostas Tzoumas-2]

From what I have been "overhearing" from the Graph API development, some
hurdles were type extractor errors that could not be deciphered easily, and
the inability to bring back data easily back to the client.

But it would be great if Vasia could elaborate!

On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
> I guess this is mostly directed at Vasia, since you are the main
> developer behind the graph API right now. (Correct me if I'm wrong).
>
> Could you share some of the difficulties that you encountered while
> developing the API and some of the stuff that bothers you. I want to
> work on making our core API better for developing specialised APIs.
>
> Cheers,
> Aljoscha
>

[Vasiliki Kalavri]

Hello Aljoscha and everyone,

Andra and Daniel might also want to give input :)

- As Kostas already said, we've had some issues with the TypeExtractor, but
these were mostly due to our lack of understanding of how it works. It
would be great if we could document the functionality and provide some tips
on usage of different methods. For example, when to use createTypeInfo and
when getReturnTypes? When is it necessary to implement ResultTypeQueryable?
etc.

- An ugly/bothering issue is not being able to return a scalar value from
an operation and having to work with a single-value DataSet. I understand
this is not trivial to support and we are currently working around this
using broadcast variables.

- The lack of a count operator (which is work in progress, no?)

- Not being able to easily access the group key inside a reduce/coGroup
UDF. If I'm not mistaken, there was an issue/discussion on this as well.

If anything else comes to mind, I'll let you know!
Overall, I think we've had very few problems and nothing for which we
(Stephan) couldn't find a workaround :-)

Cheers,
-V.

PS: I believe we'll be able to make the Graph API (code name Gelly) PR very
very soon ^^

On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:

> From what I have been "overhearing" from the Graph API development, some
> hurdles were type extractor errors that could not be deciphered easily, and
> the inability to bring back data easily back to the client.
>
> But it would be great if Vasia could elaborate!
>
> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > Hi,
> > I guess this is mostly directed at Vasia, since you are the main
> > developer behind the graph API right now. (Correct me if I'm wrong).
> >
> > Could you share some of the difficulties that you encountered while
> > developing the API and some of the stuff that bothers you. I want to
> > work on making our core API better for developing specialised APIs.
> >
> > Cheers,
> > Aljoscha
> >
>

[Henry Saputra]

Vasia,

Would you mind if I assign the Graph API JIRA ticket [1] to you?

It is nice to have owner for the ticket.

- Henry

[1] https://issues.apache.org/jira/browse/FLINK-1201

On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
<[hidden email]> wrote:

> Hello Aljoscha and everyone,
>
> Andra and Daniel might also want to give input :)
>
> - As Kostas already said, we've had some issues with the TypeExtractor, but
> these were mostly due to our lack of understanding of how it works. It
> would be great if we could document the functionality and provide some tips
> on usage of different methods. For example, when to use createTypeInfo and
> when getReturnTypes? When is it necessary to implement ResultTypeQueryable?
> etc.
>
> - An ugly/bothering issue is not being able to return a scalar value from
> an operation and having to work with a single-value DataSet. I understand
> this is not trivial to support and we are currently working around this
> using broadcast variables.
>
> - The lack of a count operator (which is work in progress, no?)
>
> - Not being able to easily access the group key inside a reduce/coGroup
> UDF. If I'm not mistaken, there was an issue/discussion on this as well.
>
> If anything else comes to mind, I'll let you know!
> Overall, I think we've had very few problems and nothing for which we
> (Stephan) couldn't find a workaround :-)
>
> Cheers,
> -V.
>
> PS: I believe we'll be able to make the Graph API (code name Gelly) PR very
> very soon ^^
>
> On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:
>
>> From what I have been "overhearing" from the Graph API development, some
>> hurdles were type extractor errors that could not be deciphered easily, and
>> the inability to bring back data easily back to the client.
>>
>> But it would be great if Vasia could elaborate!
>>
>> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>> > Hi,
>> > I guess this is mostly directed at Vasia, since you are the main
>> > developer behind the graph API right now. (Correct me if I'm wrong).
>> >
>> > Could you share some of the difficulties that you encountered while
>> > developing the API and some of the stuff that bothers you. I want to
>> > work on making our core API better for developing specialised APIs.
>> >
>> > Cheers,
>> > Aljoscha
>> >
>>

[Vasiliki Kalavri]

Assigned :-)

On 6 January 2015 at 21:20, Henry Saputra <[hidden email]> wrote:

> Vasia,
>
> Would you mind if I assign the Graph API JIRA ticket [1] to you?
>
> It is nice to have owner for the ticket.
>
> - Henry
>
> [1] https://issues.apache.org/jira/browse/FLINK-1201
>
> On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> <[hidden email]> wrote:
> > Hello Aljoscha and everyone,
> >
> > Andra and Daniel might also want to give input :)
> >
> > - As Kostas already said, we've had some issues with the TypeExtractor,
> but
> > these were mostly due to our lack of understanding of how it works. It
> > would be great if we could document the functionality and provide some
> tips
> > on usage of different methods. For example, when to use createTypeInfo
> and
> > when getReturnTypes? When is it necessary to implement
> ResultTypeQueryable?
> > etc.
> >
> > - An ugly/bothering issue is not being able to return a scalar value from
> > an operation and having to work with a single-value DataSet. I understand
> > this is not trivial to support and we are currently working around this
> > using broadcast variables.
> >
> > - The lack of a count operator (which is work in progress, no?)
> >
> > - Not being able to easily access the group key inside a reduce/coGroup
> > UDF. If I'm not mistaken, there was an issue/discussion on this as well.
> >
> > If anything else comes to mind, I'll let you know!
> > Overall, I think we've had very few problems and nothing for which we
> > (Stephan) couldn't find a workaround :-)
> >
> > Cheers,
> > -V.
> >
> > PS: I believe we'll be able to make the Graph API (code name Gelly) PR
> very
> > very soon ^^
> >
> > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:
> >
> >> From what I have been "overhearing" from the Graph API development, some
> >> hurdles were type extractor errors that could not be deciphered easily,
> and
> >> the inability to bring back data easily back to the client.
> >>
> >> But it would be great if Vasia could elaborate!
> >>
> >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
> >> wrote:
> >>
> >> > Hi,
> >> > I guess this is mostly directed at Vasia, since you are the main
> >> > developer behind the graph API right now. (Correct me if I'm wrong).
> >> >
> >> > Could you share some of the difficulties that you encountered while
> >> > developing the API and some of the stuff that bothers you. I want to
> >> > work on making our core API better for developing specialised APIs.
> >> >
> >> > Cheers,
> >> > Aljoscha
> >> >
> >>
>

[Henry Saputra]

Cool, thanks!

On Tue, Jan 6, 2015 at 12:34 PM, Vasiliki Kalavri
<[hidden email]> wrote:

> Assigned :-)
>
> On 6 January 2015 at 21:20, Henry Saputra <[hidden email]> wrote:
>
>> Vasia,
>>
>> Would you mind if I assign the Graph API JIRA ticket [1] to you?
>>
>> It is nice to have owner for the ticket.
>>
>> - Henry
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-1201
>>
>> On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
>> <[hidden email]> wrote:
>> > Hello Aljoscha and everyone,
>> >
>> > Andra and Daniel might also want to give input :)
>> >
>> > - As Kostas already said, we've had some issues with the TypeExtractor,
>> but
>> > these were mostly due to our lack of understanding of how it works. It
>> > would be great if we could document the functionality and provide some
>> tips
>> > on usage of different methods. For example, when to use createTypeInfo
>> and
>> > when getReturnTypes? When is it necessary to implement
>> ResultTypeQueryable?
>> > etc.
>> >
>> > - An ugly/bothering issue is not being able to return a scalar value from
>> > an operation and having to work with a single-value DataSet. I understand
>> > this is not trivial to support and we are currently working around this
>> > using broadcast variables.
>> >
>> > - The lack of a count operator (which is work in progress, no?)
>> >
>> > - Not being able to easily access the group key inside a reduce/coGroup
>> > UDF. If I'm not mistaken, there was an issue/discussion on this as well.
>> >
>> > If anything else comes to mind, I'll let you know!
>> > Overall, I think we've had very few problems and nothing for which we
>> > (Stephan) couldn't find a workaround :-)
>> >
>> > Cheers,
>> > -V.
>> >
>> > PS: I believe we'll be able to make the Graph API (code name Gelly) PR
>> very
>> > very soon ^^
>> >
>> > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:
>> >
>> >> From what I have been "overhearing" from the Graph API development, some
>> >> hurdles were type extractor errors that could not be deciphered easily,
>> and
>> >> the inability to bring back data easily back to the client.
>> >>
>> >> But it would be great if Vasia could elaborate!
>> >>
>> >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
>> >> wrote:
>> >>
>> >> > Hi,
>> >> > I guess this is mostly directed at Vasia, since you are the main
>> >> > developer behind the graph API right now. (Correct me if I'm wrong).
>> >> >
>> >> > Could you share some of the difficulties that you encountered while
>> >> > developing the API and some of the stuff that bothers you. I want to
>> >> > work on making our core API better for developing specialised APIs.
>> >> >
>> >> > Cheers,
>> >> > Aljoscha
>> >> >
>> >>
>>

[Henry Saputra]

Just curious, is the new Graph APIs suppose to replace flink-spargel?

- Henry

On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
<[hidden email]> wrote:

> Hello Aljoscha and everyone,
>
> Andra and Daniel might also want to give input :)
>
> - As Kostas already said, we've had some issues with the TypeExtractor, but
> these were mostly due to our lack of understanding of how it works. It
> would be great if we could document the functionality and provide some tips
> on usage of different methods. For example, when to use createTypeInfo and
> when getReturnTypes? When is it necessary to implement ResultTypeQueryable?
> etc.
>
> - An ugly/bothering issue is not being able to return a scalar value from
> an operation and having to work with a single-value DataSet. I understand
> this is not trivial to support and we are currently working around this
> using broadcast variables.
>
> - The lack of a count operator (which is work in progress, no?)
>
> - Not being able to easily access the group key inside a reduce/coGroup
> UDF. If I'm not mistaken, there was an issue/discussion on this as well.
>
> If anything else comes to mind, I'll let you know!
> Overall, I think we've had very few problems and nothing for which we
> (Stephan) couldn't find a workaround :-)
>
> Cheers,
> -V.
>
> PS: I believe we'll be able to make the Graph API (code name Gelly) PR very
> very soon ^^
>
> On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:
>
>> From what I have been "overhearing" from the Graph API development, some
>> hurdles were type extractor errors that could not be deciphered easily, and
>> the inability to bring back data easily back to the client.
>>
>> But it would be great if Vasia could elaborate!
>>
>> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>> > Hi,
>> > I guess this is mostly directed at Vasia, since you are the main
>> > developer behind the graph API right now. (Correct me if I'm wrong).
>> >
>> > Could you share some of the difficulties that you encountered while
>> > developing the API and some of the stuff that bothers you. I want to
>> > work on making our core API better for developing specialised APIs.
>> >
>> > Cheers,
>> > Aljoscha
>> >
>>

[Vasiliki Kalavri]

Spargel will be included in the graph API.
We are already using spargel, but we will move its classes to the graph
API, change them to work with Vertex and Edge types and implement some
extensions. This is the last big change we need to make and then we can
include it in flink-addons :-)

-V.

On 6 January 2015 at 22:09, Henry Saputra <[hidden email]> wrote:

> Just curious, is the new Graph APIs suppose to replace flink-spargel?
>
> - Henry
>
> On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> <[hidden email]> wrote:
> > Hello Aljoscha and everyone,
> >
> > Andra and Daniel might also want to give input :)
> >
> > - As Kostas already said, we've had some issues with the TypeExtractor,
> but
> > these were mostly due to our lack of understanding of how it works. It
> > would be great if we could document the functionality and provide some
> tips
> > on usage of different methods. For example, when to use createTypeInfo
> and
> > when getReturnTypes? When is it necessary to implement
> ResultTypeQueryable?
> > etc.
> >
> > - An ugly/bothering issue is not being able to return a scalar value from
> > an operation and having to work with a single-value DataSet. I understand
> > this is not trivial to support and we are currently working around this
> > using broadcast variables.
> >
> > - The lack of a count operator (which is work in progress, no?)
> >
> > - Not being able to easily access the group key inside a reduce/coGroup
> > UDF. If I'm not mistaken, there was an issue/discussion on this as well.
> >
> > If anything else comes to mind, I'll let you know!
> > Overall, I think we've had very few problems and nothing for which we
> > (Stephan) couldn't find a workaround :-)
> >
> > Cheers,
> > -V.
> >
> > PS: I believe we'll be able to make the Graph API (code name Gelly) PR
> very
> > very soon ^^
> >
> > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]> wrote:
> >
> >> From what I have been "overhearing" from the Graph API development, some
> >> hurdles were type extractor errors that could not be deciphered easily,
> and
> >> the inability to bring back data easily back to the client.
> >>
> >> But it would be great if Vasia could elaborate!
> >>
> >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <[hidden email]>
> >> wrote:
> >>
> >> > Hi,
> >> > I guess this is mostly directed at Vasia, since you are the main
> >> > developer behind the graph API right now. (Correct me if I'm wrong).
> >> >
> >> > Could you share some of the difficulties that you encountered while
> >> > developing the API and some of the stuff that bothers you. I want to
> >> > work on making our core API better for developing specialised APIs.
> >> >
> >> > Cheers,
> >> > Aljoscha
> >> >
> >>
>

[Kostas Tzoumas-2]

Perhaps we should have both code paths (Spargel and Spargel inside Graph
API) in the code for a while to enable current Spargel users to migrate
their applications before the breaking change.

This means that you have Spargel inside Graph API as you plan to, but you
do not remove the current Spargel classes in your PR.

What do you think?

Kostas



On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <[hidden email]
> wrote:

> Spargel will be included in the graph API.
> We are already using spargel, but we will move its classes to the graph
> API, change them to work with Vertex and Edge types and implement some
> extensions. This is the last big change we need to make and then we can
> include it in flink-addons :-)
>
> -V.
>
> On 6 January 2015 at 22:09, Henry Saputra <[hidden email]> wrote:
>
> > Just curious, is the new Graph APIs suppose to replace flink-spargel?
> >
> > - Henry
> >
> > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > <[hidden email]> wrote:
> > > Hello Aljoscha and everyone,
> > >
> > > Andra and Daniel might also want to give input :)
> > >
> > > - As Kostas already said, we've had some issues with the TypeExtractor,
> > but
> > > these were mostly due to our lack of understanding of how it works. It
> > > would be great if we could document the functionality and provide some
> > tips
> > > on usage of different methods. For example, when to use createTypeInfo
> > and
> > > when getReturnTypes? When is it necessary to implement
> > ResultTypeQueryable?
> > > etc.
> > >
> > > - An ugly/bothering issue is not being able to return a scalar value
> from
> > > an operation and having to work with a single-value DataSet. I
> understand
> > > this is not trivial to support and we are currently working around this
> > > using broadcast variables.
> > >
> > > - The lack of a count operator (which is work in progress, no?)
> > >
> > > - Not being able to easily access the group key inside a reduce/coGroup
> > > UDF. If I'm not mistaken, there was an issue/discussion on this as
> well.
> > >
> > > If anything else comes to mind, I'll let you know!
> > > Overall, I think we've had very few problems and nothing for which we
> > > (Stephan) couldn't find a workaround :-)
> > >
> > > Cheers,
> > > -V.
> > >
> > > PS: I believe we'll be able to make the Graph API (code name Gelly) PR
> > very
> > > very soon ^^
> > >
> > > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]>
> wrote:
> > >
> > >> From what I have been "overhearing" from the Graph API development,
> some
> > >> hurdles were type extractor errors that could not be deciphered
> easily,
> > and
> > >> the inability to bring back data easily back to the client.
> > >>
> > >> But it would be great if Vasia could elaborate!
> > >>
> > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> [hidden email]>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> > I guess this is mostly directed at Vasia, since you are the main
> > >> > developer behind the graph API right now. (Correct me if I'm wrong).
> > >> >
> > >> > Could you share some of the difficulties that you encountered while
> > >> > developing the API and some of the stuff that bothers you. I want to
> > >> > work on making our core API better for developing specialised APIs.
> > >> >
> > >> > Cheers,
> > >> > Aljoscha
> > >> >
> > >>
> >
>

[Vasiliki Kalavri]

+1 makes sense :)

On 7 January 2015 at 09:51, Kostas Tzoumas <[hidden email]> wrote:

> Perhaps we should have both code paths (Spargel and Spargel inside Graph
> API) in the code for a while to enable current Spargel users to migrate
> their applications before the breaking change.
>
> This means that you have Spargel inside Graph API as you plan to, but you
> do not remove the current Spargel classes in your PR.
>
> What do you think?
>
> Kostas
>
>
>
> On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <
> [hidden email]
> > wrote:
>
> > Spargel will be included in the graph API.
> > We are already using spargel, but we will move its classes to the graph
> > API, change them to work with Vertex and Edge types and implement some
> > extensions. This is the last big change we need to make and then we can
> > include it in flink-addons :-)
> >
> > -V.
> >
> > On 6 January 2015 at 22:09, Henry Saputra <[hidden email]>
> wrote:
> >
> > > Just curious, is the new Graph APIs suppose to replace flink-spargel?
> > >
> > > - Henry
> > >
> > > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > > <[hidden email]> wrote:
> > > > Hello Aljoscha and everyone,
> > > >
> > > > Andra and Daniel might also want to give input :)
> > > >
> > > > - As Kostas already said, we've had some issues with the
> TypeExtractor,
> > > but
> > > > these were mostly due to our lack of understanding of how it works.
> It
> > > > would be great if we could document the functionality and provide
> some
> > > tips
> > > > on usage of different methods. For example, when to use
> createTypeInfo
> > > and
> > > > when getReturnTypes? When is it necessary to implement
> > > ResultTypeQueryable?
> > > > etc.
> > > >
> > > > - An ugly/bothering issue is not being able to return a scalar value
> > from
> > > > an operation and having to work with a single-value DataSet. I
> > understand
> > > > this is not trivial to support and we are currently working around
> this
> > > > using broadcast variables.
> > > >
> > > > - The lack of a count operator (which is work in progress, no?)
> > > >
> > > > - Not being able to easily access the group key inside a
> reduce/coGroup
> > > > UDF. If I'm not mistaken, there was an issue/discussion on this as
> > well.
> > > >
> > > > If anything else comes to mind, I'll let you know!
> > > > Overall, I think we've had very few problems and nothing for which we
> > > > (Stephan) couldn't find a workaround :-)
> > > >
> > > > Cheers,
> > > > -V.
> > > >
> > > > PS: I believe we'll be able to make the Graph API (code name Gelly)
> PR
> > > very
> > > > very soon ^^
> > > >
> > > > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]>
> > wrote:
> > > >
> > > >> From what I have been "overhearing" from the Graph API development,
> > some
> > > >> hurdles were type extractor errors that could not be deciphered
> > easily,
> > > and
> > > >> the inability to bring back data easily back to the client.
> > > >>
> > > >> But it would be great if Vasia could elaborate!
> > > >>
> > > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> > [hidden email]>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> > I guess this is mostly directed at Vasia, since you are the main
> > > >> > developer behind the graph API right now. (Correct me if I'm
> wrong).
> > > >> >
> > > >> > Could you share some of the difficulties that you encountered
> while
> > > >> > developing the API and some of the stuff that bothers you. I want
> to
> > > >> > work on making our core API better for developing specialised
> APIs.
> > > >> >
> > > >> > Cheers,
> > > >> > Aljoscha
> > > >> >
> > > >>
> > >
> >
>

[Stephan Ewen]

+1 as well
Am 07.01.2015 10:04 schrieb "Vasiliki Kalavri" <[hidden email]>:

> +1 makes sense :)
>
> On 7 January 2015 at 09:51, Kostas Tzoumas <[hidden email]> wrote:
>
> > Perhaps we should have both code paths (Spargel and Spargel inside Graph
> > API) in the code for a while to enable current Spargel users to migrate
> > their applications before the breaking change.
> >
> > This means that you have Spargel inside Graph API as you plan to, but you
> > do not remove the current Spargel classes in your PR.
> >
> > What do you think?
> >
> > Kostas
> >
> >
> >
> > On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <
> > [hidden email]
> > > wrote:
> >
> > > Spargel will be included in the graph API.
> > > We are already using spargel, but we will move its classes to the graph
> > > API, change them to work with Vertex and Edge types and implement some
> > > extensions. This is the last big change we need to make and then we can
> > > include it in flink-addons :-)
> > >
> > > -V.
> > >
> > > On 6 January 2015 at 22:09, Henry Saputra <[hidden email]>
> > wrote:
> > >
> > > > Just curious, is the new Graph APIs suppose to replace flink-spargel?
> > > >
> > > > - Henry
> > > >
> > > > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > > > <[hidden email]> wrote:
> > > > > Hello Aljoscha and everyone,
> > > > >
> > > > > Andra and Daniel might also want to give input :)
> > > > >
> > > > > - As Kostas already said, we've had some issues with the
> > TypeExtractor,
> > > > but
> > > > > these were mostly due to our lack of understanding of how it works.
> > It
> > > > > would be great if we could document the functionality and provide
> > some
> > > > tips
> > > > > on usage of different methods. For example, when to use
> > createTypeInfo
> > > > and
> > > > > when getReturnTypes? When is it necessary to implement
> > > > ResultTypeQueryable?
> > > > > etc.
> > > > >
> > > > > - An ugly/bothering issue is not being able to return a scalar
> value
> > > from
> > > > > an operation and having to work with a single-value DataSet. I
> > > understand
> > > > > this is not trivial to support and we are currently working around
> > this
> > > > > using broadcast variables.
> > > > >
> > > > > - The lack of a count operator (which is work in progress, no?)
> > > > >
> > > > > - Not being able to easily access the group key inside a
> > reduce/coGroup
> > > > > UDF. If I'm not mistaken, there was an issue/discussion on this as
> > > well.
> > > > >
> > > > > If anything else comes to mind, I'll let you know!
> > > > > Overall, I think we've had very few problems and nothing for which
> we
> > > > > (Stephan) couldn't find a workaround :-)
> > > > >
> > > > > Cheers,
> > > > > -V.
> > > > >
> > > > > PS: I believe we'll be able to make the Graph API (code name Gelly)
> > PR
> > > > very
> > > > > very soon ^^
> > > > >
> > > > > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]>
> > > wrote:
> > > > >
> > > > >> From what I have been "overhearing" from the Graph API
> development,
> > > some
> > > > >> hurdles were type extractor errors that could not be deciphered
> > > easily,
> > > > and
> > > > >> the inability to bring back data easily back to the client.
> > > > >>
> > > > >> But it would be great if Vasia could elaborate!
> > > > >>
> > > > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> > > [hidden email]>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> > I guess this is mostly directed at Vasia, since you are the main
> > > > >> > developer behind the graph API right now. (Correct me if I'm
> > wrong).
> > > > >> >
> > > > >> > Could you share some of the difficulties that you encountered
> > while
> > > > >> > developing the API and some of the stuff that bothers you. I
> want
> > to
> > > > >> > work on making our core API better for developing specialised
> > APIs.
> > > > >> >
> > > > >> > Cheers,
> > > > >> > Aljoscha
> > > > >> >
> > > > >>
> > > >
> > >
> >
>

[Márton Balassi]

+1

On Wed, Jan 7, 2015 at 10:12 AM, Stephan Ewen <[hidden email]> wrote:

> +1 as well
> Am 07.01.2015 10:04 schrieb "Vasiliki Kalavri" <[hidden email]
> >:
>
> > +1 makes sense :)
> >
> > On 7 January 2015 at 09:51, Kostas Tzoumas <[hidden email]> wrote:
> >
> > > Perhaps we should have both code paths (Spargel and Spargel inside
> Graph
> > > API) in the code for a while to enable current Spargel users to migrate
> > > their applications before the breaking change.
> > >
> > > This means that you have Spargel inside Graph API as you plan to, but
> you
> > > do not remove the current Spargel classes in your PR.
> > >
> > > What do you think?
> > >
> > > Kostas
> > >
> > >
> > >
> > > On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <
> > > [hidden email]
> > > > wrote:
> > >
> > > > Spargel will be included in the graph API.
> > > > We are already using spargel, but we will move its classes to the
> graph
> > > > API, change them to work with Vertex and Edge types and implement
> some
> > > > extensions. This is the last big change we need to make and then we
> can
> > > > include it in flink-addons :-)
> > > >
> > > > -V.
> > > >
> > > > On 6 January 2015 at 22:09, Henry Saputra <[hidden email]>
> > > wrote:
> > > >
> > > > > Just curious, is the new Graph APIs suppose to replace
> flink-spargel?
> > > > >
> > > > > - Henry
> > > > >
> > > > > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > > > > <[hidden email]> wrote:
> > > > > > Hello Aljoscha and everyone,
> > > > > >
> > > > > > Andra and Daniel might also want to give input :)
> > > > > >
> > > > > > - As Kostas already said, we've had some issues with the
> > > TypeExtractor,
> > > > > but
> > > > > > these were mostly due to our lack of understanding of how it
> works.
> > > It
> > > > > > would be great if we could document the functionality and provide
> > > some
> > > > > tips
> > > > > > on usage of different methods. For example, when to use
> > > createTypeInfo
> > > > > and
> > > > > > when getReturnTypes? When is it necessary to implement
> > > > > ResultTypeQueryable?
> > > > > > etc.
> > > > > >
> > > > > > - An ugly/bothering issue is not being able to return a scalar
> > value
> > > > from
> > > > > > an operation and having to work with a single-value DataSet. I
> > > > understand
> > > > > > this is not trivial to support and we are currently working
> around
> > > this
> > > > > > using broadcast variables.
> > > > > >
> > > > > > - The lack of a count operator (which is work in progress, no?)
> > > > > >
> > > > > > - Not being able to easily access the group key inside a
> > > reduce/coGroup
> > > > > > UDF. If I'm not mistaken, there was an issue/discussion on this
> as
> > > > well.
> > > > > >
> > > > > > If anything else comes to mind, I'll let you know!
> > > > > > Overall, I think we've had very few problems and nothing for
> which
> > we
> > > > > > (Stephan) couldn't find a workaround :-)
> > > > > >
> > > > > > Cheers,
> > > > > > -V.
> > > > > >
> > > > > > PS: I believe we'll be able to make the Graph API (code name
> Gelly)
> > > PR
> > > > > very
> > > > > > very soon ^^
> > > > > >
> > > > > > On 6 January 2015 at 11:24, Kostas Tzoumas <[hidden email]>
> > > > wrote:
> > > > > >
> > > > > >> From what I have been "overhearing" from the Graph API
> > development,
> > > > some
> > > > > >> hurdles were type extractor errors that could not be deciphered
> > > > easily,
> > > > > and
> > > > > >> the inability to bring back data easily back to the client.
> > > > > >>
> > > > > >> But it would be great if Vasia could elaborate!
> > > > > >>
> > > > > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> > > > [hidden email]>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi,
> > > > > >> > I guess this is mostly directed at Vasia, since you are the
> main
> > > > > >> > developer behind the graph API right now. (Correct me if I'm
> > > wrong).
> > > > > >> >
> > > > > >> > Could you share some of the difficulties that you encountered
> > > while
> > > > > >> > developing the API and some of the stuff that bothers you. I
> > want
> > > to
> > > > > >> > work on making our core API better for developing specialised
> > > APIs.
> > > > > >> >
> > > > > >> > Cheers,
> > > > > >> > Aljoscha
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>