Question about StreamGraph

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about StreamGraph

Aljoscha Krettek-2
Hi,
why do we have the StreamGraph where the vertices, edges and all the
other information about the operators is stored in separate Maps? Why
can we not use a graph of isolated nodes that contain all required
information? To me this system seems quite fragile and unnecessarily
complex.

Again, this might be a stupid question and I'm probably overlooking
something again. :D

Cheers,
Aljoscha
Reply | Threaded
Open this post in threaded view
|

Re: Question about StreamGraph

Gyula Fóra
Hey,

You are right, it would probably be better that way. Adding a Map<Id, Node>
and then get the properties from the Node object.

If you feel like doing some refactoring, go right ahead :D Otherwise could
you please open a JIRA for this?

Gyula

On Fri, Apr 10, 2015 at 4:56 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
> why do we have the StreamGraph where the vertices, edges and all the
> other information about the operators is stored in separate Maps? Why
> can we not use a graph of isolated nodes that contain all required
> information? To me this system seems quite fragile and unnecessarily
> complex.
>
> Again, this might be a stupid question and I'm probably overlooking
> something again. :D
>
> Cheers,
> Aljoscha
>
Reply | Threaded
Open this post in threaded view
|

Re: Question about StreamGraph

Aljoscha Krettek-2
I'm thinking about it. But why a Map of Nodes, simply having a graph
of node objects would be sufficient. In fact, the Operators themselves
are already the graph the describes a job, right?

On Fri, Apr 10, 2015 at 5:38 PM, Gyula Fóra <[hidden email]> wrote:

> Hey,
>
> You are right, it would probably be better that way. Adding a Map<Id, Node>
> and then get the properties from the Node object.
>
> If you feel like doing some refactoring, go right ahead :D Otherwise could
> you please open a JIRA for this?
>
> Gyula
>
> On Fri, Apr 10, 2015 at 4:56 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> Hi,
>> why do we have the StreamGraph where the vertices, edges and all the
>> other information about the operators is stored in separate Maps? Why
>> can we not use a graph of isolated nodes that contain all required
>> information? To me this system seems quite fragile and unnecessarily
>> complex.
>>
>> Again, this might be a stupid question and I'm probably overlooking
>> something again. :D
>>
>> Cheers,
>> Aljoscha
>>
Reply | Threaded
Open this post in threaded view
|

Re: Question about StreamGraph

Gyula Fóra
Hey,

Unlike the batch case the streaming operators don't describe the graph.
They don't know their input nodes etc. Maybe we can rework that at some
point as well. We used to build the job graph as we added the operators
incrementally that's why it's like that.

So now the stream graph has the graph structure and some other
properties only.

Gyula

On Friday, April 10, 2015, Aljoscha Krettek <[hidden email]> wrote:

> I'm thinking about it. But why a Map of Nodes, simply having a graph
> of node objects would be sufficient. In fact, the Operators themselves
> are already the graph the describes a job, right?
>
> On Fri, Apr 10, 2015 at 5:38 PM, Gyula Fóra <[hidden email]
> <javascript:;>> wrote:
> > Hey,
> >
> > You are right, it would probably be better that way. Adding a Map<Id,
> Node>
> > and then get the properties from the Node object.
> >
> > If you feel like doing some refactoring, go right ahead :D Otherwise
> could
> > you please open a JIRA for this?
> >
> > Gyula
> >
> > On Fri, Apr 10, 2015 at 4:56 PM, Aljoscha Krettek <[hidden email]
> <javascript:;>>
> > wrote:
> >
> >> Hi,
> >> why do we have the StreamGraph where the vertices, edges and all the
> >> other information about the operators is stored in separate Maps? Why
> >> can we not use a graph of isolated nodes that contain all required
> >> information? To me this system seems quite fragile and unnecessarily
> >> complex.
> >>
> >> Again, this might be a stupid question and I'm probably overlooking
> >> something again. :D
> >>
> >> Cheers,
> >> Aljoscha
> >>
>