Hi all,
one of the operations we want to implement for the flink graph api is mapVertices, i.e. applying a mapper to the vertices of the graph. The current implementation assumes that the vertex value type remains the same: https://github.com/project-flink/flink-graph/blob/master/src/main/java/flink/graphs/Graph.java . However, we would like to be able to change the vertex value type. When trying this out, we got this error: Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ApplyMapperToVertex' could not be determined. This is most likely a type erasure problem. The type extraction currently supports types with generic variables only in cases where all variables in the return type can be deduced from the input type(s). How could we solve this and support changing the vertex value type? Thank you! Cheers, Vasia. |
Hi Vasiliki,
your error is a very common problem we have together with types. The problem is that Java does type erasure, which means that return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); becomes return vertices.map(new ApplyMapperToVertex(mapper)); Therefore we don't have the types. But since we have "implements MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the output through the input (which is always known). The return type must be provided manually e.g. by implementing the interface ResultTypeQueryable or by the a new API operator (see my last mailing list mail). Regards, Timo On 30.10.2014 11:59, Vasiliki Kalavri wrote: > Hi all, > > one of the operations we want to implement for the flink graph api is > mapVertices, i.e. applying a mapper to the vertices of the graph. > The current implementation assumes that the vertex value type remains the > same: > https://github.com/project-flink/flink-graph/blob/master/src/main/java/flink/graphs/Graph.java > . > However, we would like to be able to change the vertex value type. When > trying this out, we got this error: > > Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ApplyMapperToVertex' > could not be determined. This is most likely a type erasure problem. The > type extraction currently supports types with generic variables only in > cases where all variables in the return type can be deduced from the input > type(s). > > How could we solve this and support changing the vertex value type? > Thank you! > > Cheers, > Vasia. > |
Hi Timo,
thank you for the explanation! I guess I will try implementing ResultTypeQueryable then :) Cheers, Vasia. On 30 October 2014 13:01, Timo Walther <[hidden email]> wrote: > Hi Vasiliki, > > your error is a very common problem we have together with types. The > problem is that Java does type erasure, which means that > > return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); > > becomes > > return vertices.map(new ApplyMapperToVertex(mapper)); > > Therefore we don't have the types. But since we have "implements > MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the > output through the input (which is always known). > > The return type must be provided manually e.g. by implementing the > interface ResultTypeQueryable or by the a new API operator (see my last > mailing list mail). > > Regards, > Timo > > > > On 30.10.2014 11:59, Vasiliki Kalavri wrote: > >> Hi all, >> >> one of the operations we want to implement for the flink graph api is >> mapVertices, i.e. applying a mapper to the vertices of the graph. >> The current implementation assumes that the vertex value type remains the >> same: >> https://github.com/project-flink/flink-graph/blob/master/ >> src/main/java/flink/graphs/Graph.java >> . >> However, we would like to be able to change the vertex value type. When >> trying this out, we got this error: >> >> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ >> ApplyMapperToVertex' >> could not be determined. This is most likely a type erasure problem. The >> type extraction currently supports types with generic variables only in >> cases where all variables in the return type can be deduced from the input >> type(s). >> >> How could we solve this and support changing the vertex value type? >> Thank you! >> >> Cheers, >> Vasia. >> >> > |
Hello again,
I tried out your suggestion and made a variation of the mapVertices method, which takes the class of the new value as an argument. In order to extract the type from the class, I use the TypeExtractor.getForClass() method. Is this what you had in mind? When testing, this seems to work fine for basic types and for a custom custom class I tried, but I get an error when trying to change the value type to a Tuple. Do I have to deal with Tuples separately or is there a better way to extract the type? Thanks! Cheers, V. On 31 October 2014 01:16, Vasiliki Kalavri <[hidden email]> wrote: > Hi Timo, > > thank you for the explanation! > I guess I will try implementing ResultTypeQueryable then :) > > Cheers, > Vasia. > > On 30 October 2014 13:01, Timo Walther <[hidden email]> wrote: > >> Hi Vasiliki, >> >> your error is a very common problem we have together with types. The >> problem is that Java does type erasure, which means that >> >> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); >> >> becomes >> >> return vertices.map(new ApplyMapperToVertex(mapper)); >> >> Therefore we don't have the types. But since we have "implements >> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the >> output through the input (which is always known). >> >> The return type must be provided manually e.g. by implementing the >> interface ResultTypeQueryable or by the a new API operator (see my last >> mailing list mail). >> >> Regards, >> Timo >> >> >> >> On 30.10.2014 11:59, Vasiliki Kalavri wrote: >> >>> Hi all, >>> >>> one of the operations we want to implement for the flink graph api is >>> mapVertices, i.e. applying a mapper to the vertices of the graph. >>> The current implementation assumes that the vertex value type remains the >>> same: >>> https://github.com/project-flink/flink-graph/blob/master/ >>> src/main/java/flink/graphs/Graph.java >>> . >>> However, we would like to be able to change the vertex value type. When >>> trying this out, we got this error: >>> >>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ >>> ApplyMapperToVertex' >>> could not be determined. This is most likely a type erasure problem. The >>> type extraction currently supports types with generic variables only in >>> cases where all variables in the return type can be deduced from the >>> input >>> type(s). >>> >>> How could we solve this and support changing the vertex value type? >>> Thank you! >>> >>> Cheers, >>> Vasia. >>> >>> >> > |
Hey,
the TypeExtractor.getForClass() is only intended for primitive types, if you want to extract all kinds of types you should use TypeExtractor.createTypeInfo(Type t) (a class is also a type). However, I think it would be better if your method takes TypeInformation as an argument instead of a class, because otherwise the user has to extend tuple in order to provide the tuple field types. Regards, Timo On 08.11.2014 19:04, Vasiliki Kalavri wrote: > Hello again, > > I tried out your suggestion and made a variation of the mapVertices method, > which takes the class of the new value as an argument. > In order to extract the type from the class, I use > the TypeExtractor.getForClass() method. > Is this what you had in mind? > When testing, this seems to work fine for basic types and for a custom > custom class I tried, but I get an error when trying to change the value > type to a Tuple. Do I have to deal with Tuples separately or is there a > better way to extract the type? > > Thanks! > > Cheers, > V. > > On 31 October 2014 01:16, Vasiliki Kalavri <[hidden email]> > wrote: > >> Hi Timo, >> >> thank you for the explanation! >> I guess I will try implementing ResultTypeQueryable then :) >> >> Cheers, >> Vasia. >> >> On 30 October 2014 13:01, Timo Walther <[hidden email]> wrote: >> >>> Hi Vasiliki, >>> >>> your error is a very common problem we have together with types. The >>> problem is that Java does type erasure, which means that >>> >>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); >>> >>> becomes >>> >>> return vertices.map(new ApplyMapperToVertex(mapper)); >>> >>> Therefore we don't have the types. But since we have "implements >>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the >>> output through the input (which is always known). >>> >>> The return type must be provided manually e.g. by implementing the >>> interface ResultTypeQueryable or by the a new API operator (see my last >>> mailing list mail). >>> >>> Regards, >>> Timo >>> >>> >>> >>> On 30.10.2014 11:59, Vasiliki Kalavri wrote: >>> >>>> Hi all, >>>> >>>> one of the operations we want to implement for the flink graph api is >>>> mapVertices, i.e. applying a mapper to the vertices of the graph. >>>> The current implementation assumes that the vertex value type remains the >>>> same: >>>> https://github.com/project-flink/flink-graph/blob/master/ >>>> src/main/java/flink/graphs/Graph.java >>>> . >>>> However, we would like to be able to change the vertex value type. When >>>> trying this out, we got this error: >>>> >>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ >>>> ApplyMapperToVertex' >>>> could not be determined. This is most likely a type erasure problem. The >>>> type extraction currently supports types with generic variables only in >>>> cases where all variables in the return type can be deduced from the >>>> input >>>> type(s). >>>> >>>> How could we solve this and support changing the vertex value type? >>>> Thank you! >>>> >>>> Cheers, >>>> Vasia. >>>> >>>> |
Hi Vasia!
I think in your case, it would be even better to use "getMapReturnTypes( MapFunction<IN, OUT> mapInterface, TypeInformation<IN> inType)" That way you can pass the function that you wrap (the one that maps the vertex values) and get its produced type. But I think the example shows that we need much better tooling for types hidden in nested functions... Stephan On Mon, Nov 10, 2014 at 11:00 AM, Timo Walther <[hidden email]> wrote: > Hey, > > the TypeExtractor.getForClass() is only intended for primitive types, if > you want to extract all kinds of types you should use > TypeExtractor.createTypeInfo(Type t) (a class is also a type). However, I > think it would be better if your method takes TypeInformation as an > argument instead of a class, because otherwise the user has to extend tuple > in order to provide the tuple field types. > > Regards, > Timo > > > On 08.11.2014 19:04, Vasiliki Kalavri wrote: > >> Hello again, >> >> I tried out your suggestion and made a variation of the mapVertices >> method, >> which takes the class of the new value as an argument. >> In order to extract the type from the class, I use >> the TypeExtractor.getForClass() method. >> Is this what you had in mind? >> When testing, this seems to work fine for basic types and for a custom >> custom class I tried, but I get an error when trying to change the value >> type to a Tuple. Do I have to deal with Tuples separately or is there a >> better way to extract the type? >> >> Thanks! >> >> Cheers, >> V. >> >> On 31 October 2014 01:16, Vasiliki Kalavri <[hidden email]> >> wrote: >> >> Hi Timo, >>> >>> thank you for the explanation! >>> I guess I will try implementing ResultTypeQueryable then :) >>> >>> Cheers, >>> Vasia. >>> >>> On 30 October 2014 13:01, Timo Walther <[hidden email]> wrote: >>> >>> Hi Vasiliki, >>>> >>>> your error is a very common problem we have together with types. The >>>> problem is that Java does type erasure, which means that >>>> >>>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); >>>> >>>> becomes >>>> >>>> return vertices.map(new ApplyMapperToVertex(mapper)); >>>> >>>> Therefore we don't have the types. But since we have "implements >>>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers the >>>> output through the input (which is always known). >>>> >>>> The return type must be provided manually e.g. by implementing the >>>> interface ResultTypeQueryable or by the a new API operator (see my last >>>> mailing list mail). >>>> >>>> Regards, >>>> Timo >>>> >>>> >>>> >>>> On 30.10.2014 11:59, Vasiliki Kalavri wrote: >>>> >>>> Hi all, >>>>> >>>>> one of the operations we want to implement for the flink graph api is >>>>> mapVertices, i.e. applying a mapper to the vertices of the graph. >>>>> The current implementation assumes that the vertex value type remains >>>>> the >>>>> same: >>>>> https://github.com/project-flink/flink-graph/blob/master/ >>>>> src/main/java/flink/graphs/Graph.java >>>>> . >>>>> However, we would like to be able to change the vertex value type. When >>>>> trying this out, we got this error: >>>>> >>>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ >>>>> ApplyMapperToVertex' >>>>> could not be determined. This is most likely a type erasure problem. >>>>> The >>>>> type extraction currently supports types with generic variables only in >>>>> cases where all variables in the return type can be deduced from the >>>>> input >>>>> type(s). >>>>> >>>>> How could we solve this and support changing the vertex value type? >>>>> Thank you! >>>>> >>>>> Cheers, >>>>> Vasia. >>>>> >>>>> >>>>> > |
Thanks Stephan!
Indeed, this worked fine :) Cheers, V. On 10 November 2014 12:26, Stephan Ewen <[hidden email]> wrote: > Hi Vasia! > > I think in your case, it would be even better to use "getMapReturnTypes( > MapFunction<IN, OUT> mapInterface, TypeInformation<IN> inType)" > > That way you can pass the function that you wrap (the one that maps the > vertex values) and get its produced type. > > > But I think the example shows that we need much better tooling for types > hidden in nested functions... > > Stephan > > > On Mon, Nov 10, 2014 at 11:00 AM, Timo Walther <[hidden email]> wrote: > > > Hey, > > > > the TypeExtractor.getForClass() is only intended for primitive types, if > > you want to extract all kinds of types you should use > > TypeExtractor.createTypeInfo(Type t) (a class is also a type). However, I > > think it would be better if your method takes TypeInformation as an > > argument instead of a class, because otherwise the user has to extend > tuple > > in order to provide the tuple field types. > > > > Regards, > > Timo > > > > > > On 08.11.2014 19:04, Vasiliki Kalavri wrote: > > > >> Hello again, > >> > >> I tried out your suggestion and made a variation of the mapVertices > >> method, > >> which takes the class of the new value as an argument. > >> In order to extract the type from the class, I use > >> the TypeExtractor.getForClass() method. > >> Is this what you had in mind? > >> When testing, this seems to work fine for basic types and for a custom > >> custom class I tried, but I get an error when trying to change the value > >> type to a Tuple. Do I have to deal with Tuples separately or is there a > >> better way to extract the type? > >> > >> Thanks! > >> > >> Cheers, > >> V. > >> > >> On 31 October 2014 01:16, Vasiliki Kalavri <[hidden email]> > >> wrote: > >> > >> Hi Timo, > >>> > >>> thank you for the explanation! > >>> I guess I will try implementing ResultTypeQueryable then :) > >>> > >>> Cheers, > >>> Vasia. > >>> > >>> On 30 October 2014 13:01, Timo Walther <[hidden email]> wrote: > >>> > >>> Hi Vasiliki, > >>>> > >>>> your error is a very common problem we have together with types. The > >>>> problem is that Java does type erasure, which means that > >>>> > >>>> return vertices.map(new ApplyMapperToVertex<K, VV>(mapper)); > >>>> > >>>> becomes > >>>> > >>>> return vertices.map(new ApplyMapperToVertex(mapper)); > >>>> > >>>> Therefore we don't have the types. But since we have "implements > >>>> MapFunction<Tuple2<K, VV>, Tuple2<K, VV>>" the type extractor infers > the > >>>> output through the input (which is always known). > >>>> > >>>> The return type must be provided manually e.g. by implementing the > >>>> interface ResultTypeQueryable or by the a new API operator (see my > last > >>>> mailing list mail). > >>>> > >>>> Regards, > >>>> Timo > >>>> > >>>> > >>>> > >>>> On 30.10.2014 11:59, Vasiliki Kalavri wrote: > >>>> > >>>> Hi all, > >>>>> > >>>>> one of the operations we want to implement for the flink graph api is > >>>>> mapVertices, i.e. applying a mapper to the vertices of the graph. > >>>>> The current implementation assumes that the vertex value type remains > >>>>> the > >>>>> same: > >>>>> https://github.com/project-flink/flink-graph/blob/master/ > >>>>> src/main/java/flink/graphs/Graph.java > >>>>> . > >>>>> However, we would like to be able to change the vertex value type. > When > >>>>> trying this out, we got this error: > >>>>> > >>>>> Type of TypeVariable 'NV' in 'class flink.graphs.Graph$ > >>>>> ApplyMapperToVertex' > >>>>> could not be determined. This is most likely a type erasure problem. > >>>>> The > >>>>> type extraction currently supports types with generic variables only > in > >>>>> cases where all variables in the return type can be deduced from the > >>>>> input > >>>>> type(s). > >>>>> > >>>>> How could we solve this and support changing the vertex value type? > >>>>> Thank you! > >>>>> > >>>>> Cheers, > >>>>> Vasia. > >>>>> > >>>>> > >>>>> > > > |
Free forum by Nabble | Edit this page |