Hi all!
We discovered a nice way to give TypeHints in the Google Cloud Dataflow SDK, in a way that would fit Flink perfectly. I created a JIRA for that: https://issues.apache.org/jira/browse/FLINK-2788 Since this is more powerful and type safe than the String/Parser way of giving hints, I was wondering whether we should add this and deprecate the String variant. If we do that, 1.0 is the time to do that. What do you think about this idea? @Timo Walther Since you worked a lot on types/parser/etc - what is your take on this? Greetings, Stephan |
Hi Stephan,
looking at the TypeHint, I got reminded on how Gson (a Json Parser) handles this situation of parsing generics. See here for an overview https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types Seems like this method was rediscovered :) And maybe there are some tricks that can be learned from the implementation I'm all in favor for "hard" types over string literals. Johannes P.S. Apparently GWT uses the same "trick" to handle generics ... -----Ursprüngliche Nachricht----- Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Stephan Ewen Gesendet: Mittwoch, 11. November 2015 15:53 An: [hidden email] Betreff: [DISCUSSION] Type hints versus TypeInfoParser Hi all! We discovered a nice way to give TypeHints in the Google Cloud Dataflow SDK, in a way that would fit Flink perfectly. I created a JIRA for that: https://issues.apache.org/jira/browse/FLINK-2788 Since this is more powerful and type safe than the String/Parser way of giving hints, I was wondering whether we should add this and deprecate the String variant. If we do that, 1.0 is the time to do that. What do you think about this idea? @Timo Walther Since you worked a lot on types/parser/etc - what is your take on this? Greetings, Stephan |
Big +1
Of course, we had the initial talk about it… :D > On 11 Nov 2015, at 19:33, Kirschnick, Johannes <[hidden email]> wrote: > > Hi Stephan, > > looking at the TypeHint, I got reminded on how Gson (a Json Parser) handles this situation of parsing generics. > > See here for an overview > https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types > > Seems like this method was rediscovered :) And maybe there are some tricks that can be learned from the implementation > > I'm all in favor for "hard" types over string literals. > > Johannes > > P.S. > Apparently GWT uses the same "trick" to handle generics ... > > -----Ursprüngliche Nachricht----- > Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Stephan Ewen > Gesendet: Mittwoch, 11. November 2015 15:53 > An: [hidden email] > Betreff: [DISCUSSION] Type hints versus TypeInfoParser > > Hi all! > > We discovered a nice way to give TypeHints in the Google Cloud Dataflow SDK, in a way that would fit Flink perfectly. I created a JIRA for that: > https://issues.apache.org/jira/browse/FLINK-2788 > > Since this is more powerful and type safe than the String/Parser way of giving hints, I was wondering whether we should add this and deprecate the String variant. If we do that, 1.0 is the time to do that. > > What do you think about this idea? > > @Timo Walther Since you worked a lot on types/parser/etc - what is your take on this? > > Greetings, > Stephan |
+1
It's still hacky but we don't have better alternatives. I'm not 100% sure if we can get rid of the parser. I think it's still a nice way for quickly defining the fields of a POJO if the type extractor fails to analyze it. But actually I don't know an example where it fails. Regards, Timo On 11.11.2015 19:56, Aljoscha Krettek wrote: > Big +1 > > Of course, we had the initial talk about it… :D >> On 11 Nov 2015, at 19:33, Kirschnick, Johannes <[hidden email]> wrote: >> >> Hi Stephan, >> >> looking at the TypeHint, I got reminded on how Gson (a Json Parser) handles this situation of parsing generics. >> >> See here for an overview >> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >> >> Seems like this method was rediscovered :) And maybe there are some tricks that can be learned from the implementation >> >> I'm all in favor for "hard" types over string literals. >> >> Johannes >> >> P.S. >> Apparently GWT uses the same "trick" to handle generics ... >> >> -----Ursprüngliche Nachricht----- >> Von: [hidden email] [mailto:[hidden email]] Im Auftrag von Stephan Ewen >> Gesendet: Mittwoch, 11. November 2015 15:53 >> An: [hidden email] >> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >> >> Hi all! >> >> We discovered a nice way to give TypeHints in the Google Cloud Dataflow SDK, in a way that would fit Flink perfectly. I created a JIRA for that: >> https://issues.apache.org/jira/browse/FLINK-2788 >> >> Since this is more powerful and type safe than the String/Parser way of giving hints, I was wondering whether we should add this and deprecate the String variant. If we do that, 1.0 is the time to do that. >> >> What do you think about this idea? >> >> @Timo Walther Since you worked a lot on types/parser/etc - what is your take on this? >> >> Greetings, >> Stephan |
If we want to drop any of the possible ".returns(...)" methods (like class
or string) we should do that now. The "class" method is probably redundant anyways, as the hints are only necessary in generics are involved (where class does not help anyways). @Timo: What can the String do what the TypeHint cannot do? On Wed, Nov 11, 2015 at 9:06 PM, Timo Walther <[hidden email]> wrote: > +1 > > It's still hacky but we don't have better alternatives. > > I'm not 100% sure if we can get rid of the parser. I think it's still a > nice way for quickly defining the fields of a POJO if the type extractor > fails to analyze it. But actually I don't know an example where it fails. > > Regards, > Timo > > > On 11.11.2015 19:56, Aljoscha Krettek wrote: > >> Big +1 >> >> Of course, we had the initial talk about it… :D >> >>> On 11 Nov 2015, at 19:33, Kirschnick, Johannes < >>> [hidden email]> wrote: >>> >>> Hi Stephan, >>> >>> looking at the TypeHint, I got reminded on how Gson (a Json Parser) >>> handles this situation of parsing generics. >>> >>> See here for an overview >>> >>> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >>> >>> Seems like this method was rediscovered :) And maybe there are some >>> tricks that can be learned from the implementation >>> >>> I'm all in favor for "hard" types over string literals. >>> >>> Johannes >>> >>> P.S. >>> Apparently GWT uses the same "trick" to handle generics ... >>> >>> -----Ursprüngliche Nachricht----- >>> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >>> von Stephan Ewen >>> Gesendet: Mittwoch, 11. November 2015 15:53 >>> An: [hidden email] >>> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >>> >>> Hi all! >>> >>> We discovered a nice way to give TypeHints in the Google Cloud Dataflow >>> SDK, in a way that would fit Flink perfectly. I created a JIRA for that: >>> https://issues.apache.org/jira/browse/FLINK-2788 >>> >>> Since this is more powerful and type safe than the String/Parser way of >>> giving hints, I was wondering whether we should add this and deprecate the >>> String variant. If we do that, 1.0 is the time to do that. >>> >>> What do you think about this idea? >>> >>> @Timo Walther Since you worked a lot on types/parser/etc - what is your >>> take on this? >>> >>> Greetings, >>> Stephan >>> >> > |
If the TypeExtractor is not able to handle the fields of a Pojo
correctly, the String parser is quite useful to say "org.my.Pojo<field1=Tuple2<String,Long>>". Doing this with TypeInformation classes is terrible. But I think this a corner case we can ignore. I would also remove the ".returns(String)" in favor of TypeHints, but let the TypeInfoParser remain in a typeutils package (internal tool). Regards, Timo On 18.11.2015 16:47, Stephan Ewen wrote: > If we want to drop any of the possible ".returns(...)" methods (like class > or string) we should do that now. > > The "class" method is probably redundant anyways, as the hints are only > necessary in generics are involved (where class does not help anyways). > > @Timo: What can the String do what the TypeHint cannot do? > > On Wed, Nov 11, 2015 at 9:06 PM, Timo Walther <[hidden email]> wrote: > >> +1 >> >> It's still hacky but we don't have better alternatives. >> >> I'm not 100% sure if we can get rid of the parser. I think it's still a >> nice way for quickly defining the fields of a POJO if the type extractor >> fails to analyze it. But actually I don't know an example where it fails. >> >> Regards, >> Timo >> >> >> On 11.11.2015 19:56, Aljoscha Krettek wrote: >> >>> Big +1 >>> >>> Of course, we had the initial talk about it… :D >>> >>>> On 11 Nov 2015, at 19:33, Kirschnick, Johannes < >>>> [hidden email]> wrote: >>>> >>>> Hi Stephan, >>>> >>>> looking at the TypeHint, I got reminded on how Gson (a Json Parser) >>>> handles this situation of parsing generics. >>>> >>>> See here for an overview >>>> >>>> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >>>> >>>> Seems like this method was rediscovered :) And maybe there are some >>>> tricks that can be learned from the implementation >>>> >>>> I'm all in favor for "hard" types over string literals. >>>> >>>> Johannes >>>> >>>> P.S. >>>> Apparently GWT uses the same "trick" to handle generics ... >>>> >>>> -----Ursprüngliche Nachricht----- >>>> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >>>> von Stephan Ewen >>>> Gesendet: Mittwoch, 11. November 2015 15:53 >>>> An: [hidden email] >>>> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >>>> >>>> Hi all! >>>> >>>> We discovered a nice way to give TypeHints in the Google Cloud Dataflow >>>> SDK, in a way that would fit Flink perfectly. I created a JIRA for that: >>>> https://issues.apache.org/jira/browse/FLINK-2788 >>>> >>>> Since this is more powerful and type safe than the String/Parser way of >>>> giving hints, I was wondering whether we should add this and deprecate the >>>> String variant. If we do that, 1.0 is the time to do that. >>>> >>>> What do you think about this idea? >>>> >>>> @Timo Walther Since you worked a lot on types/parser/etc - what is your >>>> take on this? >>>> >>>> Greetings, >>>> Stephan >>>> |
I think the TypeHints case can cover this:
public class MyPojo<T, R> { public T field1; public R field2; } If you say '.returns(new TypeHint<MyPojo<String, Double>>() {})' this creates an anonymous subclass of the TypeHint, which has the types that T and R bind to, which allows one to construct the POJO type info properly. (Not sure if all that is implemented in the TypeExtractor, though). What do you think? Stephan On Wed, Nov 18, 2015 at 6:03 PM, Timo Walther <[hidden email]> wrote: > If the TypeExtractor is not able to handle the fields of a Pojo correctly, > the String parser is quite useful to say > "org.my.Pojo<field1=Tuple2<String,Long>>". Doing this with TypeInformation > classes is terrible. But I think this a corner case we can ignore. I would > also remove the ".returns(String)" in favor of TypeHints, but let the > TypeInfoParser remain in a typeutils package (internal tool). > > Regards, > Timo > > > On 18.11.2015 16:47, Stephan Ewen wrote: > >> If we want to drop any of the possible ".returns(...)" methods (like class >> or string) we should do that now. >> >> The "class" method is probably redundant anyways, as the hints are only >> necessary in generics are involved (where class does not help anyways). >> >> @Timo: What can the String do what the TypeHint cannot do? >> >> On Wed, Nov 11, 2015 at 9:06 PM, Timo Walther <[hidden email]> wrote: >> >> +1 >>> >>> It's still hacky but we don't have better alternatives. >>> >>> I'm not 100% sure if we can get rid of the parser. I think it's still a >>> nice way for quickly defining the fields of a POJO if the type extractor >>> fails to analyze it. But actually I don't know an example where it fails. >>> >>> Regards, >>> Timo >>> >>> >>> On 11.11.2015 19:56, Aljoscha Krettek wrote: >>> >>> Big +1 >>>> >>>> Of course, we had the initial talk about it… :D >>>> >>>> On 11 Nov 2015, at 19:33, Kirschnick, Johannes < >>>>> [hidden email]> wrote: >>>>> >>>>> Hi Stephan, >>>>> >>>>> looking at the TypeHint, I got reminded on how Gson (a Json Parser) >>>>> handles this situation of parsing generics. >>>>> >>>>> See here for an overview >>>>> >>>>> >>>>> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >>>>> >>>>> Seems like this method was rediscovered :) And maybe there are some >>>>> tricks that can be learned from the implementation >>>>> >>>>> I'm all in favor for "hard" types over string literals. >>>>> >>>>> Johannes >>>>> >>>>> P.S. >>>>> Apparently GWT uses the same "trick" to handle generics ... >>>>> >>>>> -----Ursprüngliche Nachricht----- >>>>> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >>>>> von Stephan Ewen >>>>> Gesendet: Mittwoch, 11. November 2015 15:53 >>>>> An: [hidden email] >>>>> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >>>>> >>>>> Hi all! >>>>> >>>>> We discovered a nice way to give TypeHints in the Google Cloud Dataflow >>>>> SDK, in a way that would fit Flink perfectly. I created a JIRA for >>>>> that: >>>>> https://issues.apache.org/jira/browse/FLINK-2788 >>>>> >>>>> Since this is more powerful and type safe than the String/Parser way of >>>>> giving hints, I was wondering whether we should add this and deprecate >>>>> the >>>>> String variant. If we do that, 1.0 is the time to do that. >>>>> >>>>> What do you think about this idea? >>>>> >>>>> @Timo Walther Since you worked a lot on types/parser/etc - what is your >>>>> take on this? >>>>> >>>>> Greetings, >>>>> Stephan >>>>> >>>>> > |
All that you have mentioned is implemented in the TypeExtractor. I just
mean corner cases e.g. if you have a POJO public class MyPojo { public Object field1; public Object field2; public Tuple2 field3; } Where the TypeExtractor can not analyze anything. Then you may want to provide the TypeInfo manually. TypeInfoParser makes it easy to specify the types of the fields of POJOs manually (but only as an internal feature). But as I said this just a corner case. Timo On 18.11.2015 18:43, Stephan Ewen wrote: > I think the TypeHints case can cover this: > > public class MyPojo<T, R> { > public T field1; > public R field2; > } > > If you say '.returns(new TypeHint<MyPojo<String, Double>>() {})' this > creates an anonymous subclass of the TypeHint, which has the types that T > and R bind to, which allows one to construct the POJO type info properly. > (Not sure if all that is implemented in the TypeExtractor, though). > > What do you think? > > Stephan > > > > > On Wed, Nov 18, 2015 at 6:03 PM, Timo Walther <[hidden email]> wrote: > >> If the TypeExtractor is not able to handle the fields of a Pojo correctly, >> the String parser is quite useful to say >> "org.my.Pojo<field1=Tuple2<String,Long>>". Doing this with TypeInformation >> classes is terrible. But I think this a corner case we can ignore. I would >> also remove the ".returns(String)" in favor of TypeHints, but let the >> TypeInfoParser remain in a typeutils package (internal tool). >> >> Regards, >> Timo >> >> >> On 18.11.2015 16:47, Stephan Ewen wrote: >> >>> If we want to drop any of the possible ".returns(...)" methods (like class >>> or string) we should do that now. >>> >>> The "class" method is probably redundant anyways, as the hints are only >>> necessary in generics are involved (where class does not help anyways). >>> >>> @Timo: What can the String do what the TypeHint cannot do? >>> >>> On Wed, Nov 11, 2015 at 9:06 PM, Timo Walther <[hidden email]> wrote: >>> >>> +1 >>>> It's still hacky but we don't have better alternatives. >>>> >>>> I'm not 100% sure if we can get rid of the parser. I think it's still a >>>> nice way for quickly defining the fields of a POJO if the type extractor >>>> fails to analyze it. But actually I don't know an example where it fails. >>>> >>>> Regards, >>>> Timo >>>> >>>> >>>> On 11.11.2015 19:56, Aljoscha Krettek wrote: >>>> >>>> Big +1 >>>>> Of course, we had the initial talk about it… :D >>>>> >>>>> On 11 Nov 2015, at 19:33, Kirschnick, Johannes < >>>>>> [hidden email]> wrote: >>>>>> >>>>>> Hi Stephan, >>>>>> >>>>>> looking at the TypeHint, I got reminded on how Gson (a Json Parser) >>>>>> handles this situation of parsing generics. >>>>>> >>>>>> See here for an overview >>>>>> >>>>>> >>>>>> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >>>>>> >>>>>> Seems like this method was rediscovered :) And maybe there are some >>>>>> tricks that can be learned from the implementation >>>>>> >>>>>> I'm all in favor for "hard" types over string literals. >>>>>> >>>>>> Johannes >>>>>> >>>>>> P.S. >>>>>> Apparently GWT uses the same "trick" to handle generics ... >>>>>> >>>>>> -----Ursprüngliche Nachricht----- >>>>>> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >>>>>> von Stephan Ewen >>>>>> Gesendet: Mittwoch, 11. November 2015 15:53 >>>>>> An: [hidden email] >>>>>> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >>>>>> >>>>>> Hi all! >>>>>> >>>>>> We discovered a nice way to give TypeHints in the Google Cloud Dataflow >>>>>> SDK, in a way that would fit Flink perfectly. I created a JIRA for >>>>>> that: >>>>>> https://issues.apache.org/jira/browse/FLINK-2788 >>>>>> >>>>>> Since this is more powerful and type safe than the String/Parser way of >>>>>> giving hints, I was wondering whether we should add this and deprecate >>>>>> the >>>>>> String variant. If we do that, 1.0 is the time to do that. >>>>>> >>>>>> What do you think about this idea? >>>>>> >>>>>> @Timo Walther Since you worked a lot on types/parser/etc - what is your >>>>>> take on this? >>>>>> >>>>>> Greetings, >>>>>> Stephan >>>>>> >>>>>> |
Ah, I see. Sounds a bit corner case to me, actually.
On Thu, Nov 19, 2015 at 2:20 PM, Timo Walther <[hidden email]> wrote: > All that you have mentioned is implemented in the TypeExtractor. I just > mean corner cases e.g. if you have a POJO > > public class MyPojo { > public Object field1; > public Object field2; > public Tuple2 field3; > } > > Where the TypeExtractor can not analyze anything. Then you may want to > provide the TypeInfo manually. TypeInfoParser makes it easy to specify the > types of the fields of POJOs manually (but only as an internal feature). > But as I said this just a corner case. > > Timo > > > > On 18.11.2015 18:43, Stephan Ewen wrote: > >> I think the TypeHints case can cover this: >> >> public class MyPojo<T, R> { >> public T field1; >> public R field2; >> } >> >> If you say '.returns(new TypeHint<MyPojo<String, Double>>() {})' this >> creates an anonymous subclass of the TypeHint, which has the types that T >> and R bind to, which allows one to construct the POJO type info properly. >> (Not sure if all that is implemented in the TypeExtractor, though). >> >> What do you think? >> >> Stephan >> >> >> >> >> On Wed, Nov 18, 2015 at 6:03 PM, Timo Walther <[hidden email]> wrote: >> >> If the TypeExtractor is not able to handle the fields of a Pojo correctly, >>> the String parser is quite useful to say >>> "org.my.Pojo<field1=Tuple2<String,Long>>". Doing this with >>> TypeInformation >>> classes is terrible. But I think this a corner case we can ignore. I >>> would >>> also remove the ".returns(String)" in favor of TypeHints, but let the >>> TypeInfoParser remain in a typeutils package (internal tool). >>> >>> Regards, >>> Timo >>> >>> >>> On 18.11.2015 16:47, Stephan Ewen wrote: >>> >>> If we want to drop any of the possible ".returns(...)" methods (like >>>> class >>>> or string) we should do that now. >>>> >>>> The "class" method is probably redundant anyways, as the hints are only >>>> necessary in generics are involved (where class does not help anyways). >>>> >>>> @Timo: What can the String do what the TypeHint cannot do? >>>> >>>> On Wed, Nov 11, 2015 at 9:06 PM, Timo Walther <[hidden email]> >>>> wrote: >>>> >>>> +1 >>>> >>>>> It's still hacky but we don't have better alternatives. >>>>> >>>>> I'm not 100% sure if we can get rid of the parser. I think it's still a >>>>> nice way for quickly defining the fields of a POJO if the type >>>>> extractor >>>>> fails to analyze it. But actually I don't know an example where it >>>>> fails. >>>>> >>>>> Regards, >>>>> Timo >>>>> >>>>> >>>>> On 11.11.2015 19:56, Aljoscha Krettek wrote: >>>>> >>>>> Big +1 >>>>> >>>>>> Of course, we had the initial talk about it… :D >>>>>> >>>>>> On 11 Nov 2015, at 19:33, Kirschnick, Johannes < >>>>>> >>>>>>> [hidden email]> wrote: >>>>>>> >>>>>>> Hi Stephan, >>>>>>> >>>>>>> looking at the TypeHint, I got reminded on how Gson (a Json Parser) >>>>>>> handles this situation of parsing generics. >>>>>>> >>>>>>> See here for an overview >>>>>>> >>>>>>> >>>>>>> >>>>>>> https://sites.google.com/site/gson/gson-user-guide#TOC-Serializing-and-Deserializing-Generic-Types >>>>>>> >>>>>>> Seems like this method was rediscovered :) And maybe there are some >>>>>>> tricks that can be learned from the implementation >>>>>>> >>>>>>> I'm all in favor for "hard" types over string literals. >>>>>>> >>>>>>> Johannes >>>>>>> >>>>>>> P.S. >>>>>>> Apparently GWT uses the same "trick" to handle generics ... >>>>>>> >>>>>>> -----Ursprüngliche Nachricht----- >>>>>>> Von: [hidden email] [mailto:[hidden email]] Im Auftrag >>>>>>> von Stephan Ewen >>>>>>> Gesendet: Mittwoch, 11. November 2015 15:53 >>>>>>> An: [hidden email] >>>>>>> Betreff: [DISCUSSION] Type hints versus TypeInfoParser >>>>>>> >>>>>>> Hi all! >>>>>>> >>>>>>> We discovered a nice way to give TypeHints in the Google Cloud >>>>>>> Dataflow >>>>>>> SDK, in a way that would fit Flink perfectly. I created a JIRA for >>>>>>> that: >>>>>>> https://issues.apache.org/jira/browse/FLINK-2788 >>>>>>> >>>>>>> Since this is more powerful and type safe than the String/Parser way >>>>>>> of >>>>>>> giving hints, I was wondering whether we should add this and >>>>>>> deprecate >>>>>>> the >>>>>>> String variant. If we do that, 1.0 is the time to do that. >>>>>>> >>>>>>> What do you think about this idea? >>>>>>> >>>>>>> @Timo Walther Since you worked a lot on types/parser/etc - what is >>>>>>> your >>>>>>> take on this? >>>>>>> >>>>>>> Greetings, >>>>>>> Stephan >>>>>>> >>>>>>> >>>>>>> > |
Free forum by Nabble | Edit this page |