Hi everybody,
I am working currently on a tutorial/article about how/when/why to use Parquet on Flink. You can find the pdf version here: https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf The git repository with all the code examples can be found here: https://github.com/FelixNeutatz/parquet-flinktacular/ What do you think about it? I am happy about every feedback I can get :) Have a nice Sunday, Felix |
Wow, very nice work!
It looks impressive at first sight, no comments :-) Just one question: Do you want this to be a standalone tutorial, or are you thinking of putting this into the Flink wiki, or some code even into "flink-contrib" ? Stephan On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz <[hidden email]> wrote: > Hi everybody, > > I am working currently on a tutorial/article about how/when/why to use > Parquet on Flink. > > You can find the pdf version here: > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > The git repository with all the code examples can be found here: > https://github.com/FelixNeutatz/parquet-flinktacular/ > > What do you think about it? I am happy about every feedback I can get :) > > Have a nice Sunday, > > Felix > |
The intention was to post it on the blog, but if you think it would better
fit into the wiki, that would be also fine :) About the code: I have not thought about putting it to contrib-package, but I can. What do you think is suitable for flink-contrib? Best regards, Felix 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email]>: > Wow, very nice work! > > It looks impressive at first sight, no comments :-) > > Just one question: Do you want this to be a standalone tutorial, or are you > thinking of putting this into the Flink wiki, or some code even into > "flink-contrib" ? > > Stephan > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz <[hidden email]> > wrote: > > > Hi everybody, > > > > I am working currently on a tutorial/article about how/when/why to use > > Parquet on Flink. > > > > You can find the pdf version here: > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > The git repository with all the code examples can be found here: > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > What do you think about it? I am happy about every feedback I can get :) > > > > Have a nice Sunday, > > > > Felix > > > |
Looks very nice! Would love to see a blog post on that!
On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz <[hidden email]> wrote: > The intention was to post it on the blog, but if you think it would better > fit into the wiki, that would be also fine :) > > About the code: I have not thought about putting it to contrib-package, but > I can. What do you think is suitable for flink-contrib? > > Best regards, > > Felix > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email]>: > > > Wow, very nice work! > > > > It looks impressive at first sight, no comments :-) > > > > Just one question: Do you want this to be a standalone tutorial, or are > you > > thinking of putting this into the Flink wiki, or some code even into > > "flink-contrib" ? > > > > Stephan > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz <[hidden email]> > > wrote: > > > > > Hi everybody, > > > > > > I am working currently on a tutorial/article about how/when/why to use > > > Parquet on Flink. > > > > > > You can find the pdf version here: > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > The git repository with all the code examples can be found here: > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > What do you think about it? I am happy about every feedback I can get > :) > > > > > > Have a nice Sunday, > > > > > > Felix > > > > > > |
Very nice article!
How about adding the full article to the wiki and having a shorter version as a blog post (with a link to the wiki)? Adding the code to contrib would be great! 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email]>: > Looks very nice! Would love to see a blog post on that! > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz <[hidden email]> > wrote: > > > The intention was to post it on the blog, but if you think it would > better > > fit into the wiki, that would be also fine :) > > > > About the code: I have not thought about putting it to contrib-package, > but > > I can. What do you think is suitable for flink-contrib? > > > > Best regards, > > > > Felix > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email]>: > > > > > Wow, very nice work! > > > > > > It looks impressive at first sight, no comments :-) > > > > > > Just one question: Do you want this to be a standalone tutorial, or are > > you > > > thinking of putting this into the Flink wiki, or some code even into > > > "flink-contrib" ? > > > > > > Stephan > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz <[hidden email]> > > > wrote: > > > > > > > Hi everybody, > > > > > > > > I am working currently on a tutorial/article about how/when/why to > use > > > > Parquet on Flink. > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > The git repository with all the code examples can be found here: > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > What do you think about it? I am happy about every feedback I can get > > :) > > > > > > > > Have a nice Sunday, > > > > > > > > Felix > > > > > > > > > > |
Hi Felix,
Very nice informative read. +1 for a short blog post and a full version in the wiki. +1 for putting this into flink-contrib On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email]> wrote: > Very nice article! > How about adding the full article to the wiki and having a shorter version > as a blog post (with a link to the wiki)? > Adding the code to contrib would be great! > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email]>: > > > Looks very nice! Would love to see a blog post on that! > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz <[hidden email]> > > wrote: > > > > > The intention was to post it on the blog, but if you think it would > > better > > > fit into the wiki, that would be also fine :) > > > > > > About the code: I have not thought about putting it to contrib-package, > > but > > > I can. What do you think is suitable for flink-contrib? > > > > > > Best regards, > > > > > > Felix > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email]>: > > > > > > > Wow, very nice work! > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > Just one question: Do you want this to be a standalone tutorial, or > are > > > you > > > > thinking of putting this into the Flink wiki, or some code even into > > > > "flink-contrib" ? > > > > > > > > Stephan > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > [hidden email]> > > > > wrote: > > > > > > > > > Hi everybody, > > > > > > > > > > I am working currently on a tutorial/article about how/when/why to > > use > > > > > Parquet on Flink. > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > The git repository with all the code examples can be found here: > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > What do you think about it? I am happy about every feedback I can > get > > > :) > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > Felix > > > > > > > > > > > > > > > |
+1 to the idea.
Awesome work, Felix On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> wrote: > Hi Felix, > > Very nice informative read. > > +1 for a short blog post and a full version in the wiki. > +1 for putting this into flink-contrib > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] > <javascript:;>> wrote: > > > Very nice article! > > How about adding the full article to the wiki and having a shorter > version > > as a blog post (with a link to the wiki)? > > Adding the code to contrib would be great! > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] > <javascript:;>>: > > > > > Looks very nice! Would love to see a blog post on that! > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz <[hidden email] > <javascript:;>> > > > wrote: > > > > > > > The intention was to post it on the blog, but if you think it would > > > better > > > > fit into the wiki, that would be also fine :) > > > > > > > > About the code: I have not thought about putting it to > contrib-package, > > > but > > > > I can. What do you think is suitable for flink-contrib? > > > > > > > > Best regards, > > > > > > > > Felix > > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > <javascript:;>>: > > > > > > > > > Wow, very nice work! > > > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > > > Just one question: Do you want this to be a standalone tutorial, or > > are > > > > you > > > > > thinking of putting this into the Flink wiki, or some code even > into > > > > > "flink-contrib" ? > > > > > > > > > > Stephan > > > > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > > [hidden email] <javascript:;>> > > > > > wrote: > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > I am working currently on a tutorial/article about how/when/why > to > > > use > > > > > > Parquet on Flink. > > > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > > > The git repository with all the code examples can be found here: > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > > > What do you think about it? I am happy about every feedback I can > > get > > > > :) > > > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > > > > > > |
+1 for a blog post
On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra <[hidden email]> wrote: > +1 to the idea. > > Awesome work, Felix > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> wrote: > > > Hi Felix, > > > > Very nice informative read. > > > > +1 for a short blog post and a full version in the wiki. > > +1 for putting this into flink-contrib > > > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] > > <javascript:;>> wrote: > > > > > Very nice article! > > > How about adding the full article to the wiki and having a shorter > > version > > > as a blog post (with a link to the wiki)? > > > Adding the code to contrib would be great! > > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] > > <javascript:;>>: > > > > > > > Looks very nice! Would love to see a blog post on that! > > > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < > [hidden email] > > <javascript:;>> > > > > wrote: > > > > > > > > > The intention was to post it on the blog, but if you think it would > > > > better > > > > > fit into the wiki, that would be also fine :) > > > > > > > > > > About the code: I have not thought about putting it to > > contrib-package, > > > > but > > > > > I can. What do you think is suitable for flink-contrib? > > > > > > > > > > Best regards, > > > > > > > > > > Felix > > > > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > > <javascript:;>>: > > > > > > > > > > > Wow, very nice work! > > > > > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > > > > > Just one question: Do you want this to be a standalone tutorial, > or > > > are > > > > > you > > > > > > thinking of putting this into the Flink wiki, or some code even > > into > > > > > > "flink-contrib" ? > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > > > [hidden email] <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > > > I am working currently on a tutorial/article about how/when/why > > to > > > > use > > > > > > > Parquet on Flink. > > > > > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > > > > > The git repository with all the code examples can be found > here: > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > > > > > What do you think about it? I am happy about every feedback I > can > > > get > > > > > :) > > > > > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
I was looking at this great example and I'd like to ask you which
serialization framework is the best if I have to serialize Tuple3<String,String,byte[]> with Parquet. The syntax I like the most is the Thrift one but I can't see all the pros and cons of using it and I'd like to hear your opinion here. Thanks in advance, Flavio On Fri, Apr 10, 2015 at 2:52 PM, Robert Metzger <[hidden email]> wrote: > +1 for a blog post > > On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra <[hidden email]> > wrote: > > > +1 to the idea. > > > > Awesome work, Felix > > > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> wrote: > > > > > Hi Felix, > > > > > > Very nice informative read. > > > > > > +1 for a short blog post and a full version in the wiki. > > > +1 for putting this into flink-contrib > > > > > > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] > > > <javascript:;>> wrote: > > > > > > > Very nice article! > > > > How about adding the full article to the wiki and having a shorter > > > version > > > > as a blog post (with a link to the wiki)? > > > > Adding the code to contrib would be great! > > > > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] > > > <javascript:;>>: > > > > > > > > > Looks very nice! Would love to see a blog post on that! > > > > > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < > > [hidden email] > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > The intention was to post it on the blog, but if you think it > would > > > > > better > > > > > > fit into the wiki, that would be also fine :) > > > > > > > > > > > > About the code: I have not thought about putting it to > > > contrib-package, > > > > > but > > > > > > I can. What do you think is suitable for flink-contrib? > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Felix > > > > > > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > > > <javascript:;>>: > > > > > > > > > > > > > Wow, very nice work! > > > > > > > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > > > > > > > Just one question: Do you want this to be a standalone > tutorial, > > or > > > > are > > > > > > you > > > > > > > thinking of putting this into the Flink wiki, or some code even > > > into > > > > > > > "flink-contrib" ? > > > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > > > > [hidden email] <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > > > > > I am working currently on a tutorial/article about > how/when/why > > > to > > > > > use > > > > > > > > Parquet on Flink. > > > > > > > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > > > > > > > The git repository with all the code examples can be found > > here: > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > > > > > > > What do you think about it? I am happy about every feedback I > > can > > > > get > > > > > > :) > > > > > > > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Hi Flavio,
in Thrift you can try: struct FlavioTuple { 1: optional string f1; 2: optional string f2; 3: optional list<byte> f3; } See: http://diwakergupta.github.io/thrift-missing-guide/ I like Thrift the most, because the API for Thrift in Parquet is the easiest. Have fun with Parquet :) Best regards, Felix 2015-04-24 12:28 GMT+02:00 Flavio Pompermaier <[hidden email]>: > I was looking at this great example and I'd like to ask you which > serialization framework is the best if I have to serialize > Tuple3<String,String,byte[]> with Parquet. > The syntax I like the most is the Thrift one but I can't see all the pros > and cons of using it and I'd like to hear your opinion here. > > Thanks in advance, > Flavio > > On Fri, Apr 10, 2015 at 2:52 PM, Robert Metzger <[hidden email]> > wrote: > > > +1 for a blog post > > > > On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra <[hidden email]> > > wrote: > > > > > +1 to the idea. > > > > > > Awesome work, Felix > > > > > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> wrote: > > > > > > > Hi Felix, > > > > > > > > Very nice informative read. > > > > > > > > +1 for a short blog post and a full version in the wiki. > > > > +1 for putting this into flink-contrib > > > > > > > > > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] > > > > <javascript:;>> wrote: > > > > > > > > > Very nice article! > > > > > How about adding the full article to the wiki and having a shorter > > > > version > > > > > as a blog post (with a link to the wiki)? > > > > > Adding the code to contrib would be great! > > > > > > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] > > > > <javascript:;>>: > > > > > > > > > > > Looks very nice! Would love to see a blog post on that! > > > > > > > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < > > > [hidden email] > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > The intention was to post it on the blog, but if you think it > > would > > > > > > better > > > > > > > fit into the wiki, that would be also fine :) > > > > > > > > > > > > > > About the code: I have not thought about putting it to > > > > contrib-package, > > > > > > but > > > > > > > I can. What do you think is suitable for flink-contrib? > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > > > > <javascript:;>>: > > > > > > > > > > > > > > > Wow, very nice work! > > > > > > > > > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > > > > > > > > > Just one question: Do you want this to be a standalone > > tutorial, > > > or > > > > > are > > > > > > > you > > > > > > > > thinking of putting this into the Flink wiki, or some code > even > > > > into > > > > > > > > "flink-contrib" ? > > > > > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > > > > > [hidden email] <javascript:;>> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > > > > > > > I am working currently on a tutorial/article about > > how/when/why > > > > to > > > > > > use > > > > > > > > > Parquet on Flink. > > > > > > > > > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > > > > > > > > > The git repository with all the code examples can be found > > > here: > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > > > > > > > > > What do you think about it? I am happy about every > feedback I > > > can > > > > > get > > > > > > > :) > > > > > > > > > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Thanks Felix,
Thanks fir the response! I'm looking forward to use it! On Apr 24, 2015 9:01 PM, "Felix Neutatz" <[hidden email]> wrote: > Hi Flavio, > > in Thrift you can try: > > struct FlavioTuple { > 1: optional string f1; > 2: optional string f2; > 3: optional list<byte> f3; > } > > See: http://diwakergupta.github.io/thrift-missing-guide/ > > I like Thrift the most, because the API for Thrift in Parquet is the > easiest. > > Have fun with Parquet :) > > Best regards, > > Felix > > 2015-04-24 12:28 GMT+02:00 Flavio Pompermaier <[hidden email]>: > > > I was looking at this great example and I'd like to ask you which > > serialization framework is the best if I have to serialize > > Tuple3<String,String,byte[]> with Parquet. > > The syntax I like the most is the Thrift one but I can't see all the pros > > and cons of using it and I'd like to hear your opinion here. > > > > Thanks in advance, > > Flavio > > > > On Fri, Apr 10, 2015 at 2:52 PM, Robert Metzger <[hidden email]> > > wrote: > > > > > +1 for a blog post > > > > > > On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra <[hidden email] > > > > > wrote: > > > > > > > +1 to the idea. > > > > > > > > Awesome work, Felix > > > > > > > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> > wrote: > > > > > > > > > Hi Felix, > > > > > > > > > > Very nice informative read. > > > > > > > > > > +1 for a short blog post and a full version in the wiki. > > > > > +1 for putting this into flink-contrib > > > > > > > > > > > > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] > > > > > <javascript:;>> wrote: > > > > > > > > > > > Very nice article! > > > > > > How about adding the full article to the wiki and having a > shorter > > > > > version > > > > > > as a blog post (with a link to the wiki)? > > > > > > Adding the code to contrib would be great! > > > > > > > > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] > > > > > <javascript:;>>: > > > > > > > > > > > > > Looks very nice! Would love to see a blog post on that! > > > > > > > > > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < > > > > [hidden email] > > > > > <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > The intention was to post it on the blog, but if you think it > > > would > > > > > > > better > > > > > > > > fit into the wiki, that would be also fine :) > > > > > > > > > > > > > > > > About the code: I have not thought about putting it to > > > > > contrib-package, > > > > > > > but > > > > > > > > I can. What do you think is suitable for flink-contrib? > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > > > > > <javascript:;>>: > > > > > > > > > > > > > > > > > Wow, very nice work! > > > > > > > > > > > > > > > > > > It looks impressive at first sight, no comments :-) > > > > > > > > > > > > > > > > > > Just one question: Do you want this to be a standalone > > > tutorial, > > > > or > > > > > > are > > > > > > > > you > > > > > > > > > thinking of putting this into the Flink wiki, or some code > > even > > > > > into > > > > > > > > > "flink-contrib" ? > > > > > > > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > > > > > > [hidden email] <javascript:;>> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > > > > > > > > > I am working currently on a tutorial/article about > > > how/when/why > > > > > to > > > > > > > use > > > > > > > > > > Parquet on Flink. > > > > > > > > > > > > > > > > > > > > You can find the pdf version here: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > > > > > > > > > > > > > > > > > > > > The git repository with all the code examples can be > found > > > > here: > > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > > > > > > > > > > > > > > > > > > > > What do you think about it? I am happy about every > > feedback I > > > > can > > > > > > get > > > > > > > > :) > > > > > > > > > > > > > > > > > > > > Have a nice Sunday, > > > > > > > > > > > > > > > > > > > > Felix > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
Hi Felix,
I was looking at your code and I don't see any parquet.block.size settings. Do you think it is safe to keep defaults? On Fri, Apr 24, 2015 at 11:19 PM, Flavio Pompermaier <[hidden email]> wrote: > Thanks Felix, > Thanks fir the response! > I'm looking forward to use it! > On Apr 24, 2015 9:01 PM, "Felix Neutatz" <[hidden email]> wrote: > >> Hi Flavio, >> >> in Thrift you can try: >> >> struct FlavioTuple { >> 1: optional string f1; >> 2: optional string f2; >> 3: optional list<byte> f3; >> } >> >> See: http://diwakergupta.github.io/thrift-missing-guide/ >> >> I like Thrift the most, because the API for Thrift in Parquet is the >> easiest. >> >> Have fun with Parquet :) >> >> Best regards, >> >> Felix >> >> 2015-04-24 12:28 GMT+02:00 Flavio Pompermaier <[hidden email]>: >> >> > I was looking at this great example and I'd like to ask you which >> > serialization framework is the best if I have to serialize >> > Tuple3<String,String,byte[]> with Parquet. >> > The syntax I like the most is the Thrift one but I can't see all the >> pros >> > and cons of using it and I'd like to hear your opinion here. >> > >> > Thanks in advance, >> > Flavio >> > >> > On Fri, Apr 10, 2015 at 2:52 PM, Robert Metzger <[hidden email]> >> > wrote: >> > >> > > +1 for a blog post >> > > >> > > On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra < >> [hidden email]> >> > > wrote: >> > > >> > > > +1 to the idea. >> > > > >> > > > Awesome work, Felix >> > > > >> > > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> >> wrote: >> > > > >> > > > > Hi Felix, >> > > > > >> > > > > Very nice informative read. >> > > > > >> > > > > +1 for a short blog post and a full version in the wiki. >> > > > > +1 for putting this into flink-contrib >> > > > > >> > > > > >> > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske <[hidden email] >> > > > > <javascript:;>> wrote: >> > > > > >> > > > > > Very nice article! >> > > > > > How about adding the full article to the wiki and having a >> shorter >> > > > > version >> > > > > > as a blog post (with a link to the wiki)? >> > > > > > Adding the code to contrib would be great! >> > > > > > >> > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas <[hidden email] >> > > > > <javascript:;>>: >> > > > > > >> > > > > > > Looks very nice! Would love to see a blog post on that! >> > > > > > > >> > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < >> > > > [hidden email] >> > > > > <javascript:;>> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > The intention was to post it on the blog, but if you think >> it >> > > would >> > > > > > > better >> > > > > > > > fit into the wiki, that would be also fine :) >> > > > > > > > >> > > > > > > > About the code: I have not thought about putting it to >> > > > > contrib-package, >> > > > > > > but >> > > > > > > > I can. What do you think is suitable for flink-contrib? >> > > > > > > > >> > > > > > > > Best regards, >> > > > > > > > >> > > > > > > > Felix >> > > > > > > > >> > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] >> > > > > <javascript:;>>: >> > > > > > > > >> > > > > > > > > Wow, very nice work! >> > > > > > > > > >> > > > > > > > > It looks impressive at first sight, no comments :-) >> > > > > > > > > >> > > > > > > > > Just one question: Do you want this to be a standalone >> > > tutorial, >> > > > or >> > > > > > are >> > > > > > > > you >> > > > > > > > > thinking of putting this into the Flink wiki, or some code >> > even >> > > > > into >> > > > > > > > > "flink-contrib" ? >> > > > > > > > > >> > > > > > > > > Stephan >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < >> > > > > > [hidden email] <javascript:;>> >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hi everybody, >> > > > > > > > > > >> > > > > > > > > > I am working currently on a tutorial/article about >> > > how/when/why >> > > > > to >> > > > > > > use >> > > > > > > > > > Parquet on Flink. >> > > > > > > > > > >> > > > > > > > > > You can find the pdf version here: >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf >> > > > > > > > > > >> > > > > > > > > > The git repository with all the code examples can be >> found >> > > > here: >> > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ >> > > > > > > > > > >> > > > > > > > > > What do you think about it? I am happy about every >> > feedback I >> > > > can >> > > > > > get >> > > > > > > > :) >> > > > > > > > > > >> > > > > > > > > > Have a nice Sunday, >> > > > > > > > > > >> > > > > > > > > > Felix >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > |
Hi Flavio,
changing the block size may help. But I haven't played around with it yet. Best regards, Felix Am 11.05.2015 12:23 nachm. schrieb "Flavio Pompermaier" < [hidden email]>: > Hi Felix, > I was looking at your code and I don't see any parquet.block.size settings. > Do you think it is safe to keep defaults? > > On Fri, Apr 24, 2015 at 11:19 PM, Flavio Pompermaier <[hidden email] > > > wrote: > > > Thanks Felix, > > Thanks fir the response! > > I'm looking forward to use it! > > On Apr 24, 2015 9:01 PM, "Felix Neutatz" <[hidden email]> wrote: > > > >> Hi Flavio, > >> > >> in Thrift you can try: > >> > >> struct FlavioTuple { > >> 1: optional string f1; > >> 2: optional string f2; > >> 3: optional list<byte> f3; > >> } > >> > >> See: http://diwakergupta.github.io/thrift-missing-guide/ > >> > >> I like Thrift the most, because the API for Thrift in Parquet is the > >> easiest. > >> > >> Have fun with Parquet :) > >> > >> Best regards, > >> > >> Felix > >> > >> 2015-04-24 12:28 GMT+02:00 Flavio Pompermaier <[hidden email]>: > >> > >> > I was looking at this great example and I'd like to ask you which > >> > serialization framework is the best if I have to serialize > >> > Tuple3<String,String,byte[]> with Parquet. > >> > The syntax I like the most is the Thrift one but I can't see all the > >> pros > >> > and cons of using it and I'd like to hear your opinion here. > >> > > >> > Thanks in advance, > >> > Flavio > >> > > >> > On Fri, Apr 10, 2015 at 2:52 PM, Robert Metzger <[hidden email]> > >> > wrote: > >> > > >> > > +1 for a blog post > >> > > > >> > > On Tue, Apr 7, 2015 at 5:17 PM, Henry Saputra < > >> [hidden email]> > >> > > wrote: > >> > > > >> > > > +1 to the idea. > >> > > > > >> > > > Awesome work, Felix > >> > > > > >> > > > On Tuesday, April 7, 2015, Maximilian Michels <[hidden email]> > >> wrote: > >> > > > > >> > > > > Hi Felix, > >> > > > > > >> > > > > Very nice informative read. > >> > > > > > >> > > > > +1 for a short blog post and a full version in the wiki. > >> > > > > +1 for putting this into flink-contrib > >> > > > > > >> > > > > > >> > > > > On Tue, Apr 7, 2015 at 1:46 PM, Fabian Hueske < > [hidden email] > >> > > > > <javascript:;>> wrote: > >> > > > > > >> > > > > > Very nice article! > >> > > > > > How about adding the full article to the wiki and having a > >> shorter > >> > > > > version > >> > > > > > as a blog post (with a link to the wiki)? > >> > > > > > Adding the code to contrib would be great! > >> > > > > > > >> > > > > > 2015-04-07 12:45 GMT+02:00 Kostas Tzoumas < > [hidden email] > >> > > > > <javascript:;>>: > >> > > > > > > >> > > > > > > Looks very nice! Would love to see a blog post on that! > >> > > > > > > > >> > > > > > > On Mon, Apr 6, 2015 at 7:19 PM, Felix Neutatz < > >> > > > [hidden email] > >> > > > > <javascript:;>> > >> > > > > > > wrote: > >> > > > > > > > >> > > > > > > > The intention was to post it on the blog, but if you think > >> it > >> > > would > >> > > > > > > better > >> > > > > > > > fit into the wiki, that would be also fine :) > >> > > > > > > > > >> > > > > > > > About the code: I have not thought about putting it to > >> > > > > contrib-package, > >> > > > > > > but > >> > > > > > > > I can. What do you think is suitable for flink-contrib? > >> > > > > > > > > >> > > > > > > > Best regards, > >> > > > > > > > > >> > > > > > > > Felix > >> > > > > > > > > >> > > > > > > > 2015-04-06 14:57 GMT+02:00 Stephan Ewen <[hidden email] > >> > > > > <javascript:;>>: > >> > > > > > > > > >> > > > > > > > > Wow, very nice work! > >> > > > > > > > > > >> > > > > > > > > It looks impressive at first sight, no comments :-) > >> > > > > > > > > > >> > > > > > > > > Just one question: Do you want this to be a standalone > >> > > tutorial, > >> > > > or > >> > > > > > are > >> > > > > > > > you > >> > > > > > > > > thinking of putting this into the Flink wiki, or some > code > >> > even > >> > > > > into > >> > > > > > > > > "flink-contrib" ? > >> > > > > > > > > > >> > > > > > > > > Stephan > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > On Sun, Apr 5, 2015 at 3:30 PM, Felix Neutatz < > >> > > > > > [hidden email] <javascript:;>> > >> > > > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > > Hi everybody, > >> > > > > > > > > > > >> > > > > > > > > > I am working currently on a tutorial/article about > >> > > how/when/why > >> > > > > to > >> > > > > > > use > >> > > > > > > > > > Parquet on Flink. > >> > > > > > > > > > > >> > > > > > > > > > You can find the pdf version here: > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://github.com/FelixNeutatz/parquet-flinktacular/blob/master/tutorial/parquet_flinktacular.pdf > >> > > > > > > > > > > >> > > > > > > > > > The git repository with all the code examples can be > >> found > >> > > > here: > >> > > > > > > > > > https://github.com/FelixNeutatz/parquet-flinktacular/ > >> > > > > > > > > > > >> > > > > > > > > > What do you think about it? I am happy about every > >> > feedback I > >> > > > can > >> > > > > > get > >> > > > > > > > :) > >> > > > > > > > > > > >> > > > > > > > > > Have a nice Sunday, > >> > > > > > > > > > > >> > > > > > > > > > Felix > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > |
Free forum by Nabble | Edit this page |