[DISCUSS] First release of flink-shaded

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] First release of flink-shaded

Chesnay Schepler-3
Hello,

I would like to kick off the first release of flink-shaded.

In the current state the release would include shaded dependencies for
asm, guava and netty.

All changes required for the integration into Flink have been prepared
and tested locally and on yarn.

In order for these changes to not lie around for too long i would like
to the first release soon, as I expect
the additions of hadoop into flink-shaded to take some time/discussions
(I also want to spend some time
to understand what we're shading for hadoop and why).

Regards,
Chesnay


Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Aljoscha Krettek-2
+1

Sounds good! I have followed the process on the various issues and the new shaded repository and the changes seem straightforward (more or less ;-))

> On 3. Jul 2017, at 14:32, Chesnay Schepler <[hidden email]> wrote:
>
> Hello,
>
> I would like to kick off the first release of flink-shaded.
>
> In the current state the release would include shaded dependencies for asm, guava and netty.
>
> All changes required for the integration into Flink have been prepared and tested locally and on yarn.
>
> In order for these changes to not lie around for too long i would like to the first release soon, as I expect
> the additions of hadoop into flink-shaded to take some time/discussions (I also want to spend some time
> to understand what we're shading for hadoop and why).
>
> Regards,
> Chesnay
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Fabian Hueske-2
In reply to this post by Chesnay Schepler-3
Thanks for working on this Chesnay.
I think releasing flink-shaded and integrating it in the build before
touching the Hadoop dependencies it sounds like a good plan.

+1

Cheers, Fabian

2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:

> Hello,
>
> I would like to kick off the first release of flink-shaded.
>
> In the current state the release would include shaded dependencies for
> asm, guava and netty.
>
> All changes required for the integration into Flink have been prepared and
> tested locally and on yarn.
>
> In order for these changes to not lie around for too long i would like to
> the first release soon, as I expect
> the additions of hadoop into flink-shaded to take some time/discussions (I
> also want to spend some time
> to understand what we're shading for hadoop and why).
>
> Regards,
> Chesnay
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Stephan Ewen
Other non-Hadoop dependencies that I think are important are the Kryo
dependencies.

Do we want them in the first release, or tackle them later?

Specifically, I think we need

  - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
currently use
  - flink-shaded-kryo3 which includes kryo3 with the latest chill version




On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]> wrote:

> Thanks for working on this Chesnay.
> I think releasing flink-shaded and integrating it in the build before
> touching the Hadoop dependencies it sounds like a good plan.
>
> +1
>
> Cheers, Fabian
>
> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
>
> > Hello,
> >
> > I would like to kick off the first release of flink-shaded.
> >
> > In the current state the release would include shaded dependencies for
> > asm, guava and netty.
> >
> > All changes required for the integration into Flink have been prepared
> and
> > tested locally and on yarn.
> >
> > In order for these changes to not lie around for too long i would like to
> > the first release soon, as I expect
> > the additions of hadoop into flink-shaded to take some time/discussions
> (I
> > also want to spend some time
> > to understand what we're shading for hadoop and why).
> >
> > Regards,
> > Chesnay
> >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Chesnay Schepler-3
I would tackle Kryo later as it is exposed through the API
(StreamExecutionEnvironment#addDefaultKryoSerializer)
which will make the shading process more difficult than for the modules
that are done so far.

Technically this doesn't prevent us from adding a flink-shaded-kryo2/3
module right now, but so far I went with the approach
of creating the shaded module and actually integrating it into Flink
before submitting it to flink-shaded to make sure
everything actually works. With kryo this approach may take a while so I
would suggest to do it later.

On 03.07.2017 17:04, Stephan Ewen wrote:

> Other non-Hadoop dependencies that I think are important are the Kryo
> dependencies.
>
> Do we want them in the first release, or tackle them later?
>
> Specifically, I think we need
>
>    - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
> currently use
>    - flink-shaded-kryo3 which includes kryo3 with the latest chill version
>
>
>
>
> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]> wrote:
>
>> Thanks for working on this Chesnay.
>> I think releasing flink-shaded and integrating it in the build before
>> touching the Hadoop dependencies it sounds like a good plan.
>>
>> +1
>>
>> Cheers, Fabian
>>
>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
>>
>>> Hello,
>>>
>>> I would like to kick off the first release of flink-shaded.
>>>
>>> In the current state the release would include shaded dependencies for
>>> asm, guava and netty.
>>>
>>> All changes required for the integration into Flink have been prepared
>> and
>>> tested locally and on yarn.
>>>
>>> In order for these changes to not lie around for too long i would like to
>>> the first release soon, as I expect
>>> the additions of hadoop into flink-shaded to take some time/discussions
>> (I
>>> also want to spend some time
>>> to understand what we're shading for hadoop and why).
>>>
>>> Regards,
>>> Chesnay
>>>
>>>
>>>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Aljoscha Krettek-2
Is it even possible to shade Kryo with it being in the public API? I don’t think it is (without looking into this to deeply, though).

> On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]> wrote:
>
> I would tackle Kryo later as it is exposed through the API (StreamExecutionEnvironment#addDefaultKryoSerializer)
> which will make the shading process more difficult than for the modules that are done so far.
>
> Technically this doesn't prevent us from adding a flink-shaded-kryo2/3 module right now, but so far I went with the approach
> of creating the shaded module and actually integrating it into Flink before submitting it to flink-shaded to make sure
> everything actually works. With kryo this approach may take a while so I would suggest to do it later.
>
> On 03.07.2017 17:04, Stephan Ewen wrote:
>> Other non-Hadoop dependencies that I think are important are the Kryo
>> dependencies.
>>
>> Do we want them in the first release, or tackle them later?
>>
>> Specifically, I think we need
>>
>>   - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
>> currently use
>>   - flink-shaded-kryo3 which includes kryo3 with the latest chill version
>>
>>
>>
>>
>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]> wrote:
>>
>>> Thanks for working on this Chesnay.
>>> I think releasing flink-shaded and integrating it in the build before
>>> touching the Hadoop dependencies it sounds like a good plan.
>>>
>>> +1
>>>
>>> Cheers, Fabian
>>>
>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
>>>
>>>> Hello,
>>>>
>>>> I would like to kick off the first release of flink-shaded.
>>>>
>>>> In the current state the release would include shaded dependencies for
>>>> asm, guava and netty.
>>>>
>>>> All changes required for the integration into Flink have been prepared
>>> and
>>>> tested locally and on yarn.
>>>>
>>>> In order for these changes to not lie around for too long i would like to
>>>> the first release soon, as I expect
>>>> the additions of hadoop into flink-shaded to take some time/discussions
>>> (I
>>>> also want to spend some time
>>>> to understand what we're shading for hadoop and why).
>>>>
>>>> Regards,
>>>> Chesnay
>>>>
>>>>
>>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Ted Yu
Currently StreamExecutionEnvironment exposes Kryo thru the following:

import com.esotericsoftware.kryo.Serializer;

I think shading is possible since we can declare the Serializer class to
come from shaded Kryo namespace.

On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <[hidden email]>
wrote:

> Is it even possible to shade Kryo with it being in the public API? I don’t
> think it is (without looking into this to deeply, though).
>
> > On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]> wrote:
> >
> > I would tackle Kryo later as it is exposed through the API
> (StreamExecutionEnvironment#addDefaultKryoSerializer)
> > which will make the shading process more difficult than for the modules
> that are done so far.
> >
> > Technically this doesn't prevent us from adding a flink-shaded-kryo2/3
> module right now, but so far I went with the approach
> > of creating the shaded module and actually integrating it into Flink
> before submitting it to flink-shaded to make sure
> > everything actually works. With kryo this approach may take a while so I
> would suggest to do it later.
> >
> > On 03.07.2017 17:04, Stephan Ewen wrote:
> >> Other non-Hadoop dependencies that I think are important are the Kryo
> >> dependencies.
> >>
> >> Do we want them in the first release, or tackle them later?
> >>
> >> Specifically, I think we need
> >>
> >>   - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
> >> currently use
> >>   - flink-shaded-kryo3 which includes kryo3 with the latest chill
> version
> >>
> >>
> >>
> >>
> >> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]>
> wrote:
> >>
> >>> Thanks for working on this Chesnay.
> >>> I think releasing flink-shaded and integrating it in the build before
> >>> touching the Hadoop dependencies it sounds like a good plan.
> >>>
> >>> +1
> >>>
> >>> Cheers, Fabian
> >>>
> >>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
> >>>
> >>>> Hello,
> >>>>
> >>>> I would like to kick off the first release of flink-shaded.
> >>>>
> >>>> In the current state the release would include shaded dependencies for
> >>>> asm, guava and netty.
> >>>>
> >>>> All changes required for the integration into Flink have been prepared
> >>> and
> >>>> tested locally and on yarn.
> >>>>
> >>>> In order for these changes to not lie around for too long i would
> like to
> >>>> the first release soon, as I expect
> >>>> the additions of hadoop into flink-shaded to take some
> time/discussions
> >>> (I
> >>>> also want to spend some time
> >>>> to understand what we're shading for hadoop and why).
> >>>>
> >>>> Regards,
> >>>> Chesnay
> >>>>
> >>>>
> >>>>
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Aljoscha Krettek-2
Yes, that’s definitely possible. I was referring to the fact that this would break the user-facing API and these are methods declared as @Public, i.e. they cannot change.


> On 3. Jul 2017, at 18:45, Ted Yu <[hidden email]> wrote:
>
> Currently StreamExecutionEnvironment exposes Kryo thru the following:
>
> import com.esotericsoftware.kryo.Serializer;
>
> I think shading is possible since we can declare the Serializer class to
> come from shaded Kryo namespace.
>
> On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> Is it even possible to shade Kryo with it being in the public API? I don’t
>> think it is (without looking into this to deeply, though).
>>
>>> On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]> wrote:
>>>
>>> I would tackle Kryo later as it is exposed through the API
>> (StreamExecutionEnvironment#addDefaultKryoSerializer)
>>> which will make the shading process more difficult than for the modules
>> that are done so far.
>>>
>>> Technically this doesn't prevent us from adding a flink-shaded-kryo2/3
>> module right now, but so far I went with the approach
>>> of creating the shaded module and actually integrating it into Flink
>> before submitting it to flink-shaded to make sure
>>> everything actually works. With kryo this approach may take a while so I
>> would suggest to do it later.
>>>
>>> On 03.07.2017 17:04, Stephan Ewen wrote:
>>>> Other non-Hadoop dependencies that I think are important are the Kryo
>>>> dependencies.
>>>>
>>>> Do we want them in the first release, or tackle them later?
>>>>
>>>> Specifically, I think we need
>>>>
>>>>  - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
>>>> currently use
>>>>  - flink-shaded-kryo3 which includes kryo3 with the latest chill
>> version
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]>
>> wrote:
>>>>
>>>>> Thanks for working on this Chesnay.
>>>>> I think releasing flink-shaded and integrating it in the build before
>>>>> touching the Hadoop dependencies it sounds like a good plan.
>>>>>
>>>>> +1
>>>>>
>>>>> Cheers, Fabian
>>>>>
>>>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I would like to kick off the first release of flink-shaded.
>>>>>>
>>>>>> In the current state the release would include shaded dependencies for
>>>>>> asm, guava and netty.
>>>>>>
>>>>>> All changes required for the integration into Flink have been prepared
>>>>> and
>>>>>> tested locally and on yarn.
>>>>>>
>>>>>> In order for these changes to not lie around for too long i would
>> like to
>>>>>> the first release soon, as I expect
>>>>>> the additions of hadoop into flink-shaded to take some
>> time/discussions
>>>>> (I
>>>>>> also want to spend some time
>>>>>> to understand what we're shading for hadoop and why).
>>>>>>
>>>>>> Regards,
>>>>>> Chesnay
>>>>>>
>>>>>>
>>>>>>
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Robert Metzger
+1 to go ahead with a release.

I can offer to take care of all the PMC-related actions in the release
process.


On Mon, Jul 3, 2017 at 6:50 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Yes, that’s definitely possible. I was referring to the fact that this
> would break the user-facing API and these are methods declared as @Public,
> i.e. they cannot change.
>
>
> > On 3. Jul 2017, at 18:45, Ted Yu <[hidden email]> wrote:
> >
> > Currently StreamExecutionEnvironment exposes Kryo thru the following:
> >
> > import com.esotericsoftware.kryo.Serializer;
> >
> > I think shading is possible since we can declare the Serializer class to
> > come from shaded Kryo namespace.
> >
> > On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> >> Is it even possible to shade Kryo with it being in the public API? I
> don’t
> >> think it is (without looking into this to deeply, though).
> >>
> >>> On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]> wrote:
> >>>
> >>> I would tackle Kryo later as it is exposed through the API
> >> (StreamExecutionEnvironment#addDefaultKryoSerializer)
> >>> which will make the shading process more difficult than for the modules
> >> that are done so far.
> >>>
> >>> Technically this doesn't prevent us from adding a flink-shaded-kryo2/3
> >> module right now, but so far I went with the approach
> >>> of creating the shaded module and actually integrating it into Flink
> >> before submitting it to flink-shaded to make sure
> >>> everything actually works. With kryo this approach may take a while so
> I
> >> would suggest to do it later.
> >>>
> >>> On 03.07.2017 17:04, Stephan Ewen wrote:
> >>>> Other non-Hadoop dependencies that I think are important are the Kryo
> >>>> dependencies.
> >>>>
> >>>> Do we want them in the first release, or tackle them later?
> >>>>
> >>>> Specifically, I think we need
> >>>>
> >>>>  - flink-shaded-kryo2 which includes kryo2 plus chill in the version
> we
> >>>> currently use
> >>>>  - flink-shaded-kryo3 which includes kryo3 with the latest chill
> >> version
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]>
> >> wrote:
> >>>>
> >>>>> Thanks for working on this Chesnay.
> >>>>> I think releasing flink-shaded and integrating it in the build before
> >>>>> touching the Hadoop dependencies it sounds like a good plan.
> >>>>>
> >>>>> +1
> >>>>>
> >>>>> Cheers, Fabian
> >>>>>
> >>>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
> >>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I would like to kick off the first release of flink-shaded.
> >>>>>>
> >>>>>> In the current state the release would include shaded dependencies
> for
> >>>>>> asm, guava and netty.
> >>>>>>
> >>>>>> All changes required for the integration into Flink have been
> prepared
> >>>>> and
> >>>>>> tested locally and on yarn.
> >>>>>>
> >>>>>> In order for these changes to not lie around for too long i would
> >> like to
> >>>>>> the first release soon, as I expect
> >>>>>> the additions of hadoop into flink-shaded to take some
> >> time/discussions
> >>>>> (I
> >>>>>> also want to spend some time
> >>>>>> to understand what we're shading for hadoop and why).
> >>>>>>
> >>>>>> Regards,
> >>>>>> Chesnay
> >>>>>>
> >>>>>>
> >>>>>>
> >>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Stephan Ewen
+1

I chatted with Chesnay privately, bringing up the issue that we need to
have the License files added to the shaded jar files to properly obey BSD /
MIT licenses (similarly as we did in Flink for the 1.3.0 release).

Other than that, good to go!

On Mon, Jul 10, 2017 at 8:42 PM, Robert Metzger <[hidden email]> wrote:

> +1 to go ahead with a release.
>
> I can offer to take care of all the PMC-related actions in the release
> process.
>
>
> On Mon, Jul 3, 2017 at 6:50 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
> > Yes, that’s definitely possible. I was referring to the fact that this
> > would break the user-facing API and these are methods declared as
> @Public,
> > i.e. they cannot change.
> >
> >
> > > On 3. Jul 2017, at 18:45, Ted Yu <[hidden email]> wrote:
> > >
> > > Currently StreamExecutionEnvironment exposes Kryo thru the following:
> > >
> > > import com.esotericsoftware.kryo.Serializer;
> > >
> > > I think shading is possible since we can declare the Serializer class
> to
> > > come from shaded Kryo namespace.
> > >
> > > On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <[hidden email]>
> > > wrote:
> > >
> > >> Is it even possible to shade Kryo with it being in the public API? I
> > don’t
> > >> think it is (without looking into this to deeply, though).
> > >>
> > >>> On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]>
> wrote:
> > >>>
> > >>> I would tackle Kryo later as it is exposed through the API
> > >> (StreamExecutionEnvironment#addDefaultKryoSerializer)
> > >>> which will make the shading process more difficult than for the
> modules
> > >> that are done so far.
> > >>>
> > >>> Technically this doesn't prevent us from adding a
> flink-shaded-kryo2/3
> > >> module right now, but so far I went with the approach
> > >>> of creating the shaded module and actually integrating it into Flink
> > >> before submitting it to flink-shaded to make sure
> > >>> everything actually works. With kryo this approach may take a while
> so
> > I
> > >> would suggest to do it later.
> > >>>
> > >>> On 03.07.2017 17:04, Stephan Ewen wrote:
> > >>>> Other non-Hadoop dependencies that I think are important are the
> Kryo
> > >>>> dependencies.
> > >>>>
> > >>>> Do we want them in the first release, or tackle them later?
> > >>>>
> > >>>> Specifically, I think we need
> > >>>>
> > >>>>  - flink-shaded-kryo2 which includes kryo2 plus chill in the version
> > we
> > >>>> currently use
> > >>>>  - flink-shaded-kryo3 which includes kryo3 with the latest chill
> > >> version
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]>
> > >> wrote:
> > >>>>
> > >>>>> Thanks for working on this Chesnay.
> > >>>>> I think releasing flink-shaded and integrating it in the build
> before
> > >>>>> touching the Hadoop dependencies it sounds like a good plan.
> > >>>>>
> > >>>>> +1
> > >>>>>
> > >>>>> Cheers, Fabian
> > >>>>>
> > >>>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
> > >>>>>
> > >>>>>> Hello,
> > >>>>>>
> > >>>>>> I would like to kick off the first release of flink-shaded.
> > >>>>>>
> > >>>>>> In the current state the release would include shaded dependencies
> > for
> > >>>>>> asm, guava and netty.
> > >>>>>>
> > >>>>>> All changes required for the integration into Flink have been
> > prepared
> > >>>>> and
> > >>>>>> tested locally and on yarn.
> > >>>>>>
> > >>>>>> In order for these changes to not lie around for too long i would
> > >> like to
> > >>>>>> the first release soon, as I expect
> > >>>>>> the additions of hadoop into flink-shaded to take some
> > >> time/discussions
> > >>>>> (I
> > >>>>>> also want to spend some time
> > >>>>>> to understand what we're shading for hadoop and why).
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> Chesnay
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>
> > >>
> > >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] First release of flink-shaded

Chesnay Schepler-3
I've opened a PR to include non-apache licenses in the jars:
https://github.com/apache/flink-shaded/pull/7

On 12.07.2017 16:25, Stephan Ewen wrote:

> +1
>
> I chatted with Chesnay privately, bringing up the issue that we need to
> have the License files added to the shaded jar files to properly obey BSD /
> MIT licenses (similarly as we did in Flink for the 1.3.0 release).
>
> Other than that, good to go!
>
> On Mon, Jul 10, 2017 at 8:42 PM, Robert Metzger <[hidden email]> wrote:
>
>> +1 to go ahead with a release.
>>
>> I can offer to take care of all the PMC-related actions in the release
>> process.
>>
>>
>> On Mon, Jul 3, 2017 at 6:50 PM, Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>>> Yes, that’s definitely possible. I was referring to the fact that this
>>> would break the user-facing API and these are methods declared as
>> @Public,
>>> i.e. they cannot change.
>>>
>>>
>>>> On 3. Jul 2017, at 18:45, Ted Yu <[hidden email]> wrote:
>>>>
>>>> Currently StreamExecutionEnvironment exposes Kryo thru the following:
>>>>
>>>> import com.esotericsoftware.kryo.Serializer;
>>>>
>>>> I think shading is possible since we can declare the Serializer class
>> to
>>>> come from shaded Kryo namespace.
>>>>
>>>> On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <[hidden email]>
>>>> wrote:
>>>>
>>>>> Is it even possible to shade Kryo with it being in the public API? I
>>> don’t
>>>>> think it is (without looking into this to deeply, though).
>>>>>
>>>>>> On 3. Jul 2017, at 17:23, Chesnay Schepler <[hidden email]>
>> wrote:
>>>>>> I would tackle Kryo later as it is exposed through the API
>>>>> (StreamExecutionEnvironment#addDefaultKryoSerializer)
>>>>>> which will make the shading process more difficult than for the
>> modules
>>>>> that are done so far.
>>>>>> Technically this doesn't prevent us from adding a
>> flink-shaded-kryo2/3
>>>>> module right now, but so far I went with the approach
>>>>>> of creating the shaded module and actually integrating it into Flink
>>>>> before submitting it to flink-shaded to make sure
>>>>>> everything actually works. With kryo this approach may take a while
>> so
>>> I
>>>>> would suggest to do it later.
>>>>>> On 03.07.2017 17:04, Stephan Ewen wrote:
>>>>>>> Other non-Hadoop dependencies that I think are important are the
>> Kryo
>>>>>>> dependencies.
>>>>>>>
>>>>>>> Do we want them in the first release, or tackle them later?
>>>>>>>
>>>>>>> Specifically, I think we need
>>>>>>>
>>>>>>>   - flink-shaded-kryo2 which includes kryo2 plus chill in the version
>>> we
>>>>>>> currently use
>>>>>>>   - flink-shaded-kryo3 which includes kryo3 with the latest chill
>>>>> version
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <[hidden email]>
>>>>> wrote:
>>>>>>>> Thanks for working on this Chesnay.
>>>>>>>> I think releasing flink-shaded and integrating it in the build
>> before
>>>>>>>> touching the Hadoop dependencies it sounds like a good plan.
>>>>>>>>
>>>>>>>> +1
>>>>>>>>
>>>>>>>> Cheers, Fabian
>>>>>>>>
>>>>>>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <[hidden email]>:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I would like to kick off the first release of flink-shaded.
>>>>>>>>>
>>>>>>>>> In the current state the release would include shaded dependencies
>>> for
>>>>>>>>> asm, guava and netty.
>>>>>>>>>
>>>>>>>>> All changes required for the integration into Flink have been
>>> prepared
>>>>>>>> and
>>>>>>>>> tested locally and on yarn.
>>>>>>>>>
>>>>>>>>> In order for these changes to not lie around for too long i would
>>>>> like to
>>>>>>>>> the first release soon, as I expect
>>>>>>>>> the additions of hadoop into flink-shaded to take some
>>>>> time/discussions
>>>>>>>> (I
>>>>>>>>> also want to spend some time
>>>>>>>>> to understand what we're shading for hadoop and why).
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Chesnay
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>