Building Flink takes long time now =(

[Henry Saputra]

Seemed like after few commits between today and yesterday, building
Flink takes very long time.

I used to be able to run mvn clean install -DskipTests in about 17
mins but after pull this morning, it has been 25 mins and the build
has not complete yet.

I am using MacOSX with JDK7

- Henry

[Henry Saputra]

Ah interesting, on my Linux box it only took about 8 mins.

Something wrong with my MBP env =(

- Henry

On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <[hidden email]> wrote:

> Seemed like after few commits between today and yesterday, building
> Flink takes very long time.
>
> I used to be able to run mvn clean install -DskipTests in about 17
> mins but after pull this morning, it has been 25 mins and the build
> has not complete yet.
>
> I am using MacOSX with JDK7
>
> - Henry

[Stephan Ewen]

Hey Henry!

Recently, the compilation process was changed to use the maven shade plugin
for all projects. This helps to hide libraries that often conflict with
versions used by user code (Guava, ASM, netty) and provide a smoother Flink
experience for users.

Robert documented it in the wiki as well.

This does indeed increase the packaging time when building Flink.

Greetings,
Stephan
Am 13.03.2015 05:20 schrieb "Henry Saputra" <[hidden email]>:

> Ah interesting, on my Linux box it only took about 8 mins.
>
> Something wrong with my MBP env =(
>
> - Henry
>
> On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <[hidden email]>
> wrote:
> > Seemed like after few commits between today and yesterday, building
> > Flink takes very long time.
> >
> > I used to be able to run mvn clean install -DskipTests in about 17
> > mins but after pull this morning, it has been 25 mins and the build
> > has not complete yet.
> >
> > I am using MacOSX with JDK7
> >
> > - Henry
>

[Henry Saputra]

Hi Stephan,

Yep, i am aware abt the new shade changes.
Didn't expect the latency addition would be so large :(

On Friday, March 13, 2015, Stephan Ewen <[hidden email]> wrote:

> Hey Henry!
>
> Recently, the compilation process was changed to use the maven shade plugin
> for all projects. This helps to hide libraries that often conflict with
> versions used by user code (Guava, ASM, netty) and provide a smoother Flink
> experience for users.
>
> Robert documented it in the wiki as well.
>
> This does indeed increase the packaging time when building Flink.
>
> Greetings,
> Stephan
> Am 13.03.2015 05:20 schrieb "Henry Saputra" <[hidden email]
> <javascript:;>>:
>
> > Ah interesting, on my Linux box it only took about 8 mins.
> >
> > Something wrong with my MBP env =(
> >
> > - Henry
> >
> > On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <[hidden email]
> <javascript:;>>
> > wrote:
> > > Seemed like after few commits between today and yesterday, building
> > > Flink takes very long time.
> > >
> > > I used to be able to run mvn clean install -DskipTests in about 17
> > > mins but after pull this morning, it has been 25 mins and the build
> > > has not complete yet.
> > >
> > > I am using MacOSX with JDK7
> > >
> > > - Henry
> >
>

[Henry Saputra]

In my MBP (OSX) is about 40mins to do mvn clean install -DskipTests now =(

Is this about ball park with others or just my machine?

Somehow in my Linux box is about 8 mins, which is surprising.

- Henry

On Fri, Mar 13, 2015 at 1:03 AM, Stephan Ewen <[hidden email]> wrote:

> Hey Henry!
>
> Recently, the compilation process was changed to use the maven shade plugin
> for all projects. This helps to hide libraries that often conflict with
> versions used by user code (Guava, ASM, netty) and provide a smoother Flink
> experience for users.
>
> Robert documented it in the wiki as well.
>
> This does indeed increase the packaging time when building Flink.
>
> Greetings,
> Stephan
> Am 13.03.2015 05:20 schrieb "Henry Saputra" <[hidden email]>:
>
>> Ah interesting, on my Linux box it only took about 8 mins.
>>
>> Something wrong with my MBP env =(
>>
>> - Henry
>>
>> On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <[hidden email]>
>> wrote:
>> > Seemed like after few commits between today and yesterday, building
>> > Flink takes very long time.
>> >
>> > I used to be able to run mvn clean install -DskipTests in about 17
>> > mins but after pull this morning, it has been 25 mins and the build
>> > has not complete yet.
>> >
>> > I am using MacOSX with JDK7
>> >
>> > - Henry
>>

[Fabian Hueske-2]

Just ran "mvn clean install -DskipTests" in 6:46min on my MBP. So 8mins on
Linux sounds more reasonable than 40mins.
Without skipping tests it should be around 18min.

2015-03-13 19:03 GMT+01:00 Henry Saputra <[hidden email]>:

> In my MBP (OSX) is about 40mins to do mvn clean install -DskipTests now =(
>
> Is this about ball park with others or just my machine?
>
> Somehow in my Linux box is about 8 mins, which is surprising.
>
> - Henry
>
> On Fri, Mar 13, 2015 at 1:03 AM, Stephan Ewen <[hidden email]> wrote:
> > Hey Henry!
> >
> > Recently, the compilation process was changed to use the maven shade
> plugin
> > for all projects. This helps to hide libraries that often conflict with
> > versions used by user code (Guava, ASM, netty) and provide a smoother
> Flink
> > experience for users.
> >
> > Robert documented it in the wiki as well.
> >
> > This does indeed increase the packaging time when building Flink.
> >
> > Greetings,
> > Stephan
> > Am 13.03.2015 05:20 schrieb "Henry Saputra" <[hidden email]>:
> >
> >> Ah interesting, on my Linux box it only took about 8 mins.
> >>
> >> Something wrong with my MBP env =(
> >>
> >> - Henry
> >>
> >> On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <
> [hidden email]>
> >> wrote:
> >> > Seemed like after few commits between today and yesterday, building
> >> > Flink takes very long time.
> >> >
> >> > I used to be able to run mvn clean install -DskipTests in about 17
> >> > mins but after pull this morning, it has been 25 mins and the build
> >> > has not complete yet.
> >> >
> >> > I am using MacOSX with JDK7
> >> >
> >> > - Henry
> >>
>

[Henry Saputra]

Thanks Fabian, now I know something wrong with my machine =(


On Fri, Mar 13, 2015 at 11:18 AM, Fabian Hueske <[hidden email]> wrote:

> Just ran "mvn clean install -DskipTests" in 6:46min on my MBP. So 8mins on
> Linux sounds more reasonable than 40mins.
> Without skipping tests it should be around 18min.
>
> 2015-03-13 19:03 GMT+01:00 Henry Saputra <[hidden email]>:
>
>> In my MBP (OSX) is about 40mins to do mvn clean install -DskipTests now =(
>>
>> Is this about ball park with others or just my machine?
>>
>> Somehow in my Linux box is about 8 mins, which is surprising.
>>
>> - Henry
>>
>> On Fri, Mar 13, 2015 at 1:03 AM, Stephan Ewen <[hidden email]> wrote:
>> > Hey Henry!
>> >
>> > Recently, the compilation process was changed to use the maven shade
>> plugin
>> > for all projects. This helps to hide libraries that often conflict with
>> > versions used by user code (Guava, ASM, netty) and provide a smoother
>> Flink
>> > experience for users.
>> >
>> > Robert documented it in the wiki as well.
>> >
>> > This does indeed increase the packaging time when building Flink.
>> >
>> > Greetings,
>> > Stephan
>> > Am 13.03.2015 05:20 schrieb "Henry Saputra" <[hidden email]>:
>> >
>> >> Ah interesting, on my Linux box it only took about 8 mins.
>> >>
>> >> Something wrong with my MBP env =(
>> >>
>> >> - Henry
>> >>
>> >> On Thu, Mar 12, 2015 at 11:37 AM, Henry Saputra <
>> [hidden email]>
>> >> wrote:
>> >> > Seemed like after few commits between today and yesterday, building
>> >> > Flink takes very long time.
>> >> >
>> >> > I used to be able to run mvn clean install -DskipTests in about 17
>> >> > mins but after pull this morning, it has been 25 mins and the build
>> >> > has not complete yet.
>> >> >
>> >> > I am using MacOSX with JDK7
>> >> >
>> >> > - Henry
>> >>
>>