(DEPRECATED) Apache Flink Mailing List archive.

Travis CI

Classic

List

Threaded

8 messages Options

Greg Hogan

Travis CI

We're getting the dreaded "The job exceeded the maximum time limit for
jobs, and has been terminated." error for some recent Travis-CI builds.
https://travis-ci.org/apache/flink/builds/174615801

The docs state that termination will occur when "A job takes longer than 50
minutes on travis-ci.org", which applies to Flink as well as user GitHub
accounts.
https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts

The jobs are running much faster now, as our builds have consistently taken
over an hour and up to an hour and a half. The following recently
successful build looks to be a mix of fast and slow jobs.
https://travis-ci.org/apache/flink/builds/174596630

Greg

Ufuk Celebi-2

Re: Travis CI

I also just noticed it today. We used to work with the 120 minutes limit and the builds took way longer as you said. I don't know what's going on here...

It might be related to some issues they reported a few hours ago (https://twitter.com/traviscistatus), but I can't tell.

I really hope that the new env is temporary (although the reduced build time is indeed nice ;)). Let's monitor this...

– Ufuk

On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email]) wrote:

> We're getting the dreaded "The job exceeded the maximum time limit for
> jobs, and has been terminated." error for some recent Travis-CI builds.
> https://travis-ci.org/apache/flink/builds/174615801
>
> The docs state that termination will occur when "A job takes longer than 50
> minutes on travis-ci.org", which applies to Flink as well as user GitHub
> accounts.
> https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts
>
> The jobs are running much faster now, as our builds have consistently taken
> over an hour and up to an hour and a half. The following recently
> successful build looks to be a mix of fast and slow jobs.
> https://travis-ci.org/apache/flink/builds/174596630
>
> Greg
>

Henry Saputra

Re: Travis CI

Building Flink from root now takes long time due to more and more modules
that we have =(
For me if takes about 50 minutes or more when doing "mvn clean install
-DskipTests"

Should we start adding Maven profiles to help exclude some Maven modules,
like connectors and flink-libraries by default ?

Do you guys see the same behavior?

- Henry

On Thu, Nov 10, 2016 at 1:35 PM, Ufuk Celebi <[hidden email]> wrote:

> I also just noticed it today. We used to work with the 120 minutes limit
> and the builds took way longer as you said. I don't know what's going on
> here...
>
> It might be related to some issues they reported a few hours ago (
> https://twitter.com/traviscistatus), but I can't tell.
>
> I really hope that the new env is temporary (although the reduced build
> time is indeed nice ;)). Let's monitor this...
>
> – Ufuk
>
> On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email]) wrote:
> > We're getting the dreaded "The job exceeded the maximum time limit for
> > jobs, and has been terminated." error for some recent Travis-CI builds.
> > https://travis-ci.org/apache/flink/builds/174615801
> >
> > The docs state that termination will occur when "A job takes longer than
> 50
> > minutes on travis-ci.org", which applies to Flink as well as user GitHub
> > accounts.
> > https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts
> >
> > The jobs are running much faster now, as our builds have consistently
> taken
> > over an hour and up to an hour and a half. The following recently
> > successful build looks to be a mix of fast and slow jobs.
> > https://travis-ci.org/apache/flink/builds/174596630
> >
> > Greg
> >
>
>

Robert Metzger

Re: Travis CI

On my machine, I need ~10 minutes to do a clean install without tests. Not
sure what is causing the slow builds in your environment.
I think our builds are both IO (shading) and CPU (Scala compiler)
intensive. If one is not well equipped you'll have these build times.

I contacted the Travis support to hear if there's anything we can do
regarding the time limit.

On Wed, Dec 14, 2016 at 3:52 AM, Henry Saputra <[hidden email]>
wrote:

> Building Flink from root now takes long time due to more and more modules
> that we have =(
> For me if takes about 50 minutes or more when doing "mvn clean install
> -DskipTests"
>
> Should we start adding Maven profiles to help exclude some Maven modules,
> like connectors and flink-libraries by default ?
>
> Do you guys see the same behavior?
>
> - Henry
>
> On Thu, Nov 10, 2016 at 1:35 PM, Ufuk Celebi <[hidden email]> wrote:
>
> > I also just noticed it today. We used to work with the 120 minutes limit
> > and the builds took way longer as you said. I don't know what's going on
> > here...
> >
> > It might be related to some issues they reported a few hours ago (
> > https://twitter.com/traviscistatus), but I can't tell.
> >
> > I really hope that the new env is temporary (although the reduced build
> > time is indeed nice ;)). Let's monitor this...
> >
> > – Ufuk
> >
> > On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email]) wrote:
> > > We're getting the dreaded "The job exceeded the maximum time limit for
> > > jobs, and has been terminated." error for some recent Travis-CI builds.
> > > https://travis-ci.org/apache/flink/builds/174615801
> > >
> > > The docs state that termination will occur when "A job takes longer
> than
> > 50
> > > minutes on travis-ci.org", which applies to Flink as well as user
> GitHub
> > > accounts.
> > > https://docs.travis-ci.com/user/customizing-the-build/#Build-Timeouts
> > >
> > > The jobs are running much faster now, as our builds have consistently
> > taken
> > > over an hour and up to an hour and a half. The following recently
> > > successful build looks to be a mix of fast and slow jobs.
> > > https://travis-ci.org/apache/flink/builds/174596630
> > >
> > > Greg
> > >
> >
> >
>

Stephan Ewen

Re: Travis CI

I am starting to think that we should go a step further from "build
profiles".
How about splitting the repository up into multiple repositories that can
be built independently? We could for example do

- flink-core (core, apis, runtime, clients)
- flink-libraries (gelly, ml, cep, table, scala shell, python)
- flink-connectors
- flink-contrib

I think we should still retain one community and one PMC, so these would
not be sub-projects, but simply independently organized repositories and
build units, for better maintainability.

I am curious to hear what others think. If there are no strong concerns, I
would start a dedicated discussion thread about that after the 1.2 release.

Greetings,
Stephan

PS: We need a git wizard to help us migrate modules without loosing commit
history. I think it can work with some sophisticated "git filter-branch"
magic...

On Wed, Dec 14, 2016 at 11:34 AM, Robert Metzger <[hidden email]>
wrote:

> On my machine, I need ~10 minutes to do a clean install without tests. Not
> sure what is causing the slow builds in your environment.
> I think our builds are both IO (shading) and CPU (Scala compiler)
> intensive. If one is not well equipped you'll have these build times.
>
> I contacted the Travis support to hear if there's anything we can do
> regarding the time limit.
>
> On Wed, Dec 14, 2016 at 3:52 AM, Henry Saputra <[hidden email]>
> wrote:
>
> > Building Flink from root now takes long time due to more and more modules
> > that we have =(
> > For me if takes about 50 minutes or more when doing "mvn clean install
> > -DskipTests"
> >
> > Should we start adding Maven profiles to help exclude some Maven modules,
> > like connectors and flink-libraries by default ?
> >
> > Do you guys see the same behavior?
> >
> > - Henry
> >
> > On Thu, Nov 10, 2016 at 1:35 PM, Ufuk Celebi <[hidden email]> wrote:
> >
> > > I also just noticed it today. We used to work with the 120 minutes
> limit
> > > and the builds took way longer as you said. I don't know what's going
> on
> > > here...
> > >
> > > It might be related to some issues they reported a few hours ago (
> > > https://twitter.com/traviscistatus), but I can't tell.
> > >
> > > I really hope that the new env is temporary (although the reduced build
> > > time is indeed nice ;)). Let's monitor this...
> > >
> > > – Ufuk
> > >
> > > On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email])
> wrote:
> > > > We're getting the dreaded "The job exceeded the maximum time limit
> for
> > > > jobs, and has been terminated." error for some recent Travis-CI
> builds.
> > > > https://travis-ci.org/apache/flink/builds/174615801
> > > >
> > > > The docs state that termination will occur when "A job takes longer
> > than
> > > 50
> > > > minutes on travis-ci.org", which applies to Flink as well as user
> > GitHub
> > > > accounts.
> > > > https://docs.travis-ci.com/user/customizing-the-build/#
> Build-Timeouts
> > > >
> > > > The jobs are running much faster now, as our builds have consistently
> > > taken
> > > > over an hour and up to an hour and a half. The following recently
> > > > successful build looks to be a mix of fast and slow jobs.
> > > > https://travis-ci.org/apache/flink/builds/174596630
> > > >
> > > > Greg
> > > >
> > >
> > >
> >
>

Henry Saputra

Re: Travis CI

In reply to this post by Robert Metzger

Thanks for the insight, Robert.

I have new MacbookPro so I think should have enough CPU to build.

Seemed like the long time taken mostly on shading and pulling the world
(dependencies)

Do you have anything in your settings.xml to set the updatePolicy for
central repo?

- Henry

On Wed, Dec 14, 2016 at 2:34 AM, Robert Metzger <[hidden email]> wrote:

Till Rohrmann

Re: Travis CI

I'm in favour of splitting the project up into multiple repositories. This
will decrease building and testing time locally as well as on Travis. The
downside, however, will be that if not built together, conflicting changes
between different modules might go unnoticed until the release or the next
time someone commits to the respective sub repository.

Cheers,
Till

On Wed, Dec 14, 2016 at 4:44 PM, Henry Saputra <[hidden email]>
wrote:

> Thanks for the insight, Robert.
>
> I have new MacbookPro so I think should have enough CPU to build.
>
> Seemed like the long time taken mostly on shading and pulling the world
> (dependencies)
>
> Do you have anything in your settings.xml to set the updatePolicy for
> central repo?
>
> - Henry
>
> On Wed, Dec 14, 2016 at 2:34 AM, Robert Metzger <[hidden email]>
> wrote:
>
> > On my machine, I need ~10 minutes to do a clean install without tests.
> Not
> > sure what is causing the slow builds in your environment.
> > I think our builds are both IO (shading) and CPU (Scala compiler)
> > intensive. If one is not well equipped you'll have these build times.
> >
> > I contacted the Travis support to hear if there's anything we can do
> > regarding the time limit.
> >
> > On Wed, Dec 14, 2016 at 3:52 AM, Henry Saputra <[hidden email]>
> > wrote:
> >
> > > Building Flink from root now takes long time due to more and more
> modules
> > > that we have =(
> > > For me if takes about 50 minutes or more when doing "mvn clean install
> > > -DskipTests"
> > >
> > > Should we start adding Maven profiles to help exclude some Maven
> modules,
> > > like connectors and flink-libraries by default ?
> > >
> > > Do you guys see the same behavior?
> > >
> > > - Henry
> > >
> > > On Thu, Nov 10, 2016 at 1:35 PM, Ufuk Celebi <[hidden email]> wrote:
> > >
> > > > I also just noticed it today. We used to work with the 120 minutes
> > limit
> > > > and the builds took way longer as you said. I don't know what's going
> > on
> > > > here...
> > > >
> > > > It might be related to some issues they reported a few hours ago (
> > > > https://twitter.com/traviscistatus), but I can't tell.
> > > >
> > > > I really hope that the new env is temporary (although the reduced
> build
> > > > time is indeed nice ;)). Let's monitor this...
> > > >
> > > > – Ufuk
> > > >
> > > > On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email])
> > wrote:
> > > > > We're getting the dreaded "The job exceeded the maximum time limit
> > for
> > > > > jobs, and has been terminated." error for some recent Travis-CI
> > builds.
> > > > > https://travis-ci.org/apache/flink/builds/174615801
> > > > >
> > > > > The docs state that termination will occur when "A job takes longer
> > > than
> > > > 50
> > > > > minutes on travis-ci.org", which applies to Flink as well as user
> > > GitHub
> > > > > accounts.
> > > > > https://docs.travis-ci.com/user/customizing-the-build/#
> > Build-Timeouts
> > > > >
> > > > > The jobs are running much faster now, as our builds have
> consistently
> > > > taken
> > > > > over an hour and up to an hour and a half. The following recently
> > > > > successful build looks to be a mix of fast and slow jobs.
> > > > > https://travis-ci.org/apache/flink/builds/174596630
> > > > >
> > > > > Greg
> > > > >
> > > >
> > > >
> > >
> >
>

Stephan Ewen

Re: Travis CI

It's going to be a fun discussion. The most obvious questions have been
brought up already: Release cycles and API stability towards usage by code
in other repositories.

Looking forward :-)

On Thu, Dec 15, 2016 at 11:48 AM, Till Rohrmann <[hidden email]>
wrote:

> I'm in favour of splitting the project up into multiple repositories. This
> will decrease building and testing time locally as well as on Travis. The
> downside, however, will be that if not built together, conflicting changes
> between different modules might go unnoticed until the release or the next
> time someone commits to the respective sub repository.
>
> Cheers,
> Till
>
> On Wed, Dec 14, 2016 at 4:44 PM, Henry Saputra <[hidden email]>
> wrote:
>
> > Thanks for the insight, Robert.
> >
> > I have new MacbookPro so I think should have enough CPU to build.
> >
> > Seemed like the long time taken mostly on shading and pulling the world
> > (dependencies)
> >
> > Do you have anything in your settings.xml to set the updatePolicy for
> > central repo?
> >
> > - Henry
> >
> > On Wed, Dec 14, 2016 at 2:34 AM, Robert Metzger <[hidden email]>
> > wrote:
> >
> > > On my machine, I need ~10 minutes to do a clean install without tests.
> > Not
> > > sure what is causing the slow builds in your environment.
> > > I think our builds are both IO (shading) and CPU (Scala compiler)
> > > intensive. If one is not well equipped you'll have these build times.
> > >
> > > I contacted the Travis support to hear if there's anything we can do
> > > regarding the time limit.
> > >
> > > On Wed, Dec 14, 2016 at 3:52 AM, Henry Saputra <
> [hidden email]>
> > > wrote:
> > >
> > > > Building Flink from root now takes long time due to more and more
> > modules
> > > > that we have =(
> > > > For me if takes about 50 minutes or more when doing "mvn clean
> install
> > > > -DskipTests"
> > > >
> > > > Should we start adding Maven profiles to help exclude some Maven
> > modules,
> > > > like connectors and flink-libraries by default ?
> > > >
> > > > Do you guys see the same behavior?
> > > >
> > > > - Henry
> > > >
> > > > On Thu, Nov 10, 2016 at 1:35 PM, Ufuk Celebi <[hidden email]> wrote:
> > > >
> > > > > I also just noticed it today. We used to work with the 120 minutes
> > > limit
> > > > > and the builds took way longer as you said. I don't know what's
> going
> > > on
> > > > > here...
> > > > >
> > > > > It might be related to some issues they reported a few hours ago (
> > > > > https://twitter.com/traviscistatus), but I can't tell.
> > > > >
> > > > > I really hope that the new env is temporary (although the reduced
> > build
> > > > > time is indeed nice ;)). Let's monitor this...
> > > > >
> > > > > – Ufuk
> > > > >
> > > > > On 10 November 2016 at 22:15:51, Greg Hogan ([hidden email])
> > > wrote:
> > > > > > We're getting the dreaded "The job exceeded the maximum time
> limit
> > > for
> > > > > > jobs, and has been terminated." error for some recent Travis-CI
> > > builds.
> > > > > > https://travis-ci.org/apache/flink/builds/174615801
> > > > > >
> > > > > > The docs state that termination will occur when "A job takes
> longer
> > > > than
> > > > > 50
> > > > > > minutes on travis-ci.org", which applies to Flink as well as
> user
> > > > GitHub
> > > > > > accounts.
> > > > > > https://docs.travis-ci.com/user/customizing-the-build/#
> > > Build-Timeouts
> > > > > >
> > > > > > The jobs are running much faster now, as our builds have
> > consistently
> > > > > taken
> > > > > > over an hour and up to an hour and a half. The following recently
> > > > > > successful build looks to be a mix of fast and slow jobs.
> > > > > > https://travis-ci.org/apache/flink/builds/174596630
> > > > > >
> > > > > > Greg
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>