Hi all!
We have by now quite some tests that are not really suitable for execution as part of the regular build process. - Some tests just take very long, for example some of the sorter tests with multiple recursive merges. - Some tests depend on external components. These components are sometimes not stable enough for reliable tests. An example is the ElasticSearchITCase How about we add a dedicated module for these tests, that we execute during release testing, but not as part of the regular build tests? Greetings, Stephan |
+1 for your proposal Stephan. Seems like a good idea to improve build
stability as well as guarding components like the ElasticSearchSink by a test, even though it’s not regularly executed. On Tue, Sep 29, 2015 at 12:34 PM, Stephan Ewen <[hidden email]> wrote: > Hi all! > > We have by now quite some tests that are not really suitable for execution > as part of the regular build process. > > - Some tests just take very long, for example some of the sorter tests > with multiple recursive merges. > > - Some tests depend on external components. These components are > sometimes not stable enough for reliable tests. An example is the > ElasticSearchITCase > > How about we add a dedicated module for these tests, that we execute during > release testing, but not as part of the regular build tests? > > Greetings, > Stephan > |
I agree that some tests are pretty resource intensive and/or unstable...
However I fear that we are ending up fixing our connectors for every release (potentially delaying releases and forcing people to fix tests for changes they didn't introduce) We could try out adding another Travis build which is only executing those heavy tests... On Tue, Sep 29, 2015 at 1:40 PM, Till Rohrmann <[hidden email]> wrote: > +1 for your proposal Stephan. Seems like a good idea to improve build > stability as well as guarding components like the ElasticSearchSink by a > test, even though it’s not regularly executed. > > > On Tue, Sep 29, 2015 at 12:34 PM, Stephan Ewen <[hidden email]> wrote: > > > Hi all! > > > > We have by now quite some tests that are not really suitable for > execution > > as part of the regular build process. > > > > - Some tests just take very long, for example some of the sorter tests > > with multiple recursive merges. > > > > - Some tests depend on external components. These components are > > sometimes not stable enough for reliable tests. An example is the > > ElasticSearchITCase > > > > How about we add a dedicated module for these tests, that we execute > during > > release testing, but not as part of the regular build tests? > > > > Greetings, > > Stephan > > > |
@Robert: This does not mean that we move all connector tests to that
package, but only the ones that we cannot fully control (like ElasticSearch) On Tue, Sep 29, 2015 at 3:49 PM, Robert Metzger <[hidden email]> wrote: > I agree that some tests are pretty resource intensive and/or unstable... > > However I fear that we are ending up fixing our connectors for every > release (potentially delaying releases and forcing people to fix tests for > changes they didn't introduce) > We could try out adding another Travis build which is only executing those > heavy tests... > > > > On Tue, Sep 29, 2015 at 1:40 PM, Till Rohrmann <[hidden email]> > wrote: > > > +1 for your proposal Stephan. Seems like a good idea to improve build > > stability as well as guarding components like the ElasticSearchSink by a > > test, even though it’s not regularly executed. > > > > > > On Tue, Sep 29, 2015 at 12:34 PM, Stephan Ewen <[hidden email]> wrote: > > > > > Hi all! > > > > > > We have by now quite some tests that are not really suitable for > > execution > > > as part of the regular build process. > > > > > > - Some tests just take very long, for example some of the sorter > tests > > > with multiple recursive merges. > > > > > > - Some tests depend on external components. These components are > > > sometimes not stable enough for reliable tests. An example is the > > > ElasticSearchITCase > > > > > > How about we add a dedicated module for these tests, that we execute > > during > > > release testing, but not as part of the regular build tests? > > > > > > Greetings, > > > Stephan > > > > > > |
Free forum by Nabble | Edit this page |