How to handle tests that need docker?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How to handle tests that need docker?

Niels Basjes
Hi,

I'm working together with Richard Deurwaarder on the PubSub Source and Sink
https://issues.apache.org/jira/browse/FLINK-9311

We currently have most of the code done and we also have several tests that
verify the behavior by actually reading and writing messages to and from
PubSub.
These tests use a docker image provided by Google that includes the pubsub
emulator.

Starting and stopping this thing is really fast.
Starting up, creating topic, running a few tests and cleaning up takes
about 5 seconds.

Although the complete set of tests run on my machine in about 20-25 seconds
it does mean that anyone doing that MUST have docker installed locally.

So my question to the committers: Do we simply leave these tests "always
on" or do you want them to be "off by default" or something else.
I case you do not want the "always on" then please indicate how we should
implement the "on"/"off" switch for these tests.

Thanks.

--
Best regards / Met vriendelijke groeten,

Niels Basjes
Reply | Threaded
Open this post in threaded view
|

ODP: How to handle tests that need docker?

Dominik Wosiński
Imho,
Such tests should be off by default as they impose some requirements on the people trying to build Flink from source. And one would have to install docker only for the tests to pass. But of course they should be enabled for CI builds.

Best Regards,
Dominik.

Wysłane z aplikacji Poczta dla Windows 10

Od: Niels Basjes
Wysłano: czwartek, 16 sierpnia 2018 14:05
Do: [hidden email]
Temat: How to handle tests that need docker?

Hi,

I'm working together with Richard Deurwaarder on the PubSub Source and Sink
https://issues.apache.org/jira/browse/FLINK-9311

We currently have most of the code done and we also have several tests that
verify the behavior by actually reading and writing messages to and from
PubSub.
These tests use a docker image provided by Google that includes the pubsub
emulator.

Starting and stopping this thing is really fast.
Starting up, creating topic, running a few tests and cleaning up takes
about 5 seconds.

Although the complete set of tests run on my machine in about 20-25 seconds
it does mean that anyone doing that MUST have docker installed locally.

So my question to the committers: Do we simply leave these tests "always
on" or do you want them to be "off by default" or something else.
I case you do not want the "always on" then please indicate how we should
implement the "on"/"off" switch for these tests.

Thanks.

--
Best regards / Met vriendelijke groeten,

Niels Basjes

Reply | Threaded
Open this post in threaded view
|

Re: How to handle tests that need docker?

Till Rohrmann
I agree with Dominik. We could add this test to
`flink-end-to-end-test/run-pre-commit-tests.sh` or
`flink-end-to-end-test/run-nightly-tests.sh` like we did for the
`flink-end-to-end-test/test-scripts/test_docker_embedded_job.sh` and
`flinke-end-to-end-test/test-scripts/test_yarn_kerberos_docker.sh`.

Cheers,
Till

On Thu, Aug 16, 2018 at 2:10 PM Dominik Wosiński <[hidden email]> wrote:

> Imho,
> Such tests should be off by default as they impose some requirements on
> the people trying to build Flink from source. And one would have to install
> docker only for the tests to pass. But of course they should be enabled for
> CI builds.
>
> Best Regards,
> Dominik.
>
> Wysłane z aplikacji Poczta dla Windows 10
>
> Od: Niels Basjes
> Wysłano: czwartek, 16 sierpnia 2018 14:05
> Do: [hidden email]
> Temat: How to handle tests that need docker?
>
> Hi,
>
> I'm working together with Richard Deurwaarder on the PubSub Source and Sink
> https://issues.apache.org/jira/browse/FLINK-9311
>
> We currently have most of the code done and we also have several tests that
> verify the behavior by actually reading and writing messages to and from
> PubSub.
> These tests use a docker image provided by Google that includes the pubsub
> emulator.
>
> Starting and stopping this thing is really fast.
> Starting up, creating topic, running a few tests and cleaning up takes
> about 5 seconds.
>
> Although the complete set of tests run on my machine in about 20-25 seconds
> it does mean that anyone doing that MUST have docker installed locally.
>
> So my question to the committers: Do we simply leave these tests "always
> on" or do you want them to be "off by default" or something else.
> I case you do not want the "always on" then please indicate how we should
> implement the "on"/"off" switch for these tests.
>
> Thanks.
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to handle tests that need docker?

Gary Yao
In reply to this post by Niels Basjes
Hi,

If you are writing JUnit tests, you could use testcontainers [1] to start
the
PubSub container. Then you could introduce a build profile [2] that enables
certain JUnit tests by using Categories [3] or the maven-failsafe-plugin
[4].

Best,
Gary

[1] https://www.testcontainers.org/usage/generic_containers.html#benefits
[2]
https://maven.apache.org/guides/introduction/introduction-to-profiles.html
[3] https://github.com/junit-team/junit4/wiki/categories
[4]
https://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html

On Thu, Aug 16, 2018 at 2:05 PM, Niels Basjes <[hidden email]> wrote:

> Hi,
>
> I'm working together with Richard Deurwaarder on the PubSub Source and Sink
> https://issues.apache.org/jira/browse/FLINK-9311
>
> We currently have most of the code done and we also have several tests that
> verify the behavior by actually reading and writing messages to and from
> PubSub.
> These tests use a docker image provided by Google that includes the pubsub
> emulator.
>
> Starting and stopping this thing is really fast.
> Starting up, creating topic, running a few tests and cleaning up takes
> about 5 seconds.
>
> Although the complete set of tests run on my machine in about 20-25 seconds
> it does mean that anyone doing that MUST have docker installed locally.
>
> So my question to the committers: Do we simply leave these tests "always
> on" or do you want them to be "off by default" or something else.
> I case you do not want the "always on" then please indicate how we should
> implement the "on"/"off" switch for these tests.
>
> Thanks.
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>