What are the pros and cons of running application testing during vs. after a docker build?

Our dockerfile creates the application environment (e.g. env variables, install libraries with apt-get etc.). It also pulls and copies the python application code from its git repo.

However, we are debating whether the application tests (some unit, some integration) should be put within a docker RUN command (e.g. RUN /bin/bash -c "source activate cool_env; pytest") or after the build using the CI stack (e.g. Jenkins, Openshift) to execute the tests on the built container.

What are the pros and cons of each?

I support J.Doe's suggestion for separate Docker files for each of the build and test stage. Such approach also allows you to:

• re-run the test stage for whatever reason (known intermittent failures, for example) without re-doing the build stage

• run multiple different test stages in parallel using the same build - for example instead of running a single long test stage made up of serialized tests, for a potentially significant overall pipeline speeding up.

StackExchange DevOps Q#1455, answer score: 3