How to avoid wasting time/resources in a CI/CD workflow?

Ideally, a team should be able to run all the pipelines on every commit and automatically deploy to production.

However, I don't understand how to most effeciently use our resources so that hours are wasted running tests on code which is about to be overwritten by the next commit.

If a build or test takes 20 minutes to go through the pipeline, the system gets "clogged".

What mechanism is used to solve this problem? Do tests/jobs get canceled as new code gets pushed? If so, then how do you deploy multiple times a day? Add to this the need for Pull requests or sign off. What intermediate steps can be taken until it's possible to automate everything?

We break ours down into stages, and each stage pulls, rather than pushes.

Our build takes 20 minutes. So if I commit once a minute, I will still only get three builds an hour, but each build will contain 20 commits.

The tests then pull on the latest successful builds and run them, and the same for the deployment scripts.

The main point is that we do NOT build every commit individually. We do NOT test every build individually.

I probably got all this from Continuous Delivery by Jez Humble and David Farley.

StackExchange DevOps Q#10308, answer score: 5