HiveBrain v1.2.0
Get Started
← Back to all entries
snippetdockerMinor

How to generate a list of every external URL accessed during docker build?

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
duringdockereverygenerateexternalhowlistaccessedbuildurl

Problem

Is there a straight forward way to generate a .txt file containing each URL (or corresponding IP) visited during a docker build process?
Example

An example list might look like this:

# urls.txt
https://hub.docker.com/
https://www.npmjs.com/


(but a real case using an extensive Dockerfile may be more complex and contain many more URLs/IPs).
Things I tried so far

  1. Installing and using an intrusion detection engine



IDSs I attempted to install and use were feature rich and probably more complex than required for what I'm attempting. Configuring IDS engines is not trivial (especially for someone with little network knowledge), so I'd like to explore simpler solutions, if possible.

  1. Collecting and parsing docker build logs



I captured build logs with this:
docker build --no-cache --progress=plain . &> build.log


and extracted lines containing a URL with this regular expression:
(http|ftp|https):\/\/([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:\/~+#-]*[\w@?^=%&\/~+#-])


A lot of URLs are found, but... i) it doesn't find URLs which aren't explicitly stated in the build logs, and ii) it finds URLs which aren't actually accessed but which are simply stated in logs (i.e. logs along the lines of "For more info, visit https://www.example.com".
Question

How can I generate a nice clean list of each external URL visited during a docker build process?

Solution

One thing that pops my mind is using either of Wireshark, tshark, or tcpdump for capturing network packets originating at docker daemon while docker build is running.

After recording the traffic, need to filter & process the captured data.

Context

StackExchange DevOps Q#17369, answer score: 3

Revisions (0)

No revisions yet.