Parallel huggingface_hub downloads on a small VPS get OOM-killed silently by the Xet downloader — disable Xet, and beware systemd killing the whole tmux scope

Running N parallel workers that each hf_hub_download a ~10 GB file (local_dir mode) on a 16 GB machine: within ~40 seconds memory peaks at ~15 GB, the kernel OOM-kills one python worker, and because the workers were launched inside a tmux session under a systemd user scope, systemd marks the scope "Failed with result oom-kill" and tears down EVERYTHING in it — the launcher/driver included. Symptoms are deceptive: the per-worker logs show no traceback, the driver never logs a retry, network traffic simply freezes, and ps shows no python at all. It looks like a stall or a rate limit, not a crash.

Read dmesg -T | grep -i "out of memory" and journalctl --since "15 min ago" | grep -iE "oom|scope" before assuming a network problem — the entry "hf-xet-N invoked oom-killer" names the culprit: huggingface_hub's Xet-based downloader buffers ~2 GB per concurrent download. Fix: export HF_HUB_DISABLE_XET=1 in the environment that starts the tmux server (the workers inherit it; make sure the old tmux server is dead first, tmux kill-server, otherwise a new session does not inherit the new env). Plain HTTP downloads then use tens of MB per worker while still reaching ~250 MB/s aggregate. Alternatives: lower HF_XET_NUM_CONCURRENT_RANGE_GETS, or fewer workers. Also on fresh Ubuntu 26.04 images, python3 -m venv fails until apt install python3-venv (python 3.14 lacks ensurepip) even though import venv succeeds, so a bootstrap check on the venv module alone is not enough.

Xet downloads fetch many byte ranges concurrently and hold them in memory before assembling the file; multiplied by N workers this exceeds RAM. systemd's oom-kill handling of a scope is all-or-nothing.

Batch-reading a multi-hundred-GB Hugging Face dataset on a rented 8-vCPU/16 GB box with one downloader+reader process per file.