Cache Management
The resolved cache directory stores imported and prepared runtime artifacts. It comes from explicit x-slurm.cache_dir, then profile/default settings, then $HOME/.cache/hpc-compose. For real cluster runs, it must be visible from both the submission host and compute nodes.
Choose A Cache Path
Use a project scratch, work, or shared filesystem path:
export CACHE_DIR=/cluster/shared/hpc-compose-cache
mkdir -p "$CACHE_DIR"
test -w "$CACHE_DIR"
You can record that path in project settings instead of every compose file:
hpc-compose setup --profile-name dev --cache-dir "$CACHE_DIR" --default-profile dev --non-interactive
Do not use /tmp, /var/tmp, /private/tmp, or /dev/shm. Validation may accept those strings, but preflight reports them as unsafe because compute nodes must reuse artifacts prepared before submission.
Inspect Cache State
hpc-compose cache list
hpc-compose cache inspect -f compose.yaml
hpc-compose cache inspect -f compose.yaml --service app
Use cache inspect to answer:
- which artifact is being reused
- whether a prepared image came from a cached manifest
- whether a service rebuilds on every prepare because prepare mounts are present
Prune Cache Entries
Prune old entries by age:
hpc-compose --profile dev cache prune --age 14
Prune artifacts not referenced by the current plan:
hpc-compose cache prune --all-unused -f compose.yaml
Prune one cache directory directly:
hpc-compose cache prune --age 7 --cache-dir '<shared-cache-dir>'
--age and --all-unused are mutually exclusive.
Rendezvous Records
Cross-job rendezvous uses the same shared cache root:
<cache_dir>/rendezvous/<name>/latest.json
These records are small endpoint descriptors, not runtime images. They are pruned separately:
hpc-compose rendezvous list --cache-dir "$CACHE_DIR"
hpc-compose rendezvous prune --cache-dir "$CACHE_DIR"
Provider cleanup removes latest.json only when the finishing job still owns it, so an older provider cannot erase a newer provider’s record.
After Upgrading
Cache keys include the tool version, so upgrading hpc-compose invalidates existing cached artifacts. Expect a full rebuild on the next prepare or up, then optionally prune old entries:
hpc-compose cache prune --age 0