Quickstart

This is the shortest safe path from an empty shell to a static plan, a first real Slurm run, and one-command failure triage.

If Slurm terms such as sbatch, srun, allocation, job step, Pyxis, or Enroot are unfamiliar, read Slurm And Container Basics before the first real cluster run.

1. Install The CLI

Installation is the single owner of install, verify, mirror, and source-build commands. Install the CLI from there, confirm hpc-compose --version works, then return here.

2. Learn The Safe Authoring Path First

The safe authoring path runs entirely on a laptop, workstation, or login node — new writes a local starter spec and plan is purely static (no sbatch, no image import):

hpc-compose new --template minimal-batch --name my-app --output compose.yaml
hpc-compose plan -f compose.yaml
hpc-compose plan --show-script -f compose.yaml

plan validates the spec and resolves service order; plan --show-script adds the rendered batch script. Run that block first on macOS, a laptop, or any machine where you want to evaluate the authoring model before touching a real cluster. The Overview page covers the same walkthrough with full expected output.

If you want a guided learning path instead of a single starter template, run the Spec Metamorphosis tutorial:

hpc-compose evolve --output compose.yaml

The normal workflow to remember is:

hpc-compose plan -f compose.yaml
hpc-compose up -f compose.yaml
hpc-compose debug -f compose.yaml --preflight

3. Choose A Starting Spec

Use the built-in starter templates when you want a fresh compose.yaml with your application name filled in:

hpc-compose new \
  --template minimal-batch \
  --name my-app \
  --output compose.yaml

Add --cache-dir '<shared-cache-dir>' when you want the generated file to include an explicit x-slurm.cache_dir. Otherwise the plan uses the active settings cache default or $HOME/.cache/hpc-compose.

From a source checkout, you can also inspect a known-good repository example:

hpc-compose plan -f examples/minimal-batch.yaml

The Examples page is the single selection guide for beginner, LLM, training, distributed, and pipeline workflows.

Use Spec Metamorphosis when you want to learn those concepts progressively in one evolving valid spec.

4. Pick And Test A Cache Directory

cache_dir is optional in the spec, but real clusters usually need a site-specific shared path because image preparation happens before the job starts and compute nodes must later see those artifacts.

Ask your cluster documentation or support team for a project scratch, work, or shared filesystem path, then test it:

export CACHE_DIR=/cluster/shared/hpc-compose-cache
mkdir -p "$CACHE_DIR"
test -w "$CACHE_DIR"

Persist it in project settings when you want the same value every time:

hpc-compose setup --profile-name dev --cache-dir "$CACHE_DIR" --default-profile dev --non-interactive

Or keep using an environment-backed explicit spec value and persist it next to your copied spec:

printf 'CACHE_DIR=%s\n' "$CACHE_DIR" > .env

Do not use /tmp, /var/tmp, /private/tmp, or /dev/shm for x-slurm.cache_dir. Validation may accept those strings, but preflight reports them as unsafe because prepare happens before runtime and compute nodes must later see the cached artifacts.

5. Before Your First Cluster Run

For Pyxis, srun --help should mention --container-image.

Everything above is safe on any machine. Everything below requires a real Slurm submission host.

The steps up to here only author specs, prepare a cache path, and read static plans. From this point the commands call sbatch, srun, and the runtime backend, so run them only on a supported Linux Slurm submission host.

6. Submit On A Real Cluster

When you move to a supported Linux submission host, the normal run is:

hpc-compose up -f compose.yaml

up runs preflight, prepares missing artifacts, renders the batch script, submits it through sbatch, then follows scheduler state and tracked logs. On the first run (or after cache eviction) the prepare step imports your container image with enroot — a multi-GB download, then extract and squashfs build — which can take several minutes; later runs reuse the cache, and an interactive terminal streams live import sub-progress. On an interactive TTY it opens the full-screen watch UI; otherwise it falls back to line-oriented output. Add --watch-queue when you want line-oriented queue polling until the Slurm job reaches RUNNING before the normal watch view opens; --queue-warn-after <DURATION> controls the one-time long-pending warning. The watch UI holds the final screen on failures by default; use --hold-on-exit never|failure|always to tune that behavior. Use hpc-compose up --detach -f compose.yaml when you want submit-and-return behavior.

Success looks like:

• the job is submitted or launched
• a tracked job id is recorded
• the watch UI or text follower shows scheduler progress
• status, ps, and logs can reconnect to the tracked run later

7. If The First Cluster Run Fails

The longer symptom guide is Troubleshooting.

8. Revisit A Tracked Run Later

hpc-compose jobs list
hpc-compose status -f compose.yaml
hpc-compose ps -f compose.yaml
hpc-compose watch -f compose.yaml
hpc-compose stats -f compose.yaml
hpc-compose logs -f compose.yaml --follow

Use jobs list first when you need to rediscover tracked runs under the current repo tree. Use ps for a stable per-service snapshot, watch to reconnect to the live UI, and logs --follow for a text-only follower.

From A Source Checkout

If you are developing from a local checkout instead of an installed binary:

cargo build --release
target/release/hpc-compose validate -f examples/minimal-batch.yaml
target/release/hpc-compose plan -f examples/minimal-batch.yaml
target/release/hpc-compose plan --show-script -f examples/minimal-batch.yaml

Read Next

• Installation
• Support Matrix
• Why hpc-compose
• Slurm And Container Basics
• Examples
• Runtime Backends
• Runbook
• Troubleshooting