Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

hpc-compose

hpc-compose logo

hpc-compose turns a Compose-like spec into a single Slurm job that runs one or more services through Enroot and Pyxis.

Quickstart Execution model Runbook Spec reference Supported Slurm model Examples

hpc-compose is intentionally not a full Docker Compose implementation. It focuses on the subset that maps cleanly to one Slurm allocation, plus either single-node services or one allocation-wide distributed service without a separate orchestration layer.

Start Here

  1. Read Quickstart for the shortest install-and-run path.
  2. Read Execution model to understand what runs on the login node, what runs on the compute node, and which paths must be shared.
  3. Use Runbook when adapting a real workload to a real cluster.
  4. Use Examples when you want the closest known-good starting point.
  5. Use Spec reference when you need exact field behavior or validation rules.
  6. Use Supported Slurm model when you need the product boundary spelled out clearly.

What it is for

  • One Slurm allocation per application
  • Single-node jobs and constrained multi-node distributed runs
  • Optional helper services pinned to the allocation’s primary node
  • Remote images such as redis:7 or existing local .sqsh images
  • Optional image customization on the login node through x-enroot.prepare
  • Shared cache management for imported and prepared images
  • Readiness-gated startup across dependent services

What it does not support

  • Compose build:
  • ports
  • custom Docker networks / network_mode
  • restart policies
  • deploy
  • arbitrary multi-node orchestration or partial-node service placement
  • mixed string/array entrypoint + command combinations in ambiguous cases

If you need image customization, use image: plus x-enroot.prepare, not build:.

Fast path

name: hello

x-slurm:
  time: "00:10:00"
  mem: 4G

services:
  app:
    image: python:3.11-slim
    command: python -c "print('Hello from Slurm!')"

hpc-compose submit --watch -f compose.yaml

submit --watch is the normal run. Break out inspect, preflight, or prepare as the debugging flow when you are validating a new spec for the first time or isolating a failure.