# Rancher Desktop: the supported way to run Linux containers on Windows

**Rancher Desktop is the preferred Linux-container runtime on this host.** It
runs containers in a WSL2 distro, so the Linux CI image can be run locally and a
CI failure reproduced on the dev box instead of guessed at through 25-minute
pipeline round trips.

It does **not** replace the Windows-container lane. Windows containers still go
through Stevedore's `docker.exe` — see [`windows-builds.md`](windows-builds.md).
The two coexist: different runtimes, different images, different jobs.

## Install

```pwsh
winget install -e --id SUS.RancherDesktop
```

Verify it is up:

```pwsh
& "C:\Program Files\Rancher Desktop\resources\resources\win32\bin\rdctl.exe" version
```

`rdctl client version: v1.23.1, targeting server version: v1` means the backend
is running. It provisions two WSL distros, `rancher-desktop` and
`rancher-desktop-data`, which are its own — leave them alone.

## Use `nerdctl`, not `docker`

Rancher Desktop defaults its container engine to **containerd**, and the CLI for
containerd is `nerdctl`. This is the single most common thing to get wrong here,
because `docker.exe` also ships in the same directory and *appears* to work
while talking to a completely different engine.

```pwsh
$nerdctl = "C:\Program Files\Rancher Desktop\resources\resources\win32\bin\nerdctl.exe"
& $nerdctl --namespace default run --rm alpine:3.20 uname -a
# Linux ... 6.18.33.2-microsoft-standard-WSL2 ... x86_64 Linux
```

That `uname` output is the check that matters: it proves a *Linux* kernel is
serving the container. Running `docker.exe info` on this host instead reports
`OSType=windows`, because the default docker context points at the Windows
engine used by the Stevedore lane. Both CLIs are on the box; only one of them is
talking to Linux.

`--namespace default` is worth passing explicitly. containerd namespaces are
real isolation, and images pulled into one namespace are invisible from another
— an image can be present and still "not found".

If you would rather use the `docker` CLI, switch Rancher Desktop's container
engine from `containerd` to `dockerd (moby)` in its settings. Then `docker`
targets Rancher and `nerdctl` stops being the entry point. Pick one and stay
with it; mixing them is how you end up debugging a missing image that is sitting
in the other engine's store.

## The image: always `:latest`

**Use `ghcr.io/kataglyphis/kataglyphis_beschleuniger:latest` for Linux
builds, in CI and locally.**

`:latest` is the default multi-arch manifest: `linux/amd64`, `linux/arm64`
and `linux/riscv64`, re-shipped with fresh per-arch digests on every validating
rebuild (see the CHANGELOG entries that name them). The per-arch wrappers
behind it (`:latest-amd64`, `:latest-arm64`, `:latest-riscv64`) are internal
build tags; variant manifests, when published, are `:latest-<variant>` (e.g.
`nvidia`, `rocm`). The amd64/arm64 wrappers carry Hailo, so there is no
separate Hailo tag.

Until 2026-09-22 this manifest was called `:latest-cross`; that name is
**retired**: no release publishes it any more. The fleet resolves the ref
through `@develop` (since 2026-09-25), whose `versions.env` names `:latest`;
a lane left at `@main` still names `:latest-cross`, whose tags were deleted on
2026-09-29 (AGENTS.md § Image and tag naming), and fails at the pull. Move any lane still naming it to `:latest`. (The `:latest` of before 2026-08-27 was
a different, dead native-lane index — its children had 404'd for months and it
was deleted in the 2026-08-27 registry cleanup; the tag name was then reused
for the cross-lane manifest.)

`.github/workflows/python-ci-linux.yml` runs its containerized steps — static
analysis, tests, docs, packaging — in `:latest` on the x64 and arm64 runners
(`CONTAINER_IMAGE` in the job's `env`, one image for both rows; the `plan` job
writes the matrix rows from the `arches` input, and `prepare-linux-ci-host` and
the `run-in-linux-container` steps consume the image; the upload steps run on the
host). **Keep local runs on the same tag**, or reproducing a CI failure
locally proves nothing.

`:latest` is not pinned by digest, so it still floats. Pinning would make
CI properly reproducible and is worth doing; it is not done yet.

## Reproducing a CI step locally

The Linux workflow mounts the repository at `/workspace` and runs the scripts in
`scripts/linux/`. The same shape works locally:

```bash
bash third_party/ANTfrastructure/linux/scripts/run-in-ci-image.sh . -- \
  bash ./scripts/linux/cmake-configure-build.sh --preset linux-debug-clang --build-dir /tmp/build
```

Its options, and why this recipe is no longer four lines of `nerdctl`, are in
[`shared-script-libraries.md`](shared-script-libraries.md).

Notes that will save time:

- **The image is large.** The first pull moves several GB; budget for it rather
  than assuming the command has hung.
- **Bind-mounting from a Dev Drive does not work** for the Windows lane and the
  same caution applies here — see the tar-fallback note in
  [`windows-builds.md`](windows-builds.md). If a mount behaves strangely, test
  with a path on a normal NTFS volume before blaming the container.
- **Build outputs land in the mounted tree**, so a Linux build directory will
  appear next to the Windows ones. Keep them under distinct `--build-dir` names
  (`build-linux`, `build-asan-clang`) or the two toolchains will fight over one
  CMake cache.

### An empty mount is not a missing drive

Two independent things silently produce a mount that resolves fine and contains
*nothing* — no error, no warning, exit code 0. Both were hit on 2026-09-03 with
a repo on `D:` (a ReFS Dev Drive), and each one alone is enough.

**1. The drive must be visible inside containerd's own mount namespace.**
Rancher's distro automounts only the drives WSL gave it at boot — here just
`C:` — and containerd does not run in the distro's init namespace:

```
containerd            mnt:[4026532289]
init / wsl -d session mnt:[4026532204]
```

A `mount -t drvfs D: /mnt/d` from a plain `wsl -d rancher-desktop -u root`
session therefore lands in the *wrong* namespace and changes nothing that
containerd can see. Mount it where containerd actually looks:

```pwsh
wsl -d rancher-desktop -u root -e sh -c 'pid=$(ps -eo pid,comm | awk "\$2==\"containerd\" {print \$1; exit}"); nsenter -t "$pid" -m -- sh -c "mkdir -p /mnt/d && mount -t drvfs D: /mnt/d"'
```

Transient — gone when the VM restarts, and it touches a distro this page
otherwise says to leave alone. Redo it per boot, or keep the repo on `C:`.

**2. Pass the Windows path, never the WSL path.** nerdctl translates
`D:\path` itself and resolves the result in containerd's namespace. Handing it
the already-translated `/mnt/d/path` bypasses that translation and binds an
empty directory instead — which containerd helpfully *creates* for you, so the
path then exists and stays empty:

```pwsh
-v "D:\GitHub\Repo:/workspace"        # correct
-v "/mnt/d/GitHub/Repo:/workspace"    # resolves, mounts nothing, no error
```

**Do not diagnose this with `-v /mnt:/hostmnt`.** That probe shows an almost
empty tree — no `c`, no real `d` — and looks like proof the drives are missing.
It is measuring containerd's raw namespace, not the path Rancher actually takes,
and it will send you after the wrong cause. Probe with a Windows path against a
directory you know has contents (`-v "C:\ProgramData:/probe"`), then compare.

This also qualifies the Dev Drive caution above: `D:` here **is** ReFS, and once
both points are satisfied the bind mount lists the tree normally. Before blaming
ReFS for an empty mount, rule out the namespace and the path form — they present
identically and are far more likely.

## The orientation AGENTS.md carried

Moved out of `AGENTS.md` on 2026-09-15 (owner decision D10), unedited except for this heading and the relative links. The RULES stayed there; this is the reference behind them.

**Rancher Desktop is the preferred Linux-container runtime on this host**, and
the way to reproduce a Linux CI failure locally instead of guessing through
pipeline round trips. It does not replace the Windows lane — Windows containers
still go through Stevedore's `docker.exe`. Full details:
[`docs/rancher-desktop-linux-containers.md`](rancher-desktop-linux-containers.md).

```pwsh
$nerdctl = "C:\Program Files\Rancher Desktop\resources\resources\win32\bin\nerdctl.exe"
& $nerdctl --namespace default run --rm alpine:3.20 uname -a   # expect ...WSL2... x86_64 Linux
```

- **Use `nerdctl`, not `docker`.** Rancher defaults to the **containerd** engine,
  and `docker.exe` ships in the same directory while talking to a different
  engine entirely — `docker info` on this host reports `OSType=windows`, because
  the default context is the Windows lane. Both CLIs are present; only one is
  talking to Linux. (Switching Rancher's engine to `dockerd (moby)` flips this —
  pick one and stay with it.)
- Pass `--namespace default` explicitly. containerd namespaces are real
  isolation, so an image pulled into another namespace is genuinely "not found".
- **Linux builds use `ghcr.io/kataglyphis/kataglyphis_beschleuniger:latest`,
  in CI *and* locally.** It publishes amd64/arm64/riscv64.
  `python-ci-linux.yml` sets `CONTAINER_IMAGE` to `:latest`; if a local run
  uses a different tag, reproducing
  a CI failure proves nothing. The tag is not digest-pinned, so it still floats.

## When to reach for this

- A CI step fails and the logs are not enough. This is the main case, and it is
  worth the disk: the alternative is pushing commits to watch a pipeline.
- Something is suspected to be toolchain-specific — the Linux lane runs
  ASan/UBSan fuzzing that the Windows box does not, so a class of bug is only
  ever observable there.
- Verifying a fix before pushing, rather than after.

## WSL containers (`wslc`) are not a replacement yet

Evaluated 2026-10-05 against WSL 3.0.1, whose `wslc.exe` went GA on 2026-09-29.
Each session is its own utility VM running dockerd 25.0.3, it needs no admin, and
it leaves Stevedore's Windows containers alone. **Keep Rancher Desktop for every
lane.** wslc rejects `--privileged`, `--platform` and `--device`, and those are
what the lanes pass:
- `--privileged`: the native and web lanes (microsoft/WSL#41545).
- `--platform`: both local drivers, on every run (#41123).
- `--device`: cameras and `/dev/dri` (#40988).

Its arm64 emulation does not survive between containers either: binfmt
registered through its inner docker is gone at the next run.

| Pain point here | wslc |
| --- | --- |
| Pulls through the Windows shim fail on `docker-credential-undefined` | fixed: credentials go to Windows Credential Manager |
| [`-v name:/path` is a bind](#-v-namepath-is-a-bind-under-windows-nerdctl) | fixed: a real named volume, still root-owned, so the chown step stays |
| [An empty mount](#an-empty-mount-is-not-a-missing-drive) | probably fixed: each bind source gets its own virtiofs share (untested, no D:) |
| uid 1001 cannot `chmod`/`touch -d` host-made files | the same, so the named-volume rule stays |
| arm64 under QEMU | worse: no `--platform`, binfmt does not persist |

Bind mounts are 2–5x faster than Rancher's 9p: 2000 small writes take 1.7 s
instead of 4.3–4.9 s, `stat` is 4.8x faster, and sequential reads run at
~900 MiB/s against ~240 MiB/s. A bind mount is still 25–30x slower than
container-local disk on both engines. The one shape it can run today is the
Dart-only loop (no `--privileged`, and `--platform` can be dropped on an x64
host). OmniAccelerANT's `Invoke-DartChecks.ps1 -Engine wslc` pilots exactly that
(2026-10-06). Warm, on the same checkout and image:

| Dart-only step | wslc | Rancher nerdctl |
| --- | --- | --- |
| `dart format` + `flutter test` | 23 s | 32 s |
| `flutter analyze` (whole workspace) | 146 s | 371 s |

The first wslc run took 554 s, most of it the pull. Check again when #41545 and
#41123 land. Until then the family image is a second ~30 GB copy on `C:`
(`session.storagePath` in `%LOCALAPPDATA%\wslc\settings.yaml` moves it).

## Persisting the cargo cache

The `:latest` image runs as uid 1001. Its `/usr/local/cargo` belongs to that uid
now (it was root-owned until the consumer-contract fixes:
[`artifact-copy-completeness.md`](artifact-copy-completeness.md#the-rust-toolchain-must-be-writable-by-the-runtime-user)),
but the crate registry and every build's target tree still live in the
throwaway container, so every fresh container rebuilds all Rust dependencies
from scratch. Point it at a named volume instead:

```bash
nerdctl volume create cargo-cache        # once

MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*' rdctl shell nerdctl run --rm --user root \
  --mount type=volume,source=cargo-cache,target=/cargo-cache \
  -v /mnt/d/path/to/repo:/workspace -w /workspace \
  ghcr.io/kataglyphis/kataglyphis_beschleuniger:latest \
  bash -c 'bash scripts/linux/cmake-configure-build.sh \
     --preset linux-debug-clang --build-dir /tmp/build --cargo-cache-dir /cargo-cache'
```

Three details that are easy to get wrong:

- **`--build-dir` must be container-native** (`/tmp/...`), not a path on the
  bind-mounted host tree. CMake's FetchContent `file RENAME` and cargo's
  temp-file cleanup both fail on that filesystem — the build dies partway
  through with a permission error on a stale artifact.
- The build driver also redirects `CARGO_TARGET_DIR` onto the same volume, so
  compiled artifacts stay off the host mount for the same reason.
- `--user root` sidesteps volume ownership. To avoid it, chown the volume to
  the image's uid once:
  ```bash
  rdctl shell nerdctl run --rm --user root \
    --mount type=volume,source=cargo-cache,target=/cargo-cache \
    alpine:3.20 chown -R 1001:1001 /cargo-cache
  ```
- The long `--mount` form is not a style choice. See
  [§ `-v name:/path` is a bind under Windows nerdctl](#-v-namepath-is-a-bind-under-windows-nerdctl)
  below: the short form silently mounts a directory instead of the volume.

The `MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*'` prefix is required when
invoking nerdctl from Git Bash — without it the `-v` argument is rewritten
into a Windows path and the run fails with "expected an absolute path".

After the first build the registry and compiled dependencies are reused and
subsequent runs are dramatically faster.

## `-v name:/path` is a bind under Windows nerdctl

**`-v name:/path` is not a named volume here.** Windows `nerdctl` reads it as a
bind of `$PWD/name`, creates that directory silently, and mounts it; running
`nerdctl volume create name` beforehand changes nothing — the volume exists and
is never mounted.

It was measured on a consumer's local Linux lane: after five runs `%TEMP%` held
`…-workspace-build/` and four siblings at 729 MB each, while the volume of that
name, mounted through `--mount type=volume,…`, was empty. One run started from a
repo root left a 151 MB directory of that name *inside the checkout*. The whole
point of the volume — keeping a write-heavy build tree off `drvfs` — was never in
effect locally, and nothing failed only because the write-heavy steps had already
been moved to `/tmp`.

Always use the long form, which nerdctl cannot reinterpret as a path:

```bash
nerdctl run --rm --mount type=volume,source=<name>,target=/path ...
```

CI is unaffected: the Linux engine there resolves the short form correctly. One
cosmetic consequence of mounting a build directory: a cleaner (`flutter clean`,
`rm -rf build`) then logs `Device or resource busy (errno 16)` on the mount point
itself — it empties the directory and cannot unlink it. Not a failure to chase.

## Registering arm64 emulation in Rancher's VM (per VM boot)

Rancher's VM starts with **no emulators at all**: `binfmt` reports
`"emulators": null` and only `linux/amd64` variants under `supported`. An arm64
container therefore runs x86-64 binaries and dies with `rustc: 1: ELF: not
found`. Register once per VM boot:

```bash
nerdctl run --rm --privileged tonistiigi/binfmt --install arm64
nerdctl run --rm --privileged tonistiigi/binfmt          # verify: qemu-aarch64 listed
nerdctl run --rm --platform linux/arm64 alpine uname -m  # verify: aarch64
```

Like the `D:` mount in containerd's namespace, this does **not** survive a VM
restart. The arm64 layers are a separate pull — roughly 6 GB over the wire and
30 GB on disk beside the amd64 copy (`nerdctl pull --platform linux/arm64 …`) —
and every compile then runs under emulation, so expect it to be far slower than
the native x64 path.

## Emulated arm64 cannot run bubblewrap or static-PIE

Under `qemu-user`, two packaging formats fail for reasons that have nothing to do
with this repository or the image. Both were verified 2026-09-05 on a consumer,
after a full arm64 build that compiled Rust and C++ without a single error:

- `bwrap: Creating new namespace failed, likely because the kernel does not
  support user namespaces`. The kernel does support them
  (`/proc/sys/user/max_user_namespaces` is 123100) and `--privileged` was passed;
  `qemu-user` simply does not carry `unshare(CLONE_NEWUSER)` through, and
  flatpak-builder sandboxes every module with bubblewrap.
- `/usr/local/bin/appimagetool: cannot execute binary file: Exec format error`.
  The binary is the correct architecture (`ELF aarch64, static-pie linked`);
  `qemu-user` cannot load static-PIE executables.

Real arm64 runners are unaffected, so a CI arm64 row that packages successfully
proves nothing about the emulated path and vice versa. Locally, treat a failing
flatpak/AppImage step on arm64 as expected and check for those two messages
before investigating anything else.

## Long-running work: detached containers + tmux

A build or training run that outlives your terminal should not be tied to it.
Start the container detached, then use tmux *inside* it so the process survives
both disconnects.

```bash
nerdctl run -dit --gpus '"device=0"' -p 8501:8501 \
  -v "$PWD:/workspace" -w /workspace \
  --name work ghcr.io/kataglyphis/kataglyphis_beschleuniger:latest
```

| Flag | Why |
|---|---|
| `-d` | Detached — runs in the background |
| `-i` | Keeps STDIN open, so a shell attached later is usable |
| `-t` | Allocates a pseudo-TTY |
| `--name` | Stable handle; without it you are copying container ids around |

Attach, start a tmux session, run the job, then leave:

```bash
nerdctl exec -it work bash
tmux                      # then start the long job, e.g. python train.py
# detach from tmux with Ctrl+B then D, or just `exit` the shell
```

The container and the tmux session keep running. Reconnect later with:

```bash
nerdctl exec -it work bash
tmux attach
```

Useful handles:

```bash
nerdctl ps                # what is running
nerdctl stop work
nerdctl start work
nerdctl rm work
tmux ls                   # inside the container: list sessions
```

Name the session when you run more than one at a time — `tmux` alone gives
you an unnamed session that is awkward to pick out later:

```bash
tmux new -s build          # create
# Ctrl+B then D to detach
tmux attach -t build       # come back to that specific one
```

Two alternatives to tmux, for reference: `Ctrl+P` then `Ctrl+Q` detaches from an
attached container without stopping it (this is a container-level detach, not a
process-level one), and `nohup cmd > cmd.log 2>&1 &` backgrounds a single
process without a session manager. tmux is preferable when you want to look at
the running output again.

> `nerdctl` is largely command-compatible with `docker`. In most cases
> substituting one for the other is enough — but see the namespace caveat above.

## Fixing file ownership after a container writes to a bind mount

A container running as a different uid than your host user leaves files you
cannot edit. Rather than chowning from the host, do it from inside the container
where you have root.

Find the host uid/gid you want the files to end up as:

```bash
id -u && id -g
# ...or read it out of /etc/passwd:
grep "^$USER:" /etc/passwd
# youruser:x:1000:1000::/home/youruser:/bin/bash
#            ^^^^ ^^^^  uid  gid
```

Then, inside the container:

```bash
chown -R 1000:1000 /workspace/some/path
```

The same trick applies to named volumes — see
[Persisting the cargo cache](#persisting-the-cargo-cache), which chowns
`cargo-cache` to the image's uid 1001 once instead of running everything as
root.

## Setting up WSL2 itself

Rancher Desktop supplies its own distro, but a second WSL distro is often
wanted — for a Linux-side toolchain, or to reproduce a CI step outside the
container.

### Installing without the Microsoft Store

Store-less installs are the norm on managed or offline machines:

```powershell
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
```

Reboot, then import a rootfs tarball from
[Ubuntu Cloud Images](https://cloud-images.ubuntu.com/wsl/):

```powershell
mkdir C:\WSL\Ubuntu24.04
wsl --import Ubuntu24.04 C:\WSL\Ubuntu24.04 .\ubuntu-24.04-wsl-rootfs.tar.gz
wsl -d Ubuntu24.04
```

An imported distro starts as root with no user account:

```bash
passwd
adduser <youruser>
usermod -aG sudo <youruser>
```

```powershell
wsl -d Ubuntu24.04 -u <youruser>
wsl --set-default Ubuntu24.04
```

### Stop Windows executables shadowing Linux ones

By default WSL appends the whole Windows `PATH`, so `python`, `node` or `cmake`
can resolve to a Windows binary inside a Linux shell — with confusing results in
a build script. In `/etc/wsl.conf`:

```ini
[interop]
enabled=true
appendWindowsPath=false
```

Windows executables remain callable by full path; they just stop winning `PATH`
lookups. Restart the distro (`wsl --shutdown`) for it to take effect.

### The virtual disk only grows

WSL2's ext4 disk never shrinks on its own. Reclaim it the same way as the
[Docker VHD](windows-builds.md):

```powershell
wsl --shutdown
cd C:\WSL\Ubuntu24.04
Optimize-VHD -Path .\ext4.vhdx -Mode Full
```

### No network inside WSL once a VPN connects

A known WSL networking interaction rather than a VPN misconfiguration. Install
the latest WSL from [its releases](https://github.com/microsoft/WSL/releases)
rather than relying on the inbox version:

```powershell
Add-AppxPackage .\Microsoft.WSL_<version>_x64_ARM64.msixbundle
```

Then set the networking mode in WSL Settings. Microsoft's
[troubleshooting note](https://learn.microsoft.com/en-us/windows/wsl/troubleshooting#wsl-has-no-network-connectivity-once-connected-to-a-vpn)
covers the remaining cases.

### USB passthrough (cameras, probes, boards)

Needed whenever a device has to be visible to a Linux toolchain — see
[Microsoft's guide](https://learn.microsoft.com/en-us/windows/wsl/connect-usb):

```powershell
usbipd list
usbipd attach --wsl --busid 1-1.2
```

Attaching is not the whole story: the device appears on the USB bus but the
kernel module still has to claim it. For a UVC camera that shows up with no
`/dev/video*`:

```bash
sudo modprobe uvcvideo videodev
lsmod | grep -E 'uvcvideo|videodev'
ls -l /dev/video*
v4l2-ctl --list-devices
dmesg | tail -n 40
```

If the device is listed but still unbound, bind its interfaces by hand — replace
the vendor id with your own from `lsusb`:

```bash
for d in /sys/bus/usb/devices/*; do
  [ -f "$d/idVendor" ] || continue
  [ "$(cat "$d/idVendor")" = "174f" ] || continue
  for iface in "$d"/*:*; do
    [ -e "$iface" ] || continue
    echo -n "$(basename "$iface")" | sudo tee /sys/bus/usb/drivers/uvcvideo/bind
  done
done
```