cameleer-server/docs/superpowers/plans/2026-04-27-init-container-jar-fetch.md

# Init-Container JAR Fetch + ArtifactStore Abstraction Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Replace host-bind-mounted JAR delivery with an init-container download from Cameleer over HTTP, sitting behind a new `ArtifactStore` abstraction. Closes the last open hardening gap from issue #152 (`withUsernsMode`) and gives the storage layer a clean migration path to OCI later (tracked separately).

**Architecture:**
- New `ArtifactStore` interface in `cameleer-server-core` with one filesystem implementation today; the runtime never touches the filesystem path again.
- Cameleer serves JARs over HTTP via short-lived HMAC-signed URLs (no JWT/RBAC dependency on the loader).
- Tenant deploys now run a `cameleer-runtime-loader` container (busybox + 30-line `entrypoint.sh`) first, sharing a per-replica named Docker volume with the main container. Loader downloads `app.jar`, exits 0, main container starts with the volume mounted read-only at `/app/jars/`.
- `withUsernsMode("host:1000:65536")` lands once host file ownership is no longer in the picture (everything in the container's UID namespace inside the named volume).
- `agent.jar` stays baked into the runtime base image — out of scope for this change.

**Tech Stack:** Java 17, Spring Boot 3.4.3, docker-java client, Postgres (Flyway), busybox-based loader image, HMAC-SHA256 URL signing.

---

## File Structure

### New files

| Path | Responsibility |
|---|---|
| `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactStore.java` | Interface: put/get/exists/delete/downloadUrl over an opaque coordinate key |
| `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactCoordinates.java` | Value type: `(tenantId, appId, version)` — OCI-friendly so a future `OciArtifactStore` reuses it |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/FilesystemArtifactStore.java` | Filesystem impl preserving today's `{jarStoragePath}/{appId}/v{version}/app.jar` layout |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/web/ArtifactDownloadTokenSigner.java` | HMAC-SHA256 signer/verifier for download URLs; key = HMAC(jwtSecret, "cameleer-artifact-token-v1") |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/web/ArtifactDownloadController.java` | `GET /api/v1/artifacts/{appVersionId}?exp=...&sig=...` — token-validated stream |
| `cameleer-runtime-loader/Dockerfile` | busybox base, copies entrypoint.sh, runs as UID 1000 |
| `cameleer-runtime-loader/entrypoint.sh` | wget the signed URL into `/app/jars/app.jar`, verify size, exit |
| `cameleer-runtime-loader/README.md` | Build + push instructions (one-liner) |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/FilesystemArtifactStoreTest.java` | put/get round-trip, delete, listing, idempotency |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/web/ArtifactDownloadTokenSignerTest.java` | sign + verify, expiry, tamper detection |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/web/ArtifactDownloadControllerTest.java` | 200 with valid sig, 401 wrong sig, 410 expired, 404 missing |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/runtime/DockerRuntimeOrchestratorLoaderTest.java` | Loader-pattern launch contract: per-replica volume created, loader runs first, main mounts RO, userns_mode applied |

### Modified files

| Path | Why |
|---|---|
| `cameleer-server-core/src/main/java/com/cameleer/server/core/runtime/ContainerRequest.java` | Drop `jarPath`/`jarVolumeName`/`jarVolumeMountPath`; add `appVersionId`, `artifactDownloadUrl`, `loaderImage` |
| `cameleer-server-core/src/main/java/com/cameleer/server/core/runtime/AppService.java` | `uploadJar` writes via `ArtifactStore.put`; `resolveJarPath` removed (callers now use store directly or the download URL flow) |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/retention/JarRetentionJob.java` | Deletes via `ArtifactStore.delete` |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/runtime/DeploymentExecutor.java` | Generates signed URL, threads through `ContainerRequest`; preflight no longer touches filesystem |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/runtime/DockerRuntimeOrchestrator.java` | Per-replica named volume, run loader synchronously, start main with volume RO at `/app/jars/`, add `withUsernsMode` |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/config/RuntimeBeanConfig.java` | Wire `ArtifactStore` bean, drop `jarStoragePath` from `AppService` ctor |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/security/SecurityConfig.java` | `permitAll` for `/api/v1/artifacts/**` (controller does HMAC validation) |
| `.claude/rules/docker-orchestration.md`, `.claude/rules/core-classes.md`, `.claude/rules/app-classes.md` | Update class/endpoint maps |
| `cameleer-server-app/src/main/resources/application.yml` | New properties: `cameleer.server.runtime.loaderimage`, `cameleer.server.runtime.artifacttokenttlseconds` |

---

## Task 1: Add `ArtifactCoordinates` value type

**Files:**
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactCoordinates.java`
- Test: `cameleer-server-core/src/test/java/com/cameleer/server/core/storage/ArtifactCoordinatesTest.java`

- [ ] **Step 1: Write the failing test**

```java
package com.cameleer.server.core.storage;

import org.junit.jupiter.api.Test;
import java.util.UUID;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;

class ArtifactCoordinatesTest {
    @Test
    void rejectsNullsAndNonPositiveVersion() {
        UUID appId = UUID.randomUUID();
        assertThatThrownBy(() -> new ArtifactCoordinates(null, appId, 1))
                .isInstanceOf(IllegalArgumentException.class);
        assertThatThrownBy(() -> new ArtifactCoordinates("default", null, 1))
                .isInstanceOf(IllegalArgumentException.class);
        assertThatThrownBy(() -> new ArtifactCoordinates("default", appId, 0))
                .isInstanceOf(IllegalArgumentException.class);
    }

    @Test
    void filesystemKeyMatchesLegacyLayout() {
        UUID appId = UUID.fromString("11111111-1111-1111-1111-111111111111");
        ArtifactCoordinates c = new ArtifactCoordinates("default", appId, 7);
        assertThat(c.filesystemKey()).isEqualTo("11111111-1111-1111-1111-111111111111/v7/app.jar");
    }

    @Test
    void ociRefIsDeterministic() {
        UUID appId = UUID.fromString("11111111-1111-1111-1111-111111111111");
        ArtifactCoordinates c = new ArtifactCoordinates("acme", appId, 7);
        assertThat(c.ociRef()).isEqualTo("acme/11111111-1111-1111-1111-111111111111:v7");
    }
}
```

- [ ] **Step 2: Run the test**

Run: `mvn -pl cameleer-server-core test -Dtest=ArtifactCoordinatesTest`
Expected: FAIL — class missing.

- [ ] **Step 3: Implement `ArtifactCoordinates`**

```java
package com.cameleer.server.core.storage;

import java.util.Objects;
import java.util.UUID;

/**
 * Opaque coordinate that addresses one stored JAR. Designed so a future
 * OCI-backed store can reuse the same coordinate without schema changes —
 * see issue (Zot follow-up).
 */
public record ArtifactCoordinates(String tenantId, UUID appId, int version) {
    public ArtifactCoordinates {
        Objects.requireNonNull(tenantId, "tenantId");
        Objects.requireNonNull(appId, "appId");
        if (version <= 0) throw new IllegalArgumentException("version must be > 0, got " + version);
    }

    /** Path-fragment used by FilesystemArtifactStore. Mirrors today's layout. */
    public String filesystemKey() {
        return appId + "/v" + version + "/app.jar";
    }

    /** OCI reference used by a future registry-backed store. Stable so coordinates
     * survive a backend swap. */
    public String ociRef() {
        return tenantId + "/" + appId + ":v" + version;
    }
}
```

- [ ] **Step 4: Re-run test — expect PASS**

- [ ] **Step 5: Commit**

```bash
git add cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactCoordinates.java \
        cameleer-server-core/src/test/java/com/cameleer/server/core/storage/ArtifactCoordinatesTest.java
git commit -m "feat(storage): add ArtifactCoordinates value type"
```

---

## Task 2: Add `ArtifactStore` interface

**Files:**
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactStore.java`

No test file — pure interface; behavior is tested via `FilesystemArtifactStore` in Task 3.

- [ ] **Step 1: Write the interface**

```java
package com.cameleer.server.core.storage;

import java.io.IOException;
import java.io.InputStream;

/**
 * Storage abstraction for deployable JAR artifacts. Implementations decide where
 * the bytes live (filesystem today; OCI registry / S3 later — see Zot follow-up
 * issue). Callers never construct filesystem paths themselves.
 *
 * <p>The interface is intentionally narrow: enough to upload, fetch, and delete.
 * {@link #downloadUrl} returns the URL the loader container hits — for the
 * filesystem store today this is a Cameleer-served signed URL; for an OCI store
 * later it could be a pre-signed registry URL.
 */
public interface ArtifactStore {

    /** Persist {@code bytes} of {@code size} bytes under {@code coords}. Idempotent
     * on identical content; overwrites on same coords with different content. */
    String put(ArtifactCoordinates coords, InputStream bytes, long size) throws IOException;

    /** Open the artifact for reading. Caller closes. */
    InputStream get(ArtifactCoordinates coords) throws IOException;

    boolean exists(ArtifactCoordinates coords);

    /** Remove the artifact and any backend-specific scaffolding (empty parent dirs, etc.).
     * Silently no-ops if the artifact is already absent. */
    void delete(ArtifactCoordinates coords) throws IOException;

    /** Stable identifier for this storage location — written into
     * {@code app_versions.jar_path} so existing rows still resolve. For the
     * filesystem store this is the absolute path; for OCI it's the ref. */
    String locator(ArtifactCoordinates coords);
}
```

- [ ] **Step 2: Verify it compiles**

Run: `mvn -pl cameleer-server-core compile`

- [ ] **Step 3: Commit**

```bash
git add cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ArtifactStore.java
git commit -m "feat(storage): add ArtifactStore interface"
```

---

## Task 3: Implement `FilesystemArtifactStore`

**Files:**
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/FilesystemArtifactStore.java`
- Test: `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/FilesystemArtifactStoreTest.java`

- [ ] **Step 1: Write the failing test**

```java
package com.cameleer.server.app.storage;

import com.cameleer.server.core.storage.ArtifactCoordinates;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;

import static org.assertj.core.api.Assertions.assertThat;

class FilesystemArtifactStoreTest {

    @Test
    void roundTripsBytes(@TempDir Path tmp) throws Exception {
        FilesystemArtifactStore store = new FilesystemArtifactStore(tmp.toString());
        ArtifactCoordinates c = new ArtifactCoordinates("default", UUID.randomUUID(), 1);

        byte[] payload = "hello-jar".getBytes();
        store.put(c, new ByteArrayInputStream(payload), payload.length);

        assertThat(store.exists(c)).isTrue();
        try (InputStream in = store.get(c)) {
            assertThat(in.readAllBytes()).isEqualTo(payload);
        }
    }

    @Test
    void deleteRemovesFileAndEmptyParents(@TempDir Path tmp) throws Exception {
        FilesystemArtifactStore store = new FilesystemArtifactStore(tmp.toString());
        UUID appId = UUID.randomUUID();
        ArtifactCoordinates c = new ArtifactCoordinates("default", appId, 1);
        store.put(c, new ByteArrayInputStream("x".getBytes()), 1);

        store.delete(c);

        assertThat(store.exists(c)).isFalse();
        // {tmp}/{appId}/v1/ should be gone too
        assertThat(Files.exists(tmp.resolve(appId.toString()).resolve("v1"))).isFalse();
    }

    @Test
    void deleteIsIdempotent(@TempDir Path tmp) throws Exception {
        FilesystemArtifactStore store = new FilesystemArtifactStore(tmp.toString());
        ArtifactCoordinates c = new ArtifactCoordinates("default", UUID.randomUUID(), 1);
        store.delete(c); // no exception
        assertThat(store.exists(c)).isFalse();
    }

    @Test
    void locatorMatchesLegacyAbsolutePath(@TempDir Path tmp) {
        FilesystemArtifactStore store = new FilesystemArtifactStore(tmp.toString());
        UUID appId = UUID.fromString("11111111-1111-1111-1111-111111111111");
        ArtifactCoordinates c = new ArtifactCoordinates("default", appId, 7);
        assertThat(store.locator(c))
                .isEqualTo(tmp.resolve("11111111-1111-1111-1111-111111111111/v7/app.jar").toString());
    }
}
```

- [ ] **Step 2: Run — expect FAIL (class missing)**

Run: `mvn -pl cameleer-server-app test -Dtest=FilesystemArtifactStoreTest`

- [ ] **Step 3: Implement**

```java
package com.cameleer.server.app.storage;

import com.cameleer.server.core.storage.ArtifactCoordinates;
import com.cameleer.server.core.storage.ArtifactStore;

import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;

public class FilesystemArtifactStore implements ArtifactStore {

    private final Path root;

    public FilesystemArtifactStore(String root) {
        this.root = Path.of(root);
    }

    private Path pathOf(ArtifactCoordinates coords) {
        return root.resolve(coords.filesystemKey());
    }

    @Override
    public String put(ArtifactCoordinates coords, InputStream bytes, long size) throws IOException {
        Path target = pathOf(coords);
        Files.createDirectories(target.getParent());
        try (InputStream in = bytes) {
            Files.copy(in, target, java.nio.file.StandardCopyOption.REPLACE_EXISTING);
        }
        return target.toString();
    }

    @Override
    public InputStream get(ArtifactCoordinates coords) throws IOException {
        return Files.newInputStream(pathOf(coords));
    }

    @Override
    public boolean exists(ArtifactCoordinates coords) {
        return Files.exists(pathOf(coords));
    }

    @Override
    public void delete(ArtifactCoordinates coords) throws IOException {
        Path target = pathOf(coords);
        Files.deleteIfExists(target);
        // Sweep empty {appId}/v{n}/ then {appId}/ if now empty.
        Path versionDir = target.getParent();
        if (versionDir != null && Files.isDirectory(versionDir) && isEmpty(versionDir)) {
            Files.delete(versionDir);
            Path appDir = versionDir.getParent();
            if (appDir != null && Files.isDirectory(appDir) && isEmpty(appDir)) {
                Files.delete(appDir);
            }
        }
    }

    @Override
    public String locator(ArtifactCoordinates coords) {
        return pathOf(coords).toString();
    }

    private static boolean isEmpty(Path dir) throws IOException {
        try (var entries = Files.list(dir)) {
            return entries.findFirst().isEmpty();
        }
    }
}
```

- [ ] **Step 4: Re-run test — expect PASS**

- [ ] **Step 5: Commit**

```bash
git add cameleer-server-app/src/main/java/com/cameleer/server/app/storage/FilesystemArtifactStore.java \
        cameleer-server-app/src/test/java/com/cameleer/server/app/storage/FilesystemArtifactStoreTest.java
git commit -m "feat(storage): add FilesystemArtifactStore (one impl of ArtifactStore)"
```

---

## Task 4: Migrate `AppService.uploadJar` to use `ArtifactStore`; remove `resolveJarPath`

**Files:**
- Modify: `cameleer-server-core/src/main/java/com/cameleer/server/core/runtime/AppService.java`
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/config/RuntimeBeanConfig.java`
- Test: `cameleer-server-core/src/test/java/com/cameleer/server/core/runtime/AppServiceUploadTest.java` (new)

The store is in `cameleer-server-app` but the interface is in `cameleer-server-core`, so `AppService` only depends on the interface.

- [ ] **Step 1: Write the failing test**

```java
package com.cameleer.server.core.runtime;

import com.cameleer.server.core.storage.ArtifactCoordinates;
import com.cameleer.server.core.storage.ArtifactStore;
import org.junit.jupiter.api.Test;

import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.util.Optional;
import java.util.UUID;

import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;

class AppServiceUploadTest {

    @Test
    void uploadJarStoresViaArtifactStoreAndRecordsLocator() throws Exception {
        AppRepository appRepo = mock(AppRepository.class);
        AppVersionRepository versionRepo = mock(AppVersionRepository.class);
        ArtifactStore store = mock(ArtifactStore.class);

        UUID appId = UUID.randomUUID();
        UUID versionId = UUID.randomUUID();
        when(appRepo.findById(appId)).thenReturn(Optional.of(
                new App(appId, UUID.randomUUID(), "demo", "Demo", null)));
        when(versionRepo.findMaxVersion(appId)).thenReturn(2);
        when(store.put(any(), any(InputStream.class), anyLong())).thenReturn("/data/jars/" + appId + "/v3/app.jar");
        when(versionRepo.create(eq(appId), eq(3), anyString(), anyString(), eq("a.jar"), eq(5L)))
                .thenReturn(versionId);
        when(versionRepo.findById(versionId)).thenReturn(Optional.of(
                new AppVersion(versionId, appId, 3, "/data/jars/" + appId + "/v3/app.jar",
                        "deadbeef", "a.jar", 5L, null, null, null)));

        AppService svc = new AppService(appRepo, versionRepo, store, "default");

        byte[] body = "hello".getBytes();
        AppVersion result = svc.uploadJar(appId, "a.jar", new ByteArrayInputStream(body), body.length);

        verify(store).put(eq(new ArtifactCoordinates("default", appId, 3)),
                          any(InputStream.class), eq(5L));
        assertThat(result.version()).isEqualTo(3);
    }
}
```

- [ ] **Step 2: Run — expect FAIL (constructor mismatch)**

Run: `mvn -pl cameleer-server-core test -Dtest=AppServiceUploadTest`

- [ ] **Step 3: Refactor `AppService`**

Replace the file contents:

```java
package com.cameleer.server.core.runtime;

import com.cameleer.server.core.storage.ArtifactCoordinates;
import com.cameleer.server.core.storage.ArtifactStore;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.MessageDigest;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.regex.Pattern;

public class AppService {
    private static final Logger log = LoggerFactory.getLogger(AppService.class);

    private static final Pattern SLUG_PATTERN = Pattern.compile("^[a-z0-9][a-z0-9-]{0,63}$");

    private final AppRepository appRepo;
    private final AppVersionRepository versionRepo;
    private final ArtifactStore artifactStore;
    private final String tenantId;
    private final CreateGuard createGuard;

    public AppService(AppRepository appRepo, AppVersionRepository versionRepo,
                      ArtifactStore artifactStore, String tenantId) {
        this(appRepo, versionRepo, artifactStore, tenantId, CreateGuard.NOOP);
    }

    public AppService(AppRepository appRepo, AppVersionRepository versionRepo,
                      ArtifactStore artifactStore, String tenantId, CreateGuard createGuard) {
        this.appRepo = appRepo;
        this.versionRepo = versionRepo;
        this.artifactStore = artifactStore;
        this.tenantId = tenantId;
        this.createGuard = createGuard;
    }

    public List<App> listAll() { return appRepo.findAll(); }
    public List<App> listByEnvironment(UUID environmentId) { return appRepo.findByEnvironmentId(environmentId); }
    public App getById(UUID id) { return appRepo.findById(id).orElseThrow(() -> new IllegalArgumentException("App not found: " + id)); }
    public App getBySlug(String slug) { return appRepo.findBySlug(slug).orElseThrow(() -> new IllegalArgumentException("App not found: " + slug)); }
    public App getByEnvironmentAndSlug(UUID environmentId, String slug) {
        return appRepo.findByEnvironmentIdAndSlug(environmentId, slug)
                .orElseThrow(() -> new IllegalArgumentException("App not found in environment: " + slug));
    }
    public List<AppVersion> listVersions(UUID appId) { return versionRepo.findByAppId(appId); }

    public AppVersion getVersion(UUID versionId) {
        return versionRepo.findById(versionId)
                .orElseThrow(() -> new IllegalArgumentException("AppVersion not found: " + versionId));
    }

    public ArtifactCoordinates coordinatesFor(AppVersion version) {
        return new ArtifactCoordinates(tenantId, version.appId(), version.version());
    }

    public void updateContainerConfig(UUID id, Map<String, Object> containerConfig) {
        getById(id);
        appRepo.updateContainerConfig(id, containerConfig);
    }

    public UUID createApp(UUID environmentId, String slug, String displayName) {
        if (slug == null || !SLUG_PATTERN.matcher(slug).matches()) {
            throw new IllegalArgumentException(
                    "Invalid app slug: must match ^[a-z0-9][a-z0-9-]{0,63}$");
        }
        createGuard.check(appRepo.count());
        if (appRepo.findByEnvironmentIdAndSlug(environmentId, slug).isPresent()) {
            throw new IllegalArgumentException("App with slug '" + slug + "' already exists in this environment");
        }
        return appRepo.create(environmentId, slug, displayName);
    }

    public AppVersion uploadJar(UUID appId, String filename, InputStream jarData, long size) throws IOException {
        getById(appId);
        int nextVersion = versionRepo.findMaxVersion(appId) + 1;
        ArtifactCoordinates coords = new ArtifactCoordinates(tenantId, appId, nextVersion);

        // Buffer once for hashing + storage so we can also detect runtime type without re-reading.
        ByteArrayOutputStream buf = new ByteArrayOutputStream(Math.max(8192, (int) Math.min(size, 64 * 1024 * 1024)));
        MessageDigest digest;
        try { digest = MessageDigest.getInstance("SHA-256"); }
        catch (Exception e) { throw new RuntimeException(e); }
        try (InputStream in = jarData) {
            byte[] tmp = new byte[8192];
            int n;
            while ((n = in.read(tmp)) != -1) {
                buf.write(tmp, 0, n);
                digest.update(tmp, 0, n);
            }
        }
        byte[] bytes = buf.toByteArray();

        String locator = artifactStore.put(coords, new java.io.ByteArrayInputStream(bytes), bytes.length);
        String checksum = HexFormat.of().formatHex(digest.digest());

        UUID versionId = versionRepo.create(appId, nextVersion, locator, checksum, filename, size);

        RuntimeDetector.DetectionResult detection = RuntimeDetector.detect(new java.io.ByteArrayInputStream(bytes));
        if (detection.runtimeType() != null) {
            versionRepo.updateDetectedRuntime(versionId, detection.runtimeType().toConfigValue(), detection.mainClass());
            log.info("Uploaded JAR for app {}: version={}, size={}, sha256={}, detected={}",
                    appId, nextVersion, size, checksum, detection.runtimeType().toConfigValue());
        } else {
            log.info("Uploaded JAR for app {}: version={}, size={}, sha256={}, detected=unknown",
                    appId, nextVersion, size, checksum);
        }

        return versionRepo.findById(versionId).orElseThrow();
    }

    public void deleteApp(UUID id) {
        appRepo.delete(id);
    }
}
```

> Note: `RuntimeDetector.detect(InputStream)` may not exist today — verify and add an `InputStream` overload if needed (it currently takes a `Path`). If adding the overload is non-trivial, write the bytes to a temp file in this method and pass the path; document the temp-file lifetime.

- [ ] **Step 4: Update `RuntimeBeanConfig`**

```java
// In cameleer-server-app/src/main/java/com/cameleer/server/app/config/RuntimeBeanConfig.java
// Replace the AppService bean:

@Bean
public ArtifactStore artifactStore(@Value("${cameleer.server.runtime.jarstoragepath:/data/jars}") String jarStoragePath) {
    return new FilesystemArtifactStore(jarStoragePath);
}

@Bean
public AppService appService(AppRepository appRepo,
                              AppVersionRepository versionRepo,
                              ArtifactStore artifactStore,
                              @Value("${cameleer.server.tenant.id:default}") String tenantId,
                              LicenseEnforcer licenseEnforcer) {
    return new AppService(appRepo, versionRepo, artifactStore, tenantId,
            current -> licenseEnforcer.assertWithinCap("max_apps", current, 1));
}
```

- [ ] **Step 5: Run all core tests — expect PASS**

```bash
mvn -pl cameleer-server-core test
mvn -pl cameleer-server-app test -Dtest='*AppService*,*RuntimeBeanConfig*'
```

Fix any callers of removed `resolveJarPath` — should be only `DeploymentExecutor` (handled in Task 7) and the orphaned-app cleanup.

- [ ] **Step 6: Commit**

```bash
git add -A
git commit -m "refactor(core): AppService writes via ArtifactStore; remove resolveJarPath"
```

---

## Task 5: Migrate `JarRetentionJob` to `ArtifactStore`

**Files:**
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/retention/JarRetentionJob.java`
- Test: `cameleer-server-app/src/test/java/com/cameleer/server/app/retention/JarRetentionJobTest.java` (extend existing or new)

- [ ] **Step 1: Write the failing test**

```java
@Test
void cleanupDeletesViaArtifactStore() {
    EnvironmentService envSvc = mock(EnvironmentService.class);
    AppService appSvc = mock(AppService.class);
    AppVersionRepository versionRepo = mock(AppVersionRepository.class);
    DeploymentRepository deployRepo = mock(DeploymentRepository.class);
    ArtifactStore store = mock(ArtifactStore.class);

    UUID envId = UUID.randomUUID(), appId = UUID.randomUUID();
    Environment env = new Environment(envId, "dev", "Dev", false, true,
            null, 2, "slate", Instant.now(), 7, 7, 7);
    App app = new App(appId, envId, "demo", "Demo", null);
    AppVersion v1 = new AppVersion(UUID.randomUUID(), appId, 1, "ignored", "h", "a.jar", 1L, null, null, null);
    AppVersion v2 = new AppVersion(UUID.randomUUID(), appId, 2, "ignored", "h", "a.jar", 1L, null, null, null);
    AppVersion v3 = new AppVersion(UUID.randomUUID(), appId, 3, "ignored", "h", "a.jar", 1L, null, null, null);

    when(envSvc.listAll()).thenReturn(List.of(env));
    when(appSvc.listByEnvironment(envId)).thenReturn(List.of(app));
    when(versionRepo.findByAppId(appId)).thenReturn(List.of(v3, v2, v1)); // DESC
    when(deployRepo.findByAppId(appId)).thenReturn(List.of());
    when(appSvc.coordinatesFor(v1)).thenReturn(new ArtifactCoordinates("default", appId, 1));

    new JarRetentionJob(envSvc, appSvc, versionRepo, deployRepo, store).cleanupOldVersions();

    verify(store).delete(new ArtifactCoordinates("default", appId, 1));
    verify(versionRepo).delete(v1.id());
    verifyNoMoreInteractions(store);
}
```

- [ ] **Step 2: Run — expect FAIL (constructor mismatch)**

- [ ] **Step 3: Refactor**

Replace `deleteJarFile` with a delegation to `store.delete(appSvc.coordinatesFor(version))`. Add `ArtifactStore` to the constructor.

```java
private final ArtifactStore store;

public JarRetentionJob(EnvironmentService environmentService,
                       AppService appService,
                       AppVersionRepository versionRepo,
                       DeploymentRepository deploymentRepo,
                       ArtifactStore store) {
    this.environmentService = environmentService;
    this.appService = appService;
    this.versionRepo = versionRepo;
    this.deploymentRepo = deploymentRepo;
    this.store = store;
}

private int cleanupApp(App app, int retentionCount) {
    List<AppVersion> versions = versionRepo.findByAppId(app.id());
    if (versions.size() <= retentionCount) return 0;

    Set<UUID> deployedVersionIds = deploymentRepo.findByAppId(app.id()).stream()
            .map(Deployment::appVersionId).collect(Collectors.toSet());

    int deleted = 0;
    for (int i = retentionCount; i < versions.size(); i++) {
        AppVersion version = versions.get(i);
        if (deployedVersionIds.contains(version.id())) continue;
        try {
            store.delete(appService.coordinatesFor(version));
        } catch (IOException e) {
            log.warn("Failed to delete artifact for version {}: {}", version.id(), e.getMessage());
        }
        versionRepo.delete(version.id());
        deleted++;
        log.info("Deleted version v{} of app {} ({})", version.version(), app.slug(), version.id());
    }
    return deleted;
}
```

Delete the old `deleteJarFile` method and unused `Files`/`Path` imports.

- [ ] **Step 4: Re-run tests — expect PASS**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "refactor(retention): JarRetentionJob deletes via ArtifactStore"
```

---

## Task 6: HMAC `ArtifactDownloadTokenSigner`

**Files:**
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/web/ArtifactDownloadTokenSigner.java`
- Test: `cameleer-server-app/src/test/java/com/cameleer/server/app/web/ArtifactDownloadTokenSignerTest.java`

- [ ] **Step 1: Write the failing test**

```java
package com.cameleer.server.app.web;

import org.junit.jupiter.api.Test;
import java.time.Clock;
import java.time.Duration;
import java.time.Instant;
import java.time.ZoneOffset;
import java.util.UUID;

import static org.assertj.core.api.Assertions.assertThat;

class ArtifactDownloadTokenSignerTest {

    private final String secret = "test-secret-do-not-use";
    private final Instant now = Instant.parse("2026-04-27T10:00:00Z");
    private final Clock clock = Clock.fixed(now, ZoneOffset.UTC);

    @Test
    void signedTokenVerifiesWithinTtl() {
        var signer = new ArtifactDownloadTokenSigner(secret, clock);
        UUID id = UUID.randomUUID();
        var token = signer.sign(id, Duration.ofMinutes(5));
        assertThat(signer.verify(id, token.exp(), token.sig())).isTrue();
    }

    @Test
    void rejectsTamperedSignature() {
        var signer = new ArtifactDownloadTokenSigner(secret, clock);
        UUID id = UUID.randomUUID();
        var token = signer.sign(id, Duration.ofMinutes(5));
        assertThat(signer.verify(id, token.exp(), token.sig() + "x")).isFalse();
    }

    @Test
    void rejectsMismatchedAppVersionId() {
        var signer = new ArtifactDownloadTokenSigner(secret, clock);
        UUID id = UUID.randomUUID();
        var token = signer.sign(id, Duration.ofMinutes(5));
        assertThat(signer.verify(UUID.randomUUID(), token.exp(), token.sig())).isFalse();
    }

    @Test
    void rejectsExpired() {
        var signer = new ArtifactDownloadTokenSigner(secret, clock);
        UUID id = UUID.randomUUID();
        long pastExp = now.minusSeconds(1).getEpochSecond();
        String sig = signer.signRaw(id, pastExp);
        assertThat(signer.verify(id, pastExp, sig)).isFalse();
    }
}
```

- [ ] **Step 2: Run — expect FAIL**

- [ ] **Step 3: Implement**

```java
package com.cameleer.server.app.web;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.time.Clock;
import java.time.Duration;
import java.util.Base64;
import java.util.UUID;

/**
 * HMAC-SHA256 signed URL tokens for artifact downloads. Key derivation is
 * deterministic from the JWT signing secret (HMAC-SHA256(jwtSecret,
 * "cameleer-artifact-token-v1")) so server restarts don't invalidate fresh
 * tokens. The loader container does NOT carry the JWT or the bootstrap token —
 * it only carries the signed URL, which is scoped to one appVersionId and one
 * short TTL.
 */
public class ArtifactDownloadTokenSigner {

    private static final String DERIVATION = "cameleer-artifact-token-v1";

    public record SignedToken(long exp, String sig) {}

    private final byte[] key;
    private final Clock clock;

    public ArtifactDownloadTokenSigner(String jwtSecret, Clock clock) {
        this.key = deriveKey(jwtSecret);
        this.clock = clock;
    }

    private static byte[] deriveKey(String jwtSecret) {
        try {
            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(new SecretKeySpec(jwtSecret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"));
            return mac.doFinal(DERIVATION.getBytes(StandardCharsets.UTF_8));
        } catch (Exception e) {
            throw new IllegalStateException("HMAC init failed", e);
        }
    }

    public SignedToken sign(UUID appVersionId, Duration ttl) {
        long exp = clock.instant().plus(ttl).getEpochSecond();
        return new SignedToken(exp, signRaw(appVersionId, exp));
    }

    public String signRaw(UUID appVersionId, long exp) {
        try {
            Mac mac = Mac.getInstance("HmacSHA256");
            mac.init(new SecretKeySpec(key, "HmacSHA256"));
            String payload = appVersionId + ":" + exp;
            byte[] tag = mac.doFinal(payload.getBytes(StandardCharsets.UTF_8));
            return Base64.getUrlEncoder().withoutPadding().encodeToString(tag);
        } catch (Exception e) {
            throw new IllegalStateException("sign failed", e);
        }
    }

    public boolean verify(UUID appVersionId, long exp, String sig) {
        if (sig == null || sig.isBlank()) return false;
        if (clock.instant().getEpochSecond() > exp) return false;
        String expected = signRaw(appVersionId, exp);
        // constant-time compare
        return java.security.MessageDigest.isEqual(
                expected.getBytes(StandardCharsets.UTF_8),
                sig.getBytes(StandardCharsets.UTF_8));
    }
}
```

- [ ] **Step 4: Run — expect PASS**

- [ ] **Step 5: Commit**

```bash
git add -A
git commit -m "feat(web): add HMAC token signer for artifact downloads"
```

---

## Task 7: `ArtifactDownloadController`

**Files:**
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/web/ArtifactDownloadController.java`
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/security/SecurityConfig.java`
- Test: `cameleer-server-app/src/test/java/com/cameleer/server/app/web/ArtifactDownloadControllerTest.java`

- [ ] **Step 1: Write the failing test (`@WebMvcTest`)**

```java
@WebMvcTest(ArtifactDownloadController.class)
class ArtifactDownloadControllerTest {

    @Autowired MockMvc mvc;
    @MockBean AppService appService;
    @MockBean ArtifactStore artifactStore;
    @MockBean ArtifactDownloadTokenSigner signer;

    @Test
    void streamsBytesWhenSignatureValid() throws Exception {
        UUID id = UUID.randomUUID();
        when(signer.verify(eq(id), eq(123L), eq("good"))).thenReturn(true);
        AppVersion v = new AppVersion(id, UUID.randomUUID(), 1, "loc", "h", "a.jar", 5L, null, null, null);
        when(appService.getVersion(id)).thenReturn(v);
        when(appService.coordinatesFor(v)).thenReturn(new ArtifactCoordinates("default", v.appId(), 1));
        when(artifactStore.get(any())).thenReturn(new ByteArrayInputStream("hello".getBytes()));

        mvc.perform(get("/api/v1/artifacts/{id}", id).param("exp", "123").param("sig", "good"))
                .andExpect(status().isOk())
                .andExpect(header().string("Content-Type", "application/java-archive"))
                .andExpect(content().bytes("hello".getBytes()));
    }

    @Test
    void rejectsBadSignature() throws Exception {
        UUID id = UUID.randomUUID();
        when(signer.verify(any(), anyLong(), any())).thenReturn(false);
        mvc.perform(get("/api/v1/artifacts/{id}", id).param("exp", "123").param("sig", "bad"))
                .andExpect(status().isUnauthorized());
    }

    @Test
    void returns404WhenArtifactMissing() throws Exception {
        UUID id = UUID.randomUUID();
        when(signer.verify(any(), anyLong(), any())).thenReturn(true);
        when(appService.getVersion(id)).thenThrow(new IllegalArgumentException("AppVersion not found"));
        mvc.perform(get("/api/v1/artifacts/{id}", id).param("exp", "123").param("sig", "ok"))
                .andExpect(status().isNotFound());
    }
}
```

- [ ] **Step 2: Run — expect FAIL**

- [ ] **Step 3: Implement**

```java
package com.cameleer.server.app.web;

import com.cameleer.server.core.runtime.AppService;
import com.cameleer.server.core.runtime.AppVersion;
import com.cameleer.server.core.storage.ArtifactStore;
import org.springframework.core.io.InputStreamResource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.io.IOException;
import java.io.InputStream;
import java.util.UUID;

/**
 * Token-validated artifact download endpoint hit by the cameleer-runtime-loader
 * init container. Auth is the HMAC sig + exp on the URL — NOT JWT/bootstrap
 * token. permitAll'd in SecurityConfig because all auth is in the controller.
 */
@RestController
@RequestMapping("/api/v1/artifacts")
public class ArtifactDownloadController {

    private final AppService appService;
    private final ArtifactStore artifactStore;
    private final ArtifactDownloadTokenSigner signer;

    public ArtifactDownloadController(AppService appService,
                                       ArtifactStore artifactStore,
                                       ArtifactDownloadTokenSigner signer) {
        this.appService = appService;
        this.artifactStore = artifactStore;
        this.signer = signer;
    }

    @GetMapping("/{appVersionId}")
    public ResponseEntity<InputStreamResource> download(@PathVariable UUID appVersionId,
                                                         @RequestParam("exp") long exp,
                                                         @RequestParam("sig") String sig) throws IOException {
        if (!signer.verify(appVersionId, exp, sig)) {
            return ResponseEntity.status(401).build();
        }
        AppVersion version;
        try {
            version = appService.getVersion(appVersionId);
        } catch (IllegalArgumentException e) {
            return ResponseEntity.notFound().build();
        }
        InputStream in = artifactStore.get(appService.coordinatesFor(version));
        return ResponseEntity.ok()
                .contentType(MediaType.parseMediaType("application/java-archive"))
                .contentLength(version.jarSizeBytes())
                .body(new InputStreamResource(in));
    }
}
```

- [ ] **Step 4: Add `permitAll` to `SecurityConfig`**

In `cameleer-server-app/src/main/java/com/cameleer/server/app/security/SecurityConfig.java`, add `/api/v1/artifacts/**` to the permitAll matchers (alongside `/api/v1/auth/**`, `/api/v1/health`, etc.).

- [ ] **Step 5: Wire `ArtifactDownloadTokenSigner` bean** (in `SecurityBeanConfig` or `RuntimeBeanConfig`)

```java
@Bean
public ArtifactDownloadTokenSigner artifactDownloadTokenSigner(
        @Value("${cameleer.server.security.jwtsecret}") String jwtSecret) {
    return new ArtifactDownloadTokenSigner(jwtSecret, java.time.Clock.systemUTC());
}
```

- [ ] **Step 6: Run all tests — expect PASS**

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat(web): add ArtifactDownloadController with HMAC URL auth"
```

---

## Task 8: Build the `cameleer-runtime-loader` image

**Files:**
- Create: `cameleer-runtime-loader/Dockerfile`
- Create: `cameleer-runtime-loader/entrypoint.sh`
- Create: `cameleer-runtime-loader/README.md`

- [ ] **Step 1: Write `entrypoint.sh`**

```sh
#!/bin/sh
# cameleer-runtime-loader: fetches one JAR from a signed URL into the shared
# /app/jars/ volume, verifies size, exits. Runs in the same hardened sandbox as
# the main container (cap_drop ALL, read-only rootfs, etc.) — only /app/jars/
# is writeable.
set -eu

: "${ARTIFACT_URL:?ARTIFACT_URL is required}"
: "${ARTIFACT_EXPECTED_SIZE:?ARTIFACT_EXPECTED_SIZE is required}"

OUT=/app/jars/app.jar
mkdir -p /app/jars

echo "loader: fetching artifact (expected $ARTIFACT_EXPECTED_SIZE bytes)"
# -q quiet, -O output, --tries=3 retry transient network blips,
# --timeout=30 cap stalls. wget exits non-zero on HTTP >=400.
wget -q --tries=3 --timeout=30 -O "$OUT" "$ARTIFACT_URL"

actual=$(wc -c < "$OUT")
if [ "$actual" -ne "$ARTIFACT_EXPECTED_SIZE" ]; then
    echo "loader: size mismatch — expected $ARTIFACT_EXPECTED_SIZE, got $actual" >&2
    exit 2
fi

echo "loader: artifact written to $OUT ($actual bytes)"
```

- [ ] **Step 2: Write `Dockerfile`**

```dockerfile
# Tiny init-container image. No app code, no shell-injection surface — script
# only sees env vars set by the orchestrator.
FROM busybox:1.37-musl

# Run as non-root (UID 1000 inside the container; with userns_mode this is
# remapped to host UID ~101000 — fully unprivileged on the host).
RUN adduser -D -u 1000 loader

COPY entrypoint.sh /usr/local/bin/loader
RUN chmod +x /usr/local/bin/loader

USER loader
WORKDIR /app
ENTRYPOINT ["/usr/local/bin/loader"]
```

- [ ] **Step 3: Build + smoke-test locally**

```bash
docker build -t gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest cameleer-runtime-loader/
docker run --rm -v test-vol:/app/jars \
    -e ARTIFACT_URL=https://example.invalid/missing \
    -e ARTIFACT_EXPECTED_SIZE=0 \
    gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest || echo "expected fail OK"
```

- [ ] **Step 4: Push to gitea registry**

```bash
docker push gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest
```

- [ ] **Step 5: Write `README.md`**

```markdown
# cameleer-runtime-loader

Init container that fetches the deployable JAR into a shared volume before the
main runtime container starts. Pairs with `DockerRuntimeOrchestrator` /
(future) K8s init-container deploys.

## Build

    docker build -t gitea.siegeln.net/cameleer/cameleer-runtime-loader:<tag> .
    docker push gitea.siegeln.net/cameleer/cameleer-runtime-loader:<tag>

## Contract

- Env: `ARTIFACT_URL` (signed download URL), `ARTIFACT_EXPECTED_SIZE` (bytes).
- Volume: writes `/app/jars/app.jar`.
- Exit 0 on success; non-zero on fetch/size failure.
- Runs as UID 1000 (loader user), drops all caps, read-only rootfs except `/app/jars`.

See `docs/superpowers/plans/2026-04-27-init-container-jar-fetch.md`.
```

- [ ] **Step 6: Commit**

```bash
git add cameleer-runtime-loader/
git commit -m "feat(loader): add cameleer-runtime-loader image (busybox + entrypoint)"
```

---

## Task 9: Refactor `ContainerRequest`

**Files:**
- Modify: `cameleer-server-core/src/main/java/com/cameleer/server/core/runtime/ContainerRequest.java`

No test — record refactor; behavior tested in Task 10.

- [ ] **Step 1: Replace the record**

```java
package com.cameleer.server.core.runtime;

import java.util.List;
import java.util.Map;
import java.util.UUID;

public record ContainerRequest(
        String containerName,
        String baseImage,
        UUID appVersionId,
        String artifactDownloadUrl,
        long artifactExpectedSize,
        String loaderImage,
        String network,
        List<String> additionalNetworks,
        Map<String, String> envVars,
        Map<String, String> labels,
        long memoryLimitBytes,
        Long memoryReserveBytes,
        int cpuShares,
        Long cpuQuota,
        List<Integer> exposedPorts,
        int healthCheckPort,
        String restartPolicyName,
        int restartPolicyMaxRetries,
        String runtimeType,
        String customArgs,
        String mainClass
) {}
```

- [ ] **Step 2: Compile (will break orchestrator + executor — fixed in next tasks)**

Run: `mvn -pl cameleer-server-core compile`

- [ ] **Step 3: Commit (broken intermediate — okay because next two tasks restore green)**

```bash
git add cameleer-server-core/src/main/java/com/cameleer/server/core/runtime/ContainerRequest.java
git commit -m "refactor(core): replace ContainerRequest jar fields with appVersionId+downloadUrl"
```

> If you prefer never to commit a broken state, fold Tasks 9, 10, 11 into one commit.

---

## Task 10: `DockerRuntimeOrchestrator` loader-pattern launch

**Files:**
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/runtime/DockerRuntimeOrchestrator.java`
- Test: `cameleer-server-app/src/test/java/com/cameleer/server/app/runtime/DockerRuntimeOrchestratorLoaderTest.java`

This is the central change. The orchestrator now: (1) creates a per-replica named volume, (2) creates + starts + waits for the loader container, (3) checks loader exit code, (4) creates the main container with the volume mounted RO at `/app/jars/`, (5) cleans up the loader container on success or failure.

- [ ] **Step 1: Write the failing test**

```java
@Test
void startContainerCreatesVolumeRunsLoaderThenStartsMain() throws Exception {
    DockerClient docker = mockDockerClientWithRuntimes(Map.of());
    // capture every createContainerCmd invocation in order
    CreateContainerCmd loaderCreate = mock(CreateContainerCmd.class, Answers.RETURNS_SELF);
    CreateContainerCmd mainCreate = mock(CreateContainerCmd.class, Answers.RETURNS_SELF);
    when(docker.createContainerCmd(eq("gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest"))).thenReturn(loaderCreate);
    when(docker.createContainerCmd(eq("base-image:latest"))).thenReturn(mainCreate);

    CreateContainerResponse loaderResp = mock(CreateContainerResponse.class);
    when(loaderResp.getId()).thenReturn("loader-id");
    when(loaderCreate.exec()).thenReturn(loaderResp);
    CreateContainerResponse mainResp = mock(CreateContainerResponse.class);
    when(mainResp.getId()).thenReturn("main-id");
    when(mainCreate.exec()).thenReturn(mainResp);

    when(docker.startContainerCmd(any())).thenReturn(mock(StartContainerCmd.class));
    when(docker.removeContainerCmd(any())).thenReturn(mock(RemoveContainerCmd.class, Answers.RETURNS_SELF));
    // wait returns exit 0
    var waitCmd = mock(WaitContainerCmd.class);
    when(docker.waitContainerCmd("loader-id")).thenReturn(waitCmd);
    var waitCallback = mock(WaitContainerResultCallback.class);
    when(waitCmd.exec(any())).thenReturn(waitCallback);
    when(waitCallback.awaitStatusCode(anyLong(), any())).thenReturn(0);
    // volume create
    when(docker.createVolumeCmd()).thenReturn(mock(CreateVolumeCmd.class, Answers.RETURNS_SELF));

    var orchestrator = new DockerRuntimeOrchestrator(docker, "");
    orchestrator.startContainer(sampleRequest("base-image:latest", "https://srv/api/v1/artifacts/uuid?exp=1&sig=x", 12345L));

    // loader was created with volume RW at /app/jars and the right env
    ArgumentCaptor<HostConfig> loaderHc = ArgumentCaptor.forClass(HostConfig.class);
    verify(loaderCreate).withHostConfig(loaderHc.capture());
    assertThat(loaderHc.getValue().getBinds()).anyMatch(b ->
        b.getVolume().getPath().equals("/app/jars") && b.getAccessMode() == AccessMode.rw);

    // main was created with volume RO at /app/jars
    ArgumentCaptor<HostConfig> mainHc = ArgumentCaptor.forClass(HostConfig.class);
    verify(mainCreate).withHostConfig(mainHc.capture());
    assertThat(mainHc.getValue().getBinds()).anyMatch(b ->
        b.getVolume().getPath().equals("/app/jars") && b.getAccessMode() == AccessMode.ro);

    // loader started, waited for, removed; then main started
    InOrder order = inOrder(docker);
    order.verify(docker).startContainerCmd("loader-id");
    order.verify(docker).waitContainerCmd("loader-id");
    order.verify(docker).removeContainerCmd("loader-id");
    order.verify(docker).startContainerCmd("main-id");
}

@Test
void startContainerAbortsAndCleansUpWhenLoaderFails() {
    // wire wait to return exit 2; expect RuntimeException; expect volume + loader removed
    // and main container NEVER created.
    // ... (use the same scaffolding; assert `docker.createContainerCmd("base-image:latest")` is never called)
}

@Test
void startContainerAppliesUsernsModeToBothContainers() {
    // assert hostConfig.getUsernsMode() == "host:1000:65536" on both loader and main HostConfig captures.
}
```

- [ ] **Step 2: Run — expect FAIL**

- [ ] **Step 3: Implement**

Replace the `startContainer` method body with the loader-then-main flow. Key code:

```java
private static final String LOADER_VOLUME_MOUNT = "/app/jars";
private static final String USERNS_MODE = "host:1000:65536";

@Override
public String startContainer(ContainerRequest request) {
    String volumeName = "cameleer-jars-" + request.containerName();
    dockerClient.createVolumeCmd().withName(volumeName).exec();

    // 1) Loader container — RW on the shared volume.
    String loaderId = createAndStartLoader(request, volumeName);
    int exitCode;
    try {
        exitCode = dockerClient.waitContainerCmd(loaderId)
                .exec(new com.github.dockerjava.core.command.WaitContainerResultCallback())
                .awaitStatusCode(120, java.util.concurrent.TimeUnit.SECONDS);
    } catch (Exception e) {
        cleanup(loaderId, volumeName);
        throw new RuntimeException("Loader wait failed for " + request.containerName(), e);
    } finally {
        try { dockerClient.removeContainerCmd(loaderId).withForce(true).exec(); }
        catch (Exception e) { log.warn("Failed to remove loader {}: {}", loaderId, e.getMessage()); }
    }
    if (exitCode != 0) {
        cleanupVolume(volumeName);
        throw new RuntimeException("Loader exited " + exitCode + " for " + request.containerName());
    }

    // 2) Main container — RO on the shared volume.
    return createAndStartMain(request, volumeName);
}

private String createAndStartLoader(ContainerRequest request, String volumeName) {
    HostConfig hc = baseHardenedHostConfig()
            .withBinds(new Bind(volumeName, new Volume(LOADER_VOLUME_MOUNT), AccessMode.rw))
            .withUsernsMode(USERNS_MODE)
            .withNetworkMode(request.network());
    if (!dockerRuntime.isBlank()) hc.withRuntime(dockerRuntime);

    var create = dockerClient.createContainerCmd(request.loaderImage())
            .withName(request.containerName() + "-loader")
            .withEnv(List.of(
                    "ARTIFACT_URL=" + request.artifactDownloadUrl(),
                    "ARTIFACT_EXPECTED_SIZE=" + request.artifactExpectedSize()))
            .withHostConfig(hc);
    String id = create.exec().getId();
    dockerClient.startContainerCmd(id).exec();
    return id;
}

private String createAndStartMain(ContainerRequest request, String volumeName) {
    List<String> envList = request.envVars().entrySet().stream()
            .map(e -> e.getKey() + "=" + e.getValue()).toList();

    HostConfig hc = baseHardenedHostConfig()
            .withMemory(request.memoryLimitBytes())
            .withMemorySwap(request.memoryLimitBytes())
            .withCpuShares(request.cpuShares())
            .withNetworkMode(request.network())
            .withRestartPolicy(RestartPolicy.onFailureRestart(request.restartPolicyMaxRetries()))
            .withUsernsMode(USERNS_MODE)
            .withBinds(new Bind(volumeName, new Volume(LOADER_VOLUME_MOUNT), AccessMode.ro));
    if (!dockerRuntime.isBlank()) hc.withRuntime(dockerRuntime);
    if (request.memoryReserveBytes() != null) hc.withMemoryReservation(request.memoryReserveBytes());
    if (request.cpuQuota() != null) hc.withCpuQuota(request.cpuQuota());

    String appJarPath = LOADER_VOLUME_MOUNT + "/app.jar";
    String customArgs = request.customArgs() != null && !request.customArgs().isBlank()
            ? " " + request.customArgs() : "";
    String entrypoint = switch (request.runtimeType()) {
        case "plain-java" -> "exec java -javaagent:/app/agent.jar" + customArgs +
                " -cp " + appJarPath + " " + request.mainClass();
        case "native" -> "exec " + appJarPath + customArgs;
        default -> "exec java -javaagent:/app/agent.jar" + customArgs + " -jar " + appJarPath;
    };

    var create = dockerClient.createContainerCmd(request.baseImage())
            .withName(request.containerName())
            .withEnv(envList)
            .withLabels(request.labels() != null ? request.labels() : Map.of())
            .withHostConfig(hc)
            .withHealthcheck(new HealthCheck()
                    .withTest(List.of("CMD-SHELL",
                            "wget -qO- http://localhost:" + request.healthCheckPort() + "/cameleer/health || exit 1"))
                    .withInterval(10_000_000_000L)
                    .withTimeout(5_000_000_000L)
                    .withRetries(3)
                    .withStartPeriod(30_000_000_000L))
            .withEntrypoint("sh", "-c", entrypoint);

    if (request.exposedPorts() != null && !request.exposedPorts().isEmpty()) {
        var ports = request.exposedPorts().stream()
                .map(com.github.dockerjava.api.model.ExposedPort::tcp)
                .toArray(com.github.dockerjava.api.model.ExposedPort[]::new);
        create.withExposedPorts(ports);
    }

    String id = create.exec().getId();
    dockerClient.startContainerCmd(id).exec();
    log.info("Started container {} ({})", request.containerName(), id);
    return id;
}

/** Hardening contract from issue #152 — applied uniformly to loader + main. */
private HostConfig baseHardenedHostConfig() {
    return HostConfig.newHostConfig()
            .withCapDrop(Capability.values())
            .withSecurityOpts(List.of("no-new-privileges:true", "apparmor=docker-default"))
            .withReadonlyRootfs(true)
            .withPidsLimit(PIDS_LIMIT)
            .withTmpFs(Map.of("/tmp", TMPFS_TMP_OPTS));
}

private void cleanup(String loaderId, String volumeName) {
    try { dockerClient.removeContainerCmd(loaderId).withForce(true).exec(); } catch (Exception ignored) {}
    cleanupVolume(volumeName);
}

private void cleanupVolume(String volumeName) {
    try { dockerClient.removeVolumeCmd(volumeName).exec(); } catch (Exception ignored) {}
}
```

- [ ] **Step 4: Volume cleanup on container removal**

In `removeContainer(String containerId)`, also remove the per-replica volume. The volume name is derived from the container name; look up the container name first via `inspectContainerCmd`:

```java
@Override
public void removeContainer(String containerId) {
    String volumeName = null;
    try {
        var name = dockerClient.inspectContainerCmd(containerId).exec().getName();
        if (name != null) {
            // Docker prefixes inspected names with '/'
            volumeName = "cameleer-jars-" + name.replaceFirst("^/", "");
        }
    } catch (Exception e) { log.warn("Could not inspect {} for volume name: {}", containerId, e.getMessage()); }
    try {
        dockerClient.removeContainerCmd(containerId).withForce(true).exec();
        log.info("Removed container {}", containerId);
    } catch (Exception e) {
        log.warn("Failed to remove container {}: {}", containerId, e.getMessage());
    }
    if (volumeName != null) cleanupVolume(volumeName);
}
```

- [ ] **Step 5: Run all orchestrator tests — expect PASS**

Run: `mvn -pl cameleer-server-app test -Dtest='DockerRuntimeOrchestrator*'`

The existing `DockerRuntimeOrchestratorHardeningTest` will need updating because `startContainer` now creates a volume and runs the loader first. Either update it to mock the loader path, or rewrite its assertions to inspect the *main* container's `HostConfig` capture (which still has the full hardening contract). Both paths must keep `cap_drop ALL`, `no-new-privileges`, `apparmor`, `readonly rootfs`, `pids_limit`, `/tmp` tmpfs.

- [ ] **Step 6: Commit**

```bash
git add -A
git commit -m "feat(runtime): orchestrator launches loader init container then main; lands withUsernsMode"
```

---

## Task 11: `DeploymentExecutor` — generate signed URL, drop filesystem checks

**Files:**
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/runtime/DeploymentExecutor.java`

- [ ] **Step 1: Inject `ArtifactDownloadTokenSigner` + new config**

Add to constructor + @Value:

```java
@Value("${cameleer.server.runtime.loaderimage:gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest}")
private String loaderImage;

@Value("${cameleer.server.runtime.artifacttokenttlseconds:600}")
private long artifactTokenTtlSeconds;

@Value("${cameleer.server.runtime.artifactbaseurl:}")
private String artifactBaseUrl; // e.g. http://cameleer-server:8081 — falls back to globalServerUrl

@Autowired
private ArtifactDownloadTokenSigner artifactTokenSigner;
```

- [ ] **Step 2: Replace JAR resolution in `executeAsync`**

Drop the `String jarPath = appService.resolveJarPath(deployment.appVersionId());` line. Replace with:

```java
AppVersion appVersion = appService.getVersion(deployment.appVersionId());
String artifactBase = artifactBaseUrl.isBlank()
        ? (globalServerUrl.isBlank() ? "http://cameleer-server:8081" : globalServerUrl)
        : artifactBaseUrl;
var token = artifactTokenSigner.sign(appVersion.id(), java.time.Duration.ofSeconds(artifactTokenTtlSeconds));
String artifactUrl = artifactBase + "/api/v1/artifacts/" + appVersion.id()
        + "?exp=" + token.exp() + "&sig=" + token.sig();
long expectedSize = appVersion.jarSizeBytes() == null ? 0L : appVersion.jarSizeBytes();
```

Pass `appVersion`, `artifactUrl`, `expectedSize` into the `DeployCtx`:

```java
private record DeployCtx(
        Deployment deployment,
        App app,
        Environment env,
        ResolvedContainerConfig config,
        UUID appVersionId,
        String artifactUrl,
        long artifactExpectedSize,
        String resolvedRuntimeType,
        String mainClass,
        String generation,
        String primaryNetwork,
        List<String> additionalNets,
        Map<String, String> baseEnvVars,
        Map<String, String> prometheusLabels,
        long deployStart
) {}
```

- [ ] **Step 3: Drop the filesystem check in `preFlightChecks`**

```java
private void preFlightChecks(AppVersion appVersion, ResolvedContainerConfig config) {
    if (appVersion.jarSizeBytes() == null || appVersion.jarSizeBytes() <= 0) {
        throw new IllegalStateException("AppVersion " + appVersion.id() + " has no recorded jarSizeBytes");
    }
    if (config.memoryLimitMb() <= 0) throw new IllegalStateException("...");
    if (config.appPort() <= 0 || config.appPort() > 65535) throw new IllegalStateException("...");
    if (config.replicas() < 1) throw new IllegalStateException("...");
}
```

Remove the `Files.exists` import.

- [ ] **Step 4: Update `startReplica` to build the new `ContainerRequest`**

```java
ContainerRequest request = new ContainerRequest(
        containerName, baseImage,
        ctx.appVersionId(), ctx.artifactUrl(), ctx.artifactExpectedSize(),
        loaderImage,
        ctx.primaryNetwork(),
        ctx.additionalNets(),
        replicaEnvVars, labels,
        config.memoryLimitBytes(), config.memoryReserveBytes(),
        config.dockerCpuShares(), config.dockerCpuQuota(),
        config.exposedPorts(), agentHealthPort,
        "on-failure", 3,
        ctx.resolvedRuntimeType(), config.customArgs(), ctx.mainClass()
);
```

Drop the `volumeName`/`jarStoragePath` plumbing.

- [ ] **Step 5: Drop `jarDockerVolume` + `jarStoragePath` @Value fields**

They're unused now. Also drop the `cameleer.server.runtime.jardockervolume` doc references in CLAUDE.md.

- [ ] **Step 6: Run executor tests — expect PASS**

```bash
mvn -pl cameleer-server-app test -Dtest='DeploymentExecutor*'
```

- [ ] **Step 7: Commit**

```bash
git add -A
git commit -m "feat(runtime): DeploymentExecutor generates signed URL; drops filesystem JAR plumbing"
```

---

## Task 12: Configuration + integration smoke test

**Files:**
- Modify: `cameleer-server-app/src/main/resources/application.yml`
- New: `cameleer-server-app/src/test/java/com/cameleer/server/app/runtime/InitContainerDeployIT.java`

- [ ] **Step 1: Add new properties to `application.yml`**

```yaml
cameleer:
  server:
    runtime:
      loaderimage: ${CAMELEER_SERVER_RUNTIME_LOADERIMAGE:gitea.siegeln.net/cameleer/cameleer-runtime-loader:latest}
      artifacttokenttlseconds: ${CAMELEER_SERVER_RUNTIME_ARTIFACTTOKENTTLSECONDS:600}
      artifactbaseurl: ${CAMELEER_SERVER_RUNTIME_ARTIFACTBASEURL:}
```

Drop `jardockervolume` and (keep `jarstoragepath` for the filesystem store, but mark it as filesystem-store-specific).

- [ ] **Step 2: Write the integration test (Testcontainers + real Docker)**

Drives the whole flow: spin up the server in MockMvc/full context, upload a tiny JAR, trigger a deployment against a local Docker daemon, assert: per-replica volume created → loader exited 0 → main container running → JAR contents in `/app/jars/app.jar` match. Use `AbstractPostgresIT` as a base. Build the loader image inside the test if not present in the local cache, OR skip when not present and emit a warning.

(Detailed test code omitted — model after `cameleer-server-app/src/test/java/.../runtime/*IT.java`.)

- [ ] **Step 3: Run with Docker daemon**

```bash
mvn -pl cameleer-server-app verify -Dit.test=InitContainerDeployIT
```

- [ ] **Step 4: Commit**

```bash
git add -A
git commit -m "test(runtime): end-to-end IT for init-container deploy"
```

---

## Task 13: Update class/endpoint maps

**Files:**
- Modify: `.claude/rules/docker-orchestration.md` — describe the loader pattern, named volume per replica, `withUsernsMode`, drop the "JAR bind-mount" lines
- Modify: `.claude/rules/core-classes.md` — add `ArtifactStore`, `ArtifactCoordinates` to a new "storage/" subsection; update `ContainerRequest` field list; remove `AppService.resolveJarPath`
- Modify: `.claude/rules/app-classes.md` — add `ArtifactDownloadController` + `ArtifactDownloadTokenSigner` + `FilesystemArtifactStore`; update the flat-endpoint allow-list with `/api/v1/artifacts/{appVersionId}` (reason: "Content-addressed download. HMAC-signed URL, no JWT context")
- Modify: `CLAUDE.md` — drop the `CAMELEER_SERVER_RUNTIME_JARDOCKERVOLUME` mention, replace with the loader-image / token-TTL / artifact-base-url envs

- [ ] **Step 1: Edit each rule file**

(No code template — read the existing file and make targeted updates.)

- [ ] **Step 2: Run `gitnexus_detect_changes` to verify scope**

```bash
# This is project policy per CLAUDE.md
```

- [ ] **Step 3: Commit**

```bash
git add -A
git commit -m "docs(rules): document init-container loader + ArtifactStore; drop bind-mount"
```

---

## Task 14: Regenerate OpenAPI schema

The new `ArtifactDownloadController` adds a public endpoint — SPA types need to know about it (even if the SPA never calls it directly).

- [ ] **Step 1: Start the backend**

```bash
mvn -pl cameleer-server-app spring-boot:run
```

- [ ] **Step 2: Regenerate**

```bash
cd ui && npm run generate-api:live
```

- [ ] **Step 3: Fix any TypeScript compile errors that surface**

```bash
cd ui && npm run typecheck
```

- [ ] **Step 4: Commit**

```bash
git add ui/src/api/openapi.json ui/src/api/schema.d.ts
git commit -m "chore(ui): regenerate openapi schema for /api/v1/artifacts"
```

---

## Self-Review

- **Spec coverage** — all four "discussed" items map to tasks:
  - ArtifactStore abstraction → Tasks 1–5
  - HMAC-signed URL + controller → Tasks 6–7
  - Loader image → Task 8
  - Container request + orchestrator + executor → Tasks 9–11
  - `withUsernsMode` → embedded in Task 10 (lands once volume model is in)
  - Maven coordinates UX → **deferred**, separate plan (out of scope; documented as a follow-on in the parent issue discussion)
- **Type consistency** — `coordinatesFor(AppVersion)` is referenced in Tasks 5 and 7 with the same signature; `ArtifactDownloadTokenSigner.SignedToken(exp, sig)` reused unchanged in 6/7/11; `ContainerRequest`'s new fields used identically in 9/10/11.
- **Placeholder scan** — Task 12 step 2 ("Detailed test code omitted") is the only deliberate omission; flagged so the engineer doesn't miss it. Everything else has either runnable code or specific edits.
- **Hidden assumption to verify in Task 4 step 3:** `RuntimeDetector.detect(InputStream)` may need to be added — current code takes a `Path`. Either add the overload or stage to a temp file inside `uploadJar`.
- **Migration safety** — existing `app_versions.jar_path` rows continue to resolve because `FilesystemArtifactStore.locator()` returns the same absolute path the upload code wrote yesterday. No data migration required.