feat(skills): add update-compositions skill and registry query reference
This commit is contained in:
2 files changed
+272
No files matched your search
@@ -0,0 +1,173 @@
|
||||
---
|
||||
name: update-compositions
|
||||
description: Use when updating, upgrading, or bumping Docker container images, tags, and composition services across one or more APEX fleet nodes.
|
||||
---
|
||||
|
||||
# Updating APEX Compositions
|
||||
|
||||
This skill guides the safe, structured update of container images and compose definitions across nodes in the APEX fleet.
|
||||
|
||||
## Core Rules
|
||||
|
||||
1. **Verify access first**: Never execute commands against a node without confirming SSH connectivity and credentials.
|
||||
2. **Hold critical services**: Always ask the user which services or compositions must NOT be touched before discovering or updating images.
|
||||
3. **Classify update scope**: Separate updates into **Minor/Patch** (non-breaking) and **Major** (breaking / schema migration risks).
|
||||
4. **Gate major updates**: Never apply a Major version bump without in-depth migration research and explicit user approval.
|
||||
5. **Verify health post-recreation**: An update is not complete until containers reach `Up` and `healthy` status with clean logs.
|
||||
|
||||
---
|
||||
|
||||
## Workflow Steps
|
||||
|
||||
### Step 1: Confirm Node Access
|
||||
|
||||
1. Identify the target nodes from the user request (e.g., `icarus`, `daedalus`, `morpheus`).
|
||||
2. If access credentials, custom SSH ports, or key paths are not declared in the environment's SSH configuration, ask the user:
|
||||
- SSH hostname / IP
|
||||
- SSH port (APEX default: `2222`)
|
||||
- SSH user (APEX default: `adam`)
|
||||
- Identity file / key
|
||||
3. Test connectivity to each node before proceeding:
|
||||
```bash
|
||||
ssh -p <port> <user>@<node_fqdn> "hostname && uptime"
|
||||
```
|
||||
4. **Completion check**: All target nodes return exit code 0 on the connectivity probe.
|
||||
|
||||
---
|
||||
|
||||
### Step 2: Inquire About Critical Services & Holds
|
||||
|
||||
1. Prompt the user explicitly before inspecting or modifying any services:
|
||||
> "Which critical services or compositions on these nodes should **NOT** be updated during this run (e.g. databases, SSO/Authelia, Mail, Vaultwarden, Matrix)?"
|
||||
2. Record the freeze list (held services/compositions).
|
||||
3. Ensure all subsequent steps skip services on the freeze list.
|
||||
4. **Completion check**: User response received, and the list of excluded services is confirmed.
|
||||
|
||||
---
|
||||
|
||||
### Step 3: Inventory Running Services and Images
|
||||
|
||||
For each target node:
|
||||
|
||||
1. List active containers, current images, and status:
|
||||
```bash
|
||||
ssh <node> "docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'"
|
||||
```
|
||||
2. Locate the compose files on the node:
|
||||
- Shared core: `/srv/<node_fqdn>/compositions/apex/docker-compose.yml`
|
||||
- Node applications: `/srv/<node_fqdn>/compositions/*/docker-compose.yml`
|
||||
- Overrides: `docker-compose.override.yml` in each composition folder
|
||||
3. Map every running container to its compose declaration:
|
||||
- Image repository (e.g. `authelia/authelia`, `postgres`, `jellyfin/jellyfin`)
|
||||
- Pinned tag / SHA digest (e.g. `:4`, `:16-alpine`, `:v1.37.1`)
|
||||
- Whether the image is locally built (`build:` context present)
|
||||
4. Exclude any services identified on the freeze list from Step 2.
|
||||
5. **Completion check**: A complete table of non-held compositions, running tags, and compose file locations is constructed.
|
||||
|
||||
---
|
||||
|
||||
### Step 4: Seek Latest Tags and Classify Update Types
|
||||
|
||||
For each candidate service:
|
||||
|
||||
1. Query the container registry (Docker Hub, GitHub Container Registry `ghcr.io`, Quay, etc.) for available release tags. Refer to [references/registry-queries.md](references/registry-queries.md) for registry API patterns.
|
||||
2. Determine the latest stable versioned tag:
|
||||
- Prefer semantic versioned tags (e.g. `1.38.0`, `16.4-alpine`) matching the existing packaging flavor.
|
||||
- Ignore pre-release, alpha, beta, or release-candidate (`-rc`) tags unless the running container already pins one.
|
||||
- Do not replace a pinned version tag with bare `:latest`.
|
||||
3. Compare the current tag with the latest tag using Semantic Versioning (SemVer: `MAJOR.MINOR.PATCH`):
|
||||
|
||||
| Tag Transition | Classification | Action Path |
|
||||
| :--- | :--- | :--- |
|
||||
| `1.37.1` -> `1.37.2` | **Patch** | Proceed to Step 6 |
|
||||
| `1.37.1` -> `1.38.0` | **Minor** | Proceed to Step 6 |
|
||||
| Floating tag (e.g. `:4` or `:16`) with new digest | **Minor / In-place** | Proceed to Step 6 |
|
||||
| `16` -> `17` or `1.x` -> `2.x` | **Major** | **Route to Step 5 (Strict Gate)** |
|
||||
| Locally built image (`build:`) | **Local Build** | Git update + build (Step 6) |
|
||||
|
||||
4. **Completion check**: Every eligible service has a determined target tag and is classified as Major or Minor/Patch.
|
||||
|
||||
---
|
||||
|
||||
### Step 5: Major Update Protocol (Strict Gating)
|
||||
|
||||
For each service classified as **Major**:
|
||||
|
||||
1. **Research Update Strategy**:
|
||||
- Check upstream release notes, migration guides, and changelogs.
|
||||
- Identify:
|
||||
- Breaking configuration changes (renamed environment variables, CLI flags).
|
||||
- Storage schema migrations (e.g., PostgreSQL major version pg_upgrade requirements).
|
||||
- Deprecated dependencies or protocol changes.
|
||||
2. **Formulate Migration Plan**:
|
||||
- Outline pre-upgrade steps (e.g., explicit database dump `pg_dump` / `mysqldump` beyond normal snapshots).
|
||||
- Outline required compose file changes (volumes, environment).
|
||||
- Outline fallback/rollback steps if the upgrade fails.
|
||||
3. **Request Explicit User Approval**:
|
||||
- Present the findings to the user:
|
||||
- Target service and node.
|
||||
- Current version vs New major version.
|
||||
- Summary of breaking changes and migration strategy.
|
||||
- Ask: *"Do you approve proceeding with the major update for `<service>` on `<node>` under this strategy?"*
|
||||
4. **Gate**:
|
||||
- If approved: Proceed to Step 6 for this service.
|
||||
- If rejected or deferred: Add service to the freeze list and leave it untouched.
|
||||
5. **Completion check**: Every Major update either has documented strategy + user approval or is explicitly skipped.
|
||||
|
||||
---
|
||||
|
||||
### Step 6: Apply Updates and Recreate Containers
|
||||
|
||||
Follow node rollout order to minimize blast radius:
|
||||
1. Application nodes / workers first (e.g., `icarus`).
|
||||
2. Communications / data nodes next (e.g., `morpheus`).
|
||||
3. Core edge / gateway / SSO nodes last (e.g., `daedalus`), ensuring routing and auth remain stable during earlier rollouts.
|
||||
|
||||
For each node:
|
||||
|
||||
1. **Pre-upgrade backup**:
|
||||
- Trigger a backup of stateful volumes:
|
||||
```bash
|
||||
ssh <node> "apex backup/run"
|
||||
```
|
||||
- For database services being updated, execute an in-container logical export to host storage.
|
||||
2. **Update image tags in compose files**:
|
||||
- Modify the target `docker-compose.yml` or `docker-compose.override.yml` with the new version tag.
|
||||
3. **Pull images**:
|
||||
```bash
|
||||
ssh <node> "cd /srv/<node_fqdn>/compositions/<service> && docker compose pull"
|
||||
```
|
||||
4. **Recreate services**:
|
||||
```bash
|
||||
ssh <node> "cd /srv/<node_fqdn>/compositions/<service> && docker compose up -d"
|
||||
```
|
||||
5. **Completion check**: Compose commands finish with exit code 0.
|
||||
|
||||
---
|
||||
|
||||
### Step 7: Verify Service Health & Stability
|
||||
|
||||
Immediately after recreation on each node:
|
||||
|
||||
1. **Inspect container state**:
|
||||
```bash
|
||||
ssh <node> "docker ps -a --filter name=<container_name> --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
|
||||
```
|
||||
- Verify status is `Up`.
|
||||
- If a healthcheck is configured, verify it transitions to `(healthy)` and does NOT get stuck in `(unhealthy)` or `(starting)`.
|
||||
2. **Inspect recent logs**:
|
||||
```bash
|
||||
ssh <node> "docker logs --tail 50 <container_name>"
|
||||
```
|
||||
- Check for crash loops, panic traces, connection refused, or migration errors.
|
||||
3. **Verify HTTP/network response (if edge/web service)**:
|
||||
```bash
|
||||
curl -IfsS -k https://<service_fqdn> || true
|
||||
```
|
||||
4. **Rollback protocol**:
|
||||
- If a container enters a crash loop (`Restarting`, `Exit 1`):
|
||||
1. Capture failure logs.
|
||||
2. Revert the image tag in `docker-compose.yml` to the prior tag.
|
||||
3. Re-run `docker compose up -d`.
|
||||
4. Notify user immediately with logs and root cause.
|
||||
5. **Completion check**: All updated containers are confirmed `Up` and `healthy` with clean logs.
|
||||
@@ -0,0 +1,99 @@
|
||||
# Container Registry Tag Queries
|
||||
|
||||
This reference details how to discover available image tags across common container registries without requiring external tooling.
|
||||
|
||||
## 1. Docker Hub (`hub.docker.com`)
|
||||
|
||||
Docker Hub provides an unauthenticated v2 API for public repositories.
|
||||
|
||||
### Official Images (Library)
|
||||
For top-level images like `postgres`, `redis`, `traefik`, `mariadb`:
|
||||
|
||||
```bash
|
||||
IMAGE="postgres"
|
||||
curl -s "https://registry.hub.docker.com/v2/repositories/library/${IMAGE}/tags?page_size=100" \
|
||||
| python3 -c '
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
tags = [t["name"] for t in data.get("results", [])]
|
||||
print("\n".join(tags[:30]))
|
||||
'
|
||||
```
|
||||
|
||||
### Community / User Images
|
||||
For namespace images like `adguard/adguardhome`, `prom/prometheus`, `grafana/alloy`:
|
||||
|
||||
```bash
|
||||
ORG="adguard"
|
||||
IMAGE="adguardhome"
|
||||
curl -s "https://registry.hub.docker.com/v2/repositories/${ORG}/${IMAGE}/tags?page_size=100" \
|
||||
| python3 -c '
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
tags = [t["name"] for t in data.get("results", [])]
|
||||
print("\n".join(tags[:30]))
|
||||
'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. GitHub Container Registry (`ghcr.io`)
|
||||
|
||||
GHCR implements OCI distribution spec. Public image tags can be queried with an anonymous bearer token.
|
||||
|
||||
```bash
|
||||
IMAGE="xtls/xray-core"
|
||||
TOKEN=$(curl -s "https://ghcr.io/token?scope=repository:${IMAGE}:pull" | python3 -c 'import sys, json; print(json.load(sys.stdin)["token"])')
|
||||
curl -s -H "Authorization: Bearer ${TOKEN}" "https://ghcr.io/v2/${IMAGE}/tags/list" \
|
||||
| python3 -c '
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
tags = data.get("tags", [])
|
||||
print("\n".join(sorted(tags, reverse=True)[:30]))
|
||||
'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Quay.io (`quay.io`)
|
||||
|
||||
Quay provides a REST API:
|
||||
|
||||
```bash
|
||||
REPO="repository_name"
|
||||
curl -s "https://quay.io/api/v1/repository/${REPO}/tag/?limit=50&onlyActiveTags=true" \
|
||||
| python3 -c '
|
||||
import sys, json
|
||||
data = json.load(sys.stdin)
|
||||
tags = [t["name"] for t in data.get("tags", [])]
|
||||
print("\n".join(tags[:30]))
|
||||
'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. SemVer Matching & Sorting
|
||||
|
||||
When filtering tags to find the latest stable version matching the current container's flavor:
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
def parse_semver(tag: str):
|
||||
# Match patterns like v1.2.3, 1.2.3-alpine, 16.4
|
||||
m = re.match(r"^v?(\d+)(?:\.(\d+))?(?:\.(\d+))?(?:-(.+))?$", tag)
|
||||
if not m:
|
||||
return None
|
||||
major = int(m.group(1))
|
||||
minor = int(m.group(2) or 0)
|
||||
patch = int(m.group(3) or 0)
|
||||
flavor = m.group(4) or ""
|
||||
return (major, minor, patch, flavor)
|
||||
```
|
||||
|
||||
Rules for candidate selection:
|
||||
1. Preserve flavor: If currently running `16-alpine`, match only tags with suffix `-alpine`.
|
||||
2. Ignore pre-releases: Filter out tags containing `alpha`, `beta`, `rc`, `dev`, `test`, or git SHAs.
|
||||
3. Classify:
|
||||
- Same `major`, higher `minor` or `patch` -> **Minor / Patch**.
|
||||
- Higher `major` -> **Major** (requires Step 5 approval).
|
||||
Reference in new issue
Block a user