Split host setup into explicit entity actions
This commit is contained in:
1 parent
9d733a8f5d
commit
e452c981aa
51 files changed
+1023
-275
No files matched your search
@@ -13,7 +13,7 @@ This skill guides the safe, structured update of container images, compose defin
|
||||
2. **Hold critical services**: Always ask the user which services or compositions must NOT be touched before discovering or updating images.
|
||||
3. **Classify update scope**: Separate updates into **Minor/Patch** (non-breaking), **Major** (breaking / schema migration risks), and **Unstable SemVer / CalVer** (requiring changelog scrutiny).
|
||||
4. **Gate major and breaking updates**: Never apply a Major version bump, SemVer `0.x` minor change, or CalVer jump without migration research and explicit user approval.
|
||||
5. **Preserve APEX engine context**: Always orchestrate compose actions through `./commons/apex compose` (or with derived `APEX_*` variables and layered env files) rather than bare `docker compose`.
|
||||
5. **Preserve APEX engine context**: Always orchestrate compose actions through `./commons/apex composition/run` (or with derived `APEX_*` variables and layered env files) rather than bare `docker compose`.
|
||||
6. **Safeguard in-tree patches**: Do not bump images for containers with host bind mounts that patch application source code without explicit review.
|
||||
7. **Verify fleet observability post-rollout**: An update is not complete until containers reach `Up` and `healthy`, logs are clean, Prometheus scrape targets report `UP`, blackbox probes pass, and remote write streams are established.
|
||||
|
||||
@@ -145,7 +145,7 @@ For each node:
|
||||
1. **Pre-upgrade backup**:
|
||||
- Trigger a backup of stateful volumes:
|
||||
```bash
|
||||
ssh <node> "apex backup/run"
|
||||
ssh <node> "apex tier/backup"
|
||||
```
|
||||
- For database services being updated, execute an in-container logical export to host storage.
|
||||
2. **Update image tags in compose definitions**:
|
||||
@@ -153,12 +153,12 @@ For each node:
|
||||
3. **Pull and recreate services via APEX CLI**:
|
||||
- Always run through the node's APEX CLI to preserve derived `APEX_*` variables and layered env files:
|
||||
```bash
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex compose pull"
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex compose up -d"
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex composition/run pull"
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex composition/run up -d"
|
||||
```
|
||||
- *Note on `--force-recreate`*: When updating containers using floating tags, recovering from host reboots, or handling services with `tmpfs` state (e.g. mail spools), pass `--force-recreate` to ensure the container entrypoint runs a clean initialization:
|
||||
```bash
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex compose up -d --force-recreate"
|
||||
ssh <node> "cd /srv/<node_fqdn> && ./commons/apex composition/run up -d --force-recreate"
|
||||
```
|
||||
4. **Completion check**: Compose commands finish with exit code 0.
|
||||
|
||||
@@ -195,7 +195,7 @@ When the update scope includes host operating system packages:
|
||||
2. Reboot node: `ssh <node> "sudo reboot"`
|
||||
3. Wait for SSH availability: poll connectivity with timeout.
|
||||
4. After boot, verify systemd state (`systemctl is-system-running`) and check that Docker containers auto-started.
|
||||
5. If services with `tmpfs` mounts fail startup checks (e.g. Postfix spool directories in mail containers), run `./commons/apex compose up -d --force-recreate`.
|
||||
5. If services with `tmpfs` mounts fail startup checks (e.g. Postfix spool directories in mail containers), run `./commons/apex composition/run up -d --force-recreate`.
|
||||
6. **Completion check**: Host packages are up to date and node is online.
|
||||
|
||||
---
|
||||
@@ -229,6 +229,6 @@ Immediately after recreation and host reboots on each node:
|
||||
- If a container enters a crash loop (`Restarting`, `Exit 1`) or breaks ingress:
|
||||
1. Capture failure logs.
|
||||
2. Revert the image tag in the compose file to the prior tag.
|
||||
3. Re-run `./commons/apex compose up -d --force-recreate`.
|
||||
3. Re-run `./commons/apex composition/run up -d --force-recreate`.
|
||||
4. Notify user immediately with logs and root cause.
|
||||
6. **Completion check**: All containers are confirmed `Up` and `healthy`, external ingress functions normally, and telemetry metrics/targets are green.
|
||||
@@ -15,6 +15,7 @@ Local infrastructure operations for independently owned nodes, with a Python sta
|
||||
|
||||
```sh
|
||||
# Clone your node's repository (which pins apex as commons/)
|
||||
cd /srv
|
||||
git clone --recurse-submodules ssh://git@git.example.com:22222/infrastructure/selfhost.example.com.git
|
||||
cd selfhost.example.com && ./init.sh && source ~/.bashrc
|
||||
|
||||
@@ -25,9 +26,13 @@ APEX_NODE_HOST=selfhost.example.com
|
||||
APEX_SUBNET=198.18.0.0/24
|
||||
EOF
|
||||
|
||||
# Configure host baseline and bring up compositions
|
||||
apex configure # firewall, crowdsec, cron, systemd, routing
|
||||
apex compose up # bring node services online
|
||||
# Stock Debian Docker host, after configuring the node repository
|
||||
apex config/packages
|
||||
apex config/base
|
||||
apex config/docker
|
||||
apex config/ufw-docker
|
||||
apex config/ufw # apply the node firewall rules
|
||||
apex composition/run up # bring node services online
|
||||
```
|
||||
|
||||
## Essential Information
|
||||
@@ -43,7 +48,7 @@ You should understand the following invariants before exploring the codebase:
|
||||
|
||||
### Repository Structure
|
||||
|
||||
- **[`actions/`](./actions)**: Built-in engine actions (`compose.py`, `configure/`, `sync/`, `tiers/`, `backup/`, `utils/`).
|
||||
- **[`actions/`](./actions)**: Built-in engine actions (`composition/`, `config/`, `repository/`, `tier/`, `util/`).
|
||||
- **[`compositions/`](./compositions)**: The shared core Compose stack (`traefik`, `socket-proxy`, `alloy`, `adguard`, `xray`, `restic`).
|
||||
- **[`engine/`](./engine)**: Framework engine — action discovery, overlay resolution, context injection, and logging.
|
||||
- **[`etc/`](./etc)**: Shared static configuration, including Alloy.
|
||||
@@ -60,20 +65,21 @@ Running and maintaining services on an existing host.
|
||||
|
||||
- **Requirements**: Debian-based host with Docker, Docker Compose v2, Git, and Python 3.
|
||||
- **Workflow**:
|
||||
- Update services: `apex compose pull && apex compose up`
|
||||
- Run backups: `apex backup/run`
|
||||
- Sync repository state: `apex sync/repository`
|
||||
- Update services: `apex composition/run pull && apex composition/run up`
|
||||
- Run backups: `apex tier/backup`
|
||||
- Sync repository state: `apex repository/sync`
|
||||
|
||||
Target a composition explicitly when operating on one application:
|
||||
|
||||
```sh
|
||||
apex compose -p apex restart traefik
|
||||
apex compose -p app up -d
|
||||
apex compose --dry-run -p apex logs --tail 20 xray
|
||||
apex composition/run -p apex restart traefik
|
||||
apex composition/run -p app up -d
|
||||
apex composition/run --dry-run -p apex logs --tail 20 xray
|
||||
```
|
||||
|
||||
The existing `apex compose` spelling is an alias for `apex composition/run`.
|
||||
Wrapper options precede the action; remaining arguments go to Docker Compose.
|
||||
The positional forms `apex compose restart apex traefik` and `apex compose up -d apex`
|
||||
The positional forms `apex composition/run restart apex traefik` and `apex composition/run up -d apex`
|
||||
remain supported. Use `-p` when service names or option values could be mistaken
|
||||
for composition names. Without a target, APEX visits core first, then other
|
||||
compositions alphabetically; `down` reverses that order.
|
||||
@@ -84,7 +90,7 @@ after changing a container definition; `restart` retains the existing definition
|
||||
For direct Docker Compose use in the core composition:
|
||||
|
||||
```sh
|
||||
apex env --sync
|
||||
apex config/env --sync
|
||||
cd compositions/apex
|
||||
docker compose config --quiet
|
||||
docker compose restart traefik
|
||||
@@ -96,12 +102,27 @@ profiles) followed by synthesized node/path values, and writes with mode `0600`.
|
||||
Resync after changing `apex.env`, node identity, or tier paths. Shell environment
|
||||
values still take precedence in Docker Compose. Do not edit the generated block.
|
||||
|
||||
`eval "$(apex env)"` exports synthesized APEX values to the current shell;
|
||||
`apex env --dotenv` prints dotenv syntax, and `apex env --write generated.env`
|
||||
`eval "$(apex config/env)"` exports synthesized APEX values to the current shell;
|
||||
`apex config/env --dotenv` prints dotenv syntax, and `apex config/env --write generated.env`
|
||||
writes those values to a separate file. These three modes do not merge the
|
||||
composition's secrets or `apex.env`; use `--sync` for the complete core bridge.
|
||||
Choose one output mode per invocation.
|
||||
|
||||
### Preparing an existing host
|
||||
|
||||
Run each setup action explicitly from the node checkout. Use `apex --help` to list the shared and node-owned actions. New actions follow `<singular-entity>/<action>`. Use `apex <entity>/<action> --help` for the arguments.
|
||||
|
||||
On an installed Proxmox VE host, `config/packages` installs the administrator tools and `config/base` applies APT and journald policy. The PVE host keeps its own storage, networking, firewall, and management listeners. Run workload configuration inside the guest that owns the service.
|
||||
|
||||
To set up the operator on either platform, start with an account that can administer the host:
|
||||
|
||||
```sh
|
||||
apex config/adam --public-key-file /path/to/operator.pub
|
||||
apex repository/chown
|
||||
```
|
||||
|
||||
`config/adam` grants passwordless sudo. `repository/chown` gives adam ownership of the tracked checkout directly under `/srv` and its Git metadata. Open a new SSH connection as adam with the operator key and run `./commons/apex config/hardening` from the checkout in that session. An independent administrator login can also run the action after the operator’s authorized key and sudo access are configured. It applies the SSH public-key policy for adam and preserves the platform's existing management ports and other logins.
|
||||
|
||||
### Infrastructure Administrator
|
||||
|
||||
Build Debian 13 amd64 golden images with Packer, then supply node identity,
|
||||
@@ -115,8 +136,7 @@ packer validate ci/packer/apex-debian-13.pkr.hcl
|
||||
packer build ci/packer/apex-debian-13.pkr.hcl
|
||||
```
|
||||
|
||||
Cloud-init runs the image's bundled enrollment helper. After enrollment, run
|
||||
`apex configure` and `apex compose up` on the node.
|
||||
Cloud-init runs the image's bundled enrollment helper. After enrollment, apply the individual `config/*` actions required by the node, then run `apex composition/run up`.
|
||||
|
||||
### Developer & Contributor
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# actions/compose.py
|
||||
# actions/composition/run.py
|
||||
"""Orchestrate docker compose across the node.
|
||||
|
||||
Supports targeting all compositions in sequence (default: apex core first, services next;
|
||||
reverse on down) or targeting one with `apex compose -p apex restart traefik`.
|
||||
reverse on down) or targeting one with `apex composition/run -p apex restart traefik`.
|
||||
Wrapper options precede the action. Exports derived APEX_* vars and layers env files.
|
||||
"""
|
||||
import os
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Provision the golden-image operator while preserving authorized keys."""
|
||||
import pwd
|
||||
import grp
|
||||
from pathlib import Path
|
||||
from engine.descriptor import Meta, Opt
|
||||
|
||||
METADATA = Meta(summary="Configure the adam operator, sudo access and optional SSH public key.",
|
||||
args=[Opt("--public-key-file", "Append one OpenSSH public key from this file.")])
|
||||
|
||||
|
||||
HOME = Path("/home/adam")
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
system = ctx.sys
|
||||
key = None
|
||||
if args.public_key_file:
|
||||
key = Path(args.public_key_file).read_text().strip()
|
||||
if len(key.splitlines()) != 1 or not key.startswith(("ssh-", "ecdsa-", "sk-")):
|
||||
raise ValueError("Supply one OpenSSH public key.")
|
||||
system.run(["ssh-keygen", "-lf", "-"], input=key + "\n", capture=True)
|
||||
groups = ["sudo"]
|
||||
try:
|
||||
grp.getgrnam("docker")
|
||||
except KeyError:
|
||||
pass
|
||||
else:
|
||||
groups.append("docker")
|
||||
try:
|
||||
account = pwd.getpwnam("adam")
|
||||
except KeyError:
|
||||
if HOME.exists() or HOME.is_symlink():
|
||||
raise RuntimeError("Resolve existing /home/adam before creating the account.") from None
|
||||
system.sudo(["useradd", "--create-home", "--shell", "/bin/bash", "--user-group", "adam"])
|
||||
account = pwd.getpwnam("adam")
|
||||
if account.pw_uid < 1000 or Path(account.pw_dir) != HOME or HOME.resolve() != HOME:
|
||||
raise RuntimeError("Existing adam account has incompatible UID or home.")
|
||||
if Path(account.pw_dir).stat().st_uid != account.pw_uid:
|
||||
raise RuntimeError("The operator must own /home/adam.")
|
||||
system.sudo(["usermod", "--shell", "/bin/bash", "--append", "--groups", ",".join(groups), "adam"])
|
||||
if key:
|
||||
system.sudo(["runuser", "-u", "adam", "--", "sh", "-ec",
|
||||
'umask 077; cd "$1"; mkdir -p .ssh; chmod 700 .ssh; '
|
||||
'touch .ssh/authorized_keys; chmod 600 .ssh/authorized_keys; '
|
||||
'IFS= read -r key; grep -Fxq -- "$key" .ssh/authorized_keys || '
|
||||
'printf "\\n%s\\n" "$key" >> .ssh/authorized_keys', "sh", str(HOME)], input=key + "\n")
|
||||
content = "adam ALL=(ALL:ALL) NOPASSWD: ALL\n"
|
||||
system.sudo(["visudo", "-cf", "-"], input=content)
|
||||
path = Path("/etc/sudoers.d/99-apex-adam")
|
||||
if path.is_symlink():
|
||||
raise RuntimeError(f"Refusing redirected sudo policy: {path}")
|
||||
ctx.host.write_file(str(path), content, mode="0440", backup=False, sudo=True)
|
||||
system.sudo(["visudo", "-c"])
|
||||
system.sudo(["runuser", "-u", "adam", "--", "sudo", "-n", "true"])
|
||||
File renamed without changes.
@@ -0,0 +1,40 @@
|
||||
"""Apply the common APT archive and journald retention policy."""
|
||||
from pathlib import Path
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Configure APT archive cleanup and journald retention.")
|
||||
|
||||
|
||||
def _configure(ctx, path, legacy, content, restart=False):
|
||||
for target in (path, legacy):
|
||||
if target.is_symlink():
|
||||
raise RuntimeError(f"Refusing redirected configuration: {target}")
|
||||
previous = path.read_text() if path.exists() else None
|
||||
backup = ctx.host.backup(str(legacy), sudo=True) if legacy.exists() else None
|
||||
if previous == content and not backup:
|
||||
return
|
||||
if previous != content:
|
||||
ctx.host.write_file(str(path), content, sudo=True)
|
||||
if backup:
|
||||
ctx.sys.sudo(["rm", "--", str(legacy)])
|
||||
if restart:
|
||||
try:
|
||||
ctx.sys.sudo(["systemctl", "restart", "systemd-journald"])
|
||||
except Exception:
|
||||
if backup:
|
||||
ctx.sys.sudo(["cp", "--preserve=all", "--", backup, str(legacy)])
|
||||
if previous is None:
|
||||
ctx.sys.sudo(["rm", "--", str(path)])
|
||||
elif previous != content:
|
||||
ctx.host.write_file(str(path), previous, backup=False, sudo=True)
|
||||
raise
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
if not Path("/etc/debian_version").is_file():
|
||||
raise RuntimeError("Debian required.")
|
||||
apt = Path("/etc/apt/apt.conf.d")
|
||||
journal = Path("/etc/systemd/journald.conf.d")
|
||||
_configure(ctx, apt / "99-apex", apt / "01clean", 'APT::Keep-Downloaded-Packages "0";\n')
|
||||
_configure(ctx, journal / "99-apex.conf", journal / "maxuse.conf",
|
||||
"[Journal]\nSystemMaxUse=200M\n", restart=True)
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/configure/cron.py
|
||||
"""Port of configure/cron — install crontab from configs/cron/crontab."""
|
||||
# actions/config/cron.py
|
||||
"""Port of config/cron — install crontab from configs/cron/crontab."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Flag
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/configure/crowdsec.py
|
||||
"""Port of configure/crowdsec — install, deploy /etc/crowdsec, collections, bouncer."""
|
||||
# actions/config/crowdsec.py
|
||||
"""Port of config/crowdsec — install, deploy /etc/crowdsec, collections, bouncer."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Flag
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Install a local Docker Engine, Compose and Buildx on Debian 13."""
|
||||
from pathlib import Path
|
||||
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Verify a working Docker toolchain or install it on stock Debian 13.")
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
provisioning = Path(ctx.paths.commons) / "ci/scripts/init-node"
|
||||
ctx.sys.sudo(["python3", "-E", "-s", "-m", "provisioning.baseline", "--docker-policy"],
|
||||
cwd=str(provisioning))
|
||||
File renamed without changes.
@@ -0,0 +1,143 @@
|
||||
"""Apply the image's public-key SSH policy to adam after a verified key login."""
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import pwd
|
||||
import re
|
||||
from pathlib import Path
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Require SSH public-key authentication for adam from an independent administrator or verified adam key session.")
|
||||
CONFIG = "/etc/ssh/sshd_config"
|
||||
POLICY = """
|
||||
# APEX operator SSH policy
|
||||
Match User adam
|
||||
AuthenticationMethods publickey
|
||||
PubkeyAuthentication yes
|
||||
PasswordAuthentication no
|
||||
KbdInteractiveAuthentication no
|
||||
PermitEmptyPasswords no
|
||||
X11Forwarding no
|
||||
"""
|
||||
|
||||
|
||||
def _ancestors():
|
||||
ancestors = {}
|
||||
pid = os.getppid()
|
||||
ticks = os.sysconf("SC_CLK_TCK")
|
||||
while pid > 1:
|
||||
# comm can contain spaces and parentheses; the remaining fields start at state.
|
||||
fields = Path(f"/proc/{pid}/stat").read_text().rsplit(")", 1)[1].split()
|
||||
ancestors[pid] = int(fields[19]) * 1_000_000 // ticks
|
||||
pid = int(fields[1])
|
||||
return ancestors
|
||||
|
||||
|
||||
def _connection(ctx):
|
||||
if pwd.getpwuid(os.getuid()).pw_name != "adam":
|
||||
raise RuntimeError("Run config/hardening inside an SSH public-key session as adam.")
|
||||
ancestors = _ancestors()
|
||||
sockets = ctx.sys.sudo(["ss", "-Hntp", "state", "established"], capture=True).stdout
|
||||
connections = set()
|
||||
for line in sockets.splitlines():
|
||||
fields = line.split(None, 4)
|
||||
if len(fields) == 5 and any(int(pid) in ancestors for pid in
|
||||
re.findall(r'"sshd(?:-session)?",pid=(\d+),', fields[4])):
|
||||
connections.add((fields[2], fields[3]))
|
||||
if len(connections) != 1:
|
||||
raise RuntimeError("Cannot identify a unique established SSH socket for this process.")
|
||||
local, peer = connections.pop()
|
||||
peer, peer_port = peer.rsplit(":", 1)
|
||||
local, local_port = local.rsplit(":", 1)
|
||||
def address(value):
|
||||
ip = ipaddress.ip_address(value.strip("[]"))
|
||||
return str(getattr(ip, "ipv4_mapped", None) or ip)
|
||||
peer, local = address(peer), address(local)
|
||||
record = rf"^Accepted publickey for adam from {re.escape(peer)} port {int(peer_port)} ssh2:"
|
||||
result = ctx.sys.sudo(["journalctl", "-b", "_UID=0", "_EXE=/usr/sbin/sshd",
|
||||
"_EXE=/usr/lib/openssh/sshd-session", "_EXE=/usr/lib/openssh/sshd-auth",
|
||||
"--no-pager", "-o", "json", "--grep", record], capture=True, check=False)
|
||||
if not result.returncode:
|
||||
for line in result.stdout.splitlines():
|
||||
entry = json.loads(line)
|
||||
pid = int(entry["_PID"])
|
||||
if (pid in ancestors and int(entry["__MONOTONIC_TIMESTAMP"]) >= ancestors[pid]
|
||||
and re.match(record, entry["MESSAGE"])):
|
||||
ctx.sys.run(["sudo", "-n", "true"])
|
||||
return f"user=adam,host={peer},addr={peer},laddr={local},lport={int(local_port)}"
|
||||
raise RuntimeError("Cannot verify public-key login for this SSH process in the root sshd journal.")
|
||||
|
||||
|
||||
def _operator_access(ctx):
|
||||
account = pwd.getpwnam("adam")
|
||||
if account.pw_uid < 1000 or account.pw_dir != "/home/adam":
|
||||
raise RuntimeError("Configure a regular adam operator account first.")
|
||||
if pwd.getpwuid(os.getuid()).pw_name == "adam":
|
||||
return _connection(ctx)
|
||||
loginuid = int(Path("/proc/self/loginuid").read_text())
|
||||
if loginuid in (account.pw_uid, 4294967295) or loginuid < 0:
|
||||
raise RuntimeError("Use an independent administrator login or an SSH public-key session as adam.")
|
||||
if os.getuid() not in (0, loginuid):
|
||||
raise RuntimeError("Run from the independent administrator's own login.")
|
||||
ctx.sys.sudo(["true"])
|
||||
home = Path(account.pw_dir)
|
||||
authorized = home / ".ssh" / "authorized_keys"
|
||||
# Inspect as adam so private .ssh directories remain usable from another administrator.
|
||||
command = ["runuser", "-u", "adam", "--"]
|
||||
for path in (home, authorized.parent, authorized):
|
||||
metadata = ctx.sys.sudo([*command, "env", "LC_ALL=C", "stat", "-c", "%u %a %F", "--", str(path)],
|
||||
capture=True).stdout.strip().split(" ", 2)
|
||||
owner, mode, kind = metadata
|
||||
expected = "regular file" if path == authorized else "directory"
|
||||
if int(owner) not in (0, account.pw_uid) or int(mode, 8) & 0o022 or kind != expected:
|
||||
raise RuntimeError(f"Unsafe operator SSH path: {path}")
|
||||
keys = ctx.sys.sudo([*command, "cat", "--", str(authorized)], capture=True).stdout
|
||||
for key in keys.splitlines():
|
||||
if key.startswith(("ssh-", "ecdsa-", "sk-")) and ctx.sys.run(
|
||||
["ssh-keygen", "-lf", "-"], input=key + "\n", capture=True, check=False,
|
||||
).returncode == 0:
|
||||
break
|
||||
else:
|
||||
raise RuntimeError("Install a valid unrestricted operator public key before applying SSH hardening.")
|
||||
ctx.sys.sudo([*command, "sudo", "-n", "true"])
|
||||
connection = "user=adam,host=localhost,addr=127.0.0.1"
|
||||
paths = _effective(ctx.sys, connection).get("authorizedkeysfile", "").split()
|
||||
if not {".ssh/authorized_keys", "%h/.ssh/authorized_keys", str(authorized)}.intersection(paths):
|
||||
raise RuntimeError("sshd does not use the operator's verified authorized_keys file.")
|
||||
return connection
|
||||
|
||||
|
||||
def _effective(system, connection):
|
||||
result = system.sudo(["/usr/sbin/sshd", "-T", "-C", connection], capture=True)
|
||||
return dict(line.split(" ", 1) for line in result.stdout.splitlines() if " " in line)
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
connection = _operator_access(ctx)
|
||||
system = ctx.sys
|
||||
system.sudo(["/usr/sbin/sshd", "-t"])
|
||||
effective = _effective(system, connection)
|
||||
if effective.get("authenticationmethods") not in ("any", "publickey"):
|
||||
raise RuntimeError("Preserve the existing SSH authentication chain; reconcile it explicitly.")
|
||||
required = {"authenticationmethods": "publickey", "pubkeyauthentication": "yes",
|
||||
"passwordauthentication": "no", "kbdinteractiveauthentication": "no",
|
||||
"permitemptypasswords": "no", "x11forwarding": "no"}
|
||||
if all(effective.get(key) == value for key, value in required.items()):
|
||||
return
|
||||
if Path(CONFIG).is_symlink():
|
||||
raise RuntimeError(f"Refusing redirected SSH configuration: {CONFIG}")
|
||||
original = system.sudo(["cat", CONFIG], capture=True).stdout
|
||||
if "# APEX operator SSH policy" in original:
|
||||
raise RuntimeError("The managed SSH policy is ineffective; reconcile earlier directives.")
|
||||
ctx.host.write_file(CONFIG, original + "\n" + POLICY, sudo=True)
|
||||
try:
|
||||
system.sudo(["/usr/sbin/sshd", "-t"])
|
||||
effective = _effective(system, connection)
|
||||
if any(effective.get(key) != value for key, value in required.items()):
|
||||
raise RuntimeError("The effective SSH policy differs; inspect existing Match blocks.")
|
||||
system.reload("ssh")
|
||||
except BaseException:
|
||||
ctx.host.write_file(CONFIG, original, backup=False, sudo=True)
|
||||
system.sudo(["/usr/sbin/sshd", "-t"])
|
||||
system.reload("ssh")
|
||||
raise
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Install common Debian administrator tools without replacing host packages."""
|
||||
import re
|
||||
from pathlib import Path
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Install Debian administrator packages and enable rsyslog.")
|
||||
BASE_PACKAGES = ("curl", "wget", "git", "rsyslog", "ufw")
|
||||
|
||||
|
||||
ADMIN_PACKAGES = (
|
||||
"tcpdump", "socat", "rsync", "bind9-dnsutils", "iproute2", "net-tools",
|
||||
"traceroute", "ca-certificates", "jq", "sed", "gawk", "tar", "gzip",
|
||||
"bzip2", "xz-utils", "unzip", "htop", "iotop", "strace", "lsof",
|
||||
"sysstat", "parted", "fdisk", "pciutils", "usbutils", "smartmontools",
|
||||
"tmux", "vim", "less", "tree", "bash-completion", "python3",
|
||||
"python3-minimal", "python3-yaml", "sudo", "cron", "iptables", "ipset",
|
||||
"debian-archive-keyring", "kmod",
|
||||
)
|
||||
|
||||
def _install_missing(s, packages):
|
||||
missing = []
|
||||
for package in packages:
|
||||
result = s.run(["dpkg-query", "-W", "-f=${Status}", package],
|
||||
check=False, capture=True)
|
||||
if result.returncode or result.stdout.split()[1:] != ["ok", "installed"]:
|
||||
missing.append(package)
|
||||
if not missing:
|
||||
return
|
||||
apt = ["env", "LC_ALL=C", "DEBIAN_FRONTEND=noninteractive", "apt-get",
|
||||
"-o", "DPkg::Lock::Timeout=60", "-o", "Acquire::Retries=2",
|
||||
"-o", "Acquire::http::Timeout=30", "-o", "Acquire::https::Timeout=30"]
|
||||
s.sudo([*apt, "update", "--error-on=any"])
|
||||
command = [*apt, "install", "-y", "--no-install-recommends", "--no-upgrade",
|
||||
"--no-remove", *missing]
|
||||
plan = s.sudo([*command, "--simulate"], capture=True)
|
||||
if any(line.startswith("Remv ") or re.match(r"^Inst \S+ \[", line)
|
||||
for line in plan.stdout.splitlines()):
|
||||
raise RuntimeError("Missing packages require upgrades/removals; reconcile packages first.")
|
||||
s.sudo(command)
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
if not Path("/etc/debian_version").is_file():
|
||||
raise RuntimeError("Debian required.")
|
||||
packages = (*BASE_PACKAGES, *ADMIN_PACKAGES)
|
||||
if Path("/usr/bin/pveversion").exists():
|
||||
packages = tuple(package for package in packages if package != "ufw")
|
||||
_install_missing(ctx.sys, packages)
|
||||
if not ctx.sys.start_and_enable("rsyslog"):
|
||||
raise RuntimeError("rsyslog failed to start.")
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/configure/routing.py
|
||||
"""Port of configure/routing — idempotent node routing (ip_forward, fwmark rule, table 100)."""
|
||||
# actions/config/routing.py
|
||||
"""Port of config/routing — idempotent node routing (ip_forward, fwmark rule, table 100)."""
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Idempotent node-level routing rules and routes.")
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/configure/systemd.py
|
||||
"""Port of configure/systemd — deploy + enable units from configs/systemd."""
|
||||
# actions/config/systemd.py
|
||||
"""Port of config/systemd — deploy + enable units from configs/systemd."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Flag
|
||||
|
||||
File renamed without changes.
@@ -0,0 +1,65 @@
|
||||
"""Install the pinned ufw-docker integration on Docker hosts."""
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import urllib.request
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Install the pinned Docker integration for UFW.")
|
||||
UFW_DOCKER_REVISION = "020a8699f95592561f254d8d4ad1bb40d401dfc7"
|
||||
UFW_DOCKER_SHA256 = "643e56b080567c567b4aa28650196849b2a2da5dd0473fd3e5216b0886035ab0"
|
||||
UFW_DOCKER_URL = (
|
||||
"https://raw.githubusercontent.com/chaifeng/ufw-docker/"
|
||||
+ UFW_DOCKER_REVISION + "/ufw-docker"
|
||||
)
|
||||
UFW_DOCKER_PATH = Path("/usr/local/bin/ufw-docker")
|
||||
def _check_helper_path(path):
|
||||
for parent in reversed(path.parents):
|
||||
info = parent.lstat()
|
||||
if (not stat.S_ISDIR(info.st_mode) or info.st_uid != 0 or
|
||||
info.st_mode & 0o022):
|
||||
raise RuntimeError(f"Unsafe ufw-docker ancestor: {parent}")
|
||||
try:
|
||||
info = path.lstat()
|
||||
except FileNotFoundError:
|
||||
return False
|
||||
if (not stat.S_ISREG(info.st_mode) or info.st_uid != 0 or info.st_gid != 0 or
|
||||
info.st_mode & 0o022 or info.st_nlink != 1):
|
||||
raise RuntimeError(f"Unsafe ufw-docker target: {path}")
|
||||
return True
|
||||
|
||||
|
||||
def _install_helper(ctx):
|
||||
path = UFW_DOCKER_PATH
|
||||
exists = _check_helper_path(path)
|
||||
payload = path.read_bytes() if exists else None
|
||||
if payload is None or hashlib.sha256(payload).hexdigest() != UFW_DOCKER_SHA256:
|
||||
ctx.log.info(f"Downloading pinned ufw-docker helper to {path}...")
|
||||
# The immutable HTTPS URL is authenticated by the digest below.
|
||||
with urllib.request.urlopen(UFW_DOCKER_URL, timeout=30) as response: # noqa: S310
|
||||
payload = response.read(1024 * 1024 + 1)
|
||||
if len(payload) > 1024 * 1024 or hashlib.sha256(payload).hexdigest() != UFW_DOCKER_SHA256:
|
||||
raise RuntimeError("ufw-docker checksum mismatch; refusing installation.")
|
||||
_check_helper_path(path)
|
||||
ctx.host.write_file(str(path), payload.decode("utf-8"), mode="0755",
|
||||
backup=False, sudo=True)
|
||||
elif stat.S_IMODE(path.stat().st_mode) != 0o755:
|
||||
ctx.host.write_file(str(path), payload.decode("utf-8"), mode="0755",
|
||||
backup=False, sudo=True)
|
||||
if (not _check_helper_path(path) or
|
||||
hashlib.sha256(path.read_bytes()).hexdigest() != UFW_DOCKER_SHA256 or
|
||||
stat.S_IMODE(path.stat().st_mode) != 0o755):
|
||||
raise RuntimeError("ufw-docker verification failed; refusing execution.")
|
||||
ctx.log.info("Ensuring ufw-docker routing rules are installed in UFW...")
|
||||
ctx.sys.sudo([str(path), "install"])
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
_check_helper_path(UFW_DOCKER_PATH)
|
||||
if os.path.exists("/usr/bin/pveversion"):
|
||||
raise RuntimeError("Use the Proxmox firewall on a PVE host.")
|
||||
for binary in ("ufw", "docker"):
|
||||
if not ctx.sys.ok(["which", binary]):
|
||||
raise RuntimeError(f"Install {binary} before configuring ufw-docker.")
|
||||
_install_helper(ctx)
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/configure/ufw.py
|
||||
"""Port of configure/ufw — apply docker + host ufw rules from configs/ufw."""
|
||||
# actions/config/ufw.py
|
||||
"""Port of config/ufw — apply docker + host ufw rules from configs/ufw."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Flag
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
# actions/configure.py
|
||||
"""Run every configure/* action in a stable order."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Flag, build_parser
|
||||
from engine import overlay
|
||||
|
||||
METADATA = Meta(summary="Run every configure/* action in stable order.",
|
||||
args=[Flag("--dry-run", "Pass --dry-run to sub-actions that support it.")])
|
||||
ORDER = ["configure/base", "configure/timezone", "configure/ufw", "configure/crowdsec",
|
||||
"configure/cron", "configure/systemd", "configure/routing"]
|
||||
|
||||
def run(ctx, args):
|
||||
commons_actions = os.path.join(ctx.paths.commons, "actions")
|
||||
prop = os.path.join(ctx.paths.proprietaries, "actions")
|
||||
prop = prop if os.path.isdir(prop) else None
|
||||
names = [i.name for i in overlay.discover(commons_actions, prop) if i.name.startswith("configure/")]
|
||||
ordered = [n for n in ORDER if n in names] + sorted(n for n in names if n not in ORDER)
|
||||
rc = 0
|
||||
for name in ordered:
|
||||
r = overlay.resolve(name, commons_actions, prop)
|
||||
mod = overlay.load_module(r.path, "apex_umbrella_" + name.replace("/", "_"))
|
||||
if getattr(mod, "DISABLED", None):
|
||||
ctx.log.warn(f"Skipping disabled {name}"); continue
|
||||
meta = mod.METADATA
|
||||
has_dry = any(getattr(a, "name", None) == "--dry-run" for a in meta.args)
|
||||
argv = ["--dry-run"] if (args.dry_run and has_dry) else []
|
||||
sub_args = build_parser(meta, f"apex {name}").parse_args(argv)
|
||||
ctx.log.info(f"=== {name} ===")
|
||||
try:
|
||||
mod.run(ctx, sub_args)
|
||||
except SystemExit as e:
|
||||
if e.code:
|
||||
ctx.log.error(f"{name} exited {e.code}"); rc = rc or e.code
|
||||
return rc
|
||||
@@ -1,131 +0,0 @@
|
||||
# actions/configure/base.py
|
||||
"""Port of configure/base — base packages + ufw + ufw-docker helper."""
|
||||
import hashlib
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import stat
|
||||
import urllib.request
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Configure base system packages (curl, wget, git, rsyslog, ufw, ufw-docker).")
|
||||
UFW_DOCKER_REVISION = "020a8699f95592561f254d8d4ad1bb40d401dfc7"
|
||||
UFW_DOCKER_SHA256 = "643e56b080567c567b4aa28650196849b2a2da5dd0473fd3e5216b0886035ab0"
|
||||
UFW_DOCKER_URL = (
|
||||
"https://raw.githubusercontent.com/chaifeng/ufw-docker/"
|
||||
+ UFW_DOCKER_REVISION + "/ufw-docker"
|
||||
)
|
||||
UFW_DOCKER_PATH = Path("/usr/local/bin/ufw-docker")
|
||||
BASE_PACKAGES = ("curl", "wget", "git", "rsyslog", "ufw")
|
||||
|
||||
|
||||
def _install_missing(s, packages):
|
||||
missing = []
|
||||
for package in packages:
|
||||
result = s.run(["dpkg-query", "-W", "-f=${Status}", package],
|
||||
check=False, capture=True)
|
||||
if result.returncode or result.stdout.split()[1:] != ["ok", "installed"]:
|
||||
missing.append(package)
|
||||
if not missing:
|
||||
return
|
||||
apt = ["env", "LC_ALL=C", "DEBIAN_FRONTEND=noninteractive", "apt-get",
|
||||
"-o", "DPkg::Lock::Timeout=60", "-o", "Acquire::Retries=2",
|
||||
"-o", "Acquire::http::Timeout=30", "-o", "Acquire::https::Timeout=30"]
|
||||
s.sudo([*apt, "update", "--error-on=any"])
|
||||
command = [*apt, "install", "-y", "--no-install-recommends", "--no-upgrade",
|
||||
"--no-remove", *missing]
|
||||
plan = s.sudo([*command, "--simulate"], capture=True)
|
||||
if any(line.startswith("Remv ") or re.match(r"^Inst \S+ \[", line)
|
||||
for line in plan.stdout.splitlines()):
|
||||
raise RuntimeError("Missing packages require upgrades/removals; reconcile packages first.")
|
||||
s.sudo(command)
|
||||
|
||||
|
||||
def _check_helper_path(path):
|
||||
for parent in reversed(path.parents):
|
||||
info = parent.lstat()
|
||||
if (not stat.S_ISDIR(info.st_mode) or info.st_uid != 0 or
|
||||
info.st_mode & 0o022):
|
||||
raise RuntimeError(f"Unsafe ufw-docker ancestor: {parent}")
|
||||
try:
|
||||
info = path.lstat()
|
||||
except FileNotFoundError:
|
||||
return False
|
||||
if (not stat.S_ISREG(info.st_mode) or info.st_uid != 0 or info.st_gid != 0 or
|
||||
info.st_mode & 0o022 or info.st_nlink != 1):
|
||||
raise RuntimeError(f"Unsafe ufw-docker target: {path}")
|
||||
return True
|
||||
|
||||
|
||||
def _install_helper(ctx):
|
||||
path = UFW_DOCKER_PATH
|
||||
exists = _check_helper_path(path)
|
||||
payload = path.read_bytes() if exists else None
|
||||
if payload is None or hashlib.sha256(payload).hexdigest() != UFW_DOCKER_SHA256:
|
||||
ctx.log.info(f"Downloading pinned ufw-docker helper to {path}...")
|
||||
# The immutable HTTPS URL is authenticated by the digest below.
|
||||
with urllib.request.urlopen(UFW_DOCKER_URL, timeout=30) as response: # noqa: S310
|
||||
payload = response.read(1024 * 1024 + 1)
|
||||
if len(payload) > 1024 * 1024 or hashlib.sha256(payload).hexdigest() != UFW_DOCKER_SHA256:
|
||||
raise RuntimeError("ufw-docker checksum mismatch; refusing installation.")
|
||||
_check_helper_path(path)
|
||||
ctx.host.write_file(str(path), payload.decode("utf-8"), mode="0755",
|
||||
backup=False, sudo=True)
|
||||
elif stat.S_IMODE(path.stat().st_mode) != 0o755:
|
||||
ctx.host.write_file(str(path), payload.decode("utf-8"), mode="0755",
|
||||
backup=False, sudo=True)
|
||||
if (not _check_helper_path(path) or
|
||||
hashlib.sha256(path.read_bytes()).hexdigest() != UFW_DOCKER_SHA256 or
|
||||
stat.S_IMODE(path.stat().st_mode) != 0o755):
|
||||
raise RuntimeError("ufw-docker verification failed; refusing execution.")
|
||||
ctx.log.info("Ensuring ufw-docker routing rules are installed in UFW...")
|
||||
ctx.sys.sudo([str(path), "install"])
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
log, s = ctx.log, ctx.sys
|
||||
log.info("Starting base system initialization...")
|
||||
if not os.path.isfile("/etc/debian_version"):
|
||||
log.error("Requires a Debian-based system (Debian/Ubuntu).")
|
||||
raise SystemExit(1)
|
||||
_check_helper_path(UFW_DOCKER_PATH)
|
||||
_install_missing(s, BASE_PACKAGES)
|
||||
s.ensure_running("rsyslog")
|
||||
|
||||
# Configure APT to discard downloaded archives
|
||||
apt_clean_path = "/etc/apt/apt.conf.d/99-apex"
|
||||
apt_clean_content = 'APT::Keep-Downloaded-Packages "0";\n'
|
||||
if os.path.exists("/etc/apt/apt.conf.d/01clean"):
|
||||
s.sudo(["rm", "-f", "/etc/apt/apt.conf.d/01clean"])
|
||||
current_apt = None
|
||||
if os.path.exists(apt_clean_path):
|
||||
try:
|
||||
with open(apt_clean_path, encoding="utf-8") as f:
|
||||
current_apt = f.read()
|
||||
except OSError:
|
||||
current_apt = None
|
||||
if current_apt != apt_clean_content:
|
||||
log.info("Configuring APT to discard downloaded archives...")
|
||||
s.sudo(["tee", apt_clean_path], input=apt_clean_content)
|
||||
|
||||
# Limit systemd-journald max disk usage
|
||||
journal_d = "/etc/systemd/journald.conf.d"
|
||||
journal_conf = os.path.join(journal_d, "99-apex.conf")
|
||||
journal_content = "[Journal]\nSystemMaxUse=200M\n"
|
||||
if os.path.exists(os.path.join(journal_d, "maxuse.conf")):
|
||||
s.sudo(["rm", "-f", os.path.join(journal_d, "maxuse.conf")])
|
||||
current_journal = None
|
||||
if os.path.exists(journal_conf):
|
||||
try:
|
||||
with open(journal_conf, encoding="utf-8") as f:
|
||||
current_journal = f.read()
|
||||
except OSError:
|
||||
current_journal = None
|
||||
if current_journal != journal_content:
|
||||
log.info("Configuring systemd-journald SystemMaxUse=200M...")
|
||||
s.sudo(["mkdir", "-p", journal_d])
|
||||
s.sudo(["tee", journal_conf], input=journal_content)
|
||||
s.sudo(["systemctl", "restart", "systemd-journald"])
|
||||
|
||||
_install_helper(ctx)
|
||||
log.success("Base system initialization completed.")
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Give adam ownership of the active node's tracked checkout and Git metadata."""
|
||||
import inspect
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
from engine.descriptor import Meta
|
||||
|
||||
METADATA = Meta(summary="Set ownership of tracked files and Git metadata under /srv to adam.")
|
||||
|
||||
|
||||
def _repair(root, tracked, mounts, uid, gid):
|
||||
import os
|
||||
import stat
|
||||
|
||||
directories = {""}
|
||||
for item in tracked:
|
||||
directory = os.path.dirname(item)
|
||||
while directory and directory not in directories:
|
||||
directories.add(directory)
|
||||
directory = os.path.dirname(directory)
|
||||
metadata = os.path.join(root, ".git")
|
||||
|
||||
def fail(error):
|
||||
raise error
|
||||
|
||||
for parent, dirs, files, fd in os.fwalk(root, onerror=fail, follow_symlinks=False):
|
||||
os.fchown(fd, uid, gid)
|
||||
for name in dirs.copy() + files:
|
||||
path = os.path.join(parent, name)
|
||||
relative = os.path.relpath(path, root)
|
||||
if path in mounts or not (relative in tracked or relative in directories or
|
||||
path == metadata or path.startswith(metadata + "/")):
|
||||
if name in dirs:
|
||||
dirs.remove(name)
|
||||
continue
|
||||
info = os.stat(name, dir_fd=fd, follow_symlinks=False)
|
||||
if stat.S_ISDIR(info.st_mode):
|
||||
continue
|
||||
if (stat.S_ISREG(info.st_mode) or stat.S_ISLNK(info.st_mode)) and info.st_nlink == 1:
|
||||
os.chown(name, uid, gid, dir_fd=fd, follow_symlinks=False)
|
||||
|
||||
|
||||
def run(ctx, args):
|
||||
root = Path(ctx.paths.repo_root)
|
||||
if (root.parent != Path("/srv") or root != root.resolve(strict=True) or
|
||||
not (root / "node.env").is_file() or (root / ".git").is_symlink() or
|
||||
not (root / ".git").is_dir()):
|
||||
raise RuntimeError("Expected the current node checkout directly under /srv.")
|
||||
git = ["git", "-c", f"safe.directory={root}", "-c", f"safe.directory={root}/commons",
|
||||
"-C", str(root)]
|
||||
entries = ctx.sys.run([*git, "ls-files", "--cached", "--recurse-submodules", "-z"],
|
||||
capture=True).stdout
|
||||
gitfiles = ctx.sys.run([*git, "submodule", "foreach", "--quiet", "--recursive",
|
||||
'printf "%s\\0" "$displaypath/.git"'], capture=True).stdout
|
||||
mounts = [re.sub(r"\\([0-7]{3})", lambda match: chr(int(match[1], 8)), line.split()[4])
|
||||
for line in Path("/proc/self/mountinfo").read_text().splitlines()]
|
||||
script = (inspect.getsource(_repair) + "\nimport json, pwd, sys\n"
|
||||
"root, entries, mounts = json.load(sys.stdin)\n"
|
||||
"adam = pwd.getpwnam('adam')\n"
|
||||
"_repair(root, set(entries), set(mounts), adam.pw_uid, adam.pw_gid)")
|
||||
ctx.sys.sudo(["python3", "-I", "-c", script],
|
||||
input=json.dumps([str(root), (entries + gitfiles).split("\0"), mounts]))
|
||||
File renamed without changes.
@@ -1,4 +1,4 @@
|
||||
# actions/backup/run.py
|
||||
# actions/tier/backup.py
|
||||
"""Label-driven restic backup via resticontainer (git.ermnvldmr.com/deytenit/resticontainer).
|
||||
Runs the resticontainer core service one-shot: init-if-needed -> backup (discovers
|
||||
`restic.*`-labelled services, runs their pre-hooks/stops, snapshots the union of the
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/tiers/chown.py
|
||||
"""Port of tiers/chown — ownership: {tier}/{proj} -> uid:gid; {tier}/shared/{proj} ->
|
||||
# actions/tier/chown.py
|
||||
"""Port of tier/chown — ownership: {tier}/{proj} -> uid:gid; {tier}/shared/{proj} ->
|
||||
uid:shared_gid (+sticky); base/node-infra -> root:root. Reads APEX_UID/GID from each .env."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Arg
|
||||
@@ -31,7 +31,7 @@ def run(ctx, args):
|
||||
|
||||
cp = s.run(["getent", "group", SHARED_GROUP], check=False, capture=True)
|
||||
if cp.returncode != 0:
|
||||
log.error(f"Shared group '{SHARED_GROUP}' not found. Run tiers/useradd first."); raise SystemExit(1)
|
||||
log.error(f"Shared group '{SHARED_GROUP}' not found. Run tier/useradd first."); raise SystemExit(1)
|
||||
|
||||
log.info("Ensuring base structures owned by root...")
|
||||
for tp in tiers:
|
||||
@@ -62,7 +62,7 @@ def run(ctx, args):
|
||||
gid = _env_val(env_path, "APEX_GID")
|
||||
sgid = _env_val(env_path, "APEX_SHARED_GID")
|
||||
if not (uid and gid and sgid):
|
||||
log.warn(f"Incomplete APEX_UID/GID/SHARED_GID for {proj}. Run tiers/useradd first. Skipping.")
|
||||
log.warn(f"Incomplete APEX_UID/GID/SHARED_GID for {proj}. Run tier/useradd first. Skipping.")
|
||||
continue
|
||||
log.info(f"Applying permissions for {proj} (UID={uid} GID={gid} SHARED_GID={sgid})")
|
||||
for tp in tiers:
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/tiers/link.py
|
||||
"""Port of tiers/link — @tier symlink structure for the node + each composition project."""
|
||||
# actions/tier/link.py
|
||||
"""Port of tier/link — @tier symlink structure for the node + each composition project."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Arg
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/tiers/useradd.py
|
||||
"""Port of tiers/useradd — one noroot system user per composition project; inject the
|
||||
# actions/tier/useradd.py
|
||||
"""Port of tier/useradd — one noroot system user per composition project; inject the
|
||||
APEX_UID/APEX_GID/APEX_SHARED_GID block into each project .env."""
|
||||
import os, re
|
||||
from engine.descriptor import Meta
|
||||
@@ -36,7 +36,7 @@ def run(ctx, args):
|
||||
continue
|
||||
# opt-out: a project marked .apex-root-owned runs as root and keeps its tier
|
||||
# data root-owned (e.g. seafile-mc, whose entrypoint refuses non-root-owned
|
||||
# data). Skip the noroot user + .env UID injection; tiers/chown forces root.
|
||||
# data). Skip the noroot user + .env UID injection; tier/chown forces root.
|
||||
if os.path.isfile(os.path.join(full, ".apex-root-owned")):
|
||||
log.info(f"{proj}: .apex-root-owned — runs as root; skipping noroot user")
|
||||
continue
|
||||
@@ -1,4 +1,4 @@
|
||||
# actions/sync/packages.py
|
||||
# actions/util/check-updates.py
|
||||
"""Check pending APT and running-image updates without pulling images."""
|
||||
import json
|
||||
import os
|
||||
@@ -1,5 +1,5 @@
|
||||
# actions/utils/extract-traefik-certs.py
|
||||
"""Port of utils/extract-traefik-certs — dump traefik v3 acme.json to pem via docker."""
|
||||
# actions/util/extract-traefik-certs.py
|
||||
"""Port of util/extract-traefik-certs — dump traefik v3 acme.json to pem via docker."""
|
||||
import os
|
||||
from engine.descriptor import Meta, Arg
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# actions/utils/lint-docker-compose.py
|
||||
# actions/util/lint-docker-compose.py
|
||||
"""Run dclint through Docker and validate anchor IPs:
|
||||
every ipv4_address in compositions/*/docker-compose.yml must be unique and inside the
|
||||
node's /24."""
|
||||
@@ -32,7 +32,7 @@ def run(ctx, args):
|
||||
if k in os.environ:
|
||||
env_args.extend(["-e", f"{k}={os.environ[k]}"])
|
||||
# "-e commons": never --fix inside the pinned submodule — a dirtied commons makes
|
||||
# sync/repository abort the nightly capture-up.
|
||||
# repository/sync abort the nightly capture-up.
|
||||
r = s.run(["docker", "run", "--rm", "--network", "host", *env_args, "-v", f"{repo}:/workdir", "-w", "/workdir", "node:lts-alpine",
|
||||
"npx", "--yes", "dclint", "-e", "@tier1", "-e", "@tier2", "-e", "@tier3", "-e", ".env",
|
||||
"-e", "commons", "--disable-rule", "no-build-and-image", "-r", "--fix", *targets], check=False)
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Service configuration and credentials are deployed later by apex configure/alloy.
|
||||
# Service configuration and credentials are deployed later by apex config/alloy.
|
||||
alloy_version=1.21.3
|
||||
alloy_sha256=dbbd705de8ab37dec0cbd6361a3fc7e754d6b2854b57ce672a83e6366199ba73
|
||||
[[ "$(dpkg --print-architecture)" == amd64 ]]
|
||||
|
||||
@@ -91,12 +91,13 @@ def preflight(*, strict=True):
|
||||
|
||||
|
||||
def package_versions():
|
||||
architecture = run(["dpkg", "--print-architecture"]).stdout.strip()
|
||||
result = run(["dpkg-query", "-W", "-f=${binary:Package}\t${Status}\t${Version}\n"])
|
||||
installed = {}
|
||||
for line in result.stdout.splitlines():
|
||||
fields = line.split("\t")
|
||||
if len(fields) == 3 and fields[1].split()[1:] == ["ok", "installed"]:
|
||||
installed[fields[0].removesuffix(":amd64")] = fields[2]
|
||||
installed[fields[0].removesuffix(":" + architecture)] = fields[2]
|
||||
return installed
|
||||
|
||||
|
||||
@@ -311,6 +312,88 @@ def docker_preflight(installed):
|
||||
return {**config, **DOCKER_CONFIG}
|
||||
|
||||
|
||||
DOCKER_KEY_SHA256 = "1500c1f56fa9e26b9b8f42452a553675796ade0807cdce11975eb98170b3a570"
|
||||
LOCAL_DOCKER = ["docker", "--host", "unix:///var/run/docker.sock"]
|
||||
|
||||
|
||||
def _docker_source(arch):
|
||||
key = Path("/etc/apt/keyrings/apex-docker.asc")
|
||||
source = Path("/etc/apt/sources.list.d/apex-docker.sources")
|
||||
content = ("Types: deb\nURIs: https://download.docker.com/linux/debian\nSuites: trixie\n"
|
||||
f"Components: stable\nArchitectures: {arch}\nSigned-By: {key}\n")
|
||||
for path in (key, source):
|
||||
if any(part.is_symlink() for part in (path, *path.parents)):
|
||||
raise BaselineError(f"Refusing redirected APT path: {path}")
|
||||
if source.exists() and source.read_text() != content:
|
||||
raise BaselineError(f"Reconcile existing Docker repository configuration: {source}")
|
||||
if key.exists():
|
||||
data = key.read_bytes()
|
||||
else:
|
||||
url = "https://download.docker.com/linux/debian/gpg"
|
||||
with urllib.request.urlopen(url, timeout=30) as response: # noqa: S310
|
||||
data = response.read(1024 * 1024)
|
||||
if hashlib.sha256(data).hexdigest() != DOCKER_KEY_SHA256:
|
||||
raise BaselineError("Docker repository signing key differs from the pinned release key.")
|
||||
write_managed(key, data)
|
||||
write_managed(source, content)
|
||||
|
||||
|
||||
def _docker_capabilities(root, *, complete_plugins=False):
|
||||
capabilities = {"docker_data_root": root}
|
||||
plugins = {"docker-compose-plugin": ("compose", ["compose", "version", "--short"]),
|
||||
"docker-buildx-plugin": ("buildx", ["buildx", "version"])}
|
||||
missing = []
|
||||
for package, (name, args) in plugins.items():
|
||||
result = run([*LOCAL_DOCKER, *args], check=False)
|
||||
if result.returncode or not result.stdout.strip():
|
||||
missing.append(package)
|
||||
else:
|
||||
capabilities[name] = result.stdout.strip()
|
||||
if missing:
|
||||
if complete_plugins and "docker-ce" in package_versions():
|
||||
with service_start_guard():
|
||||
install_missing(missing)
|
||||
return _docker_capabilities(root)
|
||||
raise BaselineError("Existing Docker needs functional plugins: " + ", ".join(missing))
|
||||
if not root:
|
||||
raise BaselineError("Unable to verify the local Docker data-root.")
|
||||
return capabilities
|
||||
|
||||
|
||||
@expected_errors
|
||||
def apply_docker_policy():
|
||||
if Path("/usr/bin/pveversion").exists():
|
||||
raise BaselineError("Docker installation on a Proxmox host requires separate administration.")
|
||||
release = dict(line.split("=", 1) for line in Path("/etc/os-release").read_text().splitlines()
|
||||
if "=" in line and not line.startswith("#"))
|
||||
arch = run(["dpkg", "--print-architecture"]).stdout.strip()
|
||||
if (release.get("ID", "").strip('"') != "debian" or
|
||||
release.get("VERSION_ID", "").strip('"') != "13" or arch not in ("amd64", "arm64")):
|
||||
raise BaselineError("Docker installation supports stock Debian 13 amd64 or arm64.")
|
||||
info = [*LOCAL_DOCKER, "info", "--format", "{{.DockerRootDir}}"]
|
||||
if shutil.which("docker"):
|
||||
current = run(info, check=False)
|
||||
if current.returncode == 0:
|
||||
return _docker_capabilities(current.stdout.strip(), complete_plugins=True)
|
||||
installed = package_versions()
|
||||
source = Path("/etc/apt/sources.list.d/apex-docker.sources")
|
||||
if (not source.exists() and
|
||||
(shutil.which("docker") or set(DOCKER_PACKAGES).intersection(installed))):
|
||||
raise BaselineError("Existing Docker installation is inactive; reconcile it before retrying.")
|
||||
config = docker_preflight(installed)
|
||||
with service_start_guard():
|
||||
install_missing(("ca-certificates",))
|
||||
_docker_source(arch)
|
||||
write_managed("/etc/docker/daemon.json", json.dumps(config, indent=2) + "\n")
|
||||
install_missing(DOCKER_PACKAGES)
|
||||
ensure_service("containerd")
|
||||
ensure_service("docker")
|
||||
root = run(info).stdout.strip()
|
||||
if root != DOCKER_CONFIG["data-root"]:
|
||||
raise BaselineError("Docker started with an unexpected data-root; inspect its configuration.")
|
||||
return _docker_capabilities(root)
|
||||
|
||||
|
||||
def configure_sources():
|
||||
write_managed("/etc/apt/sources.list.d/apex-zfs.sources",
|
||||
"Types: deb\nURIs: https://deb.debian.org/debian\nSuites: trixie\n"
|
||||
@@ -418,7 +501,7 @@ def repair_bouncer_credentials():
|
||||
|
||||
@expected_errors
|
||||
def apply_base_policy():
|
||||
"""The bounded configure/base policy: no Docker, storage or enrollment."""
|
||||
"""Apply the base package, logging, and firewall policy."""
|
||||
preflight(strict=False)
|
||||
with service_start_guard(), security_service_guard(BASE_PACKAGES):
|
||||
install_missing(BASE_PACKAGES)
|
||||
@@ -478,11 +561,19 @@ if __name__ == "__main__":
|
||||
import argparse
|
||||
from .state import ProvisionError, exclusive_lock
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--base-policy", action="store_true")
|
||||
scope = parser.add_mutually_exclusive_group()
|
||||
scope.add_argument("--base-policy", action="store_true")
|
||||
scope.add_argument("--docker-policy", action="store_true")
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
preflight(strict=not args.base_policy)
|
||||
preflight(strict=not (args.base_policy or args.docker_policy))
|
||||
with exclusive_lock("/var/lib/apex-init/init.lock"):
|
||||
print(json.dumps(apply_base_policy() if args.base_policy else apply_baseline(), sort_keys=True))
|
||||
if args.docker_policy:
|
||||
result = apply_docker_policy()
|
||||
elif args.base_policy:
|
||||
result = apply_base_policy()
|
||||
else:
|
||||
result = apply_baseline()
|
||||
print(json.dumps(result, sort_keys=True))
|
||||
except (BaselineError, OSError, ProvisionError) as exc:
|
||||
parser.exit(1, f"Baseline failed: {exc}\n")
|
||||
@@ -0,0 +1,120 @@
|
||||
from contextlib import nullcontext
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
from pathlib import Path
|
||||
from subprocess import CompletedProcess
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from provisioning import baseline
|
||||
|
||||
|
||||
class ManualDockerTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
self.root = Path(tmp.name).resolve()
|
||||
(self.root / "etc").mkdir()
|
||||
(self.root / "etc/os-release").write_text("ID=debian\nVERSION_ID=13\n")
|
||||
self.commands = []
|
||||
self.arch = "arm64"
|
||||
self.running = False
|
||||
self.data_root = "/custom/data"
|
||||
self.missing_buildx = False
|
||||
self.versions = {}
|
||||
self.installations = []
|
||||
|
||||
def run_command(self, cmd, **kwargs):
|
||||
self.commands.append(cmd)
|
||||
if cmd == ["dpkg", "--print-architecture"]:
|
||||
return CompletedProcess(cmd, 0, self.arch + "\n", "")
|
||||
if cmd[-1] == "{{.DockerRootDir}}":
|
||||
return CompletedProcess(cmd, 0 if self.running else 1, self.data_root + "\n", "")
|
||||
if cmd[-2:] in (["version", "--short"], ["buildx", "version"]):
|
||||
if self.missing_buildx and cmd[-2:] == ["buildx", "version"]:
|
||||
if ["docker-buildx-plugin"] not in self.installations:
|
||||
return CompletedProcess(cmd, 1, "", "plugin missing")
|
||||
return CompletedProcess(cmd, 0, "installed\n", "")
|
||||
raise AssertionError(cmd)
|
||||
|
||||
def apply(self):
|
||||
def target(path):
|
||||
path = Path(path)
|
||||
return path if self.root in path.parents else self.root / str(path).lstrip("/")
|
||||
|
||||
with (patch.object(baseline, "Path", side_effect=target),
|
||||
patch.object(baseline, "run", side_effect=self.run_command),
|
||||
patch.object(baseline, "package_versions", return_value=self.versions),
|
||||
patch.object(baseline.shutil, "which", return_value="/usr/bin/docker" if self.running else None),
|
||||
patch.object(baseline, "install_missing", side_effect=self.installations.append)):
|
||||
return baseline.apply_docker_policy()
|
||||
|
||||
def test_working_engine_keeps_local_data_root_without_writes(self):
|
||||
self.running = True
|
||||
self.versions = {"docker.io": "1"}
|
||||
self.assertEqual(self.apply()["docker_data_root"], "/custom/data")
|
||||
self.assertEqual(self.installations, [])
|
||||
self.assertFalse((self.root / "etc/docker").exists())
|
||||
|
||||
def test_pve_refused_before_packages_or_daemon(self):
|
||||
binary = self.root / "usr/bin/pveversion"
|
||||
binary.parent.mkdir(parents=True)
|
||||
binary.touch()
|
||||
with self.assertRaisesRegex(baseline.BaselineError, "Proxmox"):
|
||||
self.apply()
|
||||
self.assertEqual(self.commands, [])
|
||||
|
||||
def test_unsupported_architecture_refused(self):
|
||||
self.arch = "i386"
|
||||
with self.assertRaisesRegex(baseline.BaselineError, "amd64|arm64"):
|
||||
self.apply()
|
||||
self.assertEqual(self.installations, [])
|
||||
|
||||
def test_existing_runtime_packages_not_replaced(self):
|
||||
self.versions = {"containerd": "1.7"}
|
||||
with self.assertRaisesRegex(baseline.BaselineError, "containerd"):
|
||||
self.apply()
|
||||
self.assertEqual(self.installations, [])
|
||||
|
||||
def test_fresh_arm64_install_verifies_the_pinned_key_and_starts_local_engine(self):
|
||||
key = b"test repository signing key"
|
||||
self.data_root = "/srv/docker"
|
||||
|
||||
def start(service):
|
||||
if service == "docker":
|
||||
self.running = True
|
||||
|
||||
with (patch.object(baseline, "service_start_guard", return_value=nullcontext()),
|
||||
patch.object(baseline.urllib.request, "urlopen", return_value=io.BytesIO(key)),
|
||||
patch.object(baseline, "DOCKER_KEY_SHA256", hashlib.sha256(key).hexdigest()),
|
||||
patch.object(baseline, "ensure_service", side_effect=start)):
|
||||
self.assertEqual(self.apply()["docker_data_root"], "/srv/docker")
|
||||
source = (self.root / "etc/apt/sources.list.d/apex-docker.sources").read_text()
|
||||
self.assertIn("Architectures: arm64", source)
|
||||
self.assertIn("Signed-By: ", source)
|
||||
self.assertEqual((self.root / "etc/apt/keyrings/apex-docker.asc").read_bytes(), key)
|
||||
self.assertEqual(json.loads((self.root / "etc/docker/daemon.json").read_text())["data-root"],
|
||||
"/srv/docker")
|
||||
self.assertIn("docker-compose-plugin", self.installations[-1])
|
||||
self.assertIn("docker-buildx-plugin", self.installations[-1])
|
||||
self.assertNotIn("ufw", self.installations[-1])
|
||||
|
||||
def test_untrusted_signing_key_cannot_publish_apt_source_or_install_docker(self):
|
||||
with (patch.object(baseline, "service_start_guard", return_value=nullcontext()),
|
||||
patch.object(baseline.urllib.request, "urlopen", return_value=io.BytesIO(b"wrong key"))):
|
||||
with self.assertRaisesRegex(baseline.BaselineError, "signing key"):
|
||||
self.apply()
|
||||
self.assertFalse((self.root / "etc/apt/sources.list.d/apex-docker.sources").exists())
|
||||
self.assertFalse((self.root / "etc/docker/daemon.json").exists())
|
||||
self.assertEqual(self.installations, [("ca-certificates",)])
|
||||
|
||||
def test_working_ce_engine_only_installs_the_missing_plugin(self):
|
||||
self.running = True
|
||||
self.missing_buildx = True
|
||||
self.versions = {"docker-ce": "1"}
|
||||
with patch.object(baseline, "service_start_guard", return_value=nullcontext()):
|
||||
self.assertEqual(self.apply()["docker_data_root"], "/custom/data")
|
||||
self.assertEqual(self.installations, [["docker-buildx-plugin"]])
|
||||
self.assertFalse((self.root / "etc/docker").exists())
|
||||
+3
-1
@@ -28,7 +28,7 @@ def _resolve_layout():
|
||||
|
||||
|
||||
def _print_list(commons_actions, prop_actions):
|
||||
print("Usage: apex <group>/<action> [args...]\n\nAvailable actions:")
|
||||
print("Usage: apex <entity>/<action> [args...]\n\nAvailable actions:")
|
||||
tag = {"commons": "", "local": " (local)", "override": " (local override)"}
|
||||
for i in overlay.discover(commons_actions, prop_actions):
|
||||
mark = f" [DISABLED: {i.disabled}]" if i.disabled else ""
|
||||
@@ -44,6 +44,8 @@ def main(argv) -> int:
|
||||
return 0
|
||||
|
||||
name, rest = argv[0], argv[1:]
|
||||
if name == "compose":
|
||||
name = "composition/run"
|
||||
r = overlay.resolve(name, commons_actions, prop_actions)
|
||||
if r is None:
|
||||
sys.stderr.write(f"[global] [core] [ERROR] Unknown action: {name}\n")
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# engine/lib/crowdsec.py
|
||||
"""ctx.crowdsec — install + config deploy + collections (port of configure/crowdsec)."""
|
||||
"""ctx.crowdsec — install + config deploy + collections (port of config/crowdsec)."""
|
||||
from __future__ import annotations
|
||||
import os
|
||||
import shutil
|
||||
|
||||
+3
-3
@@ -125,13 +125,13 @@ class Host:
|
||||
# --- read/write with sudo awareness ---
|
||||
def _read(self, path: str, sudo: bool = False) -> str:
|
||||
if sudo:
|
||||
cp = self.sys.run(["sudo", "cat", path], check=True, capture=True)
|
||||
cp = self.sys.sudo(["cat", path], check=True, capture=True)
|
||||
return cp.stdout or ""
|
||||
with open(path, encoding="utf-8") as f:
|
||||
return f.read()
|
||||
|
||||
def backup(self, path: str, sudo: bool = False) -> Optional[str]:
|
||||
exists = (self.sys.ok(["sudo", "test", "-f", path]) if sudo
|
||||
exists = (self.sys.sudo(["test", "-f", path], check=False).returncode == 0 if sudo
|
||||
else os.path.isfile(path))
|
||||
if not exists:
|
||||
return None
|
||||
@@ -154,7 +154,7 @@ class Host:
|
||||
anchor: Optional[str] = None,
|
||||
old_pairs: Sequence[Tuple[str, str]] = (),
|
||||
dry_run: bool = False, sudo: bool = True) -> bool:
|
||||
exists = (self.sys.ok(["sudo", "test", "-f", target]) if sudo
|
||||
exists = (self.sys.sudo(["test", "-f", target], check=False).returncode == 0 if sudo
|
||||
else os.path.isfile(target))
|
||||
if not exists:
|
||||
self.log.error(f"Target file does not exist: {target}")
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# engine/lib/sys.py
|
||||
"""ctx.sys — command execution + systemd service management (port of root_sys.sh)."""
|
||||
from __future__ import annotations
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
@@ -21,6 +22,8 @@ class Sys:
|
||||
stderr=subprocess.PIPE if capture else None)
|
||||
|
||||
def sudo(self, cmd: Sequence[str], **kw) -> subprocess.CompletedProcess:
|
||||
if os.geteuid() == 0 and shutil.which("sudo") is None:
|
||||
return self.run(cmd, **kw)
|
||||
return self.run(["sudo", *cmd], **kw)
|
||||
|
||||
def ok(self, cmd: Sequence[str], **kw) -> bool:
|
||||
|
||||
+2
-2
@@ -67,10 +67,10 @@ class Ufw:
|
||||
self.log.warn("Neither docker nor host ufw rules directory found; nothing to apply.")
|
||||
return
|
||||
if not self.sys.ok(["sudo", "ufw", "--version"]):
|
||||
self.log.error("ufw not installed/working via sudo. Run configure/base first.")
|
||||
self.log.error("ufw not installed/working via sudo. Run config/packages first.")
|
||||
raise SystemExit(1)
|
||||
if not (os.path.isfile(UFW_DOCKER_BIN) and os.access(UFW_DOCKER_BIN, os.X_OK)):
|
||||
self.log.error(f"ufw-docker not found at {UFW_DOCKER_BIN}. Run configure/base first.")
|
||||
self.log.error(f"ufw-docker not found at {UFW_DOCKER_BIN}. Run config/ufw-docker first.")
|
||||
raise SystemExit(1)
|
||||
|
||||
if config_dir:
|
||||
|
||||
+7
-1
@@ -9,10 +9,12 @@ from __future__ import annotations
|
||||
import ast
|
||||
import importlib.util
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import List, Optional
|
||||
|
||||
ACTION_EXT = ".py"
|
||||
ACTION_NAME = re.compile(r"[A-Za-z0-9_-]+/[A-Za-z0-9_-]+")
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -40,11 +42,15 @@ def _rel_names(root: Optional[str]):
|
||||
continue
|
||||
full = os.path.join(dirpath, fn)
|
||||
rel = os.path.relpath(full, root)[: -len(ACTION_EXT)]
|
||||
out[rel.replace(os.sep, "/")] = full
|
||||
name = rel.replace(os.sep, "/")
|
||||
if ACTION_NAME.fullmatch(name):
|
||||
out[name] = full
|
||||
return out
|
||||
|
||||
|
||||
def resolve(name: str, commons_root: str, prop_root: Optional[str]) -> Optional[Resolved]:
|
||||
if not ACTION_NAME.fullmatch(name):
|
||||
return None
|
||||
rel = name + ACTION_EXT
|
||||
commons_path = os.path.join(commons_root, rel)
|
||||
c = os.path.isfile(commons_path)
|
||||
|
||||
+1
-1
@@ -6,4 +6,4 @@ else
|
||||
APEX_CMD="$(git rev-parse --show-toplevel)/commons/apex"
|
||||
[[ -x "$APEX_CMD" ]] || APEX_CMD="$(git rev-parse --show-toplevel)/apex" # standalone commons repo
|
||||
fi
|
||||
"$APEX_CMD" utils/lint-docker-compose --hook || { echo "[pre-commit] Aborting: apex utils/lint-docker-compose failed."; exit 1; }
|
||||
"$APEX_CMD" util/lint-docker-compose --hook || { echo "[pre-commit] Aborting: apex util/lint-docker-compose failed."; exit 1; }
|
||||
@@ -7,7 +7,7 @@ from unittest.mock import Mock
|
||||
|
||||
from engine import overlay
|
||||
|
||||
backup = overlay.load_module(str(Path(__file__).resolve().parents[1] / 'actions/backup/run.py'),
|
||||
backup = overlay.load_module(str(Path(__file__).resolve().parents[1] / 'actions/tier/backup.py'),
|
||||
'test_backup_action')
|
||||
|
||||
|
||||
|
||||
@@ -13,6 +13,22 @@ class TestCli(unittest.TestCase):
|
||||
self.assertEqual(r.returncode, 0)
|
||||
self.assertIn("Available actions", r.stdout)
|
||||
|
||||
def test_compose_alias_uses_canonical_action(self):
|
||||
canonical = run("composition/run", "--help")
|
||||
self.assertEqual(canonical.returncode, 0)
|
||||
alias = run("compose", "--help")
|
||||
self.assertEqual(alias.returncode, 0)
|
||||
self.assertEqual(alias.stdout, canonical.stdout)
|
||||
|
||||
def test_configuration_must_be_selected_explicitly(self):
|
||||
listing = run("--help")
|
||||
self.assertIn("config/hardening", listing.stdout)
|
||||
self.assertIn("repository/chown", listing.stdout)
|
||||
for command in ("configure", "config"):
|
||||
result = run(command)
|
||||
self.assertEqual(result.returncode, 1)
|
||||
self.assertIn("Unknown action", result.stderr)
|
||||
|
||||
def test_unknown_action_exits_one(self):
|
||||
r = run("no/such")
|
||||
self.assertEqual(r.returncode, 1)
|
||||
|
||||
@@ -6,7 +6,7 @@ import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from actions import compose
|
||||
from actions.composition import run as compose
|
||||
from engine.descriptor import build_parser
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ class TestCompose(unittest.TestCase):
|
||||
self.ctx.vars.return_value = {'APEX_NODE_HOST': 'test-host', 'OTHER': 'ignored'}
|
||||
|
||||
def run_compose(self, *argv):
|
||||
args = build_parser(compose.METADATA, 'apex compose').parse_args(argv)
|
||||
args = build_parser(compose.METADATA, 'apex composition/run').parse_args(argv)
|
||||
compose.run(self.ctx, args)
|
||||
|
||||
def visited(self):
|
||||
|
||||
@@ -7,7 +7,7 @@ import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
from engine.lib.sys import Sys
|
||||
|
||||
spec = importlib.util.spec_from_file_location('compose_lint', Path(__file__).resolve().parents[1] / 'actions/utils/lint-docker-compose.py')
|
||||
spec = importlib.util.spec_from_file_location('compose_lint', Path(__file__).resolve().parents[1] / 'actions/util/lint-docker-compose.py')
|
||||
lint = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(lint)
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ from unittest.mock import Mock
|
||||
from engine import overlay
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
alloy = overlay.load_module(str(ROOT / 'actions/configure/alloy.py'), 'test_alloy_action')
|
||||
alloy = overlay.load_module(str(ROOT / 'actions/config/alloy.py'), 'test_alloy_action')
|
||||
|
||||
|
||||
class NativeAlloy(unittest.TestCase):
|
||||
|
||||
@@ -16,16 +16,22 @@ from engine import overlay
|
||||
from engine.lib.host import Host
|
||||
from engine.lib.sys import Sys
|
||||
|
||||
base = overlay.load_module(
|
||||
str(Path(__file__).resolve().parents[1] / "actions/configure/base.py"),
|
||||
packages = overlay.load_module(
|
||||
str(Path(__file__).resolve().parents[1] / "actions/config/packages.py"),
|
||||
"apex_test_configure_base",
|
||||
)
|
||||
|
||||
|
||||
helper = overlay.load_module(
|
||||
str(Path(__file__).resolve().parents[1] / "actions/config/ufw-docker.py"),
|
||||
"apex_test_ufw_docker",
|
||||
)
|
||||
|
||||
|
||||
class TestBasePackages(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.system = Mock()
|
||||
self.installed = set(base.BASE_PACKAGES)
|
||||
self.installed = set(packages.BASE_PACKAGES)
|
||||
self.system.run.side_effect = self.query
|
||||
self.system.sudo.return_value = subprocess.CompletedProcess([], 0, stdout="")
|
||||
|
||||
@@ -35,7 +41,7 @@ class TestBasePackages(unittest.TestCase):
|
||||
return subprocess.CompletedProcess(command, 0, stdout=status)
|
||||
|
||||
def test_installed_rsyslog_and_ufw_do_not_require_executable_or_apt_update(self):
|
||||
base._install_missing(self.system, base.BASE_PACKAGES)
|
||||
packages._install_missing(self.system, packages.BASE_PACKAGES)
|
||||
self.assertEqual({call.args[0][-1] for call in self.system.run.call_args_list},
|
||||
{"curl", "wget", "git", "rsyslog", "ufw"})
|
||||
self.system.sudo.assert_not_called()
|
||||
@@ -45,7 +51,7 @@ class TestBasePackages(unittest.TestCase):
|
||||
self.system.sudo.return_value = subprocess.CompletedProcess(
|
||||
[], 0, stdout="Inst ufw (0.36.2 Debian:13 [all])\nConf ufw (0.36.2 Debian:13 [all])\n",
|
||||
)
|
||||
base._install_missing(self.system, base.BASE_PACKAGES)
|
||||
packages._install_missing(self.system, packages.BASE_PACKAGES)
|
||||
commands = [call.args[0] for call in self.system.sudo.call_args_list]
|
||||
self.assertEqual(len(commands), 3)
|
||||
self.assertIn("update", commands[0])
|
||||
@@ -79,7 +85,7 @@ class TestBasePackages(unittest.TestCase):
|
||||
return subprocess.CompletedProcess(command, 0, stdout=plan)
|
||||
|
||||
with patch("engine.lib.sys.subprocess.run", side_effect=filtered_sudo):
|
||||
base._install_missing(Sys(Mock()), ["ufw"])
|
||||
packages._install_missing(Sys(Mock()), ["ufw"])
|
||||
self.assertEqual(len(operations), 3)
|
||||
self.assertIn("update", operations[0])
|
||||
self.assertIn("--simulate", operations[1])
|
||||
@@ -94,13 +100,13 @@ class TestBasePackages(unittest.TestCase):
|
||||
self.system.sudo.reset_mock()
|
||||
self.system.sudo.return_value = subprocess.CompletedProcess([], 0, stdout=plan)
|
||||
with self.assertRaisesRegex(RuntimeError, "upgrades/removals"):
|
||||
base._install_missing(self.system, base.BASE_PACKAGES)
|
||||
packages._install_missing(self.system, packages.BASE_PACKAGES)
|
||||
self.assertEqual(self.system.sudo.call_count, 2)
|
||||
|
||||
def test_failed_package_query_is_not_treated_as_installed(self):
|
||||
self.system.run.return_value = subprocess.CompletedProcess([], 1, stdout="")
|
||||
self.system.run.side_effect = None
|
||||
base._install_missing(self.system, ["rsyslog"])
|
||||
packages._install_missing(self.system, ["rsyslog"])
|
||||
self.assertEqual(self.system.sudo.call_args.args[0][-1], "rsyslog")
|
||||
|
||||
|
||||
@@ -131,9 +137,9 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
# Model root-owned ancestry without sudo/chown; content, modes and links stay real.
|
||||
self.addCleanup(patch.stopall)
|
||||
patch.object(Path, "lstat", autospec=True, side_effect=root_lstat).start()
|
||||
patch.object(base, "UFW_DOCKER_PATH", self.path).start()
|
||||
patch.object(base, "UFW_DOCKER_SHA256", hashlib.sha256(self.payload).hexdigest()).start()
|
||||
self.download = patch.object(base.urllib.request, "urlopen").start()
|
||||
patch.object(helper, "UFW_DOCKER_PATH", self.path).start()
|
||||
patch.object(helper, "UFW_DOCKER_SHA256", hashlib.sha256(self.payload).hexdigest()).start()
|
||||
self.download = patch.object(helper.urllib.request, "urlopen").start()
|
||||
self.download.return_value = io.BytesIO(self.payload)
|
||||
|
||||
def sudo(self, command, **kwargs):
|
||||
@@ -150,27 +156,27 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
|
||||
def test_verified_existing_helper_executes_without_download(self):
|
||||
self.write_helper()
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.download.assert_not_called()
|
||||
self.assertEqual(self.system.sudo.call_args.args[0], [str(self.path), "install"])
|
||||
self.assertEqual(self.system.sudo.call_count, 1)
|
||||
|
||||
def test_wrong_executable_content_is_replaced_only_with_verified_download(self):
|
||||
self.write_helper(b"#!/bin/sh\necho wrong\n")
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.assertEqual(self.path.read_bytes(), self.payload)
|
||||
self.assertEqual(self.system.sudo.call_count, 2)
|
||||
self.download.assert_called_once_with(base.UFW_DOCKER_URL, timeout=30)
|
||||
self.download.assert_called_once_with(helper.UFW_DOCKER_URL, timeout=30)
|
||||
|
||||
def test_missing_helper_is_published_with_executable_mode(self):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.assertEqual(self.path.read_bytes(), self.payload)
|
||||
self.assertEqual(stat.S_IMODE(self.path.stat().st_mode), 0o755)
|
||||
self.assertEqual(list(self.path.parent.iterdir()), [self.path])
|
||||
|
||||
def test_verified_nonexecutable_content_is_reused_and_mode_repaired(self):
|
||||
self.write_helper(mode=0o600)
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.download.assert_not_called()
|
||||
self.assertEqual(stat.S_IMODE(self.path.stat().st_mode), 0o755)
|
||||
self.assertEqual(self.system.sudo.call_count, 2)
|
||||
@@ -178,7 +184,7 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
def test_bad_checksum_never_publishes_or_executes(self):
|
||||
self.download.return_value = io.BytesIO(b"not the pinned helper")
|
||||
with self.assertRaisesRegex(RuntimeError, "checksum mismatch"):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.system.sudo.assert_not_called()
|
||||
self.assertFalse(self.path.exists())
|
||||
|
||||
@@ -186,13 +192,13 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
self.write_helper(b"old unknown executable")
|
||||
self.download.return_value = io.BytesIO(b"bad download")
|
||||
with self.assertRaisesRegex(RuntimeError, "checksum mismatch"):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.system.sudo.assert_not_called()
|
||||
self.assertEqual(self.path.read_bytes(), b"old unknown executable")
|
||||
|
||||
def assert_rejected_before_network_or_sudo(self):
|
||||
with self.assertRaisesRegex(RuntimeError, "Unsafe ufw-docker"):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.download.assert_not_called()
|
||||
self.system.sudo.assert_not_called()
|
||||
|
||||
@@ -219,9 +225,9 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
|
||||
def test_unsafe_helper_preflight_precedes_package_network_or_mutation(self):
|
||||
self.path.parent.chmod(0o777)
|
||||
with patch.object(base.os.path, "isfile", return_value=True):
|
||||
with patch.object(helper.os.path, "isfile", return_value=True):
|
||||
with self.assertRaisesRegex(RuntimeError, "Unsafe ufw-docker"):
|
||||
base.run(self.ctx, None)
|
||||
helper.run(self.ctx, None)
|
||||
self.download.assert_not_called()
|
||||
self.system.run.assert_not_called()
|
||||
self.system.sudo.assert_not_called()
|
||||
@@ -260,7 +266,7 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
|
||||
self.download.side_effect = download
|
||||
with self.assertRaisesRegex(RuntimeError, "Unsafe ufw-docker"):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.system.sudo.assert_not_called()
|
||||
|
||||
def test_changed_content_after_publication_is_not_executed(self):
|
||||
@@ -273,7 +279,7 @@ class TestPinnedHelper(unittest.TestCase):
|
||||
|
||||
self.system.sudo.side_effect = tamper
|
||||
with self.assertRaisesRegex(RuntimeError, "verification failed"):
|
||||
base._install_helper(self.ctx)
|
||||
helper._install_helper(self.ctx)
|
||||
self.assertEqual(self.system.sudo.call_count, 1)
|
||||
|
||||
|
||||
|
||||
+2
-2
@@ -10,7 +10,7 @@ import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
from actions import env
|
||||
from actions.config import env
|
||||
from engine.descriptor import build_parser
|
||||
from engine.lib.host import Host
|
||||
|
||||
@@ -31,7 +31,7 @@ class TestEnv(unittest.TestCase):
|
||||
self.ctx.host = Host(self.ctx.log, self.ctx.sys)
|
||||
|
||||
def run_env(self, *argv):
|
||||
args = build_parser(env.METADATA, 'apex env').parse_args(argv)
|
||||
args = build_parser(env.METADATA, 'apex config/env').parse_args(argv)
|
||||
with contextlib.redirect_stdout(io.StringIO()) as out:
|
||||
env.run(self.ctx, args)
|
||||
return out.getvalue()
|
||||
|
||||
+12
-1
@@ -10,6 +10,7 @@ import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from engine.lib.host import Host, apply_block
|
||||
from engine.lib.sys import Sys
|
||||
|
||||
|
||||
def _check_backup_chmod(chmod, expected_group):
|
||||
@@ -122,6 +123,15 @@ class TestHostWrites(unittest.TestCase):
|
||||
self.assertEqual(stat.S_IMODE(backups[0].stat().st_mode), 0o600)
|
||||
self.assertEqual(path.read_text(), "new")
|
||||
|
||||
def test_root_can_perform_privileged_file_operations_before_sudo_is_installed(self):
|
||||
path = self.root / "bootstrap"
|
||||
path.write_text("first")
|
||||
host = Host(self.log, Sys(self.log))
|
||||
with patch("os.geteuid", return_value=0), patch("shutil.which", return_value=None):
|
||||
host.write_file(str(path), "second", sudo=True)
|
||||
self.assertEqual(host._read(str(path), sudo=True), "second")
|
||||
self.assertEqual(next(self.root.glob("bootstrap.*.bak")).read_text(), "first")
|
||||
|
||||
def test_backup_sets_owner_before_exposing_group_readable_content(self):
|
||||
path = self.root / "group-secret"
|
||||
path.write_text("synthetic group-only secret")
|
||||
@@ -137,6 +147,8 @@ class TestHostWrites(unittest.TestCase):
|
||||
os.utime(path, (1600000000, 1600000000))
|
||||
|
||||
def local_sudo(command, **kwargs):
|
||||
if command[:2] == ["test", "-f"]:
|
||||
return subprocess.run(command, **kwargs)
|
||||
self.assertEqual(command[:3], ["python3", "-I", "-c"])
|
||||
probe = ("import os\n" + inspect.getsource(_check_backup_chmod) +
|
||||
f"\nos.chmod = _check_backup_chmod(os.chmod, {group})\n")
|
||||
@@ -145,7 +157,6 @@ class TestHostWrites(unittest.TestCase):
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
return result
|
||||
|
||||
self.host.sys.ok.return_value = True
|
||||
self.host.sys.sudo.side_effect = local_sudo
|
||||
for sudo in (False, True):
|
||||
with self.subTest(sudo=sudo):
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import tempfile
|
||||
from types import SimpleNamespace
|
||||
import unittest
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from engine import overlay
|
||||
|
||||
|
||||
def action(name):
|
||||
return overlay.load_module(str(Path(__file__).resolve().parents[1] / f"actions/{name}.py"),
|
||||
"test_manual_" + name.replace("/", "_"))
|
||||
|
||||
|
||||
class HostSetup(unittest.TestCase):
|
||||
def test_packages_preserve_pve_firewall(self):
|
||||
packages = action("config/packages")
|
||||
ctx = SimpleNamespace(sys=Mock())
|
||||
with (patch.object(Path, "is_file", return_value=True),
|
||||
patch.object(Path, "exists", return_value=True),
|
||||
patch.object(packages, "_install_missing") as install):
|
||||
packages.run(ctx, None)
|
||||
requested = install.call_args.args[1]
|
||||
self.assertIn("sudo", requested)
|
||||
self.assertNotIn("ufw", requested)
|
||||
|
||||
def test_failed_effective_policy_restores_original_configuration(self):
|
||||
hardening = action("config/hardening")
|
||||
system, host = Mock(), Mock()
|
||||
system.sudo.return_value.stdout = "Port 22\n"
|
||||
with (patch.object(hardening, "_operator_access", return_value="verified"),
|
||||
patch.object(hardening, "_effective", return_value={"authenticationmethods": "any"})):
|
||||
with self.assertRaisesRegex(RuntimeError, "effective SSH"):
|
||||
hardening.run(SimpleNamespace(sys=system, host=host), None)
|
||||
writes = host.write_file.call_args_list
|
||||
self.assertTrue(writes[0].args[1].startswith("Port 22\n"))
|
||||
self.assertEqual(writes[-1].args[1], "Port 22\n")
|
||||
system.reload.assert_called_once_with("ssh")
|
||||
|
||||
def test_current_endpoint_must_have_authenticated_with_a_key(self):
|
||||
hardening = action("config/hardening")
|
||||
ctx = SimpleNamespace(sys=Mock())
|
||||
sockets = SimpleNamespace(stdout='0 0 192.0.2.9:22 192.0.2.8:3002 users:(("sshd",pid=60,fd=4))')
|
||||
accepted = {"_PID": "60", "__MONOTONIC_TIMESTAMP": "100",
|
||||
"MESSAGE": "Accepted publickey for adam from 192.0.2.8 port 3002 ssh2: ED25519 SHA256:example"}
|
||||
with (patch.dict(os.environ, {"SSH_CONNECTION": "192.0.2.100 4321 192.0.2.9 1234",
|
||||
"SSH_USER_AUTH": "/arbitrary/file"}),
|
||||
patch.object(hardening, "_ancestors", return_value={60: 50}),
|
||||
patch.object(hardening.pwd, "getpwuid", return_value=SimpleNamespace(pw_name="adam"))):
|
||||
for changes in ({"_PID": "42"}, {"__MONOTONIC_TIMESTAMP": "49"},
|
||||
{"MESSAGE": accepted["MESSAGE"].replace("3002", "4002")}):
|
||||
with self.subTest(changes=changes):
|
||||
ctx.sys.sudo.side_effect = [sockets, SimpleNamespace(
|
||||
returncode=0, stdout=json.dumps({**accepted, **changes}))]
|
||||
with self.assertRaisesRegex(RuntimeError, "Cannot verify"):
|
||||
hardening._connection(ctx)
|
||||
ctx.sys.sudo.side_effect = [sockets, SimpleNamespace(returncode=0, stdout=json.dumps(accepted))]
|
||||
self.assertEqual(hardening._connection(ctx),
|
||||
"user=adam,host=192.0.2.8,addr=192.0.2.8,laddr=192.0.2.9,lport=22")
|
||||
|
||||
def test_journald_restart_failure_restores_previous_config(self):
|
||||
base = action("config/base")
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary)
|
||||
target = root / "99-apex.conf"
|
||||
legacy = root / "maxuse.conf"
|
||||
target.write_text("previous\n")
|
||||
legacy.write_text("legacy\n")
|
||||
backup = root / "legacy-backup"
|
||||
host = Mock()
|
||||
host.backup.side_effect = lambda path, **kwargs: (backup.write_bytes(Path(path).read_bytes())
|
||||
and str(backup))
|
||||
host.write_file.side_effect = lambda path, content, **kwargs: Path(path).write_text(content)
|
||||
def sudo(command):
|
||||
if command[0] == "systemctl":
|
||||
raise RuntimeError("restart failed")
|
||||
if command[0] == "cp":
|
||||
Path(command[-1]).write_bytes(Path(command[-2]).read_bytes())
|
||||
else:
|
||||
Path(command[-1]).unlink()
|
||||
ctx = SimpleNamespace(sys=SimpleNamespace(sudo=sudo), host=host)
|
||||
with self.assertRaisesRegex(RuntimeError, "restart failed"):
|
||||
base._configure(ctx, target, legacy, "updated\n", restart=True)
|
||||
self.assertEqual(target.read_text(), "previous\n")
|
||||
self.assertEqual(legacy.read_text(), "legacy\n")
|
||||
|
||||
def test_unverified_session_makes_no_host_changes(self):
|
||||
hardening = action("config/hardening")
|
||||
ctx = SimpleNamespace(sys=Mock(), host=Mock())
|
||||
with (patch.object(hardening.pwd, "getpwnam", return_value=SimpleNamespace(
|
||||
pw_uid=1001, pw_dir="/home/adam")),
|
||||
patch.object(hardening.pwd, "getpwuid", return_value=SimpleNamespace(pw_name="root")),
|
||||
patch.object(Path, "read_text", return_value="1001")):
|
||||
with self.assertRaisesRegex(RuntimeError, "independent administrator"):
|
||||
hardening.run(ctx, None)
|
||||
self.assertFalse(ctx.host.mock_calls)
|
||||
self.assertFalse(ctx.sys.mock_calls)
|
||||
|
||||
def test_independent_administrator_can_harden_only_with_an_operator_key(self):
|
||||
hardening = action("config/hardening")
|
||||
account = SimpleNamespace(pw_uid=1001, pw_dir="/home/adam")
|
||||
system = Mock()
|
||||
def privileged(command, **kwargs):
|
||||
output = ""
|
||||
if "stat" in command:
|
||||
output = "1001 600 regular file" if command[-1].endswith("authorized_keys") else "1001 700 directory"
|
||||
elif "cat" in command:
|
||||
output = "ssh-ed25519 fixture"
|
||||
elif "-T" in command:
|
||||
output = "authorizedkeysfile .ssh/authorized_keys"
|
||||
return SimpleNamespace(stdout=output, returncode=0)
|
||||
system.sudo.side_effect = privileged
|
||||
ctx = SimpleNamespace(sys=system)
|
||||
with (patch.object(hardening.pwd, "getpwnam", return_value=account),
|
||||
patch.object(hardening.pwd, "getpwuid", return_value=SimpleNamespace(pw_name="root")),
|
||||
patch.object(hardening.os, "getuid", return_value=0),
|
||||
patch.object(Path, "read_text", return_value="0")):
|
||||
system.run.return_value.returncode = 1
|
||||
with self.assertRaisesRegex(RuntimeError, "valid unrestricted operator public key"):
|
||||
hardening._operator_access(ctx)
|
||||
system.run.return_value.returncode = 0
|
||||
self.assertEqual(hardening._operator_access(ctx), "user=adam,host=localhost,addr=127.0.0.1")
|
||||
|
||||
def test_chown_limits_changes_to_checkout_and_skips_mounts_and_hardlinks(self):
|
||||
chown = action("repository/chown")
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary).resolve()
|
||||
for name in (".git", "configs", "volumes"):
|
||||
(root / name).mkdir()
|
||||
for name in ("configs/node", ".git/HEAD", "private", "volumes/data"):
|
||||
(root / name).write_text("fixture")
|
||||
os.link(root / "private", root / "configs/key")
|
||||
(root / "configs/other").symlink_to(root / "volumes", target_is_directory=True)
|
||||
with patch("os.chown") as ownership, patch("os.fchown"):
|
||||
chown._repair(str(root), {"configs/node", "configs/key", "configs/other", "volumes/data"},
|
||||
{str(root / "volumes")}, os.getuid(), os.getgid())
|
||||
self.assertEqual({call.args[0] for call in ownership.call_args_list},
|
||||
{"HEAD", "node", "other"})
|
||||
self.assertTrue(all(not call.kwargs["follow_symlinks"] for call in ownership.call_args_list))
|
||||
|
||||
def test_operator_provisioning_creates_account_then_preserves_keys_on_rerun(self):
|
||||
adam = action("config/adam")
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
root = Path(temporary).resolve()
|
||||
home = root / "adam"
|
||||
account = SimpleNamespace(pw_dir=str(home), pw_uid=1001)
|
||||
key = root / "operator"
|
||||
subprocess.run(["ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-f", str(key)], check=True)
|
||||
public = key.with_suffix(".pub")
|
||||
ctx = SimpleNamespace(sys=Mock(), host=Mock())
|
||||
ctx.sys.run.side_effect = lambda cmd, **kw: subprocess.run(cmd, input=kw["input"], text=True,
|
||||
capture_output=True, check=True)
|
||||
def privileged(command, **kwargs):
|
||||
if command[0] == "useradd":
|
||||
home.mkdir()
|
||||
if command[:6] == ["runuser", "-u", "adam", "--", "sh", "-ec"]:
|
||||
subprocess.run(command[4:], text=True, input=kwargs["input"], check=True)
|
||||
ctx.sys.sudo.side_effect = privileged
|
||||
actual_stat = Path.stat
|
||||
def metadata(path, *args, **kwargs):
|
||||
stat = actual_stat(path, *args, **kwargs)
|
||||
if path == home:
|
||||
fields = list(stat)
|
||||
fields[4] = account.pw_uid
|
||||
return os.stat_result(fields)
|
||||
return stat
|
||||
with (patch.object(adam, "HOME", home), patch.object(Path, "stat", metadata),
|
||||
patch.object(adam.grp, "getgrnam", side_effect=KeyError),
|
||||
patch.object(adam.pwd, "getpwnam", side_effect=[KeyError, account, account])):
|
||||
args = SimpleNamespace(public_key_file=str(public))
|
||||
adam.run(ctx, args)
|
||||
authorized = home / ".ssh/authorized_keys"
|
||||
with authorized.open("a") as stream:
|
||||
stream.write("existing operator entry\n")
|
||||
adam.run(ctx, args)
|
||||
self.assertEqual(authorized.read_text().splitlines().count(public.read_text().strip()), 1)
|
||||
self.assertIn("existing operator entry", authorized.read_text())
|
||||
commands = [call.args[0] for call in ctx.sys.sudo.call_args_list]
|
||||
self.assertEqual(sum(cmd[0] == "useradd" for cmd in commands), 1)
|
||||
self.assertIn(["runuser", "-u", "adam", "--", "sudo", "-n", "true"], commands)
|
||||
|
||||
def test_existing_system_account_cannot_be_promoted(self):
|
||||
adam = action("config/adam")
|
||||
ctx = SimpleNamespace(sys=Mock())
|
||||
account = SimpleNamespace(pw_dir="/home/adam", pw_uid=999)
|
||||
with patch.object(adam.pwd, "getpwnam", return_value=account):
|
||||
with self.assertRaisesRegex(RuntimeError, "incompatible UID"):
|
||||
adam.run(ctx, SimpleNamespace(public_key_file=None))
|
||||
self.assertFalse(ctx.sys.mock_calls)
|
||||
|
||||
def test_invalid_operator_key_is_rejected_before_provisioning(self):
|
||||
adam = action("config/adam")
|
||||
with tempfile.NamedTemporaryFile(mode="w") as key:
|
||||
key.write("private key fixture")
|
||||
key.flush()
|
||||
ctx = SimpleNamespace(sys=Mock())
|
||||
with self.assertRaisesRegex(ValueError, "OpenSSH public key"):
|
||||
adam.run(ctx, SimpleNamespace(public_key_file=key.name))
|
||||
self.assertFalse(ctx.sys.mock_calls)
|
||||
+19
-11
@@ -17,15 +17,15 @@ class TestOverlay(unittest.TestCase):
|
||||
self.prop = tempfile.mkdtemp()
|
||||
|
||||
def test_commons_only(self):
|
||||
_write(self.commons, "configure/ufw.py", ACT.format(s="commons ufw", r="C"))
|
||||
r = overlay.resolve("configure/ufw", self.commons, self.prop)
|
||||
_write(self.commons, "config/ufw.py", ACT.format(s="commons ufw", r="C"))
|
||||
r = overlay.resolve("config/ufw", self.commons, self.prop)
|
||||
self.assertIsNotNone(r); self.assertIsNone(r.shadowed)
|
||||
self.assertTrue(r.path.startswith(self.commons))
|
||||
|
||||
def test_override_exposes_shadowed(self):
|
||||
_write(self.commons, "configure/ufw.py", ACT.format(s="commons", r="C"))
|
||||
_write(self.prop, "configure/ufw.py", ACT.format(s="local", r="L"))
|
||||
r = overlay.resolve("configure/ufw", self.commons, self.prop)
|
||||
_write(self.commons, "config/ufw.py", ACT.format(s="commons", r="C"))
|
||||
_write(self.prop, "config/ufw.py", ACT.format(s="local", r="L"))
|
||||
r = overlay.resolve("config/ufw", self.commons, self.prop)
|
||||
self.assertTrue(r.path.startswith(self.prop)) # local wins
|
||||
self.assertTrue(r.shadowed.startswith(self.commons))
|
||||
|
||||
@@ -33,17 +33,25 @@ class TestOverlay(unittest.TestCase):
|
||||
self.assertIsNone(overlay.resolve("no/such", self.commons, self.prop))
|
||||
|
||||
def test_discover_marks_sources_and_disabled(self):
|
||||
_write(self.commons, "configure/ufw.py", ACT.format(s="commons ufw", r="C"))
|
||||
_write(self.commons, "sync/repository.py", ACT.format(s="sync repo", r="C"))
|
||||
_write(self.prop, "configure/ufw.py", ACT.format(s="local ufw", r="L"))
|
||||
_write(self.commons, "config/ufw.py", ACT.format(s="commons ufw", r="C"))
|
||||
_write(self.commons, "repository/sync.py", ACT.format(s="sync repo", r="C"))
|
||||
_write(self.prop, "config/ufw.py", ACT.format(s="local ufw", r="L"))
|
||||
_write(self.prop, "sync/minecraft.py",
|
||||
'from engine.descriptor import Meta\nMETADATA = Meta(summary="mc")\nDISABLED = "wip"\ndef run(ctx, args): pass\n')
|
||||
infos = {i.name: i for i in overlay.discover(self.commons, self.prop)}
|
||||
self.assertEqual(infos["configure/ufw"].source, "override")
|
||||
self.assertEqual(infos["sync/repository"].source, "commons")
|
||||
self.assertEqual(infos["config/ufw"].source, "override")
|
||||
self.assertEqual(infos["repository/sync"].source, "commons")
|
||||
self.assertEqual(infos["sync/minecraft"].source, "local")
|
||||
self.assertEqual(infos["sync/minecraft"].disabled, "wip")
|
||||
self.assertEqual(infos["configure/ufw"].summary, "local ufw")
|
||||
self.assertEqual(infos["config/ufw"].summary, "local ufw")
|
||||
|
||||
def test_only_entity_action_paths_can_be_listed_and_dispatched(self):
|
||||
for name in ("configure", "config/nested/tool", "config/tool"):
|
||||
_write(self.prop, name + ".py", ACT.format(s="test", r="ok"))
|
||||
names = [item.name for item in overlay.discover(self.commons, self.prop)]
|
||||
self.assertEqual(names, ["config/tool"])
|
||||
for name in ("configure", "config/nested/tool", "config/../configure"):
|
||||
self.assertIsNone(overlay.resolve(name, self.commons, self.prop))
|
||||
|
||||
def test_load_module_runs(self):
|
||||
p = _write(self.commons, "x/y.py", ACT.format(s="s", r="Z"))
|
||||
|
||||
@@ -6,7 +6,9 @@ import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
from actions.sync import packages
|
||||
from importlib import import_module
|
||||
|
||||
packages = import_module("actions.util.check-updates")
|
||||
|
||||
|
||||
def digest(char):
|
||||
|
||||
@@ -5,7 +5,7 @@ import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import Mock
|
||||
from actions.sync import repository
|
||||
from actions.repository import sync as repository
|
||||
from engine.lib.sys import Sys
|
||||
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ import unittest
|
||||
from unittest.mock import Mock
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
spec = importlib.util.spec_from_file_location('timezone_action', ROOT / 'actions/configure/timezone.py')
|
||||
spec = importlib.util.spec_from_file_location('timezone_action', ROOT / 'actions/config/timezone.py')
|
||||
action = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(action)
|
||||
|
||||
|
||||
Reference in new issue
Block a user