80 lines
4.3 KiB
Markdown
80 lines
4.3 KiB
Markdown
---
|
|
title: Troubleshooting
|
|
description: Symptoms, diagnosis, and fixes for common apex failure modes.
|
|
weight: 40
|
|
---
|
|
|
|
Failure modes the [engine](/apex/glossary#engine) reports, with the exact log lines it emits.
|
|
|
|
## APEX_NODE_FQDN falls back to the hostname {#fqdn-fallback}
|
|
|
|
**Symptoms:** every action logs `APEX_NODE_FQDN not set in node.env; using hostname '<fqdn>'.`
|
|
|
|
**Diagnosis:** `node.env` does not set `APEX_NODE_FQDN`, so the engine derived the FQDN from the OS hostname. On a node whose hostname is the intended FQDN this is harmless; otherwise traefik `Host()` routing will use the wrong name.
|
|
|
|
**Solution:** set `APEX_NODE_FQDN` explicitly in `node.env` at the repository root (for example `APEX_NODE_FQDN=node1.example.com`).
|
|
|
|
## Identity exits 66 {#identity-66}
|
|
|
|
**Symptoms:** `identity: APEX_NODE_FQDN missing from node.env and no hostname` or `identity: APEX_SUBNET missing from node.env`, exit `66`.
|
|
|
|
**Diagnosis:** a required identity field cannot be resolved. The first line means neither `APEX_NODE_FQDN` nor an OS hostname yields an FQDN; the second means `APEX_SUBNET` is absent. The engine refuses to guess.
|
|
|
|
**Solution:** create or complete `node.env` at the repository root — set `APEX_SUBNET`, and set `APEX_NODE_FQDN` (or ensure the OS hostname is usable).
|
|
|
|
## configure/ufw exits 66 {#ufw-66}
|
|
|
|
**Symptoms:** `Rules directory does not exist: ...` or `Config directory does not exist: ...`, exit `66`.
|
|
|
|
**Diagnosis:** exactly one of `configs/ufw/docker/` or `configs/ufw/host/` is missing — a broken config layout rather than an intentionally rule-less node.
|
|
|
|
**Solution:** restore the missing directory (empty is acceptable: an empty `docker/` warns `No .rules files found ... nothing to apply.` and continues).
|
|
|
|
## ufw tooling missing {#ufw-tools}
|
|
|
|
**Symptoms:** `ufw not installed/working via sudo. Run configure/base first.` or `ufw-docker not found at ~/.local/bin/ufw-docker. Run configure/base first.`, exit `1`.
|
|
|
|
**Diagnosis:** the firewall layer's prerequisites are absent.
|
|
|
|
**Solution:** run `apex configure/base`, then retry `apex configure/ufw`.
|
|
|
|
## Crontab install refused or rolled back {#cron}
|
|
|
|
**Symptoms:** `Rendered crontab is empty; aborting (previous kept).` (exit `65`) or `crontab install failed; restoring previous.` (exit `1`).
|
|
|
|
**Diagnosis:** the rendered `configs/cron/crontab` was empty (template/variable problem — remember unknown `$VARS` stay literal, they do not blank the file) or `crontab` rejected the syntax.
|
|
|
|
**Solution:** inspect the printed preview, fix the template, re-run `apex configure/cron`. The previous crontab is intact in both cases.
|
|
|
|
## sync/repository aborts {#sync-aborts}
|
|
|
|
**Symptoms:** `Commons submodule at <path> has uncommitted changes. Aborting.` or `Directory '<repo>/compositions' does not exist. Cannot sync.`, exit `1`.
|
|
|
|
**Diagnosis:** the [capture-up](/apex/glossary#capture-up) commit refuses to record a broken checkout — a dirty pinned submodule or a missing [compositions](/apex/glossary#compositions) tree.
|
|
|
|
**Solution:** for a dirty submodule, `git -C commons checkout -- .` (or commit an intentional pin move); for a missing tree, restore the checkout before the next scheduled run.
|
|
|
|
## Telegram not delivered {#telegram}
|
|
|
|
**Symptoms:** `Telegram Bot URL not provided. Skipping notification.` or `Failed to send Telegram notification: <error>` — and notify-gated actions (`backup/run`, `sync/repository`, `sync/packages`) exit `1` even though their work succeeded.
|
|
|
|
**Diagnosis:** the bot URL argument was empty, or the send failed (network, proxy, revoked token). The non-zero exit is deliberate: an unattended run whose status cannot be reported is treated as failed.
|
|
|
|
**Solution:** verify the URL source file the cron line reads, network/proxy reachability, and the bot token; re-run manually to confirm delivery.
|
|
|
|
## docker compose rejects the core wrapper {#compose-include}
|
|
|
|
**Symptoms:** `docker compose` errors on the `include` keyword when evaluating `compositions/apex/docker-compose.yml`.
|
|
|
|
**Diagnosis:** the [core composition](/apex/glossary#core-composition) is consumed through `include:`, which requires Docker Compose v2.20+.
|
|
|
|
**Solution:** upgrade the Compose plugin (`docker compose version` to verify).
|
|
|
|
---
|
|
|
|
**See also:**
|
|
|
|
- [Reference: CLI](/apex/reference/cli)
|
|
- [Guide: Operate a node](/apex/guides/host-operations)
|
|
- [GitHub: apex.ermnvldmr.com](https://github.com/deytenit/apex.ermnvldmr.com)
|