docs: Update docs for the correctness (#13)
This commit is contained in:
deytenit authored and 
GitHub committed
1 parent
8dba55272c
commit
51a783f93a
60 files changed
+2456
-380
No files matched your search
@@ -4,8 +4,15 @@ description: API and technical references for developers.
|
||||
weight: 30
|
||||
---
|
||||
|
||||
Technical reference materials and APIs for the Root framework.
|
||||
Technical reference materials and APIs for the ermnvldmr.com root deployment commons framework.
|
||||
|
||||
{{% cards %}}
|
||||
{{< card link="compose" title="Compose actions reference" icon="play" subtitle="High-level entrypoint for orchestrating docker-compose." >}}
|
||||
{{< card link="configure" title="Configure actions reference" icon="arrow-down" subtitle="Host provisioning and system-level configuration." >}}
|
||||
{{< card link="initialization" title="Initialization actions reference" icon="play" subtitle="Actions for initial node setup." >}}
|
||||
{{< card link="root-api" title="ROOT_ API reference" icon="code" subtitle="Standardized environment variables and logging helpers." >}}
|
||||
{{< card link="source" title="Source services reference" icon="book-open" subtitle="Shared services built from source." >}}
|
||||
{{< card link="sync" title="Sync actions reference" icon="check" subtitle="Actions for keeping nodes up-to-date." >}}
|
||||
{{< card link="tiers" title="Tiers actions reference" icon="archive" subtitle="Storage tiers management." >}}
|
||||
{{< card link="utils" title="Utility actions reference" icon="check" subtitle="Helper scripts and utilities." >}}
|
||||
{{% /cards %}}
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: Compose Actions
|
||||
description: Standardized high-level entrypoint for orchestrating docker-compose across a node.
|
||||
weight: 54
|
||||
---
|
||||
|
||||
The `compose` action provides a standardized, high-level entrypoint for orchestrating Docker Compose files across all projects within a specific node.
|
||||
|
||||
## Core Compose Action {#actions}
|
||||
|
||||
| Action | Description |
|
||||
| :--- | :--- |
|
||||
| `compose <node> <action>` | Executes a given Docker Compose command (e.g., `up`, `down`, `restart`) across the node's infrastructure. |
|
||||
|
||||
## Orchestration Order {#order}
|
||||
|
||||
To maintain correct dependencies (especially when shutting down or starting up infrastructure-level proxies and networks), the `compose` action intelligently orders the execution:
|
||||
|
||||
### Starting Up (`up`, `restart`)
|
||||
When starting or restarting services, the action follows a **Forward Order**:
|
||||
1. **Node Infrastructure**: First, it processes the node-level `docker-compose.yml` (located at the root of the node directory).
|
||||
2. **Root Infrastructure**: Next, it processes the `root/docker-compose.yml` project, if it exists (usually contains Traefik, Socket Proxy, etc.).
|
||||
3. **Projects**: Finally, it processes all other project directories in alphabetical order.
|
||||
|
||||
### Shutting Down (`down`)
|
||||
When stopping services, the action uses a **Reverse Order** to ensure projects disconnect gracefully before the network or proxy goes down:
|
||||
1. **Projects**: All standard projects are stopped in reverse alphabetical order.
|
||||
2. **Root Infrastructure**: The `root/docker-compose.yml` project is stopped.
|
||||
3. **Node Infrastructure**: Finally, the node-level `docker-compose.yml` is brought down.
|
||||
|
||||
## Options {#options}
|
||||
|
||||
You can pass standard Docker Compose arguments to the end of the command.
|
||||
Additionally, the action supports a dry-run mode:
|
||||
|
||||
- `--dry-run`: Prints the commands that would be executed without actually running them.
|
||||
|
||||
**Example usage:**
|
||||
```bash
|
||||
root compose daedalus up -d --remove-orphans
|
||||
root compose daedalus restart --dry-run
|
||||
root compose daedalus down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Infrastructure Architecture](/root/concepts/architecture)
|
||||
- [Reference: ROOT_ API Reference](/root/reference/root-api)
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
title: Configuration Actions
|
||||
description: Host provisioning, template engine, and system-level configuration.
|
||||
weight: 53
|
||||
---
|
||||
|
||||
Configuration actions are used to provision the host and maintain system-level services. They rely heavily on the built-in template engine to generate declarative configuration files.
|
||||
|
||||
## The Template Engine {#template-engine}
|
||||
|
||||
The Root framework uses `envsubst` (from `gettext-base`) to render templates. This allows you to use standard Bash environment variables directly within your configuration files.
|
||||
|
||||
### Template Conventions
|
||||
|
||||
- **File Extension**: Templates typically use the `.template` extension (e.g., `crontab.template`).
|
||||
- **Variable Syntax**: Use `${VARIABLE_NAME}` or `$VARIABLE_NAME` to inject values.
|
||||
- **Auto-Discovery**: Many actions automatically look for templates in specific directories within the [Configs](/root/glossary#configs) repository.
|
||||
|
||||
### Rendering Helpers
|
||||
|
||||
The `root_template.sh` library provides several helpers:
|
||||
|
||||
- `root_template_render <input> <output>`: Renders a single file.
|
||||
- `root_template_render_directory <src> <dest>`: Recursively renders an entire directory structure.
|
||||
- `root_template_crontab <template>`: Renders and installs a crontab for the current user.
|
||||
|
||||
## Core Configuration Actions {#actions}
|
||||
|
||||
| Action | Description |
|
||||
| :--- | :--- |
|
||||
| `configure/base` | Installs essential packages (`sudo`, `ufw`, `docker`) and sets up basic system users. |
|
||||
| `configure/cron` | Renders and installs node-specific crontabs. |
|
||||
| `configure/crowdsec` | Sets up the CrowdSec agent and bouncers. |
|
||||
| `configure/routing` | Configures system-level routing and bridge networks. |
|
||||
| `configure/systemd` | Installs and enables systemd service units. |
|
||||
| `configure/ufw` | Renders and applies firewall rules using UFW and `ufw-docker`. |
|
||||
|
||||
## Example: Nginx Template {#example}
|
||||
|
||||
```nginx filename="nginx.conf.template"
|
||||
# This template uses ROOT_NODE environment variable
|
||||
user nginx;
|
||||
worker_processes auto;
|
||||
|
||||
error_log /var/log/nginx/error.log notice;
|
||||
pid /var/run/nginx.pid;
|
||||
|
||||
events {
|
||||
worker_connections 1024;
|
||||
}
|
||||
|
||||
http {
|
||||
include /etc/nginx/mime.types;
|
||||
default_type application/octet-stream;
|
||||
|
||||
# Injected from environment
|
||||
server_name ${ROOT_NODE}.example.com;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Infrastructure Architecture](/root/concepts/architecture)
|
||||
- [Reference: ROOT_ API Reference](/root/reference/root-api)
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: System Initialization
|
||||
description: Repository setup, Git hooks, and PATH configuration.
|
||||
weight: 51
|
||||
---
|
||||
|
||||
The Root framework requires a one-time initialization on each host to establish path linkages and repository-level configurations.
|
||||
|
||||
## The `init.sh` Script {#init-script}
|
||||
|
||||
The `init.sh` script located at the repository root is the primary bootstrap mechanism. It performs the following critical tasks:
|
||||
|
||||
1. **Submodule Initialization**: Ensures the [Commons](/root/glossary#commons) repository is present and up to date.
|
||||
2. **Git Hook Configuration**: Points `core.hooksPath` to `.operator/shared/githooks` to enable standardized pre-commit checks.
|
||||
3. **Safe Directory Registration**: Adds the repository to Git's `safe.directory` list, which is required for operations performed by different users (e.g., `root`, `adam`).
|
||||
4. **Vim Configuration**: Links a standardized `.vimrc` to both the current user and the `root` user for a consistent editing experience.
|
||||
5. **PATH Persistence**: Adds the `.operator/shared/scripts/run` directory to the user's `~/.bashrc`, making the `root` command available globally.
|
||||
|
||||
## Git Hooks {#githooks}
|
||||
|
||||
Root enforces standards through Git hooks stored in `.operator/shared/githooks/`.
|
||||
|
||||
- **Pre-commit**: Validates that all scripts follow the namespacing and structure conventions.
|
||||
- **Commit-msg**: (Optional) Enforces conventional commit messages if configured.
|
||||
|
||||
## PATH Configuration {#path}
|
||||
|
||||
The `root` command is a wrapper located in `.operator/shared/scripts/run/root`. By adding this directory to the `PATH`, the dispatcher can be invoked from any directory within the system, automatically resolving the repository root based on the current working directory or environment variables.
|
||||
|
||||
```bash
|
||||
# Example of manual PATH addition (performed automatically by init.sh)
|
||||
export PATH="$HOME/repo/.operator/shared/scripts/run:$PATH"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Guide: Initial Host Setup](/root/guides/installation)
|
||||
- [Reference: ROOT_ API Reference](/root/reference/root-api)
|
||||
@@ -4,11 +4,19 @@ description: Standardized environment variables for path resolution and node con
|
||||
weight: 50
|
||||
---
|
||||
|
||||
Every [Action](/root/glossary#action) script executed via the `root` dispatcher has access to a standardized set of environment variables. These variables are the single source of truth for path resolution and node context.
|
||||
Every [Action](/root/glossary#action) script executed via the `root` dispatcher has access to a standardized set of environment variables and Bash libraries. This provides a consistent "API" for interacting with the infrastructure.
|
||||
|
||||
## The `root` Dispatcher {#dispatcher}
|
||||
|
||||
The `root` command (located in `.operator/shared/scripts/run/root`) is the main entrypoint for all orchestration. Its primary responsibilities include:
|
||||
|
||||
1. **Context Resolution**: Locating the target [Node](/root/glossary#node) and its associated storage [Tiers](/root/glossary#tier).
|
||||
2. **Library Loading**: Sourcing the core namespaced libraries (e.g., `root_core`, `root_sys`).
|
||||
3. **Action Execution**: Discovering and running the requested action, prioritizing local overrides.
|
||||
|
||||
## Path resolution variables {#paths}
|
||||
|
||||
These variables define the physical location of the infrastructure components.
|
||||
These variables are established by the dispatcher and are available to all actions.
|
||||
|
||||
| Variable | Description |
|
||||
| :--- | :--- |
|
||||
@@ -20,40 +28,35 @@ These variables define the physical location of the infrastructure components.
|
||||
|
||||
## Node context variables {#node-context}
|
||||
|
||||
When an action targets a specific [Node](/root/glossary#node), the following variables are established by the `root_require_node` helper.
|
||||
When an action targets a specific node, the following variables are established by the `root_require_node` helper.
|
||||
|
||||
| Variable | Description |
|
||||
| :--- | :--- |
|
||||
| `ROOT_NODE` | The identifier of the target node (e.g., `icarus`). |
|
||||
| `ROOT_NODE` | The identifier of the target node (e.g., `daedalus`). |
|
||||
| `ROOT_NODE_DIR` | Absolute path to the node configuration directory. |
|
||||
| `ROOT_TIER1` | Absolute path to the [Tier 1](/root/glossary#tier) storage directory. |
|
||||
| `ROOT_TIER2` | Absolute path to the Tier 2 storage directory. |
|
||||
| `ROOT_TIER3` | Absolute path to the Tier 3 storage directory. |
|
||||
|
||||
## Logging helpers {#logging}
|
||||
## Core Logging Helpers {#logging}
|
||||
|
||||
Standardized logging uses the `[$ROOT_NODE] [$ROOT_ACTION_PATH]` prefix for consistency across the orchestration logs.
|
||||
Standardized logging ensure consistent output across all actions.
|
||||
|
||||
- `root_log_info`: Standard informational output.
|
||||
- `root_log_warn`: Warnings requiring attention but not stopping execution.
|
||||
- `root_log_warn`: Warnings requiring attention.
|
||||
- `root_log_error`: Critical errors sent to `stderr`.
|
||||
- `root_log_success`: Confirmation of successful task completion.
|
||||
|
||||
## Help system metadata {#help}
|
||||
## The Help System {#help}
|
||||
|
||||
The `root` [Dispatcher](/root/glossary#dispatcher) provides a built-in `--help` system. To keep action scripts "stupid", they do not implement their own help flags. Instead, the dispatcher parses a specific `action_metadata` block defined at the top of every action script.
|
||||
|
||||
When a user runs `root <action> --help`, the dispatcher extracts and displays this YAML-formatted block without executing the script.
|
||||
|
||||
### Example metadata
|
||||
The dispatcher provides a built-in help system by parsing an `action_metadata` block at the top of each script. This allows scripts to remain "stupid" and declarative without implementing their own CLI argument parsing.
|
||||
|
||||
```bash filename="example-action-metadata"
|
||||
action_metadata() {
|
||||
cat <<EOF
|
||||
description: "Brief summary of the task."
|
||||
description: "Brief summary of the action."
|
||||
args:
|
||||
- NODE: "The node name"
|
||||
- PARAM: "A custom parameter"
|
||||
- NODE: "The name of the target node."
|
||||
EOF
|
||||
}
|
||||
```
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: The `source/` Directory
|
||||
description: Conceptual overview of the self-contained source directory structure.
|
||||
weight: 52
|
||||
---
|
||||
|
||||
The `source/` directory at the repository root contains the core components and projects that the Root framework orchestrates. It is designed for maximum isolation and self-containment.
|
||||
|
||||
## Core Philosophy {#philosophy}
|
||||
|
||||
The `source/` directory is not just a place to store code; it is a collection of independent [Projects](/root/glossary#project). Each project within this directory should follow these principles:
|
||||
|
||||
1. **Isolation**: Each project lives in its own subdirectory (e.g., `source/my-app/`).
|
||||
2. **Self-Documentation**: Each project directory **MUST** contain its own `README.md` file describing its purpose, dependencies, and configuration.
|
||||
3. **Internal Logic**: Projects handle their own build processes, Dockerfiles, and internal configurations. The Root framework's [Actions](/root/glossary#action) simply orchestrate these existing components.
|
||||
|
||||
## Directory Structure {#structure}
|
||||
|
||||
A typical project within the `source/` directory might look like this:
|
||||
|
||||
{{< filetree/container >}}
|
||||
{{< filetree/folder name="source" state="open" >}}
|
||||
{{< filetree/folder name="happ-subscription-generator" state="open" >}}
|
||||
{{< filetree/file name="Dockerfile" >}}
|
||||
{{< filetree/file name="generator.py" >}}
|
||||
{{< filetree/file name="README.md" >}}
|
||||
{{< /filetree/folder >}}
|
||||
{{< /filetree/folder >}}
|
||||
{{< /filetree/container >}}
|
||||
|
||||
## Relationship with Actions {#actions}
|
||||
|
||||
Root actions (like `compose` or `sync`) interact with the `source/` directory by referencing paths within it. For example, the `compose` action might use a `docker-compose.yml` file that points to a `Dockerfile` located in `source/app/`.
|
||||
|
||||
This separation of concerns ensures that the orchestration logic (Actions) remains decoupled from the implementation details (Source).
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Infrastructure Architecture](/root/concepts/architecture)
|
||||
- [Reference: ROOT_ API Reference](/root/reference/root-api)
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: Synchronization Actions
|
||||
description: Repository updates, package synchronization, and data backups.
|
||||
weight: 54
|
||||
---
|
||||
|
||||
Synchronization actions ensure that the host's state remains consistent with the defined configurations in the remote repositories.
|
||||
|
||||
## Core Synchronization Actions {#actions}
|
||||
|
||||
| Action | Description |
|
||||
| :--- | :--- |
|
||||
| `sync/packages` | Synchronizes node packages and their configurations across the system. |
|
||||
| `sync/repository` | Updates the main configuration repository to its latest state from GitHub. |
|
||||
| `sync/tiers` | Triggers the node-specific backup strategy for the configured [Tiers](/root/glossary#tier). |
|
||||
|
||||
## Repository Synchronization (`sync/repository`) {#repository}
|
||||
|
||||
The `sync/repository` action is responsible for keeping the host's configuration repository up-to-date with its remote source. It typically performs a `git pull` on the main configuration repository and may optionally trigger sub-module updates for the [Commons](/root/glossary#commons) repository.
|
||||
|
||||
This ensures that any changes to the node configurations, actions, or source files are correctly pulled down to the target host.
|
||||
|
||||
## Backup Synchronization (`sync/tiers`) {#tiers}
|
||||
|
||||
The `sync/tiers` action is the primary entry point for a node's backup strategy.
|
||||
|
||||
- **No Default Implementation**: The framework itself does not provide a default backup mechanism, allowing operators to design a custom strategy suitable for their needs.
|
||||
- **Custom Overrides**: As a "Smart Dispatcher" action, you can define your backup logic by placing a custom script in `.operator/local/scripts/actions/sync/tiers`.
|
||||
- **Example Implementation**: A common practice is to use a tool like [Restic](https://restic.net/) within this custom script to perform off-site backups of critical data (Tiers 1 and 2) to an S3-compatible storage backend.
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Tiered Storage Model](/root/concepts/tiered-storage)
|
||||
- [Reference: ROOT_ API Reference](/root/reference/root-api)
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: Tier Management Actions
|
||||
description: Storage classification, symbolic linking, and privilege isolation.
|
||||
weight: 55
|
||||
---
|
||||
|
||||
Tier management actions are responsible for enforcing the framework's [Tiered Storage Model](/root/concepts/tiered-storage) and ensuring correct filesystem ownership and isolation.
|
||||
|
||||
## Core Tier Actions {#actions}
|
||||
|
||||
| Action | Description |
|
||||
| :--- | :--- |
|
||||
| `tiers/chown` | Sets correct filesystem ownership for project-specific [Non-Root Users](#useradd). |
|
||||
| `tiers/link` | Manages symbolic links between the project directories and their corresponding [Tiers](/root/glossary#tier). |
|
||||
| `tiers/useradd` | Provisions a unique system user for each project to achieve [Privilege Isolation](/root/concepts/security-model). |
|
||||
|
||||
## User Management (`tiers/useradd`) {#useradd}
|
||||
|
||||
The `tiers/useradd` action creates a dedicated, low-privileged system user for each project within a node. This user is then used to run the project's containers, ensuring that a compromise of one project does not grant the attacker access to the entire host.
|
||||
|
||||
- **Naming Convention**: Users are typically named after the project.
|
||||
- **Group Membership**: Users can optionally be added to the `docker` group or interact with Docker via other securely configured methods, depending on the operator's preference.
|
||||
|
||||
## Storage Linking (`tiers/link`) {#link}
|
||||
|
||||
The `tiers/link` action creates the symbolic link structure required for the tiered storage model. It ensures that directories like `config`, `data`, and `logs` within a project folder point to their physical locations in `@tier1`, `@tier2`, and `@tier3` respectively.
|
||||
|
||||
```bash
|
||||
# Example link structure created by tiers/link
|
||||
/srv/root/icarus/synapse/config -> /srv/storage/@tier1/synapse
|
||||
/srv/root/icarus/synapse/data -> /srv/storage/@tier2/synapse
|
||||
/srv/root/icarus/synapse/logs -> /srv/storage/@tier3/synapse
|
||||
```
|
||||
|
||||
## Ownership Management (`tiers/chown`) {#chown}
|
||||
|
||||
Once tiers and links are established, the `tiers/chown` action ensures that the physical directories in each tier are owned by the correct project user. This is critical for preventing permission-denied errors and ensuring that projects can only access their own data.
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Tiered Storage Model](/root/concepts/tiered-storage)
|
||||
- [Concept: Security & Privilege Isolation](/root/concepts/security-model)
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: Utility Actions
|
||||
description: Common helper scripts for maintenance, validation, and certificate management.
|
||||
weight: 56
|
||||
---
|
||||
|
||||
Utility actions provide common helper functions for node maintenance, validation, and project-specific tasks.
|
||||
|
||||
## Core Utility Actions {#actions}
|
||||
|
||||
| Action | Description |
|
||||
| :--- | :--- |
|
||||
| `utils/extract-traefik-certs` | Extracts SSL/TLS certificates and private keys from Traefik's `acme.json` storage. |
|
||||
| `utils/generate-happ-subscriptions` | Automatically generates HApp subscription files based on current project metadata. |
|
||||
| `utils/lint-docker-compose` | Performs a static analysis of all `docker-compose.yml` files to ensure they meet the framework's standards. |
|
||||
|
||||
## Certificate Management (`utils/extract-traefik-certs`) {#certificates}
|
||||
|
||||
Traefik stores its ACME-issued certificates (like those from Let's Encrypt) in a single `acme.json` file. The `utils/extract-traefik-certs` action parses this file and extracts individual `fullchain.pem` and `privkey.pem` files.
|
||||
|
||||
This is particularly useful when other services (e.g., mail servers, non-Dockerized applications) require the same SSL certificates.
|
||||
|
||||
## Quality Assurance (`utils/lint-docker-compose`) {#linting}
|
||||
|
||||
To maintain consistency and security, the `utils/lint-docker-compose` action validates all compose files across the infrastructure. It checks for:
|
||||
|
||||
- **Naming Standards**: Ensures project names and container names follow the `{node}-{project}` convention.
|
||||
- **Security Context**: Flags containers running as `root` without a justified override.
|
||||
- **Resource Constraints**: Identifies services lacking CPU or memory limits.
|
||||
- **Label Standards**: Validates required labels for routing (e.g., [Traefik](https://traefik.io/)) and [CrowdSec](/root/concepts/security-model#crowdsec).
|
||||
|
||||
---
|
||||
|
||||
**See also:**
|
||||
|
||||
- [Concept: Infrastructure Principles](/root/concepts/principles)
|
||||
- [Concept: Security & Privilege Isolation](/root/concepts/security-model)
|
||||
Reference in new issue
Block a user