deytenit/apex.ermnvldmr.com
1
0 Fork 0
Code Issues Pull Requests Actions Releases Wiki Activity

refactor: gate xray behind profile, expose loki via basicauth, move happ generator to daedalus

Browse Source 
- xray gets profiles: [xray] — daedalus/icarus opt in via COMPOSE_PROFILES,
  morpheus never runs it.
- loki is routed at /loki on the node FQDN behind basicauth
  (APEX_LOKI_BASIC_AUTH from node .env); /loki is loki's native API prefix
  so no strip-prefix needed.
- utils/generate-happ-subscriptions + source/ move to daedalus proprietaries;
  commons keeps only what every node shares (16 actions now).
This commit is contained in:
deytenit committed 2026-07-05 12:16:51 +03:00
1 parent 1bc448961a
commit ce93f879e7
8 files changed
+18 -346
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

PARITY.md
+5 -3
View File
@@ -19,7 +19,7 @@ action to its bash source and the behaviors that must match.
| tiers/useradd | actions/tiers/useradd | noroot-<proj> system users; noroot-shared group; APEX_UID/GID block in .env | | tiers/useradd | actions/tiers/useradd | noroot-<proj> system users; noroot-shared group; APEX_UID/GID block in .env |
| tiers/chown | actions/tiers/chown | {tier}/{proj}=uid:gid; {tier}/shared/{proj}=uid:sgid+sticky; base=root:root | | tiers/chown | actions/tiers/chown | {tier}/{proj}=uid:gid; {tier}/shared/{proj}=uid:sgid+sticky; base=root:root |
| utils/extract-traefik-certs | actions/utils/extract-traefik-certs | traefik-certs-dumper v3; 600 perms; chown caller | | utils/extract-traefik-certs | actions/utils/extract-traefik-certs | traefik-certs-dumper v3; 600 perms; chown caller |
| utils/generate-happ-subscriptions | actions/utils/generate-happ-subscriptions | build image; run generator input->output | | utils/generate-happ-subscriptions | actions/utils/generate-happ-subscriptions | MOVED to daedalus proprietaries (node-specific tooling; commons keeps only shared stuff) |
| utils/lint-docker-compose | actions/utils/lint-docker-compose | dclint --fix via docker; --hook staged-only + restage; + anchor-IP check | | utils/lint-docker-compose | actions/utils/lint-docker-compose | dclint --fix via docker; --hook staged-only + restage; + anchor-IP check |
| compose | actions/compose | apex core first then services alpha; reverse on down; --dry-run; extra passthrough | | compose | actions/compose | apex core first then services alpha; reverse on down; --dry-run; extra passthrough |
@@ -48,8 +48,10 @@ action to its bash source and the behaviors that must match.
restored via an icarus-only override layered in the node repo's include (Phase 2). restored via an icarus-only override layered in the node repo's include (Phase 2).
- icarus: ACME propagation-tuning flags and DEBUG log level drop to the shared defaults; - icarus: ACME propagation-tuning flags and DEBUG log level drop to the shared defaults;
traefik now runs noroot (APEX_UID) with file provider + HTTP/3 like daedalus. traefik now runs noroot (APEX_UID) with file provider + HTTP/3 like daedalus.
- morpheus: gains apex-xray (design: xray is a core service on every node) — a config - xray is profile-gated (`xray` in COMPOSE_PROFILES): daedalus + icarus opt in,
must exist at `${APEX_TIER1}/xray/data/config.jsonc` before cutover or it crash-loops. morpheus does not (it never ran xray — owner decision 2026-07-05).
- loki (profile `loki`, no node opts in yet) is exposed via traefik at
`https://<node-fqdn>/loki` behind basicauth (`APEX_LOKI_BASIC_AUTH` in the node .env).
- morpheus: dashboard auth changes basicauth (TRAEFIK_INTERNAL_API_AUTH) -> enclave IP - morpheus: dashboard auth changes basicauth (TRAEFIK_INTERNAL_API_AUTH) -> enclave IP
allowlist; the adguard router labels exist without a backend (502, was 404, on /adguard). allowlist; the adguard router labels exist without a backend (502, was 404, on /adguard).
- restic: per-node selective tier2 mounts replaced by whole-tier-root RO mounts - restic: per-node selective tier2 mounts replaced by whole-tier-root RO mounts
README.md
+3 -1
View File
@@ -9,9 +9,11 @@ the shared core docker-compose, consumed by every node as the `commons/` git sub
- `actions/` — grouped actions; the path IS the name (`configure/ufw`, `tiers/link`, …). - `actions/` — grouped actions; the path IS the name (`configure/ufw`, `tiers/link`, …).
- `compositions/apex/docker-compose.yml` — the shared, `${env}`-parameterized core compose. - `compositions/apex/docker-compose.yml` — the shared, `${env}`-parameterized core compose.
- `vendor/` — pinned pure-python third-party (empty; see `vendor/VENDORED.md`). - `vendor/` — pinned pure-python third-party (empty; see `vendor/VENDORED.md`).
- `source/` — independent containerized tools (happ subscription generator).
- `tests/` — stdlib `unittest` suite (`tests/run.sh`). - `tests/` — stdlib `unittest` suite (`tests/run.sh`).
Node-specific tooling lives in each node repo's `proprietaries/` — commons holds only
what every node shares.
## Usage (on a node) ## Usage (on a node)
`apex` acts on the **local checkout** — no `<node>` argument. Identity comes from the `apex` acts on the **local checkout** — no `<node>` argument. Identity comes from the
host FQDN (`<node>.a<x>.apex.ermnvldmr.com`) and `node.env`. host FQDN (`<node>.a<x>.apex.ermnvldmr.com`) and `node.env`.
actions/utils/generate-happ-subscriptions.py
-30
View File
@@ -1,30 +0,0 @@
# actions/utils/generate-happ-subscriptions.py
"""Port of utils/generate-happ-subscriptions — build + run the happ generator image."""
import os
import shutil
from engine.descriptor import Meta, Arg
METADATA = Meta(summary="Generate .psub files from a JSON config via the happ generator (docker).",
args=[Arg("input_json", "input JSON config"), Arg("output_dir", "output directory")])
IMAGE = "happ-subscription-generator"
def run(ctx, args):
log = ctx.log
src_dir = os.path.join(ctx.paths.commons, "source", "happ-subscription-generator")
inp, out = os.path.realpath(args.input_json), os.path.realpath(args.output_dir)
# GNU realpath (bash) fails unless all but the last component exist — without this
# guard a mistyped path reaches `docker run -v`, which creates it root-owned.
for label, p in (("input_json", inp), ("output_dir", out)):
parent = os.path.dirname(p)
if not os.path.isdir(parent):
log.error(f"Cannot resolve {label}: no such directory: {parent}")
raise SystemExit(1)
if not shutil.which("docker"):
log.error("docker is not installed or not in PATH.")
raise SystemExit(1)
log.info(f"Building {IMAGE}...")
ctx.sys.run(["docker", "build", "-t", IMAGE, src_dir], check=True)
log.info(f"Running generator: {inp} -> {out}")
ctx.sys.run(["docker", "run", "--rm", "-v", f"{inp}:/app/input.json:ro",
"-v", f"{out}:/app/output:rw", IMAGE, "/app/input.json", "/app/output"], check=True)
log.success("Subscription generation completed.")
compositions/apex/docker-compose.yml
+10 -1
View File
@@ -7,10 +7,11 @@ x-environment: &shared-environment
TZ: "Europe/Moscow" TZ: "Europe/Moscow"
services: services:
# --- edge (host network) --- # --- edge (host network; profile-gated — a node opts in via COMPOSE_PROFILES) ---
xray: xray:
image: ghcr.io/xtls/xray-core:26.3.27 image: ghcr.io/xtls/xray-core:26.3.27
container_name: apex-xray container_name: apex-xray
profiles: [xray]
volumes: volumes:
- ${APEX_TIER1}/xray/data/config.jsonc:/etc/xray/config.jsonc:ro - ${APEX_TIER1}/xray/data/config.jsonc:/etc/xray/config.jsonc:ro
- ${APEX_TIER1_SHARED}/apex/letsencrypt:/etc/letsencrypt/live:ro - ${APEX_TIER1_SHARED}/apex/letsencrypt:/etc/letsencrypt/live:ro
@@ -164,6 +165,14 @@ services:
enclave: {} enclave: {}
restart: unless-stopped restart: unless-stopped
user: "${APEX_UID?error}:${APEX_GID?error}" user: "${APEX_UID?error}:${APEX_GID?error}"
labels:
# /loki/* is loki's native API prefix — no strip needed. basicauth users come
# from the node's uncommitted .env (only needed when the loki profile is on).
- "traefik.enable=true"
- "traefik.http.routers.apex-loki.rule=Host(`${APEX_NODE_FQDN}`) && PathPrefix(`/loki`)"
- "traefik.http.routers.apex-loki.middlewares=apex-loki-auth"
- "traefik.http.middlewares.apex-loki-auth.basicauth.users=${APEX_LOKI_BASIC_AUTH}"
- "traefik.http.services.apex-loki.loadbalancer.server.port=3100"
networks: networks:
direct: direct:
source/happ-subscription-generator/Dockerfile
-7
View File
@@ -1,7 +0,0 @@
FROM python:3.11-alpine
WORKDIR /app
COPY generator.py .
ENTRYPOINT ["python", "/app/generator.py"]
source/happ-subscription-generator/README.md
-35
View File
@@ -1,35 +0,0 @@
# HAPP Subscription Generator
**Disclaimer:** This tool is intended exclusively for **educational purposes** and **private networking** research. It is designed to demonstrate how to programmatically generate and manage subscription configurations for distributed network components.
This tool generates `.psub` subscription files from a JSON configuration. It supports global routing via `happ://` links and multiple VLESS connection strings.
## Features
- **Global Routing**: Encodes your `routing` object into the `happ://routing/onadd/` link.
- **Templates**: Define common Xray-core settings once and reuse them across users.
- **Deep Merge**: User-specific `xray` overrides are merged with templates.
- **Auto Remarks**: Generates readable titles based on country, protocol, and node names.
- **ISO Lookup**: Automatically converts country codes (e.g., `US`) to names and emojis.
## Usage
The tool is designed to be run via the operator script:
```bash
./.operator/scripts/ops/generate-happ-subscriptions <input_json> <output_dir>
```
### JSON Format Specification
The configuration file consists of three main top-level objects:
1. **`routing`**: Global routing settings (DNS, direct/proxy rules, etc.).
2. **`templates`**: Named connection templates containing shared protocol and stream settings.
3. **`users`**: List of users, each with a unique subscription key (`psub`) and a set of configurations derived from templates.
#### Example Reference
See [example.json](./example.json) for a comprehensive template of the configuration format.
## Implementation Details
- **Logic**: Written in Python 3.11 (standard library only).
- **Isolation**: Runs inside a Docker container (`python:3.11-alpine`).
source/happ-subscription-generator/example.json
-141
View File
@@ -1,141 +0,0 @@
{
"routing": {
"name": "enterprise-secure-access",
"GlobalProxy": "false",
"RouteOrder": "block-proxy-direct",
"RemoteDNSType": "DoU",
"RemoteDNSDomain": "internal.corp",
"RemoteDNSIP": "10.50.10.10",
"DomesticDNSType": "DoH",
"DomesticDNSDomain": "https://dns.example.com/query",
"DomesticDNSIP": "1.1.1.1",
"Geoipurl": "https://example.com/data/geoip.dat",
"Geositeurl": "https://example.com/data/geosite.dat",
"LastUpdated": "2026-04-14",
"DnsHosts": {
"dns.example.com": "1.1.1.1",
"gateway.internal.corp": "10.50.10.1"
},
"DirectSites": [
"geosite:google",
"geosite:microsoft",
"geosite:private",
"domain:internal.corp",
"regexp:.*\\.local$"
],
"DirectIp": [
"geoip:private",
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16"
],
"ProxyIp": [
"10.50.0.0/16",
"10.60.0.0/16",
"172.20.0.0/14"
],
"DomainStrategy": "IPIfNonMatch",
"FakeDNS": "false",
"UseChunkFiles": "true"
},
"templates": {
"corp-vpn-v1": {
"country": "US",
"nodes": ["hq-gateway-01"],
"xray": {
"protocol": "vless",
"settings": {
"address": "vpn-hq.example.net",
"port": 443,
"encryption": "none",
"flow": "xtls-rprx-vision"
},
"streamSettings": {
"network": "raw",
"security": "tls",
"tlsSettings": {
"serverName": "vpn-hq.example.net",
"allowInsecure": false,
"fingerprint": "chrome",
"alpn": ["h2", "http/1.1"]
}
}
}
},
"corp-vpn-v2": {
"country": "UK",
"nodes": ["branch-office-uk"],
"xray": {
"protocol": "vless",
"settings": {
"address": "vpn-uk.example.net",
"port": 443,
"encryption": "none",
"flow": "xtls-rprx-vision"
},
"streamSettings": {
"network": "xhttp",
"security": "tls",
"tlsSettings": {
"serverName": "vpn-uk.example.net",
"fingerprint": "chrome",
"alpn": ["h3", "h2"]
},
"xhttpSettings": {
"mode": "stream-one",
"path": "/internal-access-bridge/"
}
}
}
},
"secure-enclave": {
"country": "DE",
"nodes": ["secure-zone-alpha"],
"xray": {
"protocol": "vless",
"settings": {
"address": "secure.example.net",
"port": 443,
"encryption": "none",
"flow": "xtls-rprx-vision"
},
"streamSettings": {
"network": "raw",
"security": "reality",
"realitySettings": {
"dest": "internal-app.example.net",
"fingerprint": "chrome",
"serverNames": [
"internal-app.example.net"
],
"publicKey": "base64-encoded-public-key-placeholder"
}
},
"mux": {
"enabled": true,
"concurrency": 8
}
}
}
},
"users": [
{
"name": "employee-01",
"psub": "sub-key-dev-01",
"configs": [
{ "template": "corp-vpn-v1", "xray": { "settings": { "id": "uuid-placeholder-01" } } },
{ "template": "corp-vpn-v2", "xray": { "settings": { "id": "uuid-placeholder-01" } } },
{ "template": "secure-enclave", "xray": { "settings": { "id": "uuid-placeholder-01" }, "streamSettings": { "realitySettings": { "shortId": "short-id-01" } } } }
]
},
{
"name": "employee-02",
"psub": "sub-key-admin-02",
"configs": [
{ "template": "corp-vpn-v1", "xray": { "settings": { "id": "uuid-placeholder-02" } } },
{ "template": "corp-vpn-v2", "xray": { "settings": { "id": "uuid-placeholder-02" } } },
{ "template": "secure-enclave", "xray": { "settings": { "id": "uuid-placeholder-02" }, "streamSettings": { "realitySettings": { "shortId": "short-id-02" } } } }
]
}
]
}
source/happ-subscription-generator/generator.py
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env python3
"""
HAPP subscription generator.
Produces one .psub file per user. Each file contains:
line 1 : happ://routing/onadd/<base64> (routing profile, unchanged behaviour)
line N : a share link for every config the user has
Design
------
The config is written 1:1 against the HAPP / Xray share-link format. A template
is just the literal pieces of a URI:
scheme -> vless:// , hy2:// , trojan:// , ss:// , socks:// , vmess://
host -> the @host part
port -> the :port part
id -> the userinfo before @ (uuid / password / auth)
params -> the ?a=b&c=d query, keys are the REAL link keys
(type, security, sni, fp, alpn, flow, encryption,
pbk, sid, spx, mode, path, host, serviceName,
headerType, mux, obfs, ... whatever the link supports)
Nothing is translated or guessed. Whatever you put in `params` is what goes in
the link, url-encoded. This keeps the generator dumb and the config honest.
Per-user `configs` entries deep-merge onto the named template, so a user only
supplies the bits that differ (their `id`, their `params.sid`, etc.).
The `#remark` is built from template metadata (country / nodes) + the user name.
"""
import json
import base64
import sys
import os
import copy
from urllib.parse import urlencode, quote
# ISO 3166-1 alpha-2 -> (flag, display name). Extend as needed.
COUNTRY_MAP = {
"LV": ("\U0001F1F1\U0001F1FB", "Latvia"),
"RU": ("\U0001F1F7\U0001F1FA", "Russia"),
"KZ": ("\U0001F1F0\U0001F1FF", "Kazakhstan"),
"BY": ("\U0001F1E7\U0001F1FE", "Belarus"),
}
def deep_merge(base, override):
"""Recursively merge `override` into a copy of `base`. Lists/scalars replace."""
out = copy.deepcopy(base)
for key, value in override.items():
if isinstance(out.get(key), dict) and isinstance(value, dict):
out[key] = deep_merge(out[key], value)
else:
out[key] = copy.deepcopy(value)
return out
def routing_link(routing):
"""happ://routing/onadd/<base64(json)>"""
payload = json.dumps(routing, separators=(",", ":")).encode()
return "happ://routing/onadd/" + base64.b64encode(payload).decode()
def build_remark(node, user_name):
code = node.get("country", "")
flag, name = COUNTRY_MAP.get(code, ("\U0001F310", code or "??"))
transport = node.get("params", {}).get("type", "tcp")
chain = " > ".join(node.get("nodes", ["unknown"]))
return f"{flag} {name} [{transport}] | {chain} | {user_name}"
def build_link(node, user_name):
"""Assemble one share link from a fully-merged node dict."""
scheme = node["scheme"]
host = node["host"]
port = node["port"]
userinfo = node.get("id", "")
# Drop empty params so the query stays clean; preserve insertion order.
params = {k: v for k, v in node.get("params", {}).items()
if v is not None and v != ""}
# Lists (e.g. alpn) -> comma-joined; everything else stringified.
flat = {}
for k, v in params.items():
flat[k] = ",".join(map(str, v)) if isinstance(v, list) else str(v)
query = urlencode(flat, safe="/:")
remark = quote(build_remark(node, user_name))
base = f"{scheme}://{userinfo}@{host}:{port}"
return f"{base}?{query}#{remark}" if query else f"{base}#{remark}"
def main():
if len(sys.argv) < 3:
print("Usage: generator.py <input_json> <output_dir>")
sys.exit(1)
input_path, output_dir = sys.argv[1], sys.argv[2]
with open(input_path, "r", encoding="utf-8") as f:
data = json.load(f)
routing = routing_link(data.get("routing", {}))
templates = data.get("templates", {})
users = data.get("users", [])
os.makedirs(output_dir, exist_ok=True)
for user in users:
name = user["name"]
out_path = os.path.join(output_dir, user.get("psub", name))
lines = [routing]
for entry in user.get("configs", []):
tmpl = entry.get("template")
node = deep_merge(templates.get(tmpl, {}), entry) if tmpl else entry
lines.append(build_link(node, name))
with open(out_path, "w", encoding="utf-8") as f:
f.write("\n".join(lines) + "\n")
print(f"Generated: {out_path}")
if __name__ == "__main__":
main()