NPM: validate canonical_https for www redirects; docs and env example
- Reject non-https, paths, and injection-prone chars in advanced_config 301 targets - E2E list: phoenix marketing note, the-order HAProxy remediation, 2026-03-27 passes - AGENTS.md: scoped Cloudflare token pointer; smom-dbis-138 dotenv load note - .env.master.example: DNS script flags and scoped token guidance Made-with: Cursor
This commit is contained in:
defiQUG
@@ -17,6 +17,8 @@ PROXMOX_TOKEN_VALUE=
|
||||
PROXMOX_ALLOW_ELEVATED=
|
||||
|
||||
# --- Cloudflare ---
|
||||
# Prefer CLOUDFLARE_API_TOKEN scoped to Zone:DNS:Edit on the zones you use (avoid global Account API key when possible).
|
||||
# Bulk DNS script: scripts/update-all-dns-to-public-ip.sh — use --dry-run and --zone-only=sankofa.nexus (etc.) before wide updates.
|
||||
CLOUDFLARE_API_TOKEN=
|
||||
CLOUDFLARE_EMAIL=
|
||||
CLOUDFLARE_API_KEY=
|
||||
|
||||
@@ -17,6 +17,7 @@ Orchestration for Proxmox VE, Chain 138 (`smom-dbis-138/`), explorers, NPMplus,
|
||||
| Ops template + JSON | `docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md`, `config/proxmox-operational-template.json` |
|
||||
| Live vs template (read-only SSH) | `bash scripts/verify/audit-proxmox-operational-template.sh` |
|
||||
| Config validation | `bash scripts/validation/validate-config-files.sh` |
|
||||
| smom-dbis-138 `.env` in bash scripts | Prefer `source smom-dbis-138/scripts/lib/deployment/dotenv.sh` + `load_deployment_env --repo-root "$PROJECT_ROOT"` (trims RPC URL line endings). From an interactive shell: `source smom-dbis-138/scripts/load-env.sh`. Proxmox root scripts: `source scripts/lib/load-project-env.sh` (also trims common RPC vars). |
|
||||
| Sankofa portal → CT 7801 (build + restart) | `./scripts/deployment/sync-sankofa-portal-7801.sh` (`--dry-run` first); sets `NEXTAUTH_URL` on CT via `sankofa-portal-ensure-nextauth-on-ct.sh` |
|
||||
| CCIP relay (r630-01 host) | Unit: `config/systemd/ccip-relay.service` → `/etc/systemd/system/ccip-relay.service`; `systemctl enable --now ccip-relay` |
|
||||
| TsunamiSwap VM 5010 check | `./scripts/deployment/tsunamiswap-vm-5010-provision.sh` (inventory only until VM exists) |
|
||||
@@ -24,7 +25,7 @@ Orchestration for Proxmox VE, Chain 138 (`smom-dbis-138/`), explorers, NPMplus,
|
||||
| Portal login + Keycloak systemd + `.env` (prints password once) | `./scripts/deployment/enable-sankofa-portal-login-7801.sh` (`--dry-run` first) |
|
||||
| Completable (no LAN) | `./scripts/run-completable-tasks-from-anywhere.sh` |
|
||||
| Operator (LAN + secrets) | `./scripts/run-all-operator-tasks-from-lan.sh` (use `--skip-backup` if `NPM_PASSWORD` unset) |
|
||||
| Cloudflare bulk DNS → `PUBLIC_IP` | `./scripts/update-all-dns-to-public-ip.sh` — use **`--dry-run`** and **`--zone-only=sankofa.nexus`** (or `d-bis.org` / `mim4u.org` / `defi-oracle.io`) to limit scope; see script header |
|
||||
| Cloudflare bulk DNS → `PUBLIC_IP` | `./scripts/update-all-dns-to-public-ip.sh` — use **`--dry-run`** and **`--zone-only=sankofa.nexus`** (or `d-bis.org` / `mim4u.org` / `defi-oracle.io`) to limit scope; see script header. Prefer scoped **`CLOUDFLARE_API_TOKEN`** (see `.env.master.example`). |
|
||||
|
||||
## Rules of engagement
|
||||
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
**Run E2E (public profile recommended):** `./scripts/verify/verify-end-to-end-routing.sh --profile=public` (from LAN with DNS or use `E2E_USE_SYSTEM_RESOLVER=1` and `/etc/hosts` per [E2E_DNS_FROM_LAN_RUNBOOK.md](E2E_DNS_FROM_LAN_RUNBOOK.md)).
|
||||
**Run E2E (private/admin):** `./scripts/verify/verify-end-to-end-routing.sh --profile=private`.
|
||||
|
||||
**Latest verified public pass:** `2026-03-26` via `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` with report at [verification_report.md](verification-evidence/e2e-verification-20260326_115013/verification_report.md). Result: exit `0`, `DNS passed: 37`, `Failed: 0`, `HTTPS passed: 22`.
|
||||
**Latest verified private/admin pass:** `2026-03-26` via `bash scripts/verify/verify-end-to-end-routing.sh --profile=private` with report at [verification_report.md](verification-evidence/e2e-verification-20260326_120939/verification_report.md). Result: exit `0`, `DNS passed: 4`, `Failed: 0`.
|
||||
**Latest verified public pass:** `2026-03-27` via `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` with report at [verification_report.md](verification-evidence/e2e-verification-20260327_120814/verification_report.md). Result: exit `0`, `DNS passed: 38`, `Failed: 0`, `HTTPS passed: 23` (includes `www.*` canonical redirect + `Location` checks).
|
||||
**Latest verified private/admin pass:** `2026-03-27` via `bash scripts/verify/verify-end-to-end-routing.sh --profile=private` with report at [verification_report.md](verification-evidence/e2e-verification-20260327_122148/verification_report.md). Result: exit `0`, `DNS passed: 4`, `Failed: 0`.
|
||||
|
||||
## Verification profiles
|
||||
|
||||
@@ -163,7 +163,7 @@ When running from outside LAN or when backends are down, the following endpoints
|
||||
| mifos.d-bis.org | 502 — Mifos (VMID 5800) unreachable from public |
|
||||
| mim4u.org, www.mim4u.org, secure.mim4u.org, training.mim4u.org | 502 — MIM4U web backends (192.168.11.37:80); non-blocking for contract/pool |
|
||||
| studio.sankofa.nexus | Historically 404 when the proxy misses `/studio/` or backend `192.168.11.72:8000`; verifier checks `/studio/`. Passed on 2026-03-26 after the NPMplus host update |
|
||||
| phoenix.sankofa.nexus, www.phoenix.sankofa.nexus | (Resolved in verifier) Phoenix API (7800) is API-first; `verify-end-to-end-routing.sh` checks `https://…/health` (200), not `/` |
|
||||
| phoenix.sankofa.nexus, www.phoenix.sankofa.nexus | (Resolved in verifier) Phoenix API (7800) is API-first; `verify-end-to-end-routing.sh` checks `https://…/health` (200), not `/`. A separate **marketing** site on the apex hostname (if desired) needs another upstream or app routes—NPM still points `phoenix.sankofa.nexus` at the Fastify API today. |
|
||||
| the-order.sankofa.nexus | 502 when NPM still points at empty **10210** / **10090**. `update-npmplus-proxy-hosts-api.sh` defaults **THE_ORDER_UPSTREAM_IP/PORT** to the Sankofa portal (7801) until you set `THE_ORDER_UPSTREAM_IP=192.168.11.39` and `THE_ORDER_UPSTREAM_PORT=80` once order-haproxy serves. Passed on 2026-03-26 with the interim portal target |
|
||||
|
||||
**Verifier behavior (2026-03):** `openssl s_client` is wrapped with `timeout` (`E2E_OPENSSL_TIMEOUT` default 15s, `E2E_OPENSSL_X509_TIMEOUT` default 5s) so `--profile=private` / `--profile=all` cannot hang. **`--profile=all`** merges private and public `E2E_OPTIONAL_WHEN_FAIL` lists for temporary regressions. Install **`wscat`** (`npm install -g wscat`) for full WSS JSON-RPC checks; the script uses `wscat -n` to match `curl -k`, and now treats a clean `wscat` exit as a successful full WebSocket check even when the tool prints no JSON output.
|
||||
@@ -180,4 +180,4 @@ When running from outside LAN or when backends are down, the following endpoints
|
||||
|------|--------|
|
||||
| **502s (dbis-admin, dbis-api, secure, mifos)** | From LAN: `./scripts/maintenance/address-all-remaining-502s.sh [--run-besu-fix] [--e2e]` or `./scripts/maintenance/run-all-maintenance-via-proxmox-ssh.sh --e2e`. If NPMplus API is unreachable: `./scripts/maintenance/fix-npmplus-services-via-proxmox-ssh.sh`. Runbook: [502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md](../00-meta/502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md). |
|
||||
| **404 studio.sankofa.nexus** | Ensure backend (VMID 7805, 192.168.11.72:8000) is up and NPMplus proxy for `studio.sankofa.nexus` points to it. See [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md), [SANKOFA_STUDIO_E2E_FLOW.md](../03-deployment/SANKOFA_STUDIO_E2E_FLOW.md), [SANKOFA_STUDIO_DEPLOYMENT.md](../03-deployment/SANKOFA_STUDIO_DEPLOYMENT.md). |
|
||||
| **the-order 502** | From LAN with `.env`: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh` (interim upstream = portal). When Order HAProxy is live: `THE_ORDER_UPSTREAM_IP=192.168.11.39 THE_ORDER_UPSTREAM_PORT=80` for that run. |
|
||||
| **the-order 502** | From LAN with `.env`: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh` (interim upstream = portal). When **order-haproxy** (VMID 10210, `192.168.11.39:80`) answers HTTP, switch: `THE_ORDER_UPSTREAM_IP=192.168.11.39 THE_ORDER_UPSTREAM_PORT=80` for that run. If `:80` does not connect from LAN, keep portal upstream until HAProxy is serving. |
|
||||
|
||||
@@ -131,6 +131,30 @@ resolve_proxy_host_id() {
|
||||
.id' 2>/dev/null | head -n1
|
||||
}
|
||||
|
||||
# www → apex redirect: only https://hostname[:port] (no path/query); rejects characters that could break nginx advanced_config.
|
||||
validate_canonical_https_redirect() {
|
||||
local url="$1"
|
||||
local ctx="${2:-canonical_https}"
|
||||
if [[ "$url" != https://* ]]; then
|
||||
echo " ❌ $ctx: canonical_https must start with https:// (got: $url)"
|
||||
return 1
|
||||
fi
|
||||
if [[ "$url" == *$'\n'* || "$url" == *$'\r'* || "$url" == *' '* || "$url" == *';'* || "$url" == *'$'* || "$url" == *'`'* ]]; then
|
||||
echo " ❌ $ctx: canonical_https contains forbidden characters (no spaces, semicolons, dollar, backticks)"
|
||||
return 1
|
||||
fi
|
||||
local rest="${url#https://}"
|
||||
if [[ "$rest" == */* ]]; then
|
||||
echo " ❌ $ctx: canonical_https must not include a path (got: $url)"
|
||||
return 1
|
||||
fi
|
||||
if ! [[ "$rest" =~ ^[a-zA-Z0-9._-]+(:[0-9]{1,5})?$ ]]; then
|
||||
echo " ❌ $ctx: canonical_https must be https://hostname or https://hostname:port (got: $url)"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# Function to add proxy host (POST) when domain does not exist
|
||||
# Optional 6th arg: canonical HTTPS apex for www-style hosts (sets advanced_config 301 → apex$request_uri)
|
||||
add_proxy_host() {
|
||||
@@ -141,6 +165,9 @@ add_proxy_host() {
|
||||
local block_exploits=${5:-false}
|
||||
local canonical_https="${6:-}"
|
||||
local adv_line=""
|
||||
if [ -n "$canonical_https" ] && ! validate_canonical_https_redirect "$canonical_https" "add_proxy_host($domain)"; then
|
||||
return 1
|
||||
fi
|
||||
if [ -n "$canonical_https" ]; then
|
||||
adv_line="return 301 ${canonical_https}\$request_uri;"
|
||||
fi
|
||||
@@ -174,7 +201,11 @@ add_proxy_host() {
|
||||
local id
|
||||
id=$(echo "$resp" | jq -r '.id // empty' 2>/dev/null)
|
||||
if [ -n "$id" ] && [ "$id" != "null" ]; then
|
||||
echo " ✅ Added: $domain -> http://${forward_host}:${forward_port} (WebSocket: $websocket)"
|
||||
if [ -n "$canonical_https" ]; then
|
||||
echo " ✅ Added: $domain -> http://${forward_host}:${forward_port} (WebSocket: $websocket) + 301 → ${canonical_https}\$request_uri"
|
||||
else
|
||||
echo " ✅ Added: $domain -> http://${forward_host}:${forward_port} (WebSocket: $websocket)"
|
||||
fi
|
||||
return 0
|
||||
else
|
||||
local err
|
||||
@@ -202,7 +233,10 @@ update_proxy_host() {
|
||||
local websocket=$3
|
||||
local block_exploits=${4:-true}
|
||||
local canonical_https="${5:-}"
|
||||
|
||||
if [ -n "$canonical_https" ] && ! validate_canonical_https_redirect "$canonical_https" "update_proxy_host($domain)"; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Parse target URL
|
||||
local scheme=$(echo "$target" | sed -E 's|^([^:]+):.*|\1|')
|
||||
local hostname=$(echo "$target" | sed -E 's|^[^/]+//([^:]+):.*|\1|')
|
||||
@@ -367,7 +401,8 @@ update_proxy_host "www.the-order.sankofa.nexus" "http://${THE_ORDER_UPSTREAM_IP}
|
||||
# Sankofa Studio (FusionAI) — VMID 7805; UI at /studio/ on same origin (port 8000). Prefer IP_SANKOFA_STUDIO from ip-addresses.conf / .env
|
||||
IP_SANKOFA_STUDIO="${IP_SANKOFA_STUDIO:-192.168.11.72}"
|
||||
SANKOFA_STUDIO_PORT="${SANKOFA_STUDIO_PORT:-8000}"
|
||||
update_proxy_host "studio.sankofa.nexus" "http://${IP_SANKOFA_STUDIO}:${SANKOFA_STUDIO_PORT}" false && updated_count=$((updated_count + 1)) || { add_proxy_host "studio.sankofa.nexus" "${IP_SANKOFA_STUDIO}" "${SANKOFA_STUDIO_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
|
||||
# block_exploits false — studio UI/API may POST; align with portal policy (avoid spurious 405 from NPM WAF)
|
||||
update_proxy_host "studio.sankofa.nexus" "http://${IP_SANKOFA_STUDIO}:${SANKOFA_STUDIO_PORT}" false false && updated_count=$((updated_count + 1)) || { add_proxy_host "studio.sankofa.nexus" "${IP_SANKOFA_STUDIO}" "${SANKOFA_STUDIO_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
|
||||
|
||||
echo ""
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
Reference in New Issue
Block a user