d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

NPM: canonical 301 for www sankofa/phoenix/the-order; E2E pass on 301/308

Browse Source 
- update-npmplus-proxy-hosts-api.sh: optional advanced_config 301 via 5th/6th args; wire www.the-order → https://the-order.sankofa.nexus; document OSJ portal and the_order repo path
- update-sankofa-npmplus-proxy-hosts.sh: same 301 for www rows via 4th pipe field
- verify-end-to-end-routing.sh: www.the-order in inventory; treat 301/308 as HTTPS pass for www.sankofa, www.phoenix, www.the-order
- configure-npmplus-domains.js: comment — avoid duplicate redirection UI rows for Sankofa www
- AGENTS.md, ALL_VMIDS_ENDPOINTS.md, E2E_ENDPOINTS_LIST.md: Order portal and www redirect notes

Made-with: Cursor
This commit is contained in:
defiQUG
2026-03-27 00:30:28 -07:00
parent b9d3c10d01
commit a36ccbbd77
7 changed files with 234 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
scripts/nginx-proxy-manager/configure-npmplus-domains.js
View File

@@ -69,9 +69,8 @@ const DOMAINS = [
// www.* domains that redirect to parent domains
const REDIRECT_DOMAINS = [
// REMOVED: Sankofa redirects - services not deployed
// { domain: 'www.sankofa.nexus', redirectTo: 'sankofa.nexus' },
// { domain: 'www.phoenix.sankofa.nexus', redirectTo: 'phoenix.sankofa.nexus' },
// Sankofa www → apex: use scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh (301 via proxy host advanced_config).
// Do not add duplicate NPM "Redirection Host" rows for www.sankofa / www.phoenix here while those names are proxy hosts with LE certs.
{ domain: 'www.mim4u.org', redirectTo: 'mim4u.org' },
];

88
scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh
View File

@@ -3,21 +3,17 @@
# Auth failures: only a short error message is printed by default. For a redacted JSON snippet set NPM_DEBUG_AUTH=1.
set -euo pipefail
# Load IP configuration
# Repo root (…/proxmox) — same as second block below; load IPs once from the right path
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
source "${PROJECT_ROOT}/config/ip-addresses.conf" 2>/dev/null || true
# Update existing NPMplus proxy hosts via API with correct VMIDs and IPs
# This script updates existing proxy hosts, not creates new ones.
# PUT payload includes only forward_* / websocket / block_exploits — existing certificate_id and ssl_forced are preserved by NPMplus.
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Preserve NPM credentials from environment so "export NPM_PASSWORD=...; ./script" works
_orig_npm_url="${NPM_URL:-}"
_orig_npm_email="${NPM_EMAIL:-}"
@@ -58,11 +54,12 @@ echo ""
# NPMplus API can stall indefinitely without --max-time (override e.g. NPM_CURL_MAX_TIME=300)
NPM_CURL_MAX_TIME="${NPM_CURL_MAX_TIME:-120}"
curl_npm() { curl -s -k --connect-timeout 10 --max-time "$NPM_CURL_MAX_TIME" "$@"; }
# -L: port 81 often 301s HTTP→HTTPS; POST /api/tokens without -L returns 400 "Payload is undefined"
curl_npm() { curl -s -k -L --connect-timeout 10 --max-time "$NPM_CURL_MAX_TIME" "$@"; }
# Connection check (NPMplus is on LAN 192.168.11.x). Try HTTP if HTTPS fails; try alternate IP .166/.167 if unreachable.
echo "🔐 Authenticating to NPMplus..."
try_connect() { curl -s -k -o /dev/null --connect-timeout 5 --max-time 15 "$1" 2>/dev/null; }
try_connect() { curl -s -k -L -o /dev/null --connect-timeout 5 --max-time 15 "$1" 2>/dev/null; }
if ! try_connect "$NPM_URL/"; then
# Try HTTP instead of HTTPS (NPM admin often listens on HTTP only on port 81)
http_url="${NPM_URL/https:/http:}"
@@ -73,7 +70,7 @@ if ! try_connect "$NPM_URL/"; then
alt_url=""
if [[ "$NPM_URL" == *"${IP_NPMPLUS_ETH0}"* ]]; then
alt_url="https://${IP_NPMPLUS}:81"
elif [[ "$NPM_URL" == *"${IP_NPMPLUS}"* ]] || [[ "$NPM_URL" == *"${IP_NPMPLUS_ETH1}"* ]]; then
elif [[ "$NPM_URL" == *"${IP_NPMPLUS}"* ]] || [[ -n "${IP_NPMPLUS_ETH1:-}" && "$NPM_URL" == *"${IP_NPMPLUS_ETH1}"* ]]; then
alt_url="https://${IP_NPMPLUS_ETH0}:81"
fi
connected=""
@@ -135,12 +132,18 @@ resolve_proxy_host_id() {
}
# Function to add proxy host (POST) when domain does not exist
# Optional 6th arg: canonical HTTPS apex for www-style hosts (sets advanced_config 301 → apex$request_uri)
add_proxy_host() {
local domain=$1
local forward_host=$2
local forward_port=$3
local websocket=$4
local block_exploits=${5:-false}
local canonical_https="${6:-}"
local adv_line=""
if [ -n "$canonical_https" ]; then
adv_line="return 301 ${canonical_https}\$request_uri;"
fi
local payload
payload=$(jq -n \
--arg domain "$domain" \
@@ -148,6 +151,7 @@ add_proxy_host() {
--argjson port "$forward_port" \
--argjson ws "$websocket" \
--argjson block_exploits "$([ "$block_exploits" = "true" ] && echo true || echo false)" \
--arg adv "$adv_line" \
'{
domain_names: [$domain],
forward_scheme: "http",
@@ -157,7 +161,7 @@ add_proxy_host() {
block_exploits: $block_exploits,
certificate_id: null,
ssl_forced: false
}' 2>/dev/null)
} + (if $adv != "" then {advanced_config: $adv} else {} end)' 2>/dev/null)
if [ -z "$payload" ]; then
echo " ❌ Failed to build payload for $domain"
return 1
@@ -180,7 +184,7 @@ add_proxy_host() {
echo " ↪ Host likely exists; refreshing list and attempting PUT update..."
PROXY_HOSTS_JSON=$(curl_npm -X GET "$NPM_URL/api/nginx/proxy-hosts" \
-H "Authorization: Bearer $TOKEN")
if update_proxy_host "$domain" "http://${forward_host}:${forward_port}" "$websocket" "$block_exploits"; then
if update_proxy_host "$domain" "http://${forward_host}:${forward_port}" "$websocket" "$block_exploits" "$canonical_https"; then
echo " ✅ Updated after duplicate-create error: $domain"
return 0
fi
@@ -191,11 +195,13 @@ add_proxy_host() {
# Function to update proxy host
# block_exploits: set false for RPC hosts (JSON-RPC uses POST to /; block_exploits can cause 405)
# Optional 5th arg: canonical HTTPS URL (no path) — sets advanced_config to 301 redirect (www → apex)
update_proxy_host() {
local domain=$1
local target=$2
local websocket=$3
local block_exploits=${4:-true}
local canonical_https="${5:-}"
# Parse target URL
local scheme=$(echo "$target" | sed -E 's|^([^:]+):.*|\1|')
@@ -208,6 +214,17 @@ update_proxy_host() {
hostname=$(echo "$target" | sed -E 's|^https://([^:]+):.*|\1|')
port=$(echo "$target" | sed -E 's|^https://[^:]+:([0-9]+).*|\1|' || echo "443")
fi
# Reject bad parses (e.g. https://:443 when forward IP env is empty) — NPM returns errors without .id and jq message is empty.
if [[ -z "$hostname" || "$hostname" == *"://"* || "$hostname" == *"/"* ]]; then
echo " ❌ Invalid forward target for $domain (check env / ip-addresses.conf): $target → host=[$hostname]"
return 1
fi
port="${port//[^0-9]/}"
if [[ -z "$port" || "$port" -lt 1 || "$port" -gt 65535 ]]; then
echo " ❌ Invalid forward port for $domain: $target (parsed port=$port)"
return 1
fi
# Get host ID (case-insensitive); refresh once if missing (stale list / race with other writers)
HOST_ID=$(resolve_proxy_host_id "$domain" "$PROXY_HOSTS_JSON")
@@ -228,19 +245,24 @@ update_proxy_host() {
# block_exploits must be false for RPC so POST to / is allowed (JSON-RPC); explicit false fixes 405
local be_json="false"
[ "$block_exploits" = "true" ] && be_json="true"
local adv_line=""
if [ -n "$canonical_https" ]; then
adv_line="return 301 ${canonical_https}\$request_uri;"
fi
UPDATE_PAYLOAD=$(jq -n \
--arg scheme "$scheme" \
--arg hostname "$hostname" \
--argjson port "$(echo "$port" | sed 's/[^0-9]//g' || echo "80")" \
--argjson websocket "$websocket" \
--argjson block_exploits "$be_json" \
--arg adv "$adv_line" \
'{
forward_scheme: $scheme,
forward_host: $hostname,
forward_port: $port,
allow_websocket_upgrade: $websocket,
block_exploits: $block_exploits
}' 2>/dev/null || echo "")
} + (if $adv != "" then {advanced_config: $adv} else {} end)' 2>/dev/null || echo "")
UPDATE_RESPONSE=$(curl_npm -X PUT "$NPM_URL/api/nginx/proxy-hosts/$HOST_ID" \
-H "Authorization: Bearer $TOKEN" \
@@ -250,10 +272,16 @@ update_proxy_host() {
UPDATE_ID=$(echo "$UPDATE_RESPONSE" | jq -r '.id // empty' 2>/dev/null || echo "")
if [ -n "$UPDATE_ID" ] && [ "$UPDATE_ID" != "null" ]; then
echo " ✅ Updated: $scheme://$hostname:$port (WebSocket: $websocket)"
if [ -n "$canonical_https" ]; then
echo " ✅ Updated: $scheme://$hostname:$port (WebSocket: $websocket) + 301 → ${canonical_https}\$request_uri"
else
echo " ✅ Updated: $scheme://$hostname:$port (WebSocket: $websocket)"
fi
return 0
else
ERROR=$(echo "$UPDATE_RESPONSE" | jq -r '.error.message // .error // "Unknown error"' 2>/dev/null || echo "$UPDATE_RESPONSE")
ERROR=$(echo "$UPDATE_RESPONSE" | jq -r '.error.message // .message // .error // empty' 2>/dev/null || echo "")
[ -z "$ERROR" ] && ERROR=$(echo "$UPDATE_RESPONSE" | head -c 400 | tr -d '\r\n')
[ -z "$ERROR" ] && ERROR="(empty API response — timeout or connection error; try NPM_CURL_MAX_TIME=300)"
echo " ❌ Failed: $ERROR"
return 1
fi
@@ -280,7 +308,9 @@ update_proxy_host "wss.tw-core.d-bis.org" "http://${RPC_THIRDWEB_ADMIN_CORE}:854
# Catch-all for foo.tw-core.d-bis.org → Besu HTTP JSON-RPC :8545 (exact rpc./wss. hosts above take precedence for nginx server_name)
update_proxy_host '*.tw-core.d-bis.org' "http://${RPC_THIRDWEB_ADMIN_CORE}:8545" true false && updated_count=$((updated_count + 1)) || { add_proxy_host '*.tw-core.d-bis.org' "${RPC_THIRDWEB_ADMIN_CORE}" 8545 true false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
# RPC Core-2 (Nathan) is on the THIRD NPMplus (192.168.11.169) — use add-rpc-core-2-npmplus-proxy.sh and update-npmplus-alltra-hybx-proxy-hosts.sh
update_proxy_host "rpc.public-0138.defi-oracle.io" "https://${RPC_THIRDWEB_PRIMARY}:443" true false && updated_count=$((updated_count + 1)) || failed_count=$((failed_count + 1))
# ThirdWeb / public-0138 edge (VMID 2400 nginx HTTPS) — default IP must match ALL_VMIDS_ENDPOINTS if env is unset
RPC_THIRDWEB_PRIMARY="${RPC_THIRDWEB_PRIMARY:-192.168.11.240}"
update_proxy_host "rpc.public-0138.defi-oracle.io" "https://${RPC_THIRDWEB_PRIMARY}:443" true false && updated_count=$((updated_count + 1)) || { sleep 2; echo " ↪ Retry rpc.public-0138.defi-oracle.io after transient NPM/API error..."; update_proxy_host "rpc.public-0138.defi-oracle.io" "https://${RPC_THIRDWEB_PRIMARY}:443" true false && updated_count=$((updated_count + 1)) || failed_count=$((failed_count + 1)); }
# rpc.defi-oracle.io / wss.defi-oracle.io → same backend as rpc-http-pub / rpc-ws-pub (VMID 2201)
update_proxy_host "rpc.defi-oracle.io" "http://${RPC_PUBLIC_1}:8545" true false && updated_count=$((updated_count + 1)) || { add_proxy_host "rpc.defi-oracle.io" "${RPC_PUBLIC_1}" 8545 true false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "wss.defi-oracle.io" "http://${RPC_PUBLIC_1}:8546" true false && updated_count=$((updated_count + 1)) || { add_proxy_host "wss.defi-oracle.io" "${RPC_PUBLIC_1}" 8546 true false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
@@ -309,6 +339,34 @@ update_proxy_host "dbis.xom-dev.phoenix.sankofa.nexus" "http://${IP_GOV_PORTALS_
update_proxy_host "iccc.xom-dev.phoenix.sankofa.nexus" "http://${IP_GOV_PORTALS_DEV}:3002" false && updated_count=$((updated_count + 1)) || { add_proxy_host "iccc.xom-dev.phoenix.sankofa.nexus" "${IP_GOV_PORTALS_DEV}" 3002 false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "omnl.xom-dev.phoenix.sankofa.nexus" "http://${IP_GOV_PORTALS_DEV}:3003" false && updated_count=$((updated_count + 1)) || { add_proxy_host "omnl.xom-dev.phoenix.sankofa.nexus" "${IP_GOV_PORTALS_DEV}" 3003 false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "xom.xom-dev.phoenix.sankofa.nexus" "http://${IP_GOV_PORTALS_DEV}:3004" false && updated_count=$((updated_count + 1)) || { add_proxy_host "xom.xom-dev.phoenix.sankofa.nexus" "${IP_GOV_PORTALS_DEV}" 3004 false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
# Sankofa portal (Next.js CT 7801) and Phoenix API (Fastify CT 7800) — not Blockscout / SolaceScanScout (that is explorer.d-bis.org / IP_BLOCKSCOUT:80)
# Public URL policy: https://sankofa.nexus = sovereign technology utility (portal); https://phoenix.sankofa.nexus = Phoenix division (API host; marketing site may share hostname later).
# www.sankofa.nexus → 301 https://sankofa.nexus$request_uri; www.phoenix → phoenix; www.the-order → the-order (NPM advanced_config).
IP_SANKOFA_PORTAL="${IP_SANKOFA_PORTAL:-${IP_SERVICE_51:-192.168.11.51}}"
IP_SANKOFA_PHOENIX_API="${IP_SANKOFA_PHOENIX_API:-${IP_SERVICE_50:-192.168.11.50}}"
SANKOFA_PORTAL_PORT="${SANKOFA_PORTAL_PORT:-3000}"
SANKOFA_PHOENIX_API_PORT="${SANKOFA_PHOENIX_API_PORT:-4000}"
update_proxy_host "sankofa.nexus" "http://${IP_SANKOFA_PORTAL}:${SANKOFA_PORTAL_PORT}" false false && updated_count=$((updated_count + 1)) || { add_proxy_host "sankofa.nexus" "${IP_SANKOFA_PORTAL}" "${SANKOFA_PORTAL_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "www.sankofa.nexus" "http://${IP_SANKOFA_PORTAL}:${SANKOFA_PORTAL_PORT}" false false "https://sankofa.nexus" && updated_count=$((updated_count + 1)) || { add_proxy_host "www.sankofa.nexus" "${IP_SANKOFA_PORTAL}" "${SANKOFA_PORTAL_PORT}" false false "https://sankofa.nexus" && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "phoenix.sankofa.nexus" "http://${IP_SANKOFA_PHOENIX_API}:${SANKOFA_PHOENIX_API_PORT}" false false && updated_count=$((updated_count + 1)) || { add_proxy_host "phoenix.sankofa.nexus" "${IP_SANKOFA_PHOENIX_API}" "${SANKOFA_PHOENIX_API_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "www.phoenix.sankofa.nexus" "http://${IP_SANKOFA_PHOENIX_API}:${SANKOFA_PHOENIX_API_PORT}" false false "https://phoenix.sankofa.nexus" && updated_count=$((updated_count + 1)) || { add_proxy_host "www.phoenix.sankofa.nexus" "${IP_SANKOFA_PHOENIX_API}" "${SANKOFA_PHOENIX_API_PORT}" false false "https://phoenix.sankofa.nexus" && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
# Keycloak (CT 7802) — portal SSO; NPM must forward X-Forwarded-* (Keycloak KC_PROXY_HEADERS=xforwarded on upstream)
IP_KEYCLOAK="${IP_KEYCLOAK:-192.168.11.52}"
update_proxy_host "keycloak.sankofa.nexus" "http://${IP_KEYCLOAK}:8080" false false && updated_count=$((updated_count + 1)) || { add_proxy_host "keycloak.sankofa.nexus" "${IP_KEYCLOAK}" 8080 false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
# the-order.sankofa.nexus — public hostname for the Sovereign Military Order of Malta (OSJ) management portal (secure auth).
# Application source (operator workstation): repo the_order at ~/projects/the_order (e.g. /home/intlc/projects/the_order).
# Ideal upstream: VMID 10210 order-haproxy @ IP_ORDER_HAPROXY:80. When HAProxy/order edge is not serving, NPM may 502.
# Interim default: same Next.js upstream as sankofa.nexus (7801). Switch: THE_ORDER_UPSTREAM_IP=192.168.11.39 THE_ORDER_UPSTREAM_PORT=80.
# www.the-order.sankofa.nexus → 301 https://the-order.sankofa.nexus$request_uri (same pattern as www.sankofa / www.phoenix).
IP_ORDER_HAPROXY="${IP_ORDER_HAPROXY:-192.168.11.39}"
THE_ORDER_UPSTREAM_IP="${THE_ORDER_UPSTREAM_IP:-${IP_SANKOFA_PORTAL}}"
THE_ORDER_UPSTREAM_PORT="${THE_ORDER_UPSTREAM_PORT:-${SANKOFA_PORTAL_PORT}}"
update_proxy_host "the-order.sankofa.nexus" "http://${THE_ORDER_UPSTREAM_IP}:${THE_ORDER_UPSTREAM_PORT}" false && updated_count=$((updated_count + 1)) || { add_proxy_host "the-order.sankofa.nexus" "${THE_ORDER_UPSTREAM_IP}" "${THE_ORDER_UPSTREAM_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
update_proxy_host "www.the-order.sankofa.nexus" "http://${THE_ORDER_UPSTREAM_IP}:${THE_ORDER_UPSTREAM_PORT}" false false "https://the-order.sankofa.nexus" && updated_count=$((updated_count + 1)) || { add_proxy_host "www.the-order.sankofa.nexus" "${THE_ORDER_UPSTREAM_IP}" "${THE_ORDER_UPSTREAM_PORT}" false false "https://the-order.sankofa.nexus" && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
# Sankofa Studio (FusionAI) — VMID 7805; UI at /studio/ on same origin (port 8000). Prefer IP_SANKOFA_STUDIO from ip-addresses.conf / .env
IP_SANKOFA_STUDIO="${IP_SANKOFA_STUDIO:-192.168.11.72}"
SANKOFA_STUDIO_PORT="${SANKOFA_STUDIO_PORT:-8000}"
update_proxy_host "studio.sankofa.nexus" "http://${IP_SANKOFA_STUDIO}:${SANKOFA_STUDIO_PORT}" false && updated_count=$((updated_count + 1)) || { add_proxy_host "studio.sankofa.nexus" "${IP_SANKOFA_STUDIO}" "${SANKOFA_STUDIO_PORT}" false false && updated_count=$((updated_count + 1)); } || failed_count=$((failed_count + 1))
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

78
scripts/update-sankofa-npmplus-proxy-hosts.sh
View File

@@ -1,34 +1,45 @@
#!/usr/bin/env bash
set -euo pipefail
# Load IP configuration
# Update Sankofa NPMplus proxy hosts (portal + Phoenix API) via API.
# Prefer: scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh (domain-based, runs with operator waves).
# NPM proxy host IDs below match backup backup-20260325_183932 (3–6); if your DB differs, use the main script instead.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
# shellcheck source=/dev/null
source "${PROJECT_ROOT}/config/ip-addresses.conf" 2>/dev/null || true
# Update Sankofa NPMplus proxy hosts via API
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Load environment variables
if [ -f "$PROJECT_ROOT/.env" ]; then
export $(cat "$PROJECT_ROOT/.env" | grep -v '^#' | xargs)
set -a
# shellcheck source=/dev/null
source "$PROJECT_ROOT/.env"
set +a
fi
NPM_URL="${NPM_URL:-https://${IP_NPMPLUS}:81}"
NPM_EMAIL="${NPM_EMAIL:-nsatoshi2007@hotmail.com}"
NPM_PASSWORD="${NPM_PASSWORD:-}"
NPM_CURL_MAX_TIME="${NPM_CURL_MAX_TIME:-180}"
# Sankofa proxy host mappings
IP_SANKOFA_PORTAL="${IP_SANKOFA_PORTAL:-${IP_SERVICE_51:-192.168.11.51}}"
IP_SANKOFA_PHOENIX_API="${IP_SANKOFA_PHOENIX_API:-${IP_SERVICE_50:-192.168.11.50}}"
SANKOFA_PORTAL_PORT="${SANKOFA_PORTAL_PORT:-3000}"
SANKOFA_PHOENIX_API_PORT="${SANKOFA_PHOENIX_API_PORT:-4000}"
# NPM proxy host IDs: sankofa=3, www.sankofa=4, phoenix=5, www.phoenix=6 (typical fresh install order)
SANKOFA_NPM_ID_ROOT="${SANKOFA_NPM_ID_ROOT:-3}"
SANKOFA_NPM_ID_WWW="${SANKOFA_NPM_ID_WWW:-4}"
SANKOFA_NPM_ID_PHOENIX="${SANKOFA_NPM_ID_PHOENIX:-5}"
SANKOFA_NPM_ID_WWW_PHOENIX="${SANKOFA_NPM_ID_WWW_PHOENIX:-6}"
# Optional 4th field: canonical HTTPS apex — NPM advanced_config 301 (www → apex). Matches update-npmplus-proxy-hosts-api.sh.
declare -A PROXY_HOSTS=(
["21"]="sankofa.nexus|${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-192.168.11.51}}}}}}|3000"
["22"]="www.sankofa.nexus|${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-${IP_SERVICE_51:-192.168.11.51}}}}}}|3000"
["23"]="phoenix.sankofa.nexus|${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-192.168.11.50}}}}}}|4000"
["24"]="www.phoenix.sankofa.nexus|${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-${IP_SERVICE_50:-192.168.11.50}}}}}}|4000"
["$SANKOFA_NPM_ID_ROOT"]="sankofa.nexus|${IP_SANKOFA_PORTAL}|${SANKOFA_PORTAL_PORT}|"
["$SANKOFA_NPM_ID_WWW"]="www.sankofa.nexus|${IP_SANKOFA_PORTAL}|${SANKOFA_PORTAL_PORT}|https://sankofa.nexus"
["$SANKOFA_NPM_ID_PHOENIX"]="phoenix.sankofa.nexus|${IP_SANKOFA_PHOENIX_API}|${SANKOFA_PHOENIX_API_PORT}|"
["$SANKOFA_NPM_ID_WWW_PHOENIX"]="www.phoenix.sankofa.nexus|${IP_SANKOFA_PHOENIX_API}|${SANKOFA_PHOENIX_API_PORT}|https://phoenix.sankofa.nexus"
)
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
@@ -38,7 +49,7 @@ echo ""
# Authenticate
echo "🔐 Authenticating to NPMplus..."
TOKEN_RESPONSE=$(curl -s -k -X POST "$NPM_URL/api/tokens" \
TOKEN_RESPONSE=$(curl -s -k --connect-timeout 15 --max-time "$NPM_CURL_MAX_TIME" -X POST "$NPM_URL/api/tokens" \
-H "Content-Type: application/json" \
-d "{\"identity\":\"$NPM_EMAIL\",\"secret\":\"$NPM_PASSWORD\"}")
@@ -58,11 +69,12 @@ update_proxy_host() {
local domain=$2
local target_ip=$3
local target_port=$4
local canonical_https="${5:-}"
echo "📝 Updating Proxy Host $host_id: $domain → $target_ip:$target_port"
# Get current proxy host
CURRENT_HOST=$(curl -s -k -X GET "$NPM_URL/api/nginx/proxy-hosts/$host_id" \
CURRENT_HOST=$(curl -s -k --connect-timeout 15 --max-time "$NPM_CURL_MAX_TIME" -X GET "$NPM_URL/api/nginx/proxy-hosts/$host_id" \
-H "Authorization: Bearer $TOKEN" 2>/dev/null || echo "{}")
if [ "$(echo "$CURRENT_HOST" | jq -r '.id // empty')" = "" ]; then
@@ -70,11 +82,28 @@ update_proxy_host() {
return 1
fi
# Update proxy host
UPDATE_PAYLOAD=$(echo "$CURRENT_HOST" | jq --arg ip "$target_ip" --arg port "$target_port" \
'.forward_host = $ip | .forward_port = ($port | tonumber)')
# NPMplus rejects full-document PUT (e.g. locations: null) — send only allowed forward fields.
local scheme="http"
local adv_line=""
if [ -n "$canonical_https" ]; then
adv_line="return 301 ${canonical_https}\$request_uri;"
fi
UPDATE_PAYLOAD=$(jq -n \
--arg scheme "$scheme" \
--arg hostname "$target_ip" \
--argjson port "$(echo "$target_port" | sed 's/[^0-9]//g')" \
--argjson websocket false \
--argjson block_exploits false \
--arg adv "$adv_line" \
'{
forward_scheme: $scheme,
forward_host: $hostname,
forward_port: $port,
allow_websocket_upgrade: $websocket,
block_exploits: $block_exploits
} + (if $adv != "" then {advanced_config: $adv} else {} end)')
RESPONSE=$(curl -s -k -X PUT "$NPM_URL/api/nginx/proxy-hosts/$host_id" \
RESPONSE=$(curl -s -k --connect-timeout 15 --max-time "${NPM_CURL_MAX_TIME:-120}" -X PUT "$NPM_URL/api/nginx/proxy-hosts/$host_id" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "$UPDATE_PAYLOAD")
@@ -87,6 +116,9 @@ update_proxy_host() {
else
echo "❌ Failed to update proxy host $host_id"
echo "$RESPONSE" | jq '.' 2>/dev/null || echo "$RESPONSE"
if echo "$RESPONSE" | jq -e '.error.code == 403' >/dev/null 2>&1; then
echo " (403 often means NPM user lacks permission to mutate proxy hosts; check UI role or use an admin identity.)"
fi
return 1
fi
}
@@ -96,9 +128,9 @@ SUCCESS=0
FAILED=0
for host_id in "${!PROXY_HOSTS[@]}"; do
IFS='|' read -r domain target_ip target_port <<< "${PROXY_HOSTS[$host_id]}"
IFS='|' read -r domain target_ip target_port canonical_https _ <<< "${PROXY_HOSTS[$host_id]}"
if update_proxy_host "$host_id" "$domain" "$target_ip" "$target_port"; then
if update_proxy_host "$host_id" "$domain" "$target_ip" "$target_port" "$canonical_https"; then
((SUCCESS++))
else
((FAILED++))

65
scripts/verify/verify-end-to-end-routing.sh
View File

@@ -34,6 +34,9 @@ PUBLIC_IP_FOURTH="${PUBLIC_IP_FOURTH:-76.53.10.40}"
ACCEPT_ANY_DNS="${ACCEPT_ANY_DNS:-0}"
# Use system resolver (e.g. /etc/hosts) instead of dig @8.8.8.8 — set when running from LAN with generate-e2e-hosts.sh entries
E2E_USE_SYSTEM_RESOLVER="${E2E_USE_SYSTEM_RESOLVER:-0}"
# openssl s_client has no built-in connect timeout; wrap to avoid hangs (private/wss hosts).
E2E_OPENSSL_TIMEOUT="${E2E_OPENSSL_TIMEOUT:-15}"
E2E_OPENSSL_X509_TIMEOUT="${E2E_OPENSSL_X509_TIMEOUT:-5}"
if [ "$E2E_USE_SYSTEM_RESOLVER" = "1" ]; then
ACCEPT_ANY_DNS=1
log_info "E2E_USE_SYSTEM_RESOLVER=1: using getent (respects /etc/hosts); ACCEPT_ANY_DNS=1"
@@ -77,7 +80,8 @@ declare -A DOMAIN_TYPES_ALL=(
["www.sankofa.nexus"]="web"
["phoenix.sankofa.nexus"]="web"
["www.phoenix.sankofa.nexus"]="web"
["the-order.sankofa.nexus"]="web"
["the-order.sankofa.nexus"]="web" # OSJ portal (secure auth); app: ~/projects/the_order
["www.the-order.sankofa.nexus"]="web" # 301 → https://the-order.sankofa.nexus
["studio.sankofa.nexus"]="web"
["rpc.public-0138.defi-oracle.io"]="rpc-http"
["rpc.defi-oracle.io"]="rpc-http"
@@ -162,11 +166,15 @@ else
fi
# Domains that are optional when any test fails (off-LAN, 502, unreachable); fail → skip so run passes.
_PUB_OPTIONAL_WHEN_FAIL="dapp.d-bis.org mifos.d-bis.org explorer.d-bis.org dbis-admin.d-bis.org dbis-api.d-bis.org dbis-api-2.d-bis.org secure.d-bis.org sankofa.nexus www.sankofa.nexus phoenix.sankofa.nexus www.phoenix.sankofa.nexus the-order.sankofa.nexus www.the-order.sankofa.nexus studio.sankofa.nexus mim4u.org www.mim4u.org secure.mim4u.org training.mim4u.org rpc-http-pub.d-bis.org rpc.d-bis.org rpc2.d-bis.org rpc.public-0138.defi-oracle.io rpc.defi-oracle.io ws.rpc.d-bis.org ws.rpc2.d-bis.org"
_PRIV_OPTIONAL_WHEN_FAIL="rpc-http-prv.d-bis.org rpc-ws-prv.d-bis.org rpc-fireblocks.d-bis.org ws.rpc-fireblocks.d-bis.org"
if [[ -z "${E2E_OPTIONAL_WHEN_FAIL:-}" ]]; then
if [[ "$PROFILE" == "private" ]]; then
E2E_OPTIONAL_WHEN_FAIL="rpc-http-prv.d-bis.org rpc-ws-prv.d-bis.org rpc-fireblocks.d-bis.org ws.rpc-fireblocks.d-bis.org"
E2E_OPTIONAL_WHEN_FAIL="$_PRIV_OPTIONAL_WHEN_FAIL"
elif [[ "$PROFILE" == "all" ]]; then
E2E_OPTIONAL_WHEN_FAIL="$_PRIV_OPTIONAL_WHEN_FAIL $_PUB_OPTIONAL_WHEN_FAIL"
else
E2E_OPTIONAL_WHEN_FAIL="dapp.d-bis.org mifos.d-bis.org explorer.d-bis.org dbis-admin.d-bis.org dbis-api.d-bis.org dbis-api-2.d-bis.org secure.d-bis.org sankofa.nexus www.sankofa.nexus phoenix.sankofa.nexus www.phoenix.sankofa.nexus the-order.sankofa.nexus studio.sankofa.nexus mim4u.org www.mim4u.org secure.mim4u.org training.mim4u.org rpc-http-pub.d-bis.org rpc.d-bis.org rpc2.d-bis.org rpc.public-0138.defi-oracle.io rpc.defi-oracle.io ws.rpc.d-bis.org ws.rpc2.d-bis.org"
E2E_OPTIONAL_WHEN_FAIL="$_PUB_OPTIONAL_WHEN_FAIL"
fi
else
E2E_OPTIONAL_WHEN_FAIL="${E2E_OPTIONAL_WHEN_FAIL}"
@@ -178,6 +186,12 @@ declare -A EXPECTED_IP=(
["dev.d-bis.org"]="$PUBLIC_IP_FOURTH"
["codespaces.d-bis.org"]="$PUBLIC_IP_FOURTH"
)
# HTTPS check path (default "/"). API-first hosts may 404 on /; see docs/02-architecture/EXPECTED_WEB_CONTENT.md
declare -A E2E_HTTPS_PATH=(
["phoenix.sankofa.nexus"]="/health"
["www.phoenix.sankofa.nexus"]="/health"
["studio.sankofa.nexus"]="/studio/"
)
# --list-endpoints: print selected profile endpoints and exit (no tests)
if [[ "$LIST_ENDPOINTS" == "1" ]]; then
@@ -257,7 +271,7 @@ test_domain() {
if [ "$domain_type" != "unknown" ]; then
log_info "Test 2: SSL Certificate"
cert_info=$(echo | openssl s_client -connect "$domain:443" -servername "$domain" 2>/dev/null | openssl x509 -noout -subject -issuer -dates -ext subjectAltName 2>/dev/null || echo "")
cert_info=$( (echo | timeout "$E2E_OPENSSL_TIMEOUT" openssl s_client -connect "$domain:443" -servername "$domain" 2>/dev/null) | timeout "$E2E_OPENSSL_X509_TIMEOUT" openssl x509 -noout -subject -issuer -dates -ext subjectAltName 2>/dev/null || echo "")
if [ -n "$cert_info" ]; then
cert_cn=$(echo "$cert_info" | grep "subject=" | sed -E 's/.*CN\s*=\s*([^,]*).*/\1/' | sed 's/^ *//;s/ *$//' || echo "")
@@ -301,10 +315,12 @@ test_domain() {
# Test 3: HTTPS Request
if [ "$domain_type" = "web" ] || [ "$domain_type" = "api" ]; then
log_info "Test 3: HTTPS Request"
https_path="${E2E_HTTPS_PATH[$domain]:-}"
https_url="https://${domain}${https_path}"
log_info "Test 3: HTTPS Request (${https_url})"
START_TIME=$(date +%s.%N)
http_response=$(curl -s -I -k --connect-timeout 10 -w "\n%{time_total}" "https://$domain" 2>&1 || echo "")
http_response=$(curl -s -I -k --connect-timeout 10 -w "\n%{time_total}" "$https_url" 2>&1 || echo "")
END_TIME=$(date +%s.%N)
RESPONSE_TIME=$(echo "$END_TIME - $START_TIME" | bc 2>/dev/null || echo "0")
@@ -315,8 +331,22 @@ test_domain() {
echo "$headers" > "$OUTPUT_DIR/${domain//./_}_https_headers.txt"
if [ -n "$http_code" ]; then
if [ "$http_code" -ge 200 ] && [ "$http_code" -lt 400 ]; then
log_success "HTTPS: $domain returned HTTP $http_code (Time: ${time_total}s)"
# NPM canonical www → apex (advanced_config return 301/308)
_e2e_canonical_www_redirect=""
case "$domain" in
www.sankofa.nexus|www.phoenix.sankofa.nexus|www.the-order.sankofa.nexus)
if [ "$http_code" = "301" ] || [ "$http_code" = "308" ]; then
_e2e_canonical_www_redirect=1
fi
;;
esac
if [ -n "$_e2e_canonical_www_redirect" ]; then
location_hdr=$(echo "$headers" | grep -iE '^[Ll]ocation:' | head -1 | tr -d '\r' || echo "")
log_success "HTTPS: $domain returned HTTP $http_code (canonical redirect)${https_path:+ at ${https_url}}"
result=$(echo "$result" | jq --arg code "$http_code" --arg time "$time_total" --arg loc "$location_hdr" \
'.tests.https = {"status": "pass", "http_code": ($code | tonumber), "response_time_seconds": ($time | tonumber), "canonical_redirect": true, "location_header": $loc}')
elif [ "$http_code" -ge 200 ] && [ "$http_code" -lt 400 ]; then
log_success "HTTPS: $domain returned HTTP $http_code (Time: ${time_total}s)${https_path:+ at ${https_path}}"
# Check security headers
hsts=$(echo "$headers" | grep -i "strict-transport-security" || echo "")
@@ -330,12 +360,12 @@ test_domain() {
--argjson hsts "$HAS_HSTS" --argjson csp "$HAS_CSP" --argjson xfo "$HAS_XFO" \
'.tests.https = {"status": "pass", "http_code": ($code | tonumber), "response_time_seconds": ($time | tonumber), "has_hsts": $hsts, "has_csp": $csp, "has_xfo": $xfo}')
else
log_warn "HTTPS: $domain returned HTTP $http_code (Time: ${time_total}s)"
log_warn "HTTPS: $domain returned HTTP $http_code (Time: ${time_total}s)${https_path:+ (${https_url})}"
result=$(echo "$result" | jq --arg code "$http_code" --arg time "$time_total" \
'.tests.https = {"status": "warn", "http_code": ($code | tonumber), "response_time_seconds": ($time | tonumber)}')
fi
else
log_error "HTTPS: Failed to connect to $domain"
log_error "HTTPS: Failed to connect to ${https_url}"
result=$(echo "$result" | jq --arg time "$time_total" '.tests.https = {"status": "fail", "response_time_seconds": ($time | tonumber)}')
fi
# Optional: Blockscout API check for explorer.d-bis.org (does not affect E2E pass/fail)
@@ -401,13 +431,21 @@ test_domain() {
# Check if wscat is available for full test
if command -v wscat >/dev/null 2>&1; then
log_info " Attempting full WebSocket test with wscat..."
WS_FULL_TEST=$(timeout 3 wscat -c "wss://$domain" -x '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' 2>&1 || echo "")
# -n: no TLS verify (aligns with curl -k); -w: seconds to wait for JSON-RPC response
WS_FULL_TEST=""
WS_FULL_EXIT=0
if ! WS_FULL_TEST=$(timeout 15 wscat -n -c "wss://$domain" -x '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}' -w 5 2>&1); then
WS_FULL_EXIT=$?
fi
if echo "$WS_FULL_TEST" | grep -q "result"; then
log_success "WebSocket: Full test passed"
result=$(echo "$result" | jq --arg code "$WS_RESULT" '.tests.websocket = {"status": "pass", "http_code": $code, "full_test": true}')
result=$(echo "$result" | jq --arg code "$WS_RESULT" '.tests.websocket = {"status": "pass", "http_code": $code, "full_test": true, "full_test_output": "result"}')
elif [ "$WS_FULL_EXIT" -eq 0 ]; then
log_success "WebSocket: Full test connected cleanly"
result=$(echo "$result" | jq --arg code "$WS_RESULT" '.tests.websocket = {"status": "pass", "http_code": $code, "full_test": true, "note": "wscat exited successfully without printable RPC output"}')
else
log_warn "WebSocket: Connection established but RPC test failed"
result=$(echo "$result" | jq --arg code "$WS_RESULT" '.tests.websocket = {"status": "warning", "http_code": $code, "full_test": false}')
result=$(echo "$result" | jq --arg code "$WS_RESULT" --arg exit_code "$WS_FULL_EXIT" '.tests.websocket = {"status": "warning", "http_code": $code, "full_test": false, "exit_code": $exit_code}')
fi
else
log_warn "WebSocket: Basic test (Code: $WS_RESULT) - Install wscat for full test: npm install -g wscat"
@@ -558,6 +596,7 @@ cat >> "$REPORT_FILE" <<EOF
- **Optional domains:** Domains in \`E2E_OPTIONAL_WHEN_FAIL\` (default: many d-bis.org/sankofa/mim4u/rpc) have any fail treated as skip so the run passes when off-LAN or services unreachable. Set \`E2E_OPTIONAL_WHEN_FAIL=\` (empty) for strict mode.
- WebSocket tests require \`wscat\` tool: \`npm install -g wscat\`
- OpenSSL fetch uses \`timeout\` (\`E2E_OPENSSL_TIMEOUT\` / \`E2E_OPENSSL_X509_TIMEOUT\`, defaults 15s / 5s) so \`openssl s_client\` cannot hang indefinitely
- Internal connectivity tests require access to NPMplus container
- Explorer (explorer.d-bis.org): optional Blockscout API check; use \`SKIP_BLOCKSCOUT_API=1\` to skip when backend is unreachable (e.g. off-LAN). Fix runbook: docs/03-deployment/BLOCKSCOUT_FIX_RUNBOOK.md