diff --git a/docs/00-meta/OPERATOR_READY_CHECKLIST.md b/docs/00-meta/OPERATOR_READY_CHECKLIST.md index ddf536e..9cfeb41 100644 --- a/docs/00-meta/OPERATOR_READY_CHECKLIST.md +++ b/docs/00-meta/OPERATOR_READY_CHECKLIST.md @@ -1,6 +1,6 @@ # Operator Ready Checklist — Copy-Paste Commands -**Last Updated:** 2026-03-04 +**Last Updated:** 2026-03-27 **Purpose:** Single page with exact commands to complete every pending todo. Run from **repo root** on a host with **LAN** access (and `smom-dbis-138/.env` with `PRIVATE_KEY`, `NPM_PASSWORD` where noted). **Do you have all necessary creds?** See [OPERATOR_CREDENTIALS_CHECKLIST.md](OPERATOR_CREDENTIALS_CHECKLIST.md) — per-task list of LAN, PRIVATE_KEY, NPM_PASSWORD, RPC_URL_138, SSH, LINK, gas, token balance. @@ -15,6 +15,22 @@ --- +## Completed in this session (2026-03-26) + +| Item | Result | +|------|--------| +| NPMplus recovery | VMID `10233` was wedged on `192.168.11.167:81` (TCP connect, no HTTP). `pct reboot 10233` on `r630-01` restored the expected `301` response on port `81`. | +| NPMplus API updater | `NPM_URL=https://192.168.11.167:81 bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh` completed with **39 hosts updated, 0 failed**. | +| Sankofa / Order / Studio routing | **Superseded 2026-03-27:** Order hostnames default to **order-haproxy** `http://192.168.11.39:80` (10210 → `.51:3000`). Through 2026-03-26 NPM pointed Order directly at portal `:3000`. `studio.sankofa.nexus` → `http://192.168.11.72:8000`. | +| Public E2E | Latest run `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` exited `0` with **Failed: 0**, **DNS passed: 37**, **HTTPS passed: 22**. Sankofa, Phoenix, Studio, The Order, DBIS, Mifos, and MIM4U public endpoints passed. Evidence: `docs/04-configuration/verification-evidence/e2e-verification-20260326_115013/`. | +| Private E2E | Latest run `bash scripts/verify/verify-end-to-end-routing.sh --profile=private` exited `0` with **Failed: 0** and **DNS passed: 4**. `rpc-http-prv.d-bis.org`, `rpc-fireblocks.d-bis.org`, `rpc-ws-prv.d-bis.org`, and `ws.rpc-fireblocks.d-bis.org` all passed. Evidence: `docs/04-configuration/verification-evidence/e2e-verification-20260326_120939/`. | +| NPMplus backup | Fresh backup completed: `backups/npmplus/backup-20260326_115622.tar.gz`. API exports succeeded; direct SQLite file copy and certbot path copy were partial/warn-only, but the backup manifest and compressed bundle were created successfully. | +| Blockscout verification run | `./scripts/verify/run-contract-verification-with-proxy.sh` completed; contracts were submitted or skipped if already verified. `WETH10` returned `The address is not a smart contract`; others like `Multicall`, `Aggregator`, `Proxy`, `CCIPSender`, `CCIPWETH10Bridge`, and `CCIPWETH9Bridge` submitted successfully. | +| Private RPC redirect fix | `rpc-http-prv.d-bis.org` no longer returns HTTP `301` on JSON-RPC POST. Live NPMplus host `11` was updated to `ssl_forced=false` while preserving upstream `192.168.11.211:8545`. | +| NPM creds loading | For NPM-only runs, prefer targeted `grep` of `NPM_EMAIL` / `NPM_PASSWORD` if full `.env` export triggers `Argument list too long`. | + +--- + ## 1. High: Cronos closure + reachable CCIP funding **Ref:** [CONFIG_READY_CHAINS_COMPLETION_RUNBOOK](../07-ccip/CONFIG_READY_CHAINS_COMPLETION_RUNBOOK.md) @@ -84,6 +100,8 @@ Single contract retry: `./scripts/verify/run-contract-verification-with-proxy.sh **Runbook:** [502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md](502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md) +**Current status after 2026-03-26:** no public 502s reproduced in the latest public E2E run. Use this section only if those endpoints regress. + --- ## 5. LAN: Run all operator tasks (backup + verify ± deploy ± create-vms) @@ -211,8 +229,14 @@ bash scripts/verify/backup-npmplus.sh **NPMplus RPC fix (405):** From LAN: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh`. Verify: `bash scripts/verify/verify-end-to-end-routing.sh`. +**Status (2026-03-26):** main NPMplus API update completed successfully with `39 hosts updated, 0 failed`; public E2E now passes for Sankofa root, Phoenix, Studio, and The Order. Re-run only when upstream targets or proxy definitions change. + +**Latest backup evidence:** `backups/npmplus/backup-20260326_115622.tar.gz` + **NPMplus API unreachable (167/169):** Restart Docker inside NPMplus LXC: `./scripts/maintenance/fix-npmplus-services-via-proxmox-ssh.sh` (SSH to r630-01, restarts npmplus in 10233 and 10235). +**If port 81 accepts TCP but hangs at HTTP:** reboot CT `10233` with `pct reboot 10233` on `r630-01`, then retry the API updater. + **E2E from LAN (no public DNS):** If E2E fails at DNS (`Could not resolve host`), use [E2E_DNS_FROM_LAN_RUNBOOK.md](../04-configuration/E2E_DNS_FROM_LAN_RUNBOOK.md): append `config/e2e-hosts-append.txt` to `/etc/hosts`, then run `E2E_USE_SYSTEM_RESOLVER=1 ./scripts/verify/verify-end-to-end-routing.sh --profile=public`. Revert with `sudo ./scripts/verify/remove-e2e-hosts-from-etc-hosts.sh`. **E2E profiles:** Use `--profile=public` for public endpoints (default) or `--profile=private` for private/admin RPC only. Run sequentially to avoid timestamp collision in evidence dirs. **Known E2E warnings** (502/404 and WS): [E2E_ENDPOINTS_LIST.md](../04-configuration/E2E_ENDPOINTS_LIST.md) § Known E2E warnings and Remediation. MIM4U web 502s and WS test-format warnings are **non-blocking** for contract/pool completion. @@ -221,6 +245,25 @@ bash scripts/verify/backup-npmplus.sh --- +## 8.5 PMM mesh (6s oracle / keeper / PMM–WETH poll) + +**Ref:** `smom-dbis-138/docs/integration/ORACLE_AND_KEEPER_CHAIN138.md` (PMM mesh automation) + +```bash +cd smom-dbis-138 +# .env should include: PRIVATE_KEY, AGGREGATOR_ADDRESS, PRICE_FEED_KEEPER_ADDRESS (optional: KEEPER_PRIVATE_KEY if different from PRIVATE_KEY) +./scripts/reserve/set-price-feed-keeper-interval.sh 6 # once per keeper deployment if interval was 30s +./scripts/update-oracle-price.sh # verify transmitter + gas (Besu needs explicit gas limit in script) +./scripts/reserve/sync-weth-mock-price.sh # if CHAIN138_WETH_MOCK_PRICE_FEED is set (keeper WETH path) +mkdir -p logs +nohup ./scripts/reserve/pmm-mesh-6s-automation.sh >> logs/pmm-mesh-automation.log 2>&1 & +# journalctl equivalent: tail -f logs/pmm-mesh-automation.log +``` + +**systemd:** `config/systemd/chain138-pmm-mesh-automation.service.example` — copy, set `User` and absolute paths, `enable --now`. + +--- + ## 9. Wemix token verification (Deferred) This is intentionally deferred with the rest of the Wemix path. If the chain is brought back into scope later, open [scan.wemix.com/tokens](https://scan.wemix.com/tokens); confirm WETH, USDT, USDC addresses. If different, update `config/token-mapping-multichain.json` and [WEMIX_TOKEN_VERIFICATION.md](../07-ccip/WEMIX_TOKEN_VERIFICATION.md). Then: diff --git a/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md b/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md index bcfc4d1..a971928 100644 --- a/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md +++ b/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md @@ -25,7 +25,7 @@ | Placeholder | Location | What to complete | |-------------|----------|------------------| -| **the-order.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md) | When The Order portal is deployed: add NPMplus proxy host and document IP:port in RPC_ENDPOINTS_MASTER and ALL_VMIDS_ENDPOINTS. | +| **the-order.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md) | **Done 2026-03-27:** NPM → 10210 `192.168.11.39:80` (HAProxy → portal :3000). Keep docs in sync if routing changes. | | **Sankofa cutover plan** | [SANKOFA_CUTOVER_PLAN](../04-configuration/SANKOFA_CUTOVER_PLAN.md) | Replace ``, ``, and table TBDs with actual Sankofa service IPs/ports when deployed. | | **sankofa.nexus / phoenix.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md), [DNS_NPMPLUS_VM](../04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md) | **Doc fix done:** Correct targets: sankofa → 192.168.11.51:3000 (VMID 7801), phoenix → 192.168.11.50:4000 (VMID 7800). **Operator:** Ensure NPMplus proxy hosts use these, not 192.168.11.140. Only explorer.d-bis.org → .140. | | **Public blocks #2–#6** | [NETWORK_ARCHITECTURE](../02-architecture/NETWORK_ARCHITECTURE.md), [NETWORK_CONFIGURATION_MASTER](../11-references/NETWORK_CONFIGURATION_MASTER.md) | Document when blocks are assigned or mark as “reserved”. | diff --git a/docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md b/docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md new file mode 100644 index 0000000..a0751ad --- /dev/null +++ b/docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md @@ -0,0 +1,143 @@ +# Proxmox VE — Operational deployment template + +**Last Updated:** 2026-03-25 +**Status:** Active — ties hypervisors, LAN/WAN, cluster peering, Chain 138 Besu tiers, NPMplus ingress, FQDNs, and deployment gates into one place. + +**Machine-readable:** [`config/proxmox-operational-template.json`](../../config/proxmox-operational-template.json) (sync when you change VMIDs/IPs/FQDNs). + +**Authoritative detail (do not drift):** + +- VMID, port, status tables: [`docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`](../04-configuration/ALL_VMIDS_ENDPOINTS.md) +- Shell/env single source: [`config/ip-addresses.conf`](../../config/ip-addresses.conf) +- Edge, port forwards, four NPMplus picture: [`docs/11-references/NETWORK_CONFIGURATION_MASTER.md`](../11-references/NETWORK_CONFIGURATION_MASTER.md) +- Contract deploy order / gates: [`docs/03-deployment/DEPLOYMENT_ORDER_OF_OPERATIONS.md`](DEPLOYMENT_ORDER_OF_OPERATIONS.md) + +--- + +## 1. Proxmox VE hosts (management) + +| Hostname | MGMT IP | Proxmox UI | Cluster | Role (target) | +|----------|---------|------------|---------|----------------| +| ml110 | 192.168.11.10 | https://192.168.11.10:8006 | h (legacy) | Planned WAN aggregator (OPNsense/pfSense); **migrate CT/VM off before repurpose** | +| r630-01 | 192.168.11.11 | https://192.168.11.11:8006 | h | Primary: Chain 138 RPC/CCIP-adjacent workloads, Sankofa Phoenix stack, much of DBIS | +| r630-02 | 192.168.11.12 | https://192.168.11.12:8006 | h | Firefly, MIM4U, Mifos LXC, extra NPMplus instances, supporting infra | + +**LAN:** 192.168.11.0/24, gateway **192.168.11.1** (UDM Pro), VLAN 11. Extended node IP plan (r630-03 …): `config/ip-addresses.conf` comments. + +--- + +## 2. Cluster peering (Corosync / quorum) + +| Item | Value / note | +|------|----------------| +| Cluster name | **h** (verify live: `pvecm status`) | +| Ring | Typically same L2/L3 as MGMT — **192.168.11.0/24** | +| UDP ports | **5405–5412** between all nodes (+ SSH 22, API **8006** TCP) | +| Quorum | Odd node count preferred; during ml110 removal use 2-node awareness (risk window) or add qdevice | + +Cluster and UDM: [`docs/04-configuration/UDM_PRO_PROXMOX_CLUSTER.md`](../04-configuration/UDM_PRO_PROXMOX_CLUSTER.md). **Live inventory:** [`docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [`config/proxmox-operational-template.json`](../../config/proxmox-operational-template.json). + +--- + +## 3. Chain 138 Besu — peering model (summary) + +| Layer | VMID range (typical) | IPv4 pattern | P2P | +|--------|----------------------|--------------|-----| +| Validators | 1000–1004 | 192.168.11.100–104 | 30303 — **to sentries**, not raw public | +| Sentries | 1500–1506 | .150–.154, .213–.214 | Boundary / fan-out | +| Core RPC (deploy) | 2101 | **192.168.11.211** | 8545/8546 + 30303 | +| Core RPC (Nathan core-2) | 2102 | **192.168.11.212** | NPMplus **10235** / tunnel | +| Public RPC | 2201 | **192.168.11.221** | Frontends / bridge / read-mostly | +| Named RPC | 2303–2308 | .233–.238 | Partner-dedicated | +| ThirdWeb stack | 2400–2403 | .240–.243 | Includes translator/nginx on 2400 | + +Canonical roles and adjacency rules: [`docs/02-architecture/CHAIN138_CANONICAL_NETWORK_ROLES_VALIDATORS_SENTRY_AND_RPC.md`](../02-architecture/CHAIN138_CANONICAL_NETWORK_ROLES_VALIDATORS_SENTRY_AND_RPC.md). + +--- + +## 4. NPMplus and public ingress + +| VMID | Internal IP(s) | Public IP (typical) | Purpose | +|------|----------------|---------------------|---------| +| 10233 | 192.168.11.166 / **.167** | 76.53.10.36 | Main d-bis.org, explorer, Option B RPC, MIM4U | +| 10234 | 192.168.11.168 | 76.53.10.37 | Secondary HA (confirm running) | +| 10235 | 192.168.11.169 | 76.53.10.38 (alt **76.53.10.42**) | rpc-core-2, Alltra, HYBX | +| 10236 | 192.168.11.170 | 76.53.10.40 | Dev / Codespaces tunnel, Gitea, Proxmox admin | +| 10237 | 192.168.11.171 | (tunnel/Mifos) | mifos.d-bis.org → VMID 5800 | + +UDM Pro forwards **80 / 443** (and **81** where documented) to the matching internal IP. Detail: [`docs/04-configuration/NPMPLUS_FOUR_INSTANCES_MASTER.md`](../04-configuration/NPMPLUS_FOUR_INSTANCES_MASTER.md). + +--- + +## 5. FQDN → backend (high level) + +Use the full table in **ALL_VMIDS_ENDPOINTS** (“NPMplus Endpoint Configuration Reference”). Critical correctness checks: + +- **explorer.d-bis.org** → VMID **5000**, **192.168.11.140** (not Sankofa IPs). +- **sankofa.nexus** / **phoenix.sankofa.nexus** → VMID **7801** / **7800** at **.51:3000** / **.50:4000**. +- **rpc-http-prv / rpc-ws-prv** → **2101** (.211); **rpc-http-pub / rpc-ws-pub** → **2201** (.221). +- **rpc.public-0138.defi-oracle.io** → **2400** **192.168.11.240:443** (update NPM if still pointing at decommissioned IPs). + +**the-order.sankofa.nexus:** NPMplus → order HAProxy **10210** @ **192.168.11.39:80** (proxies to Sankofa portal **192.168.11.51:3000**). See `scripts/deployment/provision-order-haproxy-10210.sh`. + +### 5.1 Order stack (live VMIDs, r630-01 unless noted) + +| VMID | Hostname | IP | Role (short) | +|------|----------|-----|----------------| +| 10030 | order-identity | 192.168.11.40 | Identity | +| 10040 | order-intake | 192.168.11.41 | Intake | +| 10050 | order-finance | 192.168.11.49 | Finance | +| 10060 | order-dataroom | 192.168.11.42 | Dataroom | +| 10070 | order-legal | **192.168.11.87** | Legal — **moved off .54 2026-03-25** (`IP_ORDER_LEGAL`); .54 is **only** VMID 7804 gov-portals | +| 10080 | order-eresidency | 192.168.11.43 | eResidency | +| 10090 | order-portal-public | 192.168.11.36 | Public portal | +| 10091 | order-portal-internal | 192.168.11.35 | Internal portal | +| 10092 | order-mcp-legal | 192.168.11.37 | MCP legal | +| 10200 | order-prometheus | 192.168.11.46 | Metrics | +| 10201 | order-grafana | 192.168.11.47 | Dashboards | +| 10202 | order-opensearch | 192.168.11.48 | Search | +| 10210 | order-haproxy | 192.168.11.39 | Edge / HAProxy | + +**Redis:** `ORDER_REDIS_IP` = 192.168.11.38 in `ip-addresses.conf` — bind to live VMID via `pct list` / audit script. + +--- + +## 6. Deployment requirements (cross-domain) + +### 6.1 Platform (Proxmox / network) + +- [ ] All cluster nodes **quorate**; storage sufficient for CT/VM disks (local-lvm / future Ceph per master plan). +- [ ] **vmbr0** VLAN-aware; each workload IP **unique** on 192.168.11.0/24 (see ALL_VMIDS conflict section). +- [ ] UDM Pro routes and port-forwards match **NETWORK_CONFIGURATION_MASTER**. +- [ ] NPMplus proxy host rows match **ALL_VMIDS** (no Blockscout IP on Sankofa hostnames). + +### 6.2 Chain 138 (contracts / ops) + +- [ ] **Core RPC** 2101 reachable: `http://192.168.11.211:8545` for **deploy only** (not public RPC). +- [ ] `smom-dbis-138/.env`: `PRIVATE_KEY`, `RPC_URL_138`, nonce discipline — **DEPLOYMENT_ORDER_OF_OPERATIONS** Phase 0. +- [ ] Optional: `./scripts/deployment/preflight-chain138-deploy.sh` before any broadcast. + +### 6.3 Application / operator + +- [ ] Repo **`.env`** + **`smom-dbis-138/.env`** for operator scripts (`scripts/lib/load-project-env.sh`). +- [ ] Blockscout / verify / NPM backup scripts per **OPERATOR_READY_CHECKLIST** when doing release ops. + +--- + +## 7. Maintaining this template + +1. Change **ALL_VMIDS_ENDPOINTS** and/or **ip-addresses.conf** first (operator truth). +2. Update **`config/proxmox-operational-template.json`** so automation (future CMDB, checks) stays aligned. +3. Run **`./scripts/validation/validate-config-files.sh`** (includes JSON shape check for the template). +4. **Live diff (read-only, SSH):** from repo root on a host with SSH to Proxmox nodes: **`bash scripts/verify/audit-proxmox-operational-template.sh`**. Compares template VMIDs to `pct`/`qm` lists on ML110 + R630s (override **`PROXMOX_HOSTS`** if needed). + +--- + +## 8. Related runbooks + +| Topic | Doc | +|-------|-----| +| Operational runbooks index | [`OPERATIONAL_RUNBOOKS.md`](OPERATIONAL_RUNBOOKS.md) | +| Phoenix / Sankofa deploy | [`PHOENIX_DEPLOYMENT_RUNBOOK.md`](PHOENIX_DEPLOYMENT_RUNBOOK.md) | +| NPMplus health | [`docs/04-configuration/NPMPLUS_QUICK_REF.md`](../04-configuration/NPMPLUS_QUICK_REF.md) | +| 13-node / HA roadmap | [`docs/02-architecture/R630_13_NODE_DOD_HA_MASTER_PLAN.md`](../02-architecture/R630_13_NODE_DOD_HA_MASTER_PLAN.md) | diff --git a/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md b/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md index 971278c..b2e0df4 100644 --- a/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md +++ b/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md @@ -252,7 +252,7 @@ The following VMIDs have been permanently removed: - `www.sankofa.nexus` → Same upstream as apex; NPM **`advanced_config`** issues **301** to **`https://sankofa.nexus`** (preserve path/query via `$request_uri`). ✅ - `phoenix.sankofa.nexus` → Routes to `http://192.168.11.50:4000` (Phoenix API/VMID 7800) ✅ - `www.phoenix.sankofa.nexus` → Same upstream; **301** to **`https://phoenix.sankofa.nexus`**. ✅ -- `the-order.sankofa.nexus` / `www.the-order.sankofa.nexus` → OSJ management portal (secure auth). App source: **the_order** at `~/projects/the_order`. NPMplus **target** order-haproxy `http://192.168.11.39:80` (VMID **10210**) when that stack is serving. Until then, `update-npmplus-proxy-hosts-api.sh` defaults upstream to Sankofa portal `http://192.168.11.51:3000` (7801); override with `THE_ORDER_UPSTREAM_IP` / `THE_ORDER_UPSTREAM_PORT` when switching to HAProxy. **`www.the-order.sankofa.nexus`** is configured for **301** to **`https://the-order.sankofa.nexus`** (same pattern as `www.sankofa` / `www.phoenix`). +- `the-order.sankofa.nexus` / `www.the-order.sankofa.nexus` → OSJ management portal (secure auth). App source: **the_order** at `~/projects/the_order`. NPMplus default upstream: **order-haproxy** `http://192.168.11.39:80` (VMID **10210**), which proxies to Sankofa portal `http://192.168.11.51:3000` (7801). Fallback: set `THE_ORDER_UPSTREAM_IP` / `THE_ORDER_UPSTREAM_PORT` to `.51` / `3000` if HAProxy is offline. **`www.the-order.sankofa.nexus`** → **301** **`https://the-order.sankofa.nexus`** (same as `www.sankofa` / `www.phoenix`). - `studio.sankofa.nexus` → Routes to `http://192.168.11.72:8000` (Sankofa Studio / VMID 7805) **Public verification evidence (2026-03-26):** `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` passed with `Failed: 0`; Sankofa root, Phoenix, Studio, and The Order returned `200`. See [verification_report.md](verification-evidence/e2e-verification-20260326_100057/verification_report.md). @@ -509,7 +509,7 @@ This section lists all endpoints that should be configured in NPMplus, extracted | `www.sankofa.nexus` | `192.168.11.51` | `http` | `3000` | ❌ No | Sankofa Portal (VMID 7801) ✅ **Deployed** | | `phoenix.sankofa.nexus` | `192.168.11.50` | `http` | `4000` | ❌ No | Phoenix API - Cloud Platform Portal (VMID 7800) ✅ **Deployed** | | `www.phoenix.sankofa.nexus` | `192.168.11.50` | `http` | `4000` | ❌ No | Phoenix API (VMID 7800) ✅ **Deployed** | -| `the-order.sankofa.nexus` | `192.168.11.39` (HAProxy) or `192.168.11.51` (interim portal) | `http` | `80` or `3000` | ❌ No | Order edge via 10210 when live; else portal (7801) per `update-npmplus-proxy-hosts-api.sh` default | +| `the-order.sankofa.nexus`, `www.the-order.sankofa.nexus` | `192.168.11.39` (10210 HAProxy; default) or `192.168.11.51` (direct portal if env override) | `http` | `80` or `3000` | ❌ No | NPM → **.39:80** by default; HAProxy → **.51:3000** | | `studio.sankofa.nexus` | `192.168.11.72` | `http` | `8000` | ❌ No | Sankofa Studio (FusionAI Creator) — VMID 7805 | ### Path-Based Routing Notes @@ -546,7 +546,7 @@ If NPMplus proxy hosts for sankofa.nexus or phoenix.sankofa.nexus currently poin --- -**Last Updated**: 2026-01-18 +**Last Updated**: 2026-03-27 **Maintained By**: Infrastructure Team --- diff --git a/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md b/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md index 93256ff..4f77c3b 100644 --- a/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md +++ b/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md @@ -291,7 +291,7 @@ nginx on VMID 2400 (192.168.11.240:443): |--------|------------------|---------------------| | `sankofa.nexus`, `www.sankofa.nexus` | 192.168.11.51:3000 (VMID 7801) | 192.168.11.140 | | `phoenix.sankofa.nexus`, `www.phoenix.sankofa.nexus` | 192.168.11.50:4000 (VMID 7800) | 192.168.11.140 | -| `the-order.sankofa.nexus` | TBD when The Order portal is deployed | 192.168.11.140 | +| `the-order.sankofa.nexus`, `www.the-order.sankofa.nexus` | 192.168.11.39:80 (10210 HAProxy → .51:3000); www → 301 apex | 192.168.11.140 | **Action:** If any Sankofa/Phoenix proxy host in NPMplus points to 192.168.11.140 (Blockscout), update it to the correct IP:port above. Only `explorer.d-bis.org` should point to 192.168.11.140. diff --git a/docs/04-configuration/E2E_ENDPOINTS_LIST.md b/docs/04-configuration/E2E_ENDPOINTS_LIST.md index 934a2a5..807997b 100644 --- a/docs/04-configuration/E2E_ENDPOINTS_LIST.md +++ b/docs/04-configuration/E2E_ENDPOINTS_LIST.md @@ -31,7 +31,7 @@ | www.sankofa.nexus | web | https://www.sankofa.nexus | **301** to `https://sankofa.nexus` (canonical apex; NPM `advanced_config`). | | phoenix.sankofa.nexus | web | https://phoenix.sankofa.nexus | Phoenix API (7800); E2E uses `/health` for HTTPS check. | | www.phoenix.sankofa.nexus | web | https://www.phoenix.sankofa.nexus | **301** to `https://phoenix.sankofa.nexus` (canonical apex; NPM `advanced_config`). | -| the-order.sankofa.nexus | web | https://the-order.sankofa.nexus | OSJ (Sovereign Military Order of Malta) management portal behind secure auth; app source repo **the_order** at `~/projects/the_order` (NPM upstream: order-haproxy 10210 when live, else interim portal 7801 per `update-npmplus-proxy-hosts-api.sh`). | +| the-order.sankofa.nexus | web | https://the-order.sankofa.nexus | OSJ management portal (secure auth); app **the_order** at `~/projects/the_order`. NPM upstream default: **order-haproxy** VMID **10210** `http://192.168.11.39:80` → portal **192.168.11.51:3000** (`provision-order-haproxy-10210.sh`). Override with `THE_ORDER_UPSTREAM_*` for direct portal if 10210 is down. | | www.the-order.sankofa.nexus | web | https://www.the-order.sankofa.nexus | **301** to `https://the-order.sankofa.nexus` (canonical apex; NPM `advanced_config`). | | studio.sankofa.nexus | web | https://studio.sankofa.nexus | Sankofa Studio (FusionAI Creator) at VMID 7805. | | cacti-alltra.d-bis.org | web | https://cacti-alltra.d-bis.org | Cacti monitoring UI for Alltra. | diff --git a/docs/04-configuration/RPC_ENDPOINTS_MASTER.md b/docs/04-configuration/RPC_ENDPOINTS_MASTER.md index e0ba66d..502f32d 100644 --- a/docs/04-configuration/RPC_ENDPOINTS_MASTER.md +++ b/docs/04-configuration/RPC_ENDPOINTS_MASTER.md @@ -135,7 +135,7 @@ See [DBIS_CORE_API_REFERENCE.md](../11-references/DBIS_CORE_API_REFERENCE.md). ### Sankofa Services (sankofa.nexus) -**Config TBD:** When The Order portal is deployed or Sankofa cutover is completed, update this table and [SANKOFA_CUTOVER_PLAN.md](SANKOFA_CUTOVER_PLAN.md) with actual IP:port and NPMplus proxy backends. +**NPMplus backends:** See [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md). The Order uses **10210** (HAProxy) in front of the portal. | Domain | Protocol | Target VMID | Target IP | Target Port | WebSocket | Notes | |--------|----------|-------------|-----------|-------------|-----------|-------| @@ -143,7 +143,8 @@ See [DBIS_CORE_API_REFERENCE.md](../11-references/DBIS_CORE_API_REFERENCE.md). | `www.sankofa.nexus` | Redirect | - | - | - | ❌ No | Redirects to sankofa.nexus | | `phoenix.sankofa.nexus` | HTTP | 7800 | 192.168.11.50 | 4000 | ❌ No | Phoenix API | | `www.phoenix.sankofa.nexus` | Redirect | - | - | - | ❌ No | Redirects to phoenix.sankofa.nexus | -| `the-order.sankofa.nexus` | HTTP | TBD | TBD | TBD | ❌ No | ⚠️ Placeholder — not yet configured; add when The Order portal is deployed | +| `the-order.sankofa.nexus` | HTTP | 10210 | 192.168.11.39 | 80 | ❌ No | HAProxy → portal 7801 (192.168.11.51:3000); provision: `scripts/deployment/provision-order-haproxy-10210.sh` | +| `www.the-order.sankofa.nexus` | Redirect | - | - | - | ❌ No | 301 → `https://the-order.sankofa.nexus` (NPM advanced_config) | --- @@ -183,7 +184,8 @@ secure.mim4u.org → http://192.168.11.37:80 training.mim4u.org → http://192.168.11.37:80 sankofa.nexus → http://192.168.11.51:3000 phoenix.sankofa.nexus → http://192.168.11.50:4000 -the-order.sankofa.nexus → (TBD — add when The Order portal is deployed) +the-order.sankofa.nexus → http://192.168.11.39:80 (10210 HAProxy → 192.168.11.51:3000) +www.the-order.sankofa.nexus → 301 apex (NPM) ``` ### Redirect Hosts @@ -192,6 +194,7 @@ the-order.sankofa.nexus → (TBD — add when The Order portal is deployed) www.mim4u.org → mim4u.org www.sankofa.nexus → sankofa.nexus www.phoenix.sankofa.nexus → phoenix.sankofa.nexus +www.the-order.sankofa.nexus → the-order.sankofa.nexus ``` --- diff --git a/reports/r630-02-incomplete-tasks-summary.md b/reports/r630-02-incomplete-tasks-summary.md index f9ca487..f0d8faa 100644 --- a/reports/r630-02-incomplete-tasks-summary.md +++ b/reports/r630-02-incomplete-tasks-summary.md @@ -91,7 +91,7 @@ This document lists all tasks that were mentioned or identified during the conta - [ ] Set up dashboards and alerts #### Infrastructure Services (4 containers) -- **CT 10210** (order-haproxy) - HAProxy needs installation +- **CT 10210** (order-haproxy) — **HAProxy installed 2026-03-27** (`config/haproxy/order-haproxy-10210.cfg.template`, `scripts/deployment/provision-order-haproxy-10210.sh`; unprivileged CT may need one-time host `chown -R 100000:100000` on mounted rootfs if apt fails) - **CT 10230** (order-vault) - Vault needs installation - **CT 5200** (cacti-1) - Cacti needs installation - **CT 6000** (fabric-1) - Hyperledger Fabric needs installation @@ -174,7 +174,7 @@ This document lists all tasks that were mentioned or identified during the conta **Tasks:** - [ ] Configure Order services to connect to PostgreSQL (192.168.11.44) and Redis (192.168.11.38) -- [ ] Configure DBIS services to connect to PostgreSQL (192.168.11.105) and Redis (192.168.11.120) +- [ ] Configure DBIS services to connect to PostgreSQL (192.168.11.105) and Redis (192.168.11.125) - [ ] Configure frontend services to connect to API services - [ ] Configure monitoring services to scrape targets - [ ] Configure HAProxy backends