docs: sync The Order routing (10210 HAProxy) and fix stale TBDs

- E2E, ALL_VMIDS, operator checklist, RPC_ENDPOINTS_MASTER, DNS/NPM architecture - PROXMOX deployment template: the-order wired via 10210 - Placeholders master + r630-02 incomplete summary for 10210 - CT 10210: chown /var/cache on host idmap (mandb clean) — applied on cluster Made-with: Cursor
2026-03-27 15:06:06 -07:00
parent 430431f2f6
commit a086c451c3
8 changed files with 201 additions and 12 deletions
--- a/docs/00-meta/OPERATOR_READY_CHECKLIST.md
+++ b/docs/00-meta/OPERATOR_READY_CHECKLIST.md
@@ -1,6 +1,6 @@
 # Operator Ready Checklist — Copy-Paste Commands

-**Last Updated:** 2026-03-04  
+**Last Updated:** 2026-03-27  
 **Purpose:** Single page with exact commands to complete every pending todo. Run from **repo root** on a host with **LAN** access (and `smom-dbis-138/.env` with `PRIVATE_KEY`, `NPM_PASSWORD` where noted).

 **Do you have all necessary creds?** See [OPERATOR_CREDENTIALS_CHECKLIST.md](OPERATOR_CREDENTIALS_CHECKLIST.md) — per-task list of LAN, PRIVATE_KEY, NPM_PASSWORD, RPC_URL_138, SSH, LINK, gas, token balance.
@@ -15,6 +15,22 @@

 ---

+## Completed in this session (2026-03-26)
+
+| Item | Result |
+|------|--------|
+| NPMplus recovery | VMID `10233` was wedged on `192.168.11.167:81` (TCP connect, no HTTP). `pct reboot 10233` on `r630-01` restored the expected `301` response on port `81`. |
+| NPMplus API updater | `NPM_URL=https://192.168.11.167:81 bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh` completed with **39 hosts updated, 0 failed**. |
+| Sankofa / Order / Studio routing | **Superseded 2026-03-27:** Order hostnames default to **order-haproxy** `http://192.168.11.39:80` (10210 → `.51:3000`). Through 2026-03-26 NPM pointed Order directly at portal `:3000`. `studio.sankofa.nexus` → `http://192.168.11.72:8000`. |
+| Public E2E | Latest run `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` exited `0` with **Failed: 0**, **DNS passed: 37**, **HTTPS passed: 22**. Sankofa, Phoenix, Studio, The Order, DBIS, Mifos, and MIM4U public endpoints passed. Evidence: `docs/04-configuration/verification-evidence/e2e-verification-20260326_115013/`. |
+| Private E2E | Latest run `bash scripts/verify/verify-end-to-end-routing.sh --profile=private` exited `0` with **Failed: 0** and **DNS passed: 4**. `rpc-http-prv.d-bis.org`, `rpc-fireblocks.d-bis.org`, `rpc-ws-prv.d-bis.org`, and `ws.rpc-fireblocks.d-bis.org` all passed. Evidence: `docs/04-configuration/verification-evidence/e2e-verification-20260326_120939/`. |
+| NPMplus backup | Fresh backup completed: `backups/npmplus/backup-20260326_115622.tar.gz`. API exports succeeded; direct SQLite file copy and certbot path copy were partial/warn-only, but the backup manifest and compressed bundle were created successfully. |
+| Blockscout verification run | `./scripts/verify/run-contract-verification-with-proxy.sh` completed; contracts were submitted or skipped if already verified. `WETH10` returned `The address is not a smart contract`; others like `Multicall`, `Aggregator`, `Proxy`, `CCIPSender`, `CCIPWETH10Bridge`, and `CCIPWETH9Bridge` submitted successfully. |
+| Private RPC redirect fix | `rpc-http-prv.d-bis.org` no longer returns HTTP `301` on JSON-RPC POST. Live NPMplus host `11` was updated to `ssl_forced=false` while preserving upstream `192.168.11.211:8545`. |
+| NPM creds loading | For NPM-only runs, prefer targeted `grep` of `NPM_EMAIL` / `NPM_PASSWORD` if full `.env` export triggers `Argument list too long`. |
+
+---
+
 ## 1. High: Cronos closure + reachable CCIP funding

 **Ref:** [CONFIG_READY_CHAINS_COMPLETION_RUNBOOK](../07-ccip/CONFIG_READY_CHAINS_COMPLETION_RUNBOOK.md)
@@ -84,6 +100,8 @@ Single contract retry: `./scripts/verify/run-contract-verification-with-proxy.sh

 **Runbook:** [502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md](502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md)

+**Current status after 2026-03-26:** no public 502s reproduced in the latest public E2E run. Use this section only if those endpoints regress.
+
 ---

 ## 5. LAN: Run all operator tasks (backup + verify ± deploy ± create-vms)
@@ -211,8 +229,14 @@ bash scripts/verify/backup-npmplus.sh

 **NPMplus RPC fix (405):** From LAN: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh`. Verify: `bash scripts/verify/verify-end-to-end-routing.sh`.

+**Status (2026-03-26):** main NPMplus API update completed successfully with `39 hosts updated, 0 failed`; public E2E now passes for Sankofa root, Phoenix, Studio, and The Order. Re-run only when upstream targets or proxy definitions change.
+
+**Latest backup evidence:** `backups/npmplus/backup-20260326_115622.tar.gz`
+
 **NPMplus API unreachable (167/169):** Restart Docker inside NPMplus LXC: `./scripts/maintenance/fix-npmplus-services-via-proxmox-ssh.sh` (SSH to r630-01, restarts npmplus in 10233 and 10235).

+**If port 81 accepts TCP but hangs at HTTP:** reboot CT `10233` with `pct reboot 10233` on `r630-01`, then retry the API updater.
+
 **E2E from LAN (no public DNS):** If E2E fails at DNS (`Could not resolve host`), use [E2E_DNS_FROM_LAN_RUNBOOK.md](../04-configuration/E2E_DNS_FROM_LAN_RUNBOOK.md): append `config/e2e-hosts-append.txt` to `/etc/hosts`, then run `E2E_USE_SYSTEM_RESOLVER=1 ./scripts/verify/verify-end-to-end-routing.sh --profile=public`. Revert with `sudo ./scripts/verify/remove-e2e-hosts-from-etc-hosts.sh`.

 **E2E profiles:** Use `--profile=public` for public endpoints (default) or `--profile=private` for private/admin RPC only. Run sequentially to avoid timestamp collision in evidence dirs. **Known E2E warnings** (502/404 and WS): [E2E_ENDPOINTS_LIST.md](../04-configuration/E2E_ENDPOINTS_LIST.md) § Known E2E warnings and Remediation. MIM4U web 502s and WS test-format warnings are **non-blocking** for contract/pool completion.
@@ -221,6 +245,25 @@ bash scripts/verify/backup-npmplus.sh

 ---

+## 8.5 PMM mesh (6s oracle / keeper / PMM–WETH poll)
+
+**Ref:** `smom-dbis-138/docs/integration/ORACLE_AND_KEEPER_CHAIN138.md` (PMM mesh automation)
+
+```bash
+cd smom-dbis-138
+# .env should include: PRIVATE_KEY, AGGREGATOR_ADDRESS, PRICE_FEED_KEEPER_ADDRESS (optional: KEEPER_PRIVATE_KEY if different from PRIVATE_KEY)
+./scripts/reserve/set-price-feed-keeper-interval.sh 6   # once per keeper deployment if interval was 30s
+./scripts/update-oracle-price.sh   # verify transmitter + gas (Besu needs explicit gas limit in script)
+./scripts/reserve/sync-weth-mock-price.sh   # if CHAIN138_WETH_MOCK_PRICE_FEED is set (keeper WETH path)
+mkdir -p logs
+nohup ./scripts/reserve/pmm-mesh-6s-automation.sh >> logs/pmm-mesh-automation.log 2>&1 &
+# journalctl equivalent: tail -f logs/pmm-mesh-automation.log
+```
+
+**systemd:** `config/systemd/chain138-pmm-mesh-automation.service.example` — copy, set `User` and absolute paths, `enable --now`.
+
+---
+
 ## 9. Wemix token verification (Deferred)

 This is intentionally deferred with the rest of the Wemix path. If the chain is brought back into scope later, open [scan.wemix.com/tokens](https://scan.wemix.com/tokens); confirm WETH, USDT, USDC addresses. If different, update `config/token-mapping-multichain.json` and [WEMIX_TOKEN_VERIFICATION.md](../07-ccip/WEMIX_TOKEN_VERIFICATION.md). Then:
--- a/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md
+++ b/docs/00-meta/PLACEHOLDERS_AND_COMPLETION_MASTER_LIST.md
@@ -25,7 +25,7 @@

 | Placeholder | Location | What to complete |
 |-------------|----------|------------------|
-| **the-order.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md) | When The Order portal is deployed: add NPMplus proxy host and document IP:port in RPC_ENDPOINTS_MASTER and ALL_VMIDS_ENDPOINTS. |
+| **the-order.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md) | **Done 2026-03-27:** NPM → 10210 `192.168.11.39:80` (HAProxy → portal :3000). Keep docs in sync if routing changes. |
 | **Sankofa cutover plan** | [SANKOFA_CUTOVER_PLAN](../04-configuration/SANKOFA_CUTOVER_PLAN.md) | Replace `<TARGET_IP>`, `<TARGET_PORT>`, and table TBDs with actual Sankofa service IPs/ports when deployed. |
 | **sankofa.nexus / phoenix.sankofa.nexus** | [ALL_VMIDS_ENDPOINTS](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [RPC_ENDPOINTS_MASTER](../04-configuration/RPC_ENDPOINTS_MASTER.md), [DNS_NPMPLUS_VM](../04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md) | **Doc fix done:** Correct targets: sankofa → 192.168.11.51:3000 (VMID 7801), phoenix → 192.168.11.50:4000 (VMID 7800). **Operator:** Ensure NPMplus proxy hosts use these, not 192.168.11.140. Only explorer.d-bis.org → .140. |
 | **Public blocks #2–#6** | [NETWORK_ARCHITECTURE](../02-architecture/NETWORK_ARCHITECTURE.md), [NETWORK_CONFIGURATION_MASTER](../11-references/NETWORK_CONFIGURATION_MASTER.md) | Document when blocks are assigned or mark as “reserved”. |
--- a/docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md
+++ b/docs/03-deployment/PROXMOX_VE_OPERATIONAL_DEPLOYMENT_TEMPLATE.md
@@ -0,0 +1,143 @@
+# Proxmox VE — Operational deployment template
+
+**Last Updated:** 2026-03-25  
+**Status:** Active — ties hypervisors, LAN/WAN, cluster peering, Chain 138 Besu tiers, NPMplus ingress, FQDNs, and deployment gates into one place.
+
+**Machine-readable:** [`config/proxmox-operational-template.json`](../../config/proxmox-operational-template.json) (sync when you change VMIDs/IPs/FQDNs).
+
+**Authoritative detail (do not drift):**
+
+- VMID, port, status tables: [`docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`](../04-configuration/ALL_VMIDS_ENDPOINTS.md)
+- Shell/env single source: [`config/ip-addresses.conf`](../../config/ip-addresses.conf)
+- Edge, port forwards, four NPMplus picture: [`docs/11-references/NETWORK_CONFIGURATION_MASTER.md`](../11-references/NETWORK_CONFIGURATION_MASTER.md)
+- Contract deploy order / gates: [`docs/03-deployment/DEPLOYMENT_ORDER_OF_OPERATIONS.md`](DEPLOYMENT_ORDER_OF_OPERATIONS.md)
+
+---
+
+## 1. Proxmox VE hosts (management)
+
+| Hostname | MGMT IP | Proxmox UI | Cluster | Role (target) |
+|----------|---------|------------|---------|----------------|
+| ml110 | 192.168.11.10 | https://192.168.11.10:8006 | h (legacy) | Planned WAN aggregator (OPNsense/pfSense); **migrate CT/VM off before repurpose** |
+| r630-01 | 192.168.11.11 | https://192.168.11.11:8006 | h | Primary: Chain 138 RPC/CCIP-adjacent workloads, Sankofa Phoenix stack, much of DBIS |
+| r630-02 | 192.168.11.12 | https://192.168.11.12:8006 | h | Firefly, MIM4U, Mifos LXC, extra NPMplus instances, supporting infra |
+
+**LAN:** 192.168.11.0/24, gateway **192.168.11.1** (UDM Pro), VLAN 11. Extended node IP plan (r630-03 …): `config/ip-addresses.conf` comments.
+
+---
+
+## 2. Cluster peering (Corosync / quorum)
+
+| Item | Value / note |
+|------|----------------|
+| Cluster name | **h** (verify live: `pvecm status`) |
+| Ring | Typically same L2/L3 as MGMT — **192.168.11.0/24** |
+| UDP ports | **5405–5412** between all nodes (+ SSH 22, API **8006** TCP) |
+| Quorum | Odd node count preferred; during ml110 removal use 2-node awareness (risk window) or add qdevice |
+
+Cluster and UDM: [`docs/04-configuration/UDM_PRO_PROXMOX_CLUSTER.md`](../04-configuration/UDM_PRO_PROXMOX_CLUSTER.md). **Live inventory:** [`docs/04-configuration/ALL_VMIDS_ENDPOINTS.md`](../04-configuration/ALL_VMIDS_ENDPOINTS.md), [`config/proxmox-operational-template.json`](../../config/proxmox-operational-template.json).
+
+---
+
+## 3. Chain 138 Besu — peering model (summary)
+
+| Layer | VMID range (typical) | IPv4 pattern | P2P |
+|--------|----------------------|--------------|-----|
+| Validators | 1000–1004 | 192.168.11.100–104 | 30303 — **to sentries**, not raw public |
+| Sentries | 1500–1506 | .150–.154, .213–.214 | Boundary / fan-out |
+| Core RPC (deploy) | 2101 | **192.168.11.211** | 8545/8546 + 30303 |
+| Core RPC (Nathan core-2) | 2102 | **192.168.11.212** | NPMplus **10235** / tunnel |
+| Public RPC | 2201 | **192.168.11.221** | Frontends / bridge / read-mostly |
+| Named RPC | 2303–2308 | .233–.238 | Partner-dedicated |
+| ThirdWeb stack | 2400–2403 | .240–.243 | Includes translator/nginx on 2400 |
+
+Canonical roles and adjacency rules: [`docs/02-architecture/CHAIN138_CANONICAL_NETWORK_ROLES_VALIDATORS_SENTRY_AND_RPC.md`](../02-architecture/CHAIN138_CANONICAL_NETWORK_ROLES_VALIDATORS_SENTRY_AND_RPC.md).
+
+---
+
+## 4. NPMplus and public ingress
+
+| VMID | Internal IP(s) | Public IP (typical) | Purpose |
+|------|----------------|---------------------|---------|
+| 10233 | 192.168.11.166 / **.167** | 76.53.10.36 | Main d-bis.org, explorer, Option B RPC, MIM4U |
+| 10234 | 192.168.11.168 | 76.53.10.37 | Secondary HA (confirm running) |
+| 10235 | 192.168.11.169 | 76.53.10.38 (alt **76.53.10.42**) | rpc-core-2, Alltra, HYBX |
+| 10236 | 192.168.11.170 | 76.53.10.40 | Dev / Codespaces tunnel, Gitea, Proxmox admin |
+| 10237 | 192.168.11.171 | (tunnel/Mifos) | mifos.d-bis.org → VMID 5800 |
+
+UDM Pro forwards **80 / 443** (and **81** where documented) to the matching internal IP. Detail: [`docs/04-configuration/NPMPLUS_FOUR_INSTANCES_MASTER.md`](../04-configuration/NPMPLUS_FOUR_INSTANCES_MASTER.md).
+
+---
+
+## 5. FQDN → backend (high level)
+
+Use the full table in **ALL_VMIDS_ENDPOINTS** (“NPMplus Endpoint Configuration Reference”). Critical correctness checks:
+
+- **explorer.d-bis.org** → VMID **5000**, **192.168.11.140** (not Sankofa IPs).
+- **sankofa.nexus** / **phoenix.sankofa.nexus** → VMID **7801** / **7800** at **.51:3000** / **.50:4000**.
+- **rpc-http-prv / rpc-ws-prv** → **2101** (.211); **rpc-http-pub / rpc-ws-pub** → **2201** (.221).
+- **rpc.public-0138.defi-oracle.io** → **2400** **192.168.11.240:443** (update NPM if still pointing at decommissioned IPs).
+
+**the-order.sankofa.nexus:** NPMplus → order HAProxy **10210** @ **192.168.11.39:80** (proxies to Sankofa portal **192.168.11.51:3000**). See `scripts/deployment/provision-order-haproxy-10210.sh`.
+
+### 5.1 Order stack (live VMIDs, r630-01 unless noted)
+
+| VMID | Hostname | IP | Role (short) |
+|------|----------|-----|----------------|
+| 10030 | order-identity | 192.168.11.40 | Identity |
+| 10040 | order-intake | 192.168.11.41 | Intake |
+| 10050 | order-finance | 192.168.11.49 | Finance |
+| 10060 | order-dataroom | 192.168.11.42 | Dataroom |
+| 10070 | order-legal | **192.168.11.87** | Legal — **moved off .54 2026-03-25** (`IP_ORDER_LEGAL`); .54 is **only** VMID 7804 gov-portals |
+| 10080 | order-eresidency | 192.168.11.43 | eResidency |
+| 10090 | order-portal-public | 192.168.11.36 | Public portal |
+| 10091 | order-portal-internal | 192.168.11.35 | Internal portal |
+| 10092 | order-mcp-legal | 192.168.11.37 | MCP legal |
+| 10200 | order-prometheus | 192.168.11.46 | Metrics |
+| 10201 | order-grafana | 192.168.11.47 | Dashboards |
+| 10202 | order-opensearch | 192.168.11.48 | Search |
+| 10210 | order-haproxy | 192.168.11.39 | Edge / HAProxy |
+
+**Redis:** `ORDER_REDIS_IP` = 192.168.11.38 in `ip-addresses.conf` — bind to live VMID via `pct list` / audit script.
+
+---
+
+## 6. Deployment requirements (cross-domain)
+
+### 6.1 Platform (Proxmox / network)
+
+- [ ] All cluster nodes **quorate**; storage sufficient for CT/VM disks (local-lvm / future Ceph per master plan).
+- [ ] **vmbr0** VLAN-aware; each workload IP **unique** on 192.168.11.0/24 (see ALL_VMIDS conflict section).
+- [ ] UDM Pro routes and port-forwards match **NETWORK_CONFIGURATION_MASTER**.
+- [ ] NPMplus proxy host rows match **ALL_VMIDS** (no Blockscout IP on Sankofa hostnames).
+
+### 6.2 Chain 138 (contracts / ops)
+
+- [ ] **Core RPC** 2101 reachable: `http://192.168.11.211:8545` for **deploy only** (not public RPC).
+- [ ] `smom-dbis-138/.env`: `PRIVATE_KEY`, `RPC_URL_138`, nonce discipline — **DEPLOYMENT_ORDER_OF_OPERATIONS** Phase 0.
+- [ ] Optional: `./scripts/deployment/preflight-chain138-deploy.sh` before any broadcast.
+
+### 6.3 Application / operator
+
+- [ ] Repo **`.env`** + **`smom-dbis-138/.env`** for operator scripts (`scripts/lib/load-project-env.sh`).
+- [ ] Blockscout / verify / NPM backup scripts per **OPERATOR_READY_CHECKLIST** when doing release ops.
+
+---
+
+## 7. Maintaining this template
+
+1. Change **ALL_VMIDS_ENDPOINTS** and/or **ip-addresses.conf** first (operator truth).
+2. Update **`config/proxmox-operational-template.json`** so automation (future CMDB, checks) stays aligned.
+3. Run **`./scripts/validation/validate-config-files.sh`** (includes JSON shape check for the template).
+4. **Live diff (read-only, SSH):** from repo root on a host with SSH to Proxmox nodes: **`bash scripts/verify/audit-proxmox-operational-template.sh`**. Compares template VMIDs to `pct`/`qm` lists on ML110 + R630s (override **`PROXMOX_HOSTS`** if needed).
+
+---
+
+## 8. Related runbooks
+
+| Topic | Doc |
+|-------|-----|
+| Operational runbooks index | [`OPERATIONAL_RUNBOOKS.md`](OPERATIONAL_RUNBOOKS.md) |
+| Phoenix / Sankofa deploy | [`PHOENIX_DEPLOYMENT_RUNBOOK.md`](PHOENIX_DEPLOYMENT_RUNBOOK.md) |
+| NPMplus health | [`docs/04-configuration/NPMPLUS_QUICK_REF.md`](../04-configuration/NPMPLUS_QUICK_REF.md) |
+| 13-node / HA roadmap | [`docs/02-architecture/R630_13_NODE_DOD_HA_MASTER_PLAN.md`](../02-architecture/R630_13_NODE_DOD_HA_MASTER_PLAN.md) |
--- a/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md
+++ b/docs/04-configuration/ALL_VMIDS_ENDPOINTS.md
@@ -252,7 +252,7 @@ The following VMIDs have been permanently removed:
 - `www.sankofa.nexus` → Same upstream as apex; NPM **`advanced_config`** issues **301** to **`https://sankofa.nexus`** (preserve path/query via `$request_uri`). ✅
 - `phoenix.sankofa.nexus` → Routes to `http://192.168.11.50:4000` (Phoenix API/VMID 7800) ✅
 - `www.phoenix.sankofa.nexus` → Same upstream; **301** to **`https://phoenix.sankofa.nexus`**. ✅
- `the-order.sankofa.nexus` / `www.the-order.sankofa.nexus` → OSJ management portal (secure auth). App source: **the_order** at `~/projects/the_order`. NPMplus **target** order-haproxy `http://192.168.11.39:80` (VMID **10210**) when that stack is serving. Until then, `update-npmplus-proxy-hosts-api.sh` defaults upstream to Sankofa portal `http://192.168.11.51:3000` (7801); override with `THE_ORDER_UPSTREAM_IP` / `THE_ORDER_UPSTREAM_PORT` when switching to HAProxy. **`www.the-order.sankofa.nexus`** is configured for **301** to **`https://the-order.sankofa.nexus`** (same pattern as `www.sankofa` / `www.phoenix`).
+- `the-order.sankofa.nexus` / `www.the-order.sankofa.nexus` → OSJ management portal (secure auth). App source: **the_order** at `~/projects/the_order`. NPMplus default upstream: **order-haproxy** `http://192.168.11.39:80` (VMID **10210**), which proxies to Sankofa portal `http://192.168.11.51:3000` (7801). Fallback: set `THE_ORDER_UPSTREAM_IP` / `THE_ORDER_UPSTREAM_PORT` to `.51` / `3000` if HAProxy is offline. **`www.the-order.sankofa.nexus`** → **301** **`https://the-order.sankofa.nexus`** (same as `www.sankofa` / `www.phoenix`).
 - `studio.sankofa.nexus` → Routes to `http://192.168.11.72:8000` (Sankofa Studio / VMID 7805)

 **Public verification evidence (2026-03-26):** `bash scripts/verify/verify-end-to-end-routing.sh --profile=public` passed with `Failed: 0`; Sankofa root, Phoenix, Studio, and The Order returned `200`. See [verification_report.md](verification-evidence/e2e-verification-20260326_100057/verification_report.md).
@@ -509,7 +509,7 @@ This section lists all endpoints that should be configured in NPMplus, extracted
 | `www.sankofa.nexus` | `192.168.11.51` | `http` | `3000` | ❌ No | Sankofa Portal (VMID 7801) ✅ **Deployed** |
 | `phoenix.sankofa.nexus` | `192.168.11.50` | `http` | `4000` | ❌ No | Phoenix API - Cloud Platform Portal (VMID 7800) ✅ **Deployed** |
 | `www.phoenix.sankofa.nexus` | `192.168.11.50` | `http` | `4000` | ❌ No | Phoenix API (VMID 7800) ✅ **Deployed** |
-| `the-order.sankofa.nexus` | `192.168.11.39` (HAProxy) or `192.168.11.51` (interim portal) | `http` | `80` or `3000` | ❌ No | Order edge via 10210 when live; else portal (7801) per `update-npmplus-proxy-hosts-api.sh` default |
+| `the-order.sankofa.nexus`, `www.the-order.sankofa.nexus` | `192.168.11.39` (10210 HAProxy; default) or `192.168.11.51` (direct portal if env override) | `http` | `80` or `3000` | ❌ No | NPM → **.39:80** by default; HAProxy → **.51:3000** |
 | `studio.sankofa.nexus` | `192.168.11.72` | `http` | `8000` | ❌ No | Sankofa Studio (FusionAI Creator) — VMID 7805 |

 ### Path-Based Routing Notes
@@ -546,7 +546,7 @@ If NPMplus proxy hosts for sankofa.nexus or phoenix.sankofa.nexus currently poin

 ---

-**Last Updated**: 2026-01-18  
+**Last Updated**: 2026-03-27  
 **Maintained By**: Infrastructure Team

 ---
--- a/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md
+++ b/docs/04-configuration/DNS_NPMPLUS_VM_COMPREHENSIVE_ARCHITECTURE.md
@@ -291,7 +291,7 @@ nginx on VMID 2400 (192.168.11.240:443):
 |--------|------------------|---------------------|
 | `sankofa.nexus`, `www.sankofa.nexus` | 192.168.11.51:3000 (VMID 7801) | 192.168.11.140 |
 | `phoenix.sankofa.nexus`, `www.phoenix.sankofa.nexus` | 192.168.11.50:4000 (VMID 7800) | 192.168.11.140 |
-| `the-order.sankofa.nexus` | TBD when The Order portal is deployed | 192.168.11.140 |
+| `the-order.sankofa.nexus`, `www.the-order.sankofa.nexus` | 192.168.11.39:80 (10210 HAProxy → .51:3000); www → 301 apex | 192.168.11.140 |

 **Action:** If any Sankofa/Phoenix proxy host in NPMplus points to 192.168.11.140 (Blockscout), update it to the correct IP:port above. Only `explorer.d-bis.org` should point to 192.168.11.140.

--- a/docs/04-configuration/E2E_ENDPOINTS_LIST.md
+++ b/docs/04-configuration/E2E_ENDPOINTS_LIST.md
@@ -31,7 +31,7 @@
 | www.sankofa.nexus | web | https://www.sankofa.nexus | **301** to `https://sankofa.nexus` (canonical apex; NPM `advanced_config`). |
 | phoenix.sankofa.nexus | web | https://phoenix.sankofa.nexus | Phoenix API (7800); E2E uses `/health` for HTTPS check. |
 | www.phoenix.sankofa.nexus | web | https://www.phoenix.sankofa.nexus | **301** to `https://phoenix.sankofa.nexus` (canonical apex; NPM `advanced_config`). |
-| the-order.sankofa.nexus | web | https://the-order.sankofa.nexus | OSJ (Sovereign Military Order of Malta) management portal behind secure auth; app source repo **the_order** at `~/projects/the_order` (NPM upstream: order-haproxy 10210 when live, else interim portal 7801 per `update-npmplus-proxy-hosts-api.sh`). |
+| the-order.sankofa.nexus | web | https://the-order.sankofa.nexus | OSJ management portal (secure auth); app **the_order** at `~/projects/the_order`. NPM upstream default: **order-haproxy** VMID **10210** `http://192.168.11.39:80` → portal **192.168.11.51:3000** (`provision-order-haproxy-10210.sh`). Override with `THE_ORDER_UPSTREAM_*` for direct portal if 10210 is down. |
 | www.the-order.sankofa.nexus | web | https://www.the-order.sankofa.nexus | **301** to `https://the-order.sankofa.nexus` (canonical apex; NPM `advanced_config`). |
 | studio.sankofa.nexus | web | https://studio.sankofa.nexus | Sankofa Studio (FusionAI Creator) at VMID 7805. |
 | cacti-alltra.d-bis.org | web | https://cacti-alltra.d-bis.org | Cacti monitoring UI for Alltra. |
--- a/docs/04-configuration/RPC_ENDPOINTS_MASTER.md
+++ b/docs/04-configuration/RPC_ENDPOINTS_MASTER.md
@@ -135,7 +135,7 @@ See [DBIS_CORE_API_REFERENCE.md](../11-references/DBIS_CORE_API_REFERENCE.md).

 ### Sankofa Services (sankofa.nexus)

-**Config TBD:** When The Order portal is deployed or Sankofa cutover is completed, update this table and [SANKOFA_CUTOVER_PLAN.md](SANKOFA_CUTOVER_PLAN.md) with actual IP:port and NPMplus proxy backends.
+**NPMplus backends:** See [ALL_VMIDS_ENDPOINTS.md](ALL_VMIDS_ENDPOINTS.md). The Order uses **10210** (HAProxy) in front of the portal.

 | Domain | Protocol | Target VMID | Target IP | Target Port | WebSocket | Notes |
 |--------|----------|-------------|-----------|-------------|-----------|-------|
@@ -143,7 +143,8 @@ See [DBIS_CORE_API_REFERENCE.md](../11-references/DBIS_CORE_API_REFERENCE.md).
 | `www.sankofa.nexus` | Redirect | - | - | - | ❌ No | Redirects to sankofa.nexus |
 | `phoenix.sankofa.nexus` | HTTP | 7800 | 192.168.11.50 | 4000 | ❌ No | Phoenix API |
 | `www.phoenix.sankofa.nexus` | Redirect | - | - | - | ❌ No | Redirects to phoenix.sankofa.nexus |
-| `the-order.sankofa.nexus` | HTTP | TBD | TBD | TBD | ❌ No | ⚠️ Placeholder — not yet configured; add when The Order portal is deployed |
+| `the-order.sankofa.nexus` | HTTP | 10210 | 192.168.11.39 | 80 | ❌ No | HAProxy → portal 7801 (192.168.11.51:3000); provision: `scripts/deployment/provision-order-haproxy-10210.sh` |
+| `www.the-order.sankofa.nexus` | Redirect | - | - | - | ❌ No | 301 → `https://the-order.sankofa.nexus` (NPM advanced_config) |

 ---

@@ -183,7 +184,8 @@ secure.mim4u.org           → http://192.168.11.37:80
 training.mim4u.org         → http://192.168.11.37:80
 sankofa.nexus              → http://192.168.11.51:3000
 phoenix.sankofa.nexus      → http://192.168.11.50:4000
-the-order.sankofa.nexus    → (TBD — add when The Order portal is deployed)
+the-order.sankofa.nexus    → http://192.168.11.39:80 (10210 HAProxy → 192.168.11.51:3000)
+www.the-order.sankofa.nexus → 301 apex (NPM)
 ```

 ### Redirect Hosts
@@ -192,6 +194,7 @@ the-order.sankofa.nexus    → (TBD — add when The Order portal is deployed)
 www.mim4u.org              → mim4u.org
 www.sankofa.nexus          → sankofa.nexus
 www.phoenix.sankofa.nexus  → phoenix.sankofa.nexus
+www.the-order.sankofa.nexus → the-order.sankofa.nexus
 ```

 ---
--- a/reports/r630-02-incomplete-tasks-summary.md
+++ b/reports/r630-02-incomplete-tasks-summary.md
@@ -91,7 +91,7 @@ This document lists all tasks that were mentioned or identified during the conta
 - [ ] Set up dashboards and alerts

 #### Infrastructure Services (4 containers)
- **CT 10210** (order-haproxy) - HAProxy needs installation
+- **CT 10210** (order-haproxy) — **HAProxy installed 2026-03-27** (`config/haproxy/order-haproxy-10210.cfg.template`, `scripts/deployment/provision-order-haproxy-10210.sh`; unprivileged CT may need one-time host `chown -R 100000:100000` on mounted rootfs if apt fails)
 - **CT 10230** (order-vault) - Vault needs installation
 - **CT 5200** (cacti-1) - Cacti needs installation
 - **CT 6000** (fabric-1) - Hyperledger Fabric needs installation
@@ -174,7 +174,7 @@ This document lists all tasks that were mentioned or identified during the conta

 **Tasks:**
 - [ ] Configure Order services to connect to PostgreSQL (192.168.11.44) and Redis (192.168.11.38)
- [ ] Configure DBIS services to connect to PostgreSQL (192.168.11.105) and Redis (192.168.11.120)
+- [ ] Configure DBIS services to connect to PostgreSQL (192.168.11.105) and Redis (192.168.11.125)
 - [ ] Configure frontend services to connect to API services
 - [ ] Configure monitoring services to scrape targets
 - [ ] Configure HAProxy backends