proxmox/docs/00-meta/REMAINING_WORK_DETAILED_TASKS.md

# Remaining Work — Detailed Tasks

**Last Updated:** 2026-02-05  
**Purpose:** Single checklist of every remaining task with concrete steps. Use with [FULL_PARALLEL_EXECUTION_ORDER.md](FULL_PARALLEL_EXECUTION_ORDER.md) and [WAVE2_WAVE3_OPERATOR_CHECKLIST.md](WAVE2_WAVE3_OPERATOR_CHECKLIST.md).

---

## Wave 0 — Gates / credentials (do when creds allow)

| ID | Task | Detailed steps |
|----|------|-----------------|
| **W0-1** | NPMplus RPC fix (405) | ✅ Done (2026-02-06 run). Re-run from host on LAN if needed: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh` |
| **W0-2** | Execute sendCrossChain (real) | 1) Ensure `PRIVATE_KEY` and LINK/fee token approved in `.env`. 2) Run `./scripts/bridge/run-send-cross-chain.sh <amount_eth> [recipient]` **without** `--dry-run`. 3) Example: `./scripts/bridge/run-send-cross-chain.sh 0.01` or with recipient: `./scripts/bridge/run-send-cross-chain.sh 0.01 0xYourAddress`. Bridge: `0x971cD9D156f193df8051E48043C476e53ECd4693`. |
| **W0-3** | NPMplus backup | 1) Set `NPM_PASSWORD` in `.env`. 2) When NPMplus container is up, run: `bash scripts/verify/backup-npmplus.sh` or `./scripts/backup/automated-backup.sh [--with-npmplus]`. 3) Re-run if previous backup had API/auth warnings. |

---

## ~~Post-create: Containers 2506, 2507, 2508~~ — Destroyed 2026-02-08

Containers **2506, 2507, 2508** were **removed and destroyed** on all Proxmox hosts (2026-02-08). Script: `scripts/destroy-vmids-2506-2508.sh`. RPC range is **2500–2505** only. No follow-up. See [MISSING_CONTAINERS_LIST.md](../03-deployment/MISSING_CONTAINERS_LIST.md).

### 2506 — besu-rpc-luis (Luis, 0x1)

- [x] Apply permissioned RPC configuration (Besu config) — **Done 2026-02-06:** `configure-besu-chain138-nodes.sh` run on r630-01; static-nodes.json and permissioned-nodes.json deployed.
- [x] Configure `static-nodes.json` / `permissioned-nodes.json` — Deployed (6 enodes: validators + sentries; RPC enodes not in list).
- [x] **Disable discovery** — Script sets discovery disabled for 2506 (DISCOVERY_DISABLED_VMIDS); 2506 had no config file on host so manual check if Besu uses discovery=false.
- [ ] Configure permissioned identity **0x1** (if not already in container).
- [ ] Set up **JWT authentication** (e.g. nginx reverse proxy in front of Besu).
- [ ] Verify access: Luis RPC-only, 0x1 identity.

**Scripts:** `scripts/configure-besu-chain138-nodes.sh`, `scripts/setup-new-chain138-containers.sh`; see [CHAIN138_BESU_CONFIGURATION.md](../06-besu/CHAIN138_BESU_CONFIGURATION.md).

### 2507 — besu-rpc-putu (Putu, 0x8a)

- [x] Permissioned RPC configuration — **Done 2026-02-06:** static-nodes/permissioned-nodes deployed via configure script on r630-01.
- [x] **Disable discovery** — Script sets discovery disabled for 2507.
- [ ] Configure permissioned identity **0x8a**.
- [ ] Set up **JWT authentication** (nginx reverse proxy).
- [ ] Verify access: Putu RPC-only, 0x8a identity.

### 2508 — besu-rpc-putu (Putu, 0x1)

- [x] Permissioned RPC configuration — **Done 2026-02-06:** static-nodes/permissioned-nodes deployed.
- [x] **Disable discovery** — Script sets discovery disabled for 2508.
- [ ] Configure permissioned identity **0x1**.
- [ ] Set up **JWT authentication** (nginx reverse proxy).
- [ ] Verify access: Putu RPC-only, 0x1 identity.

---

## Config cleanup (docs vs created containers) — Completed

| Task | Details |
|------|---------|
| **IP config** | Done. `config/ip-addresses.conf`: `RPC_LUIS_2="192.168.11.202"`, `RPC_PUTU_1="192.168.11.203"`, `RPC_PUTU_2="192.168.11.204"`. (RPC_LUIS_1 remains .255; fix separately if needed.) |
| **MISSING_CONTAINERS_LIST.md** | Done. Table updated to deployed IPs .202/.203/.204 and note that 2506–2508 created on r630-01. |
| **Other docs/scripts** | Done. REMAINING_WORK_DETAILED_STEPS.md, CHAIN138_JWT_AUTH_REQUIREMENTS.md, create-all-chain138-containers-direct.sh, create-chain138-containers.sh, generate-jwt-token-for-container.sh, repair-corrupted-ip-replacements.sh, fix-remaining-hardcoded-ips.sh updated to .202/.203/.204. |

---

## Wave 1 — Remaining (parallel by owner/task)

### Security (apply when ready)

| ID | Task | Details |
|----|------|---------|
| W1-1 | SSH key-based auth | Run `./scripts/security/setup-ssh-key-auth.sh --apply` after testing; disable password auth only after key auth verified (coordinate to avoid lockout). |
| W1-2 | Firewall Proxmox 8006 | Run `./scripts/security/firewall-proxmox-8006.sh --apply [CIDR]` to restrict Proxmox API to specific IPs. |

### smom / audits

| ID | Task |
|----|------|
| W1-3 | smom: Security audits VLT-024, ISO-024 |
| W1-4 | smom: Bridge integrations BRG-VLT, BRG-ISO |

### Monitoring (deploy vs config)

| ID | Task | Details |
|----|------|---------|
| W1-5 | Prometheus / alerts | Config in `config/monitoring/` (phase2-observability.sh --config-only done). Deploy and add Besu 9545 scrape targets; alert rules. |
| W1-6 | Grafana / Alertmanager | Deploy Grafana; publish via Cloudflare Access; configure Alertmanager routes. |
| W1-7 | Loki | Config present; deploy when stack is deployed (W2-1). |

### Backup

| ID | Task | Details |
|----|------|---------|
| W1-8 | NPMplus backup cron | Done. Cron installed (daily 03:00 → backup-npmplus.sh; logs to logs/npmplus-backup.log). |

### VLAN (optional)

| ID | Task |
|----|------|
| W1-9 | VLAN enablement: UDM Pro VLAN config docs; Proxmox VLAN-aware bridge design |
| W1-10 | VLAN migration plan (per-service table) |

### Documentation

| ID | Task |
|----|------|
| W1-11 | Documentation consolidation (by folder 01–12); archive old status |
| W1-12 | Quick reference cards; decision trees; config templates (ALL_IMPROVEMENTS 68–74) |
| W1-13 | Final IP assignments; service connectivity matrix; operational runbooks |

### Codebase

| ID | Task |
|----|------|
| W1-14 | dbis_core: TypeScript/Prisma fixes (parallelize by file; or defer) |
| W1-15 | smom: EnhancedSwapRouter quoter; AlltraAdapter fee TODO |
| W1-16 | smom: IRU remaining tasks |
| W1-17 | Placeholders: canonical addresses env-only; AlltraAdapter fee; smart accounts kit; quote service Fabric chainId 999; .bak deprecation (87–91) |

### Quick wins & checklist

| ID | Task |
|----|------|
| W1-18 | Add progress indicators to scripts; config validation in CI/pre-deploy |
| W1-19 | Secure validator key permissions: on Proxmox host as root `./scripts/secure-validator-keys.sh [--dry-run]` (VMIDs 1000–1004); chmod 600, chown besu |
| W1-20 | Secret management audit; input validation in scripts; security scanning (ALL_IMPROVEMENTS 48–51) |
| W1-21 | Config validation (JSON/YAML schema); config templates; env standardization (52–54) |

### Optional: MetaMask / explorer

| ID | Task |
|----|------|
| W1-22 | Token-aggregation hardening; CoinGecko submission |
| W1-23 | Chain 138 Snap: market data UI; swap quotes; bridge routes; testing & distribution |
| W1-24 | Explorer: dark mode, network selector, sync indicator |
| W1-25 | Paymaster deploy (optional); Consensys outreach |
| W1-26 | API keys: Li.Fi, Jumper, 1inch (when keys available; see API_KEYS_REQUIRED.md) |

### Improvements index (ALL_IMPROVEMENTS 1–139)

| ID | Task |
|----|------|
| W1-27 | ALL_IMPROVEMENTS 1–11 (Proxmox high) |
| W1-28 | ALL_IMPROVEMENTS 12–20 (Proxmox medium) |
| W1-29 | ALL_IMPROVEMENTS 21–30 (Proxmox low) |
| W1-30 | ALL_IMPROVEMENTS 31–35 (Quick wins) |
| W1-31 | ALL_IMPROVEMENTS 36–43 (script shebang, set -euo, shellcheck, consolidation) |
| W1-32 | ALL_IMPROVEMENTS 44–47 (doc consolidation, API doc) |
| W1-33 | ALL_IMPROVEMENTS 48–57 (security, validation, RBAC, tests, CI) |
| W1-34 | ALL_IMPROVEMENTS 58–67 (logging, metrics, health, DevContainer, backup) |
| W1-35 | ALL_IMPROVEMENTS 68–74 (docs: quick ref, decision trees, glossary) |
| W1-36 | ALL_IMPROVEMENTS 75–81 (Phase 1–4 design; missing containers list) |
| W1-37 | ALL_IMPROVEMENTS 82–86 (smom audits, BRG, CCIP AMB, dbis_core, IRU) |
| W1-38 | ALL_IMPROVEMENTS 87–91 (placeholders) |
| W1-39 | ALL_IMPROVEMENTS 92–105 (MetaMask/explorer) |
| W1-40 | ALL_IMPROVEMENTS 106–121 (Tezos/Etherlink/CCIP) |
| W1-41 | ALL_IMPROVEMENTS 122–126 (Besu/blockchain) |
| W1-42 | ALL_IMPROVEMENTS 127–130 (RPC translator) |
| W1-43 | ALL_IMPROVEMENTS 131–134 (Orchestration portal) |
| W1-44 | ALL_IMPROVEMENTS 135–139 (Maintenance — document/automate) |

**Detail:** [ALL_IMPROVEMENTS_AND_GAPS_INDEX.md](../ALL_IMPROVEMENTS_AND_GAPS_INDEX.md)

---

## Wave 2 — Infra / deploy (parallel by host or component)

| ID | Task | Detailed steps |
|----|------|----------------|
| **W2-1** | Deploy monitoring stack | Deploy Prometheus, Grafana, Loki, Alertmanager using `smom-dbis-138/monitoring/` and `scripts/monitoring/` configs. |
| **W2-2** | Grafana + alerts | After W2-1: publish Grafana via Cloudflare Access; configure Alertmanager routes. |
| **W2-3** | VLAN enablement | Apply UDM Pro VLAN config; Proxmox VLAN-aware bridge; migrate services to VLANs (by VLAN/host). See NETWORK_ARCHITECTURE.md §3–5. |
| **W2-4** | Phase 3 CCIP | 1) Deploy Ops/Admin (5400, 5401). 2) NAT pools. 3) Expand commit/execute/RMN scripts. Order: Ops first, then NAT, then scripts. See [CCIP_DEPLOYMENT_SPEC.md](../07-ccip/CCIP_DEPLOYMENT_SPEC.md). |
| **W2-5** | Phase 4 sovereign tenants | Sovereign tenant VLANs; isolation; access control (by tenant/VLAN). After W2-3. |
| **W2-6** | Missing containers 2506–2508 | ✅ Created on r630-01 with .202/.203/.204. Remaining: post-create steps above (Besu config, JWT, discovery off, identity). |
| **W2-7** | DBIS services / Hyperledger | Start DBIS services (10100–10151, etc.); additional Hyperledger per deployment runbooks (by host). |
| **W2-8** | NPMplus HA | Optional: Keepalived, secondary 10234. See NPMPLUS_HA_SETUP_GUIDE.md. |

---

## Wave 3 — After Wave 2

| ID | Task | Detailed steps |
|----|------|----------------|
| **W3-1** | CCIP Fleet full deploy | After W2-4 (Ops/Admin, NAT): deploy 16 commit (5410–5425), 16 execute (5440–5455), 7 RMN (5470–5476). |
| **W3-2** | Phase 4 tenant isolation | After W2-3/W2-5: enforce tenant isolation; access control. |

---

## Ongoing (schedule, not sequenced) — Completed

| ID | Task | Frequency | Status |
|----|------|-----------|--------|
| O-1 | Monitor explorer sync | Daily 08:00 | Cron installed via schedule-daily-weekly-cron.sh; daily-weekly-checks.sh daily |
| O-2 | Monitor RPC 2201 | Daily 08:00 | Same cron/script |
| O-3 | Config API uptime | Weekly (Sun 09:00) | Cron installed; daily-weekly-checks.sh weekly |
| O-4 | Review explorer logs | Weekly | Runbook [138] in OPERATIONAL_RUNBOOKS; O-4 procedure and pct exec 5000 journalctl documented |
| O-5 | Update token list | As needed | token-lists/lists/dbis-138.tokenlist.json; runbook [139]; TOKEN_LIST_AUTHORING_GUIDE linked |

---

## Optional one-off — Script and runbook added

| Task | Details |
|------|---------|
| Start firefly-ali-1 (6201) | Script: scripts/maintenance/start-firefly-6201.sh (--dry-run, --host). Default r630-02. In OPERATIONAL_RUNBOOKS Maintenance. |

---

## Automation complete — remaining is operator-only

All tasks that can run without LAN, SSH to Proxmox, or live credentials have been executed (config cleanup, validation, cron install, dry-runs, checklists). **What remains** requires you or a host with access:

- **Wave 0:** W0-2 sendCrossChain real (`run-send-cross-chain.sh` without `--dry-run`), W0-3 run backup when NPMplus is up.
- **Post-create 2506–2508:** **Done 2026-02-06.** Besu configure run on r630-01 and ml110: `PROXMOX_HOST=192.168.11.11 bash scripts/run-configure-besu-on-host.sh` and `PROXMOX_HOST=192.168.11.10 bash scripts/run-configure-besu-on-host.sh`. Static-nodes.json and permissioned-nodes.json deployed to all running Besu nodes; discovery disabled for 2500, 2503–2508. RPC enodes (2500–2508) are not in the enode list (extraction skipped); validators + sentries only. Remaining: JWT/nginx for 2506–2508 if required; verify discovery and identity per container.
- **Wave 1 apply:** W1-1 `setup-ssh-key-auth.sh --apply`, W1-2 `firewall-proxmox-8006.sh --apply` (per host).
- **Wave 2 & 3:** Deploy monitoring, VLAN, CCIP, Phase 4, DBIS, NPMplus HA; then CCIP Fleet and Phase 4 isolation.

Use [WAVE2_WAVE3_OPERATOR_CHECKLIST.md](WAVE2_WAVE3_OPERATOR_CHECKLIST.md) and runbooks for execution order.

---

## Validation commands (after changes)

| Check | Command |
|-------|---------|
| CI / config | `bash scripts/verify/run-all-validation.sh [--skip-genesis]` |
| Full verification | `bash scripts/verify/run-full-verification.sh` |
| E2E routing | `bash scripts/verify/verify-end-to-end-routing.sh` |
| Backend VMs | `bash scripts/verify/verify-backend-vms.sh` |
| Besu peers | `bash scripts/besu-verify-peers.sh http://192.168.11.211:8545` |

---

## Summary counts

| Category | Count |
|----------|-------|
| Wave 0 | 3 (W0-2, W0-3 remaining; W0-1 done) |
| Post-create 2506–2508 | 3 containers × checklist items |
| Config cleanup | 3 (ip-addresses.conf, MISSING_CONTAINERS_LIST, other docs) |
| Wave 1 | 44 items (W1-1 … W1-44) |
| Wave 2 | 8 (W2-1–W2-8; W2-6 create done, post-create pending) |
| Wave 3 | 2 (W3-1, W3-2) |
| Ongoing | 5 (scheduled) |

**References:** [FULL_PARALLEL_EXECUTION_ORDER.md](FULL_PARALLEL_EXECUTION_ORDER.md) · [WAVE2_WAVE3_OPERATOR_CHECKLIST.md](WAVE2_WAVE3_OPERATOR_CHECKLIST.md) · [REMAINING_ITEMS_FULL_PARALLEL_LIST.md](REMAINING_ITEMS_FULL_PARALLEL_LIST.md) · [MISSING_CONTAINERS_LIST.md](../03-deployment/MISSING_CONTAINERS_LIST.md) · [FULL_PARALLEL_RUN_LOG.md](../archive/00-meta-pruned/FULL_PARALLEL_RUN_LOG.md) (archived)