Operator and External Completion Checklist
Purpose: Items from ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md that cannot be completed from repo-only (e.g. from this workspace). They require LAN/Proxmox access, operator credentials, external services, or ongoing maintenance.
Use this checklist when you have operator or LAN access to complete the remaining recommendations.
Full plan (Wave 0–3, required/optional/recommended): COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md.
1. Proxmox / security (LAN or host access)
| # |
Action |
Command or notes |
| 1 |
Secure .env permissions |
chmod 600 .env (and subproject .env files); run from project root or use scripts/security/secure-env-permissions.sh if present |
| 2 |
Secure validator key permissions |
On host: chmod 600 keys, chown -R besu:besu validator dirs |
| 3 |
SSH key-only auth |
On Proxmox/containers: set PasswordAuthentication no, PubkeyAuthentication yes in sshd_config |
| 4 |
Firewall for Proxmox API (8006) |
Restrict to specific IPs (e.g. iptables or UDM Pro rules) |
| 5 |
Network segmentation (VLANs) |
Per NETWORK_ARCHITECTURE; configure switches, Proxmox bridges, ER605 |
| 6 |
Metrics (Prometheus, Besu 9545) |
Deploy/configure Prometheus scrape; enable Besu metrics port |
| 7 |
Health check + alerting |
Configure Alertmanager / PagerDuty / Slack per MASTER_SECRETS |
| 8 |
Automated backup + encrypted validator keys |
Run backup script; store encrypted copies off-host |
| 9 |
Backup configs + version control |
Commit configs to repo or backup store from LAN |
2. Deployment and runbooks (operator)
| # |
Action |
When |
| Priority |
Mainnet liquidity + ramps |
Fund mainnet LP and relay bridge; wire ramp API; document user flows. See MAINNET_LIQUIDITY_AND_RAMPS_PRIORITY.md and MAINNET_RAMP_USER_FLOWS.md. |
| 10 |
Integration tests for deployment scripts |
Run from LAN against staging/dev |
| 11 |
Runbooks: add/remove validator, upgrade Besu, key rotation, recovery, consensus |
Document and run from LAN when needed |
| 12–20 |
Error handling, logging, Loki, resource/network/DB tuning, CI/CD, CLI tool |
Implement on infra from operator environment |
3. Infrastructure and DNS (operator / when services go live)
| # |
Action |
Notes |
| R21 |
The Order / Sankofa NPMplus |
Done 2026-03 — see ALL_VMIDS, RPC_ENDPOINTS_MASTER, update-npmplus-proxy-hosts-api.sh |
| Sankofa cutover |
Done — SANKOFA_CUTOVER_PLAN v1.1 |
|
| Blocks #2–#6 |
Document in NETWORK_ARCHITECTURE / NETWORK_CONFIGURATION_MASTER when assigned or mark reserved |
|
| 75–81 |
VLAN enablement, observability stack, CCIP fleet, sovereign tenants, missing containers |
Per NEXT_STEPS_MASTER and deployment phases |
4. Contracts and verification (operator / after deploy)
| # |
Action |
Command |
| R1 |
Verify every deployed contract on Blockscout |
After each deploy when Blockscout reachable |
| R2 |
Keep CONTRACT_ADDRESSES_REFERENCE and ADDRESS_MATRIX_AND_STATUS updated |
When new contracts deployed or deprecated |
| R3 |
Run on-chain check |
./scripts/verify/check-contracts-on-chain-138.sh (set RPC_URL_138 from LAN) |
| R8 |
Set RPC_URL_138; run from LAN if 192.168.11.x not reachable |
Before any deploy |
| R9 |
Use GAS_PRICE=1000000000 (or current min) on Chain 138 |
Every forge script on 138 |
| R14 |
Run verification after deploy in CI |
When Blockscout reachable from runner |
5. Secrets and access (operator / security)
| # |
Action |
Notes |
| R4 |
Do not use deprecated CCIPWETH9Bridge; use 0x971c... and set env |
Always |
| R5 |
Never commit .env or private keys; rotate exposed keys |
Always |
| R7 |
Restrict deployer key and RPC admin access |
Access review |
| 48–52 |
Secret audit, input validation, security scanning, RBAC, config validation |
Run from secure env; use git-secrets, gitleaks, bandit, trivy as applicable |
6. External services and submissions
| # |
Action |
Where |
| 98 |
CoinGecko submission (Chain 138) |
CoinGecko |
| 99 |
Consensys outreach (Swaps/Bridge support) |
MetaMask/Consensys channels |
| 106–108 |
Verify Etherlink/Jumper/LiFi for chains 138, 651940, 42793 |
External APIs / docs |
| 109–121 |
Tezos/Etherlink contracts and relay services |
Deploy and run from operator env |
7. Maintenance (ongoing)
| # |
Task |
Frequency |
| 135 |
Monitor explorer sync status |
Daily |
| 136 |
Monitor RPC node health (e.g. VMID 2201) |
Daily |
| 137 |
Check config API uptime |
Weekly |
| 138 |
Review explorer logs |
Weekly |
| 139 |
Update token list |
As needed |
| R17 |
Monitor critical bridge/oracle events |
Ongoing |
| R18 |
Ensure Blockscout (VMID 5000) up and /api reachable |
Health checks |
8. Optional tools (install for full automation)
Install for scripts that recommend them:
- shellcheck —
apt install shellcheck or brew install shellcheck. With --optional, run reports findings but does not fail (CI and validate pass).
- jq — Required on Proxmox host for genesis validation;
run-via-proxmox-ssh.sh validate installs it when missing.
- wscat —
npm i -g wscat or use npx -y wscat for WebSocket RPC tests
- sqlite3, websocat, sshpass, dig, parallel — per verification evidence NEXT_STEPS_RUN_*.md
What was completed in-repo (no operator needed)
- Validate via Proxmox SSH (2026-02-23):
run-via-proxmox-ssh.sh validate runs full shellcheck on scripts/verify/*.sh and genesis validation (smom-dbis-138); installs shellcheck and jq on the host when missing. Shellcheck with --optional is non-fatal so CI and validate complete successfully.
- Mainnet liquidity & ramps priority: MAINNET_LIQUIDITY_AND_RAMPS_PRIORITY.md and MAINNET_RAMP_USER_FLOWS.md — priority plan (fund LP + relay bridge, wire ramps) and user flows for on-ramp/off-ramp; linked from ALL_NEXT_STEPS_CURRENT, NEXT_STEPS_INDEX, METAMASK_BRIDGE_SWAP, and this checklist.
- smom-dbis-138 deployment (2026-02-23): Chain 138 verification, fund CCIP/mainnet LP dry-runs, optional G1 PMM and G2 Trustless on L2s, run-all-four-gaps and CCIPLogger documented. See COMPLETION_RUN_20260223_ALL_STEPS.md and ALL_NEXT_STEPS_CURRENT.md.
- Shebang: Active (non-archive) scripts under
scripts/ updated from #!/bin/bash to #!/usr/bin/env bash.
- Script header template: 10-best-practices/SCRIPT_HEADER_TEMPLATE.md added.
- Config validation in CI:
.github/workflows/validate-config.yml already runs config validation and completable-tasks dry-run; shellcheck added as optional step.
- Documentation: Quick reference cards, troubleshooting FAQ, and decision tree already exist and are linked from ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md.
- Operator checklist: This file; R1–R24 and remaining items are in RECOMMENDATIONS_OPERATOR_CHECKLIST.md and ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md.
Related: NEXT_STEPS_OPERATOR.md, NEXT_STEPS_AND_REMAINING_TODOS.md, OPERATOR_READY_CHECKLIST.md.
70a6d66e4d