b3a8fe4496
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
- Config, docs, scripts, and backup manifests - Submodule refs unchanged (m = modified content in submodules) Made-with: Cursor
7.8 KiB
7.8 KiB
Operator and External Completion Checklist
Purpose: Items from ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md that cannot be completed from repo-only (e.g. from this workspace). They require LAN/Proxmox access, operator credentials, external services, or ongoing maintenance.
Use this checklist when you have operator or LAN access to complete the remaining recommendations.
Full plan (Wave 0–3, required/optional/recommended): COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md.
1. Proxmox / security (LAN or host access)
| # | Action | Command or notes |
|---|---|---|
| 1 | Secure .env permissions | chmod 600 .env (and subproject .env files); run from project root or use scripts/security/secure-env-permissions.sh if present |
| 2 | Secure validator key permissions | On host: chmod 600 keys, chown -R besu:besu validator dirs |
| 3 | SSH key-only auth | On Proxmox/containers: set PasswordAuthentication no, PubkeyAuthentication yes in sshd_config |
| 4 | Firewall for Proxmox API (8006) | Restrict to specific IPs (e.g. iptables or UDM Pro rules) |
| 5 | Network segmentation (VLANs) | Per NETWORK_ARCHITECTURE; configure switches, Proxmox bridges, ER605 |
| 6 | Metrics (Prometheus, Besu 9545) | Deploy/configure Prometheus scrape; enable Besu metrics port |
| 7 | Health check + alerting | Configure Alertmanager / PagerDuty / Slack per MASTER_SECRETS |
| 8 | Automated backup + encrypted validator keys | Run backup script; store encrypted copies off-host |
| 9 | Backup configs + version control | Commit configs to repo or backup store from LAN |
2. Deployment and runbooks (operator)
| # | Action | When |
|---|---|---|
| Priority | Mainnet liquidity + ramps | Fund mainnet LP and relay bridge; wire ramp API; document user flows. See MAINNET_LIQUIDITY_AND_RAMPS_PRIORITY.md and MAINNET_RAMP_USER_FLOWS.md. |
| 10 | Integration tests for deployment scripts | Run from LAN against staging/dev |
| 11 | Runbooks: add/remove validator, upgrade Besu, key rotation, recovery, consensus | Document and run from LAN when needed |
| 12–20 | Error handling, logging, Loki, resource/network/DB tuning, CI/CD, CLI tool | Implement on infra from operator environment |
3. Infrastructure and DNS (operator / when services go live)
| # | Action | Notes |
|---|---|---|
| R21 | The Order / Sankofa NPMplus proxy host | When The Order portal deployed: add proxy in NPMplus; document in RPC_ENDPOINTS_MASTER, ALL_VMIDS_ENDPOINTS |
| Sankofa cutover | Replace <TARGET_IP>, <TARGET_PORT>, TBDs in SANKOFA_CUTOVER_PLAN with actual values | |
| Blocks #2–#6 | Document in NETWORK_ARCHITECTURE / NETWORK_CONFIGURATION_MASTER when assigned or mark reserved | |
| 75–81 | VLAN enablement, observability stack, CCIP fleet, sovereign tenants, missing containers | Per NEXT_STEPS_MASTER and deployment phases |
4. Contracts and verification (operator / after deploy)
| # | Action | Command |
|---|---|---|
| R1 | Verify every deployed contract on Blockscout | After each deploy when Blockscout reachable |
| R2 | Keep CONTRACT_ADDRESSES_REFERENCE and ADDRESS_MATRIX_AND_STATUS updated | When new contracts deployed or deprecated |
| R3 | Run on-chain check | ./scripts/verify/check-contracts-on-chain-138.sh (set RPC_URL_138 from LAN) |
| R8 | Set RPC_URL_138; run from LAN if 192.168.11.x not reachable | Before any deploy |
| R9 | Use GAS_PRICE=1000000000 (or current min) on Chain 138 | Every forge script on 138 |
| R14 | Run verification after deploy in CI | When Blockscout reachable from runner |
5. Secrets and access (operator / security)
| # | Action | Notes |
|---|---|---|
| R4 | Do not use deprecated CCIPWETH9Bridge; use 0x971c... and set env | Always |
| R5 | Never commit .env or private keys; rotate exposed keys | Always |
| R7 | Restrict deployer key and RPC admin access | Access review |
| 48–52 | Secret audit, input validation, security scanning, RBAC, config validation | Run from secure env; use git-secrets, gitleaks, bandit, trivy as applicable |
6. External services and submissions
| # | Action | Where |
|---|---|---|
| 98 | CoinGecko submission (Chain 138) | CoinGecko |
| 99 | Consensys outreach (Swaps/Bridge support) | MetaMask/Consensys channels |
| 106–108 | Verify Etherlink/Jumper/LiFi for chains 138, 651940, 42793 | External APIs / docs |
| 109–121 | Tezos/Etherlink contracts and relay services | Deploy and run from operator env |
7. Maintenance (ongoing)
| # | Task | Frequency |
|---|---|---|
| 135 | Monitor explorer sync status | Daily |
| 136 | Monitor RPC node health (e.g. VMID 2201) | Daily |
| 137 | Check config API uptime | Weekly |
| 138 | Review explorer logs | Weekly |
| 139 | Update token list | As needed |
| R17 | Monitor critical bridge/oracle events | Ongoing |
| R18 | Ensure Blockscout (VMID 5000) up and /api reachable | Health checks |
8. Optional tools (install for full automation)
Install for scripts that recommend them:
- shellcheck —
apt install shellcheckorbrew install shellcheck. With--optional, run reports findings but does not fail (CI and validate pass). - jq — Required on Proxmox host for genesis validation;
run-via-proxmox-ssh.sh validateinstalls it when missing. - wscat —
npm i -g wscator usenpx -y wscatfor WebSocket RPC tests - sqlite3, websocat, sshpass, dig, parallel — per verification evidence NEXT_STEPS_RUN_*.md
What was completed in-repo (no operator needed)
- Validate via Proxmox SSH (2026-02-23):
run-via-proxmox-ssh.sh validateruns full shellcheck onscripts/verify/*.shand genesis validation (smom-dbis-138); installs shellcheck and jq on the host when missing. Shellcheck with--optionalis non-fatal so CI and validate complete successfully. - Mainnet liquidity & ramps priority: MAINNET_LIQUIDITY_AND_RAMPS_PRIORITY.md and MAINNET_RAMP_USER_FLOWS.md — priority plan (fund LP + relay bridge, wire ramps) and user flows for on-ramp/off-ramp; linked from ALL_NEXT_STEPS_CURRENT, NEXT_STEPS_INDEX, METAMASK_BRIDGE_SWAP, and this checklist.
- smom-dbis-138 deployment (2026-02-23): Chain 138 verification, fund CCIP/mainnet LP dry-runs, optional G1 PMM and G2 Trustless on L2s, run-all-four-gaps and CCIPLogger documented. See COMPLETION_RUN_20260223_ALL_STEPS.md and ALL_NEXT_STEPS_CURRENT.md.
- Shebang: Active (non-archive) scripts under
scripts/updated from#!/bin/bashto#!/usr/bin/env bash. - Script header template: 10-best-practices/SCRIPT_HEADER_TEMPLATE.md added.
- Config validation in CI:
.github/workflows/validate-config.ymlalready runs config validation and completable-tasks dry-run; shellcheck added as optional step. - Documentation: Quick reference cards, troubleshooting FAQ, and decision tree already exist and are linked from ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md.
- Operator checklist: This file; R1–R24 and remaining items are in RECOMMENDATIONS_OPERATOR_CHECKLIST.md and ALL_RECOMMENDATIONS_AND_IMPROVEMENTS_LIST.md.
Related: NEXT_STEPS_OPERATOR.md, NEXT_STEPS_AND_REMAINING_TODOS.md, OPERATOR_READY_CHECKLIST.md.