Files
proxmox/docs/00-meta/OPERATOR_READY_CHECKLIST.md
defiQUG 3f76bc9507
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
docs: update master documentation and push to Gitea (2026-03-06)
- MASTER_INDEX: Last Updated 2026-03-06; status 59/59 contracts; add NEXT_STEPS_LIST, CONTRACT_NEXT_STEPS_LIST
- docs/README, NEXT_STEPS_INDEX, 06-besu/MASTER_INDEX: Last Updated 2026-03-06
- Contract check script: 59 addresses (PMM, vault/reserve, CompliantFiatTokens); canonical CCIP/router
- New docs: EXECUTION_CHECKLIST, NEXT_STEPS_LIST, DOTENV_AUDIT, ADDITIONAL_PATHS, deployer gas runbook, WEMIX_ACQUISITION_TABLED, etc.
- Config: deployer-gas-routes, cro-wemix-swap-routes, routing-registry, token-mapping
- Scripts: check-contracts-on-chain-138, check-pmm-pool-balances-chain138, deployer-gas-auto-route, acquire-cro-and-wemix-gas
- Operator rule: operator-lan-access-check.mdc

Made-with: Cursor
2026-03-06 19:11:25 -08:00

198 lines
9.4 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Operator Ready Checklist — Copy-Paste Commands
**Last Updated:** 2026-03-04
**Purpose:** Single page with exact commands to complete every pending todo. Run from **repo root** on a host with **LAN** access (and `smom-dbis-138/.env` with `PRIVATE_KEY`, `NPM_PASSWORD` where noted).
**Do you have all necessary creds?** See [OPERATOR_CREDENTIALS_CHECKLIST.md](OPERATOR_CREDENTIALS_CHECKLIST.md) — per-task list of LAN, PRIVATE_KEY, NPM_PASSWORD, RPC_URL_138, SSH, LINK, gas, token balance.
**From anywhere (no LAN):** `./scripts/run-completable-tasks-from-anywhere.sh`
**If deployer needs gas on public chains or Wemix:** Run `./scripts/deployment/deployer-gas-auto-route.sh` (optional: `--dry-run`, `--chain 138` or `--chain 1111`). See [DEPLOYER_GAS_AUTO_ROUTE_RUNBOOK.md](../03-deployment/DEPLOYER_GAS_AUTO_ROUTE_RUNBOOK.md).
**Remaining for full network coverage (13-chain max execution):** [REMAINING_DEPLOYMENTS_FOR_FULL_NETWORK_COVERAGE.md](../03-deployment/REMAINING_DEPLOYMENTS_FOR_FULL_NETWORK_COVERAGE.md) — Phase A (mint + add liquidity 138) → B (**Cronos**, Celo, **Wemix** CCIP + LINK) → C (cW* + edge pools). **2026-03-04:** Celo + Gnosis CCIP ✅; Cronos and Wemix need deployer gas (CRO ~15, WEMIX ~0.4) then deploy + complete-config. See `./scripts/deployment/acquire-cro-and-wemix-gas.sh`.
---
## 1. High: Gnosis, Cronos, Celo, Wemix CCIP bridges
**Ref:** [CONFIG_READY_CHAINS_COMPLETION_RUNBOOK](../07-ccip/CONFIG_READY_CHAINS_COMPLETION_RUNBOOK.md)
**Prereqs:** Confirm [CCIP supports](https://docs.chain.link/ccip/supported-networks) 100, **25**, 42220, 1111. Per chain: RPC, CCIP Router, LINK, WETH9/WETH10, deployer with native gas (xDAI, **CRO ~15**, CELO, **WEMIX ~0.4**). To acquire CRO and WEMIX: `./scripts/deployment/acquire-cro-and-wemix-gas.sh`.
```bash
cd smom-dbis-138
source .env
# Per chain (set RPC_URL, CCIP_ROUTER_ADDRESS, LINK_TOKEN_ADDRESS, WETH9_ADDRESS, WETH10_ADDRESS, PRIVATE_KEY)
forge script script/deploy/bridge/DeployWETHBridges.s.sol:DeployWETHBridges --rpc-url "$RPC_URL" --broadcast -vvvv
```
Then add destinations (Chain 138 ↔ each chain) and fund with LINK — use:
```bash
DRY_RUN=1 ./scripts/deployment/complete-config-ready-chains.sh # print commands
./scripts/deployment/complete-config-ready-chains.sh # run (requires bridge addresses in .env)
```
**Add Cronos and Wemix:** Copy Cronos/Wemix vars from `smom-dbis-138/docs/deployment/ENV_CONFIG_READY_CHAINS.example` into `smom-dbis-138/.env`; fund deployer with CRO and WEMIX (see `acquire-cro-and-wemix-gas.sh`); then:
```bash
cd smom-dbis-138
./scripts/deployment/deploy-bridges-config-ready-chains.sh cronos
./scripts/deployment/deploy-bridges-config-ready-chains.sh wemix
# Set CCIPWETH9_BRIDGE_CRONOS, CCIPWETH10_BRIDGE_CRONOS, CCIPWETH9_BRIDGE_WEMIX, CCIPWETH10_BRIDGE_WEMIX in .env from output
./scripts/deployment/complete-config-ready-chains.sh
```
**Full steps:** See runbook § Step 14.
---
## 2. Medium: LINK support on Mainnet relay
**Ref:** [RELAY_BRIDGE_ADD_LINK_SUPPORT_RUNBOOK](../07-ccip/RELAY_BRIDGE_ADD_LINK_SUPPORT_RUNBOOK.md)
**Options:** A = extend CCIPRelayBridge to accept LINK; B = deploy separate LINK receiver. After implement + deploy + fund:
```bash
# In config/token-mapping.json set relaySupported: true for LINK
# Update TOKEN_MAPPING_AND_MAINNET_ADDRESSES.md and CCIP_BRIDGE_MAINNET_CONNECTION.md
# Restart relay service on r630-01: /opt/smom-dbis-138/services/relay
```
---
## 3. LAN: Blockscout verification
```bash
source smom-dbis-138/.env 2>/dev/null
./scripts/verify/run-contract-verification-with-proxy.sh
```
Single contract retry: `./scripts/verify/run-contract-verification-with-proxy.sh --only ContractName`
---
## 4. LAN: Fix E2E 502s
```bash
./scripts/maintenance/run-all-maintenance-via-proxmox-ssh.sh --e2e
# Or lighter:
./scripts/maintenance/address-all-remaining-502s.sh --run-besu-fix --e2e
```
**Runbook:** [502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md](502_DEEP_DIVE_ROOT_CAUSES_AND_FIXES.md)
---
## 5. LAN: Run all operator tasks (backup + verify ± deploy ± create-vms)
```bash
./scripts/run-all-operator-tasks-from-lan.sh --dry-run # print steps
./scripts/run-all-operator-tasks-from-lan.sh # backup + Blockscout verify
./scripts/run-all-operator-tasks-from-lan.sh --deploy # + contract deploy
./scripts/run-all-operator-tasks-from-lan.sh --create-vms # + create DBIS Core containers
./scripts/run-all-operator-tasks-from-lan.sh --deploy --create-vms
```
---
## 5a. LAN: Token-aggregation DB and migrations (VMID 5000)
If `/health` returns "database token_aggregation does not exist":
```bash
./scripts/apply-token-aggregation-fix.sh # create DB, run migrations, restart (via Proxmox)
./scripts/apply-token-aggregation-fix.sh --dry-run # print steps only
```
If VMID 5000 has no `postgres` user, run `createdb` and migrations on the host where PostgreSQL runs, or set token-aggregation `DATABASE_URL` to `explorer_db` and run `smom-dbis-138/services/token-aggregation/scripts/run-migrations.sh` there.
---
## 5b. LAN: Chain 138 next steps (Phase 2: preflight → mirror+pool → register c* as GRU → verify)
**Ref:** [DEPLOYMENT_ORDER_OF_OPERATIONS](../03-deployment/DEPLOYMENT_ORDER_OF_OPERATIONS.md) Phase 2. Use when mirror/pool/GRU registration or verify are pending.
```bash
./scripts/deployment/run-all-next-steps-chain138.sh --dry-run # print steps only
./scripts/deployment/run-all-next-steps-chain138.sh # run all (preflight, deploy mirror+pool, register c*, verify)
./scripts/deployment/run-all-next-steps-chain138.sh --skip-mirror # pool + register + verify only (set TRANSACTION_MIRROR_ADDRESS in smom-dbis-138/.env first)
```
If TransactionMirror deploy fails with **CreateCollision:** set `TRANSACTION_MIRROR_ADDRESS=0xC7f2Cf4845C6db0e1a1e91ED41Bcd0FcC1b0E141` in `smom-dbis-138/.env` and re-run with `--skip-mirror`. See [TRANSACTION_MIRROR_CHAIN138_COLLISION_FIX](../03-deployment/TRANSACTION_MIRROR_CHAIN138_COLLISION_FIX.md).
---
## 6. Low: DODO PMM on Chain 138
**Ref:** [OPTIONAL_DEPLOYMENTS_START_HERE](../07-ccip/OPTIONAL_DEPLOYMENTS_START_HERE.md) §2B
**Prereqs:** Set in `smom-dbis-138/.env`: `DODO_VENDING_MACHINE_ADDRESS`, `COMPLIANT_USDT_ADDRESS`, `COMPLIANT_USDC_ADDRESS`.
```bash
./scripts/run-optional-deployments.sh --execute --phases 7
# Or from smom-dbis-138: ./scripts/deployment/deploy-optional-future-all.sh (Phase 7 = DODO)
```
---
## 7. Low: Mainnet trustless stack (Lockbox138 + Mainnet)
**Ref:** [OPTIONAL_DEPLOYMENTS_START_HERE](../07-ccip/OPTIONAL_DEPLOYMENTS_START_HERE.md) §2C
**Prereqs:** `ETHEREUM_MAINNET_RPC`, Mainnet ETH for deployer.
```bash
cd smom-dbis-138
source .env
forge script script/bridge/trustless/DeployTrustlessBridge.s.sol:DeployTrustlessBridge \
--rpc-url "$ETHEREUM_MAINNET_RPC" --broadcast --via-ir --verify
# Then: Lockbox138 on 138; configure Lockbox138↔InboxETH; fund liquidity. See runbook §C.
```
---
## 8. Wave 0: sendCrossChain (real) and NPMplus backup
**sendCrossChain (real):** Requires `PRIVATE_KEY` and LINK approved in `.env`. Bridge: `0x971cD9D156f193df8051E48043C476e53ECd4693`.
```bash
bash scripts/bridge/run-send-cross-chain.sh 0.01 [recipient_address]
# Omit --dry-run to execute. Example: bash scripts/bridge/run-send-cross-chain.sh 0.01 0x...
```
**NPMplus backup:** Requires `NPM_PASSWORD` in `.env` and host on LAN.
```bash
bash scripts/verify/backup-npmplus.sh
# Or combined Wave 0: bash scripts/run-wave0-from-lan.sh
```
**NPMplus RPC fix (405):** From LAN: `bash scripts/nginx-proxy-manager/update-npmplus-proxy-hosts-api.sh`. Verify: `bash scripts/verify/verify-end-to-end-routing.sh`.
**NPMplus API unreachable (167/169):** Restart Docker inside NPMplus LXC: `./scripts/maintenance/fix-npmplus-services-via-proxmox-ssh.sh` (SSH to r630-01, restarts npmplus in 10233 and 10235).
**E2E from LAN (no public DNS):** If E2E fails at DNS (`Could not resolve host`), use [E2E_DNS_FROM_LAN_RUNBOOK.md](../04-configuration/E2E_DNS_FROM_LAN_RUNBOOK.md): append `config/e2e-hosts-append.txt` to `/etc/hosts`, then run `E2E_USE_SYSTEM_RESOLVER=1 ./scripts/verify/verify-end-to-end-routing.sh --profile=public`. Revert with `sudo ./scripts/verify/remove-e2e-hosts-from-etc-hosts.sh`.
**E2E profiles:** Use `--profile=public` for public endpoints (default) or `--profile=private` for private/admin RPC only. Run sequentially to avoid timestamp collision in evidence dirs. **Known E2E warnings** (502/404 and WS): [E2E_ENDPOINTS_LIST.md](../04-configuration/E2E_ENDPOINTS_LIST.md) § Known E2E warnings and Remediation. MIM4U web 502s and WS test-format warnings are **non-blocking** for contract/pool completion.
**Pre-PR validation:** Before opening PRs (Chainlist, token list, Trust Wallet), run `./scripts/run-before-pr-validations.sh` from repo root.
---
## 9. Wemix token verification (High)
Open [scan.wemix.com/tokens](https://scan.wemix.com/tokens); confirm WETH, USDT, USDC addresses. If different, update `config/token-mapping-multichain.json` and [WEMIX_TOKEN_VERIFICATION.md](../07-ccip/WEMIX_TOKEN_VERIFICATION.md). Then:
```bash
./scripts/validation/validate-config-files.sh
```
---
## References
- [COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md](COMPLETE_REQUIRED_OPTIONAL_RECOMMENDED_INDEX.md) — full plan (required, optional, recommended)
- [TODOS_CONSOLIDATED.md](TODOS_CONSOLIDATED.md) — full task list
- [NEXT_STEPS_AND_REMAINING_TODOS.md](NEXT_STEPS_AND_REMAINING_TODOS.md) — detail and completed items
- [STEPS_FROM_PROXMOX_OR_LAN_WITH_SECRETS.md](STEPS_FROM_PROXMOX_OR_LAN_WITH_SECRETS.md) — full LAN steps