# VM Restart and Verification Status

**Date:** 2026-02-03  
**Purpose:** Post-restart verification, health checks, and known issues

---

## VM Restart Summary

### Restarted Successfully (via SSH)

| Host | VMIDs Restarted | Status |
|------|-----------------|--------|
| **r630-01** (192.168.11.11) | 106, 107, 108, 10000, 10001, 10020, 10100, 10101, 10120, 10233 | ✅ All running |
| **r630-02** (192.168.11.12) | 2201, 2303, 2401, 6200, 6201, 10234 | ✅ All running |
| **ml110** (192.168.11.10) | 2301 | ✅ Recreated 2026-02-04 (was corrupted) |

### Previously Failed — Now Resolved

- **CT 2301** (ml110): **Was** corrupted rootfs (bad superblock). **Resolved 2026-02-04:** Recreated via `scripts/recreate-ct-2301.sh`. See [scripts/README.md § CT 2301](../../scripts/README.md). If container shows mount/fs errors again, use `scripts/fix-ct-2301-corrupted-rootfs.sh` for recovery options.

---

## Health Verification Results

### Backend VMs (verify-backend-vms.sh)

| VMID | Service | Status | IP | Notes |
|------|---------|--------|-----|-------|
| 2101 | besu-rpc-core-1 | ✅ Running | 192.168.11.211 | RPC health OK, 12 peers |
| 2201 | besu-rpc-public-1 | ✅ Running | 192.168.11.221 | RPC health OK; admin API disabled (by design) |
| 5000 | blockscout-1 | ✅ Running | 192.168.11.140 | Nginx active, HTTP 200 |
| 7810 | mim-web-1 | ✅ Running | 192.168.11.37 | Nginx active |
| 2400 | thirdweb-rpc-1 | ✅ Running | 192.168.11.240 | Nginx active; config path thirdweb-rpc not found (uses info-defi-oracle) |
| 10150 | dbis-api-primary | ✅ Running | 192.168.11.155 | Port 3000 listening; IP fix applied |
| 10151 | dbis-api-secondary | ✅ Running | 192.168.11.156 | Port 3000 listening; IP fix applied |
| 10130 | dbis-frontend | ⚠️ Running | - | Nginx **inactive**; requires service start |

### Besu Peer Verification

- **RPC Core (2101):** 12 peers connected ✅
- **RPC Public (2201):** admin_nodeInfo returns error — **ADMIN API disabled** for public RPC (security best practice). Use eth_* methods only.

### Static Nodes

- **master-static-nodes.json:** 30 enodes; format valid
- **RPC Core enode:** Advertises 192.168.11.250 in nodeInfo (p2p-host may differ from eth0); peers connect correctly

---

## Known Issues and Warnings

1. **CT 2301** — ✅ **Resolved 2026-02-04:** Recreated via `scripts/recreate-ct-2301.sh`. See [scripts/README.md § CT 2301](../../scripts/README.md). Besu config may need to be reinstalled (copy from 2101/2201) if not included in recreate script.
2. **dbis-frontend (10130)** — Nginx inactive; `nginx.service` not found (container may use Node.js or other server)
3. **thirdweb-rpc (2400)** — Fixed: verification script now expects `rpc-thirdweb` (exists)
4. **RPC Public admin API** — Intentionally disabled; besu-verify-peers requires admin; use RPC Core (2101) for peer verification

---

## Script Fixes Applied (2026-02-03)

- **verify-backend-vms.sh:** Parse `ip=X.X.X.X` from net0 line in pct config (fixes dbis-api 10150/10151 IP detection); sanitize nginx_status; rpc-thirdweb config path
- **fix-ct-2301-corrupted-rootfs.sh:** New script with CT 2301 recovery options
- **backup-proxmox-configs.sh:** New script; backs up config/, .env; supports --dry-run
- **create-chain138-containers.sh:** Added --dry-run
- **verify-min-gas-price.sh:** New script in scripts/verify/

---

## Related Documentation

- [NETWORK_CONFIGURATION_MASTER.md](../../docs/11-references/NETWORK_CONFIGURATION_MASTER.md)
- [VMID_IP_FIXED_REFERENCE.md](../../docs/11-references/VMID_IP_FIXED_REFERENCE.md)
- [OPERATIONAL_RUNBOOKS.md](../../docs/03-deployment/OPERATIONAL_RUNBOOKS.md)