bea1903ac9
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
Co-authored-by: Cursor <cursoragent@cursor.com>
7.0 KiB
7.0 KiB
Peer Connections — Current State and Plan
Last Updated: 2026-02-18
Purpose: Document peer topology from RPC nodes, the 9 peers not connected to RPC 2101, and a short plan.
1. Peer counts by node (admin_peers)
Queried from nodes that expose RPC on port 8545:
| Node | IP | VMID | Peer count | Notes |
|---|---|---|---|---|
| Core | 192.168.11.211 | 2101 | 24 | All 5 validators (.100–.104), 2 sentries (.153, .154), 17 others |
| Core 2 | 192.168.11.212 | 2102 | 25 | Has 214, 221; does not list 211 |
| Public | 192.168.11.221 | 2201 | 0 | P2P may be disabled or not connected to static peers |
| ALLTRA 1 | 192.168.11.172 | 2500 | 15 | Has 221; does not list 211 |
| ALLTRA 2 | 192.168.11.173 | 2501 | 25 | Has 211, 212, 221, 214 |
| ALLTRA 3 | 192.168.11.174 | 2502 | 25 | Has 211, 212, 221, 172, 173 |
Takeaways:
- 2101 is well connected (24/33 possible; max-peers=32).
- 2102 has 25 peers but 2101 (211) is not in 2102’s peer list — mutual connection 2101↔2102 is missing.
- 2201 reports 0 peers — worth confirming if P2P is intended and if static/allowlist is applied.
- 2500 (172) does not peer with 211; 2501/2502 do peer with 211.
2. The 9 IPs not connected to RPC 2101
From check-rpc-2101-all-peers.sh (allowlist vs admin_peers on 2101):
| IP | Role / VMID | Status / note |
|---|---|---|
| 192.168.11.150 | Sentry 1 (1500) | P2P only; no RPC to query; optional peer for 2101 |
| 192.168.11.151 | Sentry 2 (1501) | Same as above |
| 192.168.11.152 | Sentry 3 (1502) | Same as above |
| 192.168.11.172 | ALLTRA RPC (2500) | Has 15 peers; 2101 not in its list (one-way: 2101 doesn’t see 172) |
| 192.168.11.203 | Putu 0x8a (2507) | Destroyed — VMID moved to 2307 (192.168.11.237) |
| 192.168.11.204 | Putu 0x1 (2508) | Destroyed — VMID moved to 2308 (192.168.11.238) |
| 192.168.11.212 | Core RPC 2 (2102) | 2102 has 25 peers; 211 not in 2102’s list — mutual 2101↔2102 missing |
| 192.168.11.214 | Sentry Alltra 2 (1506) | In 2102’s and 172’s peer lists; not in 2101’s |
| 192.168.11.221 | Public RPC (2201) | Reports 0 peers (P2P likely not connected) |
So: 2 destroyed (203, 204), 1 RPC with 0 peers (221), 1 other RPC not peering with 2101 (212), 1 ALLTRA RPC (172) not seen by 2101, 3 sentries (150, 151, 152), 1 sentry (214).
3. Plan
3.1 Clean up allowlist — DONE
- Removed 192.168.11.203 and 192.168.11.204 from
permissions-nodes.tomlandstatic-nodes.json(VMIDs destroyed; Putu now 2307/2308 at .237/.238). - To apply: Run deploy and restart 2101:
./scripts/maintenance/apply-peer-plan-fixes.sh
Or deploy only:./scripts/deploy-besu-node-lists-to-all.shthen
./scripts/maintenance/fix-core-rpc-2101.sh --restart-only
3.2 Optional: 2101 ↔ 2102 mutual connection
- Goal: Have Core RPC 2101 and Core RPC 2102 (212) peer with each other.
- Check: Both use the same allowlist (permissions-nodes.toml); 212 already has 25 peers and is at its limit if max-peers=25. Ensure 2102’s config has 211 in static/allowlist and that max-peers allows an extra peer (e.g. ≥26) if needed.
- Action: Confirm 2102’s Besu config (e.g. on host for VMID 2102): same permissions/static node list as 2101, and max-peers ≥ 26. Restart 2102 if config changed. Re-run
check-rpc-2101-all-peers.shto see if 212 appears under 2101’s peers.
3.3 Optional: Public RPC 2201 P2P
- Goal: 2201 to have P2P peers (redundancy, metrics). Repo
smom-dbis-138/config/config-rpc-public.tomlhasmax-peers=32and uses permissions/static; 2201 uses/genesis/and/permissions/paths inside the container. - Current: admin_peers on 2201 returns 0 (static/allowlist may not be mounted or discovery not finding peers).
- Action: On the host for VMID 2201 (r630-02), ensure the same node lists are at the container paths and config has
max-peers=32. Restart Besu. If 2201 is intentionally RPC-only, leave as is.
3.4 Optional: More peers on 2101 (sentries, 172, 214)
- Current: 2101 has 24 connected; max-peers=32, so there is room for up to 8 more.
- Options:
- Sentries 150, 151, 152: If they are up and in the allowlist, Besu will connect when slots free up and connections succeed; no change needed unless they are down or not in allowlist.
- 172 (2500): 2500 doesn’t list 211; ensuring 2500 has 211 in its allowlist/static and has free peer slots may allow 2101↔2500.
- 214: Same idea — allowlist/static on both sides and free slots.
- Action: Run
./scripts/verify/check-rpc-2101-all-peers.shafter any config or restart to see if the “not connected” set shrinks.
3.5 Verification
- After any change:
./scripts/verify/verify-rpc-2101-approve-and-sync.sh
./scripts/verify/check-rpc-2101-all-peers.sh
./scripts/monitoring/monitor-blockchain-health.sh
4. Full-mesh peering (all 32 nodes)
Goal: Every node can peer with all other nodes (31 peers per node).
- Network size: 32 nodes in
config/besu-node-lists/permissions-nodes.tomlandstatic-nodes.json. Each node should have max-peers ≥ 31 (we use 32). - Repo changes (done): All Besu configs in the repo now use max-peers=32:
smom-dbis-138/config: validator, member, rpc-core, rpc-public, rpc-4, rpc-putu-, rpc-luis-, rpc-perm (thirdweb already 50)config/generated-node-configs: 1505–1508, 2500–2505smom-dbis-138-proxmox/templates/besu-configs: validator, sentry, rpc, rpc-core, rpc-4smom-dbis-138-proxmox/install: besu-validator-install.sh, besu-sentry-install.sh, besu-rpc-install.sh
- To get full mesh on existing nodes:
- Deploy node lists (same list on every node):
./scripts/deploy-besu-node-lists-to-all.sh - Apply max-peers=32 on each node — copy the appropriate config from the repo to each VM (e.g. validators → config-validator.toml, 2101/2102 → config-rpc-core.toml, 2201 → config-rpc-public.toml, sentries → config-sentry.toml, etc.) or run your usual config-push process so every node has
max-peers=32. - Restart Besu on all nodes so they load the new config and node lists:
./scripts/besu/restart-besu-reload-node-lists.sh
- Deploy node lists (same list on every node):
- Check: After restarts, each RPC that exposes
admin_peersshould show up to 31 peers; run./scripts/verify/check-rpc-2101-all-peers.shfor 2101.
5. Reference
- Allowlist:
config/besu-node-lists/permissions-nodes.toml - Static nodes:
config/besu-node-lists/static-nodes.json - Apply plan (deploy + restart 2101):
./scripts/maintenance/apply-peer-plan-fixes.sh - Deploy node lists to all:
./scripts/deploy-besu-node-lists-to-all.sh - Restart all Besu (reload lists):
./scripts/besu/restart-besu-reload-node-lists.sh - Check 2101 peers:
./scripts/verify/check-rpc-2101-all-peers.sh - RPC/validator runbook: RPC_AND_VALIDATOR_TESTING_RUNBOOK.md