# Deployment Checklist - Besu Nodes with New Keys

## Pre-Deployment Verification

### ✅ Key Generation Complete
- [x] 5 Validator keys generated
- [x] 4 Sentry keys generated
- [x] 3 RPC keys generated
- [x] All keys verified (`./verify-keys.sh` passed)

### ✅ Configuration Files Ready
- [x] `static-nodes.json` updated with new enode URLs
- [x] `genesis.json` ready (preserved with pre-allocated balances)
- [x] All config files synced to ml110

### ✅ Remote Sync Complete
- [x] Files synced to ml110 (192.168.11.10)
- [x] Keys verified on remote host
- [x] Backup created

## Deployment Steps

### 1. Verify Source Project Location
```bash
# On ml110 or local machine
# Ensure smom-dbis-138 is accessible from smom-dbis-138-proxmox
cd /opt/smom-dbis-138-proxmox
ls -la ../smom-dbis-138/keys/validators/
```

Expected: Should see validator-1 through validator-5 directories

### 2. Deploy Containers (if not already deployed)
```bash
cd /opt/smom-dbis-138-proxmox
./scripts/deployment/deploy-besu-nodes.sh
```

This will:
- Create LXC containers (VMIDs 1000-1004, 1500-1503, 2500-2502)
- Install Besu on each container
- Configure network settings

### 3. Copy Configuration Files and Keys
```bash
cd /opt/smom-dbis-138-proxmox
./scripts/copy-besu-config.sh ../smom-dbis-138
```

This will copy:
- `genesis.json` to all containers
- `config-validator.toml` to validators
- `config-sentry.toml` to sentries
- `config-rpc-*.toml` to RPC nodes (type-specific)
- `permissions-nodes.toml` and `permissions-accounts.toml` to all
- **Validator keys** to all validator containers

### 4. Update static-nodes.json
```bash
# The static-nodes.json should already be in the source project
# Copy it to containers or use bootstrap script
cd /opt/smom-dbis-138-proxmox
./scripts/network/bootstrap-network.sh
```

This will:
- Extract enode URLs from running nodes
- Update static-nodes.json on all containers

**Note**: If using pre-generated static-nodes.json, you can copy it directly:
```bash
cd /opt/smom-dbis-138-proxmox
pct push 1000 ../smom-dbis-138/config/static-nodes.json /etc/besu/static-nodes.json
# Repeat for all containers (1000-1004, 1500-1503, 2500-2502)
```

### 5. Configure Validator Keys

Each validator container needs its specific validator key directory:
- VMID 1000 → `/keys/validators/validator-1/`
- VMID 1001 → `/keys/validators/validator-2/`
- VMID 1002 → `/keys/validators/validator-3/`
- VMID 1003 → `/keys/validators/validator-4/`
- VMID 1004 → `/keys/validators/validator-5/`

**Note**: The `copy-besu-config.sh` script copies ALL validator keys to each validator. You may want to ensure each validator uses its specific key by updating `config-validator.toml` to reference the correct key directory.

Check `config-validator.toml` for:
```toml
miner-coinbase="/keys/validators/validator-{N}/address.txt"
```

### 6. Start Besu Services

```bash
cd /opt/smom-dbis-138-proxmox
./scripts/fix-besu-services.sh
```

Or manually:
```bash
# For validators
for vmid in 1000 1001 1002 1003 1004; do
    pct exec $vmid -- systemctl enable besu-validator.service
    pct exec $vmid -- systemctl start besu-validator.service
done

# For sentries
for vmid in 1500 1501 1502 1503; do
    pct exec $vmid -- systemctl enable besu-sentry.service
    pct exec $vmid -- systemctl start besu-sentry.service
done

# For RPC nodes
for vmid in 2500 2501 2502; do
    pct exec $vmid -- systemctl enable besu-rpc.service
    pct exec $vmid -- systemctl start besu-rpc.service
done
```

### 7. Verify Deployment

```bash
cd /opt/smom-dbis-138-proxmox
./scripts/validation/validate-deployment-comprehensive.sh
```

Or check manually:
```bash
# Check service status
for vmid in 1000 1001 1002 1003 1004; do
    echo "VMID $vmid:"
    pct exec $vmid -- systemctl status besu-validator.service --no-pager | head -5
done

# Check logs
pct exec 1000 -- journalctl -u besu-validator.service -n 50 --no-pager
```

### 8. Verify Network Connectivity

```bash
# Check if nodes can connect
pct exec 1000 -- curl -X POST -H "Content-Type: application/json" \
    --data '{"jsonrpc":"2.0","method":"net_peerCount","params":[],"id":1}' \
    http://localhost:8545

# Check block number
pct exec 1000 -- curl -X POST -H "Content-Type: application/json" \
    --data '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
    http://localhost:8545
```

### 9. Monitor Consensus

```bash
# Check validator participation
pct exec 1000 -- curl -X POST -H "Content-Type: application/json" \
    --data '{"jsonrpc":"2.0","method":"qbft_getValidatorsByBlockNumber","params":["latest"],"id":1}' \
    http://localhost:8545
```

## Troubleshooting

### Validator Keys Not Found
- Verify keys exist: `ls -la /opt/smom-dbis-138/keys/validators/`
- Check keys copied: `pct exec 1000 -- ls -la /keys/validators/`
- Verify ownership: `pct exec 1000 -- ls -la /keys/validators/validator-1/`

### Nodes Not Connecting
- Check static-nodes.json: `pct exec 1000 -- cat /etc/besu/static-nodes.json`
- Verify IP addresses match container IPs
- Check firewall rules
- Review Besu logs: `pct exec 1000 -- journalctl -u besu-validator.service -f`

### Consensus Issues
- Verify all 5 validators are running
- Check validator keys are correctly configured
- Verify genesis.json is identical on all nodes
- Check QBFT configuration in config files

## Key File Locations

### Source (ml110)
- Validators: `/opt/smom-dbis-138/keys/validators/validator-{1-5}/`
- Sentries: `/opt/smom-dbis-138/keys/sentries/sentry-{1-4}/`
- RPC: `/opt/smom-dbis-138/keys/rpc/rpc-{1-3}/`

### Target (in containers)
- Validator keys: `/keys/validators/validator-{1-5}/`
- Config files: `/etc/besu/`
- Data directory: `/data/besu/`

## Validator Addresses Reference

| Validator | Address | VMID | IP |
|-----------|---------|------|-----|
| validator-1 | `43ea6615474ac886c78182af1acbbf84346f2e9c` | 1000 | 192.168.11.100 |
| validator-2 | `05db2d6b5584285cc03cd33017c0f8da32652583` | 1001 | 192.168.11.101 |
| validator-3 | `23e1139cc8359872f8f4ef0d8f01c20355ac5f4b` | 1002 | 192.168.11.102 |
| validator-4 | `231a55a8ae9946b5dd2dc81c4c07522df42fd3ed` | 1003 | 192.168.11.103 |
| validator-5 | `c0af7f9251dc57cfb84c192c1bab20f5e312acb3` | 1004 | 192.168.11.104 |

---

**Last Updated**: 2025-12-20  
**Status**: Ready for deployment with new keys