# Flush All Validator Mempools - Instructions

**Date**: $(date)  
**Status**: ⚠️ **ACTION REQUIRED ON PROXMOX HOST**

---

## 🎯 Objective

Flush mempools on all Besu validator nodes (and RPC nodes) to clear stuck transactions blocking Ethereum Mainnet configuration.

---

## 📋 Validator Nodes

| VMID | IP Address | Service Name |
|------|------------|--------------|
| 1000 | 192.168.11.100 | besu-validator.service |
| 1001 | 192.168.11.101 | besu-validator.service |
| 1002 | 192.168.11.102 | besu-validator.service |
| 1003 | 192.168.11.103 | besu-validator.service |
| 1004 | 192.168.11.104 | besu-validator.service |

## 📋 RPC Nodes (Also Need Flushing)

| VMID | IP Address | Service Name |
|------|------------|--------------|
| 2500 | 192.168.11.250 | besu-rpc.service |
| 2501 | 192.168.11.251 | besu-rpc.service |
| 2502 | 192.168.11.252 | besu-rpc.service |

---

## 🔧 Method 1: Automated Script (Recommended)

**On the Proxmox host**, run:

```bash
cd /home/intlc/projects/proxmox
chmod +x scripts/flush-all-mempools-proxmox.sh
./scripts/flush-all-mempools-proxmox.sh
```

This script will:
1. Restart all validator services (1000-1004)
2. Restart all sentry services (1500-1503)
3. Restart all RPC services (2500-2502)
4. Verify services are running
5. Report summary

---

## 🔧 Method 2: Manual Restart (If Script Fails)

**On the Proxmox host**, run these commands:

### Validators
```bash
for vmid in 1000 1001 1002 1003 1004; do
    echo "Restarting VMID $vmid..."
    pct exec $vmid -- systemctl restart besu-validator.service
    sleep 2
done
```

### RPC Nodes
```bash
for vmid in 2500 2501 2502; do
    echo "Restarting VMID $vmid..."
    pct exec $vmid -- systemctl restart besu-rpc.service
    sleep 2
done
```

### Verify Services
```bash
for vmid in 1000 1001 1002 1003 1004 2500 2501 2502; do
    if pct exec $vmid -- pgrep -f "besu" >/dev/null 2>&1; then
        echo "✓ VMID $vmid: Besu running"
    else
        echo "✗ VMID $vmid: Besu not running"
    fi
done
```

---

## 🔧 Method 3: SSH to Each Node (If pct Not Available)

**From a machine with SSH access**, run:

```bash
# Validators
for ip in 192.168.11.100 192.168.11.101 192.168.11.102 192.168.11.103 192.168.11.104; do
    echo "Restarting $ip..."
    ssh root@$ip "systemctl restart besu-validator.service"
    sleep 2
done

# RPC Nodes
for ip in 192.168.11.250 192.168.11.251 192.168.11.252; do
    echo "Restarting $ip..."
    ssh root@$ip "systemctl restart besu-rpc.service"
    sleep 2
done
```

---

## ⏱️ Wait for Services to Stabilize

After restarting, wait 15-30 seconds for services to come back online:

```bash
# Wait and verify RPC is online
for i in {1..15}; do
    if cast block-number --rpc-url http://192.168.11.250:8545 >/dev/null 2>&1; then
        echo "✅ RPC is online!"
        break
    else
        echo "  Attempt $i/15: Waiting..."
        sleep 2
    fi
done
```

---

## ✅ Verify Mempools Are Flushed

After restarting, verify the mempool issue is resolved:

```bash
cd /home/intlc/projects/proxmox
./scripts/configure-ethereum-mainnet-final.sh
```

**Expected Result**: Transactions should succeed without "Replacement transaction underpriced" errors.

---

## 📊 Current Status

- **Issue**: Stuck transaction in mempool with extremely high gas price
- **Blocking**: Ethereum Mainnet configuration
- **Solution**: Restart Besu services to flush mempools
- **Status**: ⏳ **AWAITING MANUAL RESTART ON PROXMOX HOST**

---

## 🎯 Next Steps After Flushing

1. **Wait for services to stabilize** (15-30 seconds)
2. **Verify RPC is online**: `cast block-number --rpc-url http://192.168.11.250:8545`
3. **Configure Ethereum Mainnet**: `./scripts/configure-ethereum-mainnet-final.sh`
4. **Verify configuration**: `./scripts/test-bridge-all-7-networks.sh weth9`
5. **Expected**: 7/7 networks configured ✅

---

## 📝 Scripts Created

1. `scripts/flush-all-mempools-proxmox.sh` - Automated script for Proxmox host
2. `scripts/flush-validator-mempools.sh` - Validator-specific script
3. `scripts/configure-ethereum-mainnet-final.sh` - Configuration script (ready to run after flush)

---

**Last Updated**: $(date)  
**Action Required**: Run flush script on Proxmox host, then configure Ethereum Mainnet

