# Fix All Issues - Immediate Actions

**Date**: 2026-01-05  
**Issues Identified**: 
1. Wallet allowlist empty on all translators
2. Health checks not responding
3. RPC tests failing on VMIDs 241 and 242

---

## 🚀 Quick Fix

Run the automated fix script:

```bash
cd /home/intlc/projects/proxmox/rpc-translator-138
./scripts/fix-all-issues.sh
```

---

## 📋 Manual Fix Steps

### Step 1: Fix Web3Signer Keys

```bash
# Fix permissions
ssh root@192.168.11.11 "pct exec 107 -- bash -c 'for f in /opt/web3signer/data/keys/*.json; do [ -f \"\$f\" ] && chmod 644 \"\$f\"; done'"

# Restart Web3Signer
ssh root@192.168.11.11 "pct exec 107 -- systemctl restart web3signer && sleep 5"

# Verify keys
curl http://192.168.11.111:9000/api/v1/eth1/publicKeys
```

### Step 2: Configure Allowlist

```bash
cd /home/intlc/projects/proxmox/rpc-translator-138

# Get addresses
ADDRESSES=$(curl -s http://192.168.11.111:9000/api/v1/eth1/publicKeys | jq -r '.[]' | tr '\n' ',' | sed 's/,$//')

# Configure
./scripts/configure-wallet-allowlist.sh "$ADDRESSES"
```

### Step 3: Diagnose Health Endpoint Issues

For each translator (240, 241, 242):

```bash
# Check service status
ssh -i ~/.ssh/proxmox_translator root@192.168.11.240 "systemctl status rpc-translator-138.service --no-pager | head -15"

# Check if port is listening
ssh -i ~/.ssh/proxmox_translator root@192.168.11.240 "netstat -tlnp | grep ':9545' || ss -tlnp | grep ':9545'"

# Test health endpoint
curl -v http://192.168.11.240:9545/health

# Check logs
ssh -i ~/.ssh/proxmox_translator root@192.168.11.240 "journalctl -u rpc-translator-138.service -n 50 --no-pager | tail -20"
```

### Step 4: Restart Services if Needed

```bash
for IP in 192.168.11.240 192.168.11.241 192.168.11.242; do
    echo "Restarting $IP..."
    ssh -i ~/.ssh/proxmox_translator root@$IP "systemctl restart rpc-translator-138.service && sleep 3"
done
```

### Step 5: Verify All Services

```bash
cd /home/intlc/projects/proxmox/rpc-translator-138
./scripts/monitor-services.sh
```

---

## 🔍 Health Endpoint Troubleshooting

The health endpoint is implemented in `src/servers/http-server.ts` and should:
1. Check Besu connectivity
2. Check Redis connectivity
3. Check Web3Signer connectivity
4. Check Vault connectivity

**Possible Issues:**
- Health check might be timing out (async operations)
- Services might not be properly initialized
- Network connectivity issues
- Code might need to be rebuilt/redeployed

**Check if code is up to date:**
```bash
# On each translator VMID
ssh -i ~/.ssh/proxmox_translator root@192.168.11.240 "cd /opt/rpc-translator-138 && git log -1 --oneline"
```

**Rebuild and redeploy if needed:**
```bash
cd /home/intlc/projects/proxmox/rpc-translator-138
pnpm build
./scripts/deploy-all-vmids.sh
```

---

## ✅ Verification Checklist

- [ ] Web3Signer keys loaded (3 addresses)
- [ ] Allowlist configured on all 3 translators
- [ ] Health endpoint responding on all translators
- [ ] RPC methods working on all translators
- [ ] All services active and healthy

---

## 🎯 Expected Results

### After Step 1 (Keys)
```json
[
  "0x306290a09aefe8e7009c4fbd2662e1ee075255dc",
  "0x74b9ed9d5f37211128aec5b6de8ef5bb2762c68f",
  "0x7be3046f456a106d2ff8999ce90359dfc4c52f4c"
]
```

### After Step 2 (Allowlist)
```
VMID 192.168.11.240:
WALLET_ALLOWLIST=0x306290a09aefe8e7009c4fbd2662e1ee075255dc,0x74b9ed9d5f37211128aec5b6de8ef5bb2762c68f,0x7be3046f456a106d2ff8999ce90359dfc4c52f4c
```

### After Step 3 (Health Check)
```json
{
  "status": "ok",
  "service": "rpc-translator-138",
  "components": {
    "besu": { "healthy": true },
    "redis": { "healthy": true },
    "web3signer": { "healthy": true },
    "vault": { "healthy": true }
  }
}
```

---

**Status**: Run `./scripts/fix-all-issues.sh` to fix all issues automatically!
