proxmox/docs/archive/completion/QBFT_FINAL_RESOLUTION_SUMMARY.md

QBFT Transaction Resolution - Final Summary

Date: $(date)
Network: Hyperledger Besu QBFT
Issue: Stuck transaction blocking Ethereum Mainnet configuration

---

✅ Completed Investigation

1. Enabled TXPOOL and ADMIN RPC Methods

• ✅ TXPOOL enabled on RPC node (192.168.11.250)
• ✅ ADMIN enabled on RPC node
• ✅ Used txpool_besuTransactions to inspect transaction pool

2. Identified Stuck Transaction

• Hash: 0x359e4e1501d062e32077ca5cb854c46ef7df4b0233431befad1321c0c7a20670
• Nonce: 23
• From: 0x4A666F96fC8764181194447A7dFdb7d471b301C8
• Gas Price: 20 gwei (visible in RPC pool)
• Status: Stuck - blocks all replacement attempts

3. Attempted Resolution Methods

✅ Enabled TXPOOL

• Script: scripts/enable-txpool-rpc-ssh.sh
• Result: Successfully enabled

✅ Enabled ADMIN

• Script: scripts/enable-admin-rpc-ssh.sh
• Result: Successfully enabled

❌ Remove Transaction via RPC

• Method: admin_removeTransaction
• Result: Not available in this Besu version

❌ Replace with Higher Gas Price

• Attempted: 50,000 gwei (2,500x higher than visible 20 gwei)
• Result: Still "Replacement transaction underpriced"

---

🔍 Root Cause Analysis

Why Replacement Fails

1. Transaction on Validator Nodes: The stuck transaction is likely in validator nodes' mempools, not just the RPC node. QBFT validators maintain their own transaction pools.

2. Hidden Gas Price: The transaction visible in RPC pool shows 20 gwei, but validators may have a different version with much higher gas price (>1,000,000 gwei as previously identified).

3. QBFT Consensus: In QBFT, validators must agree on transaction ordering. A transaction stuck in validator pools cannot be easily replaced without validator coordination.

4. Transaction Persistence: Previous attempts to clear (restart, database clear) failed because:

   • Transaction is in blockchain state (nonce 23 is on-chain)
   • Validators re-broadcast the transaction
   • Network re-syncs restore the state

---

🎯 Recommended Solution

Use a Different Deployer Account

Since the current account's nonce 23 is permanently stuck in the QBFT network state, the most reliable solution is to use a different account:

# 1. Create new account (already created: 0xC13EfAe66708C7541d2D66A2527bcBF9992e7186)
# 2. Fund the new account
cast send 0xC13EfAe66708C7541d2D66A2527bcBF9992e7186 \
  --value 10ether \
  --rpc-url http://192.168.11.250:8545 \
  --private-key $PRIVATE_KEY

# 3. Update .env with new PRIVATE_KEY
# 4. Configure Ethereum Mainnet with new account
./scripts/configure-ethereum-mainnet-final.sh

---

📋 Alternative Solutions (If New Account Not Possible)

Option 1: Wait for Transaction Expiration

• Retention Period: 6 hours (default tx-pool-retention-hours)
• Risk: Transaction may persist beyond retention period if it's in blockchain state

Option 2: Coordinate Validator Restart

• Restart all validators simultaneously
• Clear all validator transaction pools
• Risk: May not work if transaction is in blockchain state

Option 3: Network Fork (Not Recommended)

• Requires network-wide coordination
• High risk of consensus issues
• Not recommended for production

---

📊 Besu QBFT-Specific Findings

Available RPC Methods

• ✅ txpool_besuTransactions - List all transactions in pool
• ❌ txpool_content - Not available
• ❌ txpool_status - Not available
• ❌ txpool_clear - Not available
• ❌ admin_removeTransaction - Not available

Transaction Pool Behavior

• QBFT validators maintain separate transaction pools
• RPC node pool is separate from validator pools
• Transaction propagation between nodes may be inconsistent
• Replacement transactions require higher gas price across all nodes

---

🛠️ Scripts Created

1. scripts/enable-txpool-rpc-ssh.sh - Enable TXPOOL via SSH
2. scripts/enable-admin-rpc-ssh.sh - Enable ADMIN via SSH
3. scripts/resolve-stuck-transaction-besu-qbft.sh - Comprehensive resolution
4. scripts/remove-stuck-transaction-besu.sh - Remove specific transaction

---

📝 Lessons Learned

1. QBFT networks require validator coordination for transaction management
2. Transaction pools are node-specific, not network-wide
3. Besu RPC methods are limited compared to Geth
4. Nonce management is critical - stuck nonces are difficult to resolve
5. Different accounts are the most reliable bypass for stuck transactions

---

🎯 Final Recommendation

Use a different deployer account to configure Ethereum Mainnet. This is the most reliable solution for QBFT networks where transaction state is distributed across validators.

---

Last Updated: $(date)
Status: ⚠️ STUCK TRANSACTION PERSISTS - USE DIFFERENT ACCOUNT