proxmox/docs/06-besu/DEPLOYMENT_STRATEGY_EVALUATION.md

Deployment Strategy Evaluation

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation

---

Date: 2025-01-20
Status: Comprehensive Analysis Complete
Purpose: Evaluate best methods to accomplish all deployment tasks successfully

---

Current Situation

✅ Resolved Issues

1. Network Configuration

   • Problem: VMID 2101 had no IP address
   • Solution: Configured IP 192.168.11.211/24 with gateway
   • Status: ✅ FIXED

2. Internet Access

   • Problem: VM had no internet connectivity
   • Solution: Gateway configured, internet restored
   • Status: ✅ FIXED

3. Dependencies

   • Problem: Missing lib/ directory (forge-std, openzeppelin)
   • Solution: Copied lib/ directory to VM
   • Status: ✅ FIXED

4. Compilation Strategy

   • Problem: 266 files too large for single compilation
   • Solution: Incremental deployment script (compile per script)
   • Status: ✅ SOLVED

❌ Current Blocking Issues

1. "Replacement transaction underpriced" Errors

   • Multiple deployment attempts creating pending transactions
   • Same nonce being reused
   • New transactions can't replace pending ones

2. Transaction Pool Issues

   • Transactions stuck in mempool
   • Unable to verify transaction status reliably
   • Besu network may have transaction pool limitations

3. Deployment Verification

   • Contracts not appearing on-chain despite transaction submission
   • Transaction hashes exist but contracts not deployed

---

Root Cause Analysis

Issue 1: Nonce Management

Problem:

• Each deployment attempt uses the same nonce
• Previous transactions with that nonce are still pending
• New transactions can't replace them without higher gas price

Evidence:

• Transaction hash: 0x421f8e0a8265fd46ccb0f18502a355aaf7c375216a200281444d14bd7e6cea08
• Current nonce: 13104
• Error: "Replacement transaction underpriced"

Issue 2: Gas Price Strategy

Problem:

• Gas prices may not be high enough to replace pending transactions
• EIP-1559 parameters may not be optimal
• Besu network may have specific gas price requirements

Current Settings:

• Max fee: 1.5 gwei (1500000000 wei)
• Priority fee: 1.4 gwei (1400000000 wei)
• Base fee: 7 wei

Issue 3: Transaction Pool Management

Problem:

• Besu transaction pool may not be clearing properly
• Pending transactions blocking new deployments
• No reliable way to check/clear mempool

---

Solution Strategies

Strategy 1: Wait and Retry (RECOMMENDED)

Approach:

1. Wait for pending transactions to be mined
2. Check transaction status periodically
3. Once mined or failed, proceed with next deployment

Pros:

• No risk of transaction conflicts
• Respects network state
• Most reliable approach

Cons:

• Requires waiting (may take time)
• Need to monitor transaction status

Implementation:

# Wait for transaction to be mined
while true; do
  STATUS=$(cast receipt $TXHASH --rpc-url $RPC 2>&1)
  if echo "$STATUS" | grep -q "status.*0x1"; then
    echo "Transaction successful!"
    break
  elif echo "$STATUS" | grep -q "status.*0x0"; then
    echo "Transaction failed, retrying..."
    break
  fi
  sleep 10
done

Strategy 2: Clear Broadcast Cache and Use Higher Nonce

Approach:

1. Clear all broadcast cache files
2. Manually increment nonce
3. Deploy with explicit nonce

Pros:

• Bypasses pending transaction issues
• Full control over nonce

Cons:

• Requires manual nonce management
• Risk of nonce gaps if transaction eventually succeeds

Implementation:

# Clear cache
rm -rf broadcast/*/138/run-*.json

# Get current nonce
NONCE=$(cast nonce $DEPLOYER --rpc-url $RPC)

# Deploy with explicit nonce (if forge supports it)
forge script ... --nonce $NONCE

Strategy 3: Significantly Increase Gas Price

Approach:

1. Use much higher gas prices (10x or more)
2. Force replacement of pending transactions
3. Accept higher deployment costs

Pros:

• May replace pending transactions
• Faster deployment

Cons:

• Higher costs
• May not work if transaction already mined
• Wastes gas if pending tx eventually succeeds

Implementation:

# Use very high gas price
MAX_FEE="15000000000"  # 15 gwei
PRIORITY="14000000000"  # 14 gwei

forge script ... --with-gas-price $MAX_FEE --priority-gas-price $PRIORITY

Strategy 4: Use Besu Admin API to Clear Mempool

Approach:

1. Use Besu admin API to inspect/clear mempool
2. Remove pending transactions
3. Deploy fresh

Pros:

• Direct control over mempool
• Can clear specific transactions

Cons:

• Requires admin access
• May not be available on all Besu nodes
• Could affect other pending transactions

Implementation:

# Check if admin API available
cast rpc txpool_besuPendingTransactions --rpc-url $RPC

# Clear mempool (if supported)
cast rpc txpool_clearPendingTransactions --rpc-url $RPC

Strategy 5: Deploy Using Direct cast send

Approach:

1. Compile contracts locally
2. Extract bytecode
3. Use cast send --create directly
4. Full control over transaction parameters

Pros:

• Complete control over transaction
• Can set explicit nonce
• Can use very high gas prices
• Bypasses forge script limitations

Cons:

• More complex setup
• Need to extract bytecode manually
• More error-prone

Implementation:

# Extract bytecode
BYTECODE=$(jq -r '.bytecode.object' out/CCIPWETH9Bridge.sol/CCIPWETH9Bridge.json)

# Deploy with cast send
cast send --create $BYTECODE \
  --rpc-url $RPC \
  --private-key $PRIVATE_KEY \
  --nonce $NONCE \
  --max-fee-per-gas $MAX_FEE \
  --priority-fee-per-gas $PRIORITY

Strategy 6: Deploy One at a Time with Verification

Approach:

1. Deploy WETH9 Bridge
2. Wait and verify deployment
3. Only proceed to WETH10 after WETH9 confirmed
4. Repeat for each contract

Pros:

• Clear progress tracking
• Can verify each step
• Easier to debug issues

Cons:

• Slower overall process
• Requires manual intervention

---

Recommended Approach

Primary Strategy: Wait and Verify (Strategy 1 + Strategy 6)

Step-by-Step:

1. Check Current Transaction Status

   # Check if WETH9 transaction is mined
   TXHASH="0x421f8e0a8265fd46ccb0f18502a355aaf7c375216a200281444d14bd7e6cea08"
   cast receipt $TXHASH --rpc-url http://localhost:8545
   

2. If Transaction Pending

   • Wait for it to be mined (check every 30 seconds)
   • If successful, verify contract deployment
   • If failed, proceed to retry

3. If Transaction Failed or Not Found

   • Clear broadcast cache
   • Retry deployment with higher gas price
   • Use explicit nonce management

4. Deploy Each Contract Sequentially

   • WETH9 Bridge → Verify → WETH10 Bridge → Verify → LINK Token → Verify

Fallback Strategy: Direct cast send (Strategy 5)

If forge script continues to fail:

1. Compile contracts
2. Extract bytecode
3. Use cast send --create with explicit nonce and high gas prices

---

Implementation Plan

Phase 1: Transaction Status Verification

1. Check all pending transaction hashes
2. Wait for them to be mined (with timeout)
3. Verify contract deployments
4. Document results

Phase 2: Clean Deployment

1. Clear all broadcast caches
2. Get current nonce
3. Deploy with explicit nonce tracking
4. Verify each deployment before proceeding

Phase 3: Alternative Methods

If Phase 2 fails:

1. Use cast send --create method
2. Extract bytecode from compiled contracts
3. Deploy directly with full control

---

Best Practices

1. Always Verify Before Proceeding

   • Check transaction receipt
   • Verify contract code on-chain
   • Confirm deployment address

2. Manage Nonces Explicitly

   • Track nonce manually if needed
   • Don't rely on automatic nonce management
   • Clear cache between attempts

3. Use Appropriate Gas Prices

   • Check network minimum gas price
   • Use 2-3x minimum for safety
   • Monitor base fee for EIP-1559

4. Monitor Transaction Pool

   • Check for pending transactions
   • Wait for clearance before retrying
   • Use admin APIs if available

5. Incremental Deployment

   • Deploy one contract at a time
   • Verify each before proceeding
   • Keep detailed logs

---

Tools and Scripts Needed

1. Transaction Status Checker

   • Monitor transaction status
   • Wait for confirmation
   • Report results

2. Nonce Manager

   • Track current nonce
   • Increment safely
   • Handle gaps

3. Gas Price Calculator

   • Dynamic gas price calculation
   • Safety multipliers
   • EIP-1559 support

4. Deployment Verifier

   • Check contract code on-chain
   • Verify deployment addresses
   • Confirm functionality

---

Success Criteria

✅ All contracts deployed:

• WETH9 Bridge at expected address
• WETH10 Bridge at generated address
• LINK Token at canonical address

✅ All transactions confirmed:

• Transaction receipts show success
• Contracts have code on-chain
• Addresses match expected values

✅ Configuration complete:

• Bridge destinations configured
• Mainnet chain selector set
• All settings verified

---

Status: Ready for implementation
Next: Execute Phase 1 - Transaction Status Verification