Files
smom-dbis-138/runbooks/node-add-remove.md
defiQUG 1fb7266469 Add Oracle Aggregator and CCIP Integration
- Introduced Aggregator.sol for Chainlink-compatible oracle functionality, including round-based updates and access control.
- Added OracleWithCCIP.sol to extend Aggregator with CCIP cross-chain messaging capabilities.
- Created .gitmodules to include OpenZeppelin contracts as a submodule.
- Developed a comprehensive deployment guide in NEXT_STEPS_COMPLETE_GUIDE.md for Phase 2 and smart contract deployment.
- Implemented Vite configuration for the orchestration portal, supporting both Vue and React frameworks.
- Added server-side logic for the Multi-Cloud Orchestration Portal, including API endpoints for environment management and monitoring.
- Created scripts for resource import and usage validation across non-US regions.
- Added tests for CCIP error handling and integration to ensure robust functionality.
- Included various new files and directories for the orchestration portal and deployment scripts.
2025-12-12 14:57:48 -08:00

3.5 KiB

Node Add/Remove Runbook

Adding a Validator Node

Prerequisites

  1. Generate validator key
  2. Update genesis extraData
  3. Configure network access
  4. Prepare node infrastructure

Steps

  1. Generate Validator Key

    ./scripts/key-management/generate-validator-keys.sh 1
    
  2. Update Genesis extraData

    • Add validator address to genesis
    • Generate new extraData with Besu tooling
    • Update genesis.json
  3. Update static-nodes.json

    • Add validator enode to static-nodes.json
    • Update on all validators and sentries
  4. Deploy Validator Node

    kubectl apply -f k8s/base/validators/statefulset.yaml
    
  5. Verify Node

    • Check node is peering
    • Verify block production
    • Monitor metrics

Rollback

If issues occur:

  1. Remove validator from static-nodes.json
  2. Update genesis extraData
  3. Restart all nodes
  4. Verify network stability

Removing a Validator Node

Steps

  1. Update static-nodes.json

    • Remove validator enode
    • Update on all validators and sentries
  2. Update Genesis extraData

    • Remove validator address from genesis
    • Generate new extraData
    • Update genesis.json
  3. Stop Validator Node

    kubectl delete statefulset besu-validator-<index> -n besu-network
    
  4. Verify Network

    • Check remaining validators are working
    • Verify block production
    • Monitor metrics

Rollback

If issues occur:

  1. Add validator back to static-nodes.json
  2. Update genesis extraData
  3. Restart validator node
  4. Verify network stability

Adding a Sentry Node

Steps

  1. Generate Node Key

    ./scripts/key-management/generate-validator-keys.sh 1
    
  2. Update static-nodes.json

    • Add sentry enode to static-nodes.json
    • Update on all validators and sentries
  3. Deploy Sentry Node

    helm install besu-sentry-<index> ./helm/besu-network -f helm/besu-network/values-sentries.yaml -n besu-network
    
  4. Verify Node

    • Check node is peering
    • Verify P2P connections
    • Monitor metrics

Removing a Sentry Node

Steps

  1. Update static-nodes.json

    • Remove sentry enode
    • Update on all validators and sentries
  2. Stop Sentry Node

    helm uninstall besu-sentry-<index> -n besu-network
    
  3. Verify Network

    • Check remaining sentries are working
    • Verify P2P connections
    • Monitor metrics

Adding an RPC Node

Steps

  1. Deploy RPC Node

    helm install besu-rpc-<index> ./helm/besu-network -f helm/besu-network/values-rpc.yaml -n besu-network
    
  2. Update Load Balancer

    • Add RPC node to load balancer pool
    • Update health checks
  3. Verify Node

    • Check RPC endpoint is responding
    • Verify sync status
    • Monitor metrics

Removing an RPC Node

Steps

  1. Remove from Load Balancer

    • Remove RPC node from load balancer pool
    • Wait for connections to drain
  2. Stop RPC Node

    helm uninstall besu-rpc-<index> -n besu-network
    
  3. Verify Network

    • Check remaining RPC nodes are working
    • Verify load balancer health
    • Monitor metrics

Troubleshooting

Node Not Peering

  1. Check network connectivity
  2. Verify static-nodes.json configuration
  3. Check firewall rules
  4. Review node logs

Node Not Syncing

  1. Check genesis file matches
  2. Verify node configuration
  3. Check disk space
  4. Review node logs

Node Performance Issues

  1. Check resource usage
  2. Verify node configuration
  3. Review metrics
  4. Consider scaling resources