smom-dbis-138/docs/deployment/DEPLOYMENT.md

# Deployment Guide

**Last Updated**: 2025-01-27  
**Status**: Active

This guide provides step-by-step instructions for deploying the DeFi Oracle Meta Mainnet (ChainID 138) on Azure Kubernetes Service (AKS).

> **Related Documentation**:
> - [Deployment Quick Start](../DEPLOYMENT_QUICK_START.md) - Fast deployment guide
> - [Deployment Checklist](DEPLOYMENT_CHECKLIST.md) - Deployment checklist
> - [Architecture Documentation](../architecture/ARCHITECTURE.md) - System architecture
> - [Configuration Index](../configuration/CONFIGURATION_INDEX.md) - Configuration guides

## Table of Contents

- [Prerequisites](#prerequisites)
- [Step 1: Generate Genesis and Keys](#step-1-generate-genesis-and-keys)
- [Step 2: Deploy Azure Infrastructure](#step-2-deploy-azure-infrastructure-admin-region--multi-region)
- [Step 3: Deploy Kubernetes Resources](#step-3-deploy-kubernetes-resources)
- [Step 4: Deploy Monitoring](#step-4-deploy-monitoring)
- [Step 5: Deploy Blockscout](#step-5-deploy-blockscout)
- [Step 6: Deploy Contracts](#step-6-deploy-contracts)
- [Step 7: Deploy Oracle Publisher](#step-7-deploy-oracle-publisher)
- [Step 8: Tatum SDK Integration](#step-8-tatum-sdk-integration)
- [Step 9: Verification](#step-9-verification)
- [Troubleshooting](#troubleshooting)
  - [Nodes not syncing](#nodes-not-syncing)
  - [RPC errors](#rpc-errors)
  - [Oracle not updating](#oracle-not-updating)
- [Next Steps](#next-steps)

## Prerequisites

- Azure CLI installed and configured
- Terraform >= 1.0
- kubectl configured for AKS
- Helm 3.x
- Besu CLI tools
- Foundry (forge, cast, anvil)

## Step 1: Generate Genesis and Keys

1. Generate validator keys:
```bash
./scripts/key-management/generate-validator-keys.sh 4
```

2. Generate oracle keys:
```bash
./scripts/key-management/generate-oracle-keys.sh
```

3. Generate genesis file:
```bash
./scripts/generate-genesis.sh
```

4. Store keys in Azure Key Vault:
```bash
./scripts/key-management/azure-keyvault-setup.sh
```

## Step 2: Deploy Azure Infrastructure (Admin Region + Multi-Region)

1. Navigate to Terraform directory:
```bash
cd terraform
```

2. Initialize Terraform:
```bash
terraform init
```

3. Create terraform.tfvars:
```bash
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values
```

4. Plan deployment for the West Europe admin cluster:
```bash
terraform plan -lock-timeout=5m
```

5. Apply infrastructure:
```bash
terraform apply -lock-timeout=5m
```

6. (Recommended) Run a **canary multi-region deployment** for a single workload region before rolling out globally:
```bash
cd ..
scripts/deployment/canary-region.sh northeurope
```

7. After the canary region is healthy, roll out to all 36 workload regions:
```bash
cd terraform
terraform plan -lock-timeout=5m
terraform apply -lock-timeout=5m
```

8. Get kubeconfig for the West Europe admin cluster (adjust if you changed names):
```bash
az aks get-credentials --resource-group az-p-wst-rg-comp-001 --name az-p-wst-aks-main --overwrite-existing
```

## Step 3: Deploy Kubernetes Resources

1. Create namespace:
```bash
kubectl apply -f k8s/base/namespace.yaml
```

2. Deploy validators:
```bash
helm install besu-validators ./helm/besu-network -f helm/besu-network/values-validators.yaml -n besu-network
```

3. Deploy sentries:
```bash
helm install besu-sentries ./helm/besu-network -f helm/besu-network/values-sentries.yaml -n besu-network
```

4. Deploy RPC nodes:
```bash
helm install besu-rpc ./helm/besu-network -f helm/besu-network/values-rpc.yaml -n besu-network
```

5. Deploy API gateway:
```bash
kubectl apply -f k8s/gateway/nginx-config.yaml
```

## Step 4: Deploy Monitoring

1. Create monitoring namespace:
```bash
kubectl create namespace monitoring
```

2. Deploy Prometheus:
```bash
kubectl apply -f monitoring/k8s/prometheus.yaml
```

3. Deploy Grafana (optional):
```bash
helm install grafana grafana/grafana -n monitoring
```

## Step 5: Deploy Blockscout

1. Deploy Blockscout database:
```bash
kubectl apply -f k8s/blockscout/deployment.yaml
```

2. Wait for database to be ready:
```bash
kubectl wait --for=condition=ready pod -l app=blockscout-db -n besu-network --timeout=300s
```

3. Blockscout will automatically run migrations on startup.

## Step 6: Deploy Contracts

1. Set environment variables:
```bash
export RPC_URL="https://rpc.d-bis.org"
export PRIVATE_KEY="your-private-key"
```

2. Deploy WETH:
```bash
./scripts/deployment/deploy-weth.sh
```

3. Deploy Multicall:
```bash
./scripts/deployment/deploy-multicall.sh
```

4. Deploy Oracle Aggregator:
```bash
forge script script/DeployOracle.s.sol --rpc-url $RPC_URL --broadcast --private-key $PRIVATE_KEY
```

## Step 7: Deploy Oracle Publisher

1. Update oracle configuration:
```bash
kubectl create configmap oracle-config --from-literal=aggregator_address=<AGGREGATOR_ADDRESS> -n besu-network
```

2. Deploy oracle publisher:
```bash
kubectl apply -f services/oracle-publisher/k8s/deployment.yaml
```

## Step 8: Tatum SDK Integration

1. Install SDK dependencies:
```bash
cd sdk
npm install
```

2. Configure environment:
```bash
cp env.example .env
# Edit .env with your RPC endpoint
```

3. Test connection:
```bash
npm run test
```

4. Run examples:
```bash
# Basic usage
npm run example:basic

# Send transaction
npm run example:transaction

# Deploy contract
npm run example:contract
```

See [Tatum SDK Integration Guide](TATUM_SDK.md) for detailed documentation.

## Step 9: Verification

1. Check node status:
```bash
kubectl get pods -n besu-network
```

2. Check block production:
```bash
kubectl logs -f besu-validator-0 -n besu-network
```

3. Test RPC endpoint:
```bash
curl -X POST https://rpc.d-bis.org \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'
```

4. Test Tatum SDK integration:
```bash
cd sdk
npm run test
npm run smoke-test
```

## Troubleshooting

### Nodes not syncing
- Check network connectivity
- Verify genesis file matches across all nodes
- Check validator keys are correctly configured

### RPC errors
- Verify RPC nodes are synced
- Check API gateway configuration
- Review rate limiting settings

### Oracle not updating
- Check oracle publisher logs
- Verify aggregator contract address
- Check private key is correctly configured

## Next Steps

- Configure monitoring alerts
- Set up backup procedures
- Review security hardening
- Document operational procedures