Sankofa/docs/smom-dbis-138/smom-dbis-138-next-steps.md

# SMOM-DBIS-138 Next Steps Guide

## Date
2025-12-08

## Status
✅ **All VMs Deployed - Ready for Configuration**

---

## Current Status

### VM Deployment
- ✅ **18 VMs Created** with VMIDs assigned
- ⚠️ **VMs in "stopped" state** - Normal during initial creation
- ⏳ **Waiting for VMs to start** - Controller is processing

### Infrastructure VMs
- ✅ nginx-proxy-vm (VMID: 118)
- ✅ cloudflare-tunnel-vm (VMID: 119)

### Application VMs
- ✅ 4 Validators (VMIDs: 132, 133, 134, 135)
- ✅ 4 Sentries (VMIDs: 127, 128, 129, 130)
- ✅ 4 RPC Nodes (VMIDs: 123, 124, 125, 126)
- ✅ Services (VMID: 131)
- ✅ Blockscout (VMID: 120)
- ✅ Monitoring (VMID: 122)
- ✅ Management (VMID: 121)

---

## Project Locations

- **Sankofa Project** (VM Deployment): `~/projects/Sankofa`
- **SMOM-DBIS-138 Project** (Blockchain Network): `~/projects/smom-dbis-138`

**Important**: The actual SMOM-DBIS-138 blockchain network configuration, contracts, and deployment scripts are in the separate project at `~/projects/smom-dbis-138`. After VMs are running, you'll need to configure them using scripts from that project.

## Step-by-Step Next Steps

### 1. Monitor VM Startup

```bash
# Watch VM status
kubectl get proxmoxvm -A -w

# Or use verification script
./scripts/verify-deployment.sh
```

**Expected**: VMs will transition from "stopped" to "running" state and receive IP addresses.

### 1.1. Collect VM IP Addresses

Once VMs have IP addresses, collect them for use in SMOM-DBIS-138 project:

```bash
# From Sankofa project root
./scripts/get-smom-vm-ips.sh
```

This will:
- Collect all VM IP addresses
- Save to `smom-vm-ips.txt` in Sankofa project
- Copy to `~/projects/smom-dbis-138/config/vm-ips.txt` for use in SMOM-DBIS-138 project

---

### 2. Configure Infrastructure VMs

#### 2.1 Nginx Proxy VM

**Get VM IP:**
```bash
kubectl get proxmoxvm nginx-proxy-vm -n default -o jsonpath='{.status.ipAddress}'
```

**SSH into VM:**
```bash
ssh admin@<nginx-proxy-ip>
```

**Configuration Steps:**
1. Install SSL certificates:
   ```bash
   sudo certbot --nginx -d your-domain.com
   ```

2. Configure backend services:
   - See: `docs/configs/nginx/README.md`
   - Create configs in `/etc/nginx/sites-available/`
   - Enable and reload nginx

3. Test configuration:
   ```bash
   sudo nginx -t
   sudo systemctl reload nginx
   ```

**Documentation**: `docs/configs/nginx/README.md`

#### 2.2 Cloudflare Tunnel VM

**Get VM IP:**
```bash
kubectl get proxmoxvm cloudflare-tunnel-vm -n default -o jsonpath='{.status.ipAddress}'
```

**SSH into VM:**
```bash
ssh admin@<cloudflare-tunnel-ip>
```

**Configuration Steps:**
1. Create tunnel in Cloudflare dashboard:
   - Go to Zero Trust → Networks → Tunnels
   - Create tunnel: `smom-dbis-138-tunnel`
   - Copy tunnel token/credentials

2. Configure credentials:
   ```bash
   sudo mkdir -p /etc/cloudflared
   sudo nano /etc/cloudflared/tunnel-credentials.json
   # Paste tunnel credentials
   ```

3. Configure tunnel:
   ```bash
   sudo cp docs/configs/cloudflare/tunnel-config.yaml /etc/cloudflared/config.yaml
   sudo nano /etc/cloudflared/config.yaml
   # Update with your settings
   ```

4. Start tunnel:
   ```bash
   sudo systemctl start cloudflared
   sudo systemctl enable cloudflared
   sudo systemctl status cloudflared
   ```

**Documentation**: `docs/configs/cloudflare/README.md`

---

### 3. Configure Application VMs

#### 3.1 Validators (4 VMs)

**Get VM IPs:**
```bash
# Use the IP collection script
./scripts/get-smom-vm-ips.sh

# Or manually
for i in 01 02 03 04; do
  echo "Validator $i:"
  kubectl get proxmoxvm smom-validator-$i -n default -o jsonpath='{.status.ipAddress}'
  echo
done
```

**Configuration Steps:**
1. **Switch to SMOM-DBIS-138 project:**
   ```bash
   cd ~/projects/smom-dbis-138
   ```

2. **Update configuration with VM IPs:**
   ```bash
   source config/vm-ips.txt  # If available
   # Or manually update .env file with validator IPs
   ```

3. **Use SMOM-DBIS-138 deployment scripts:**
   ```bash
   # Check available commands
   make help
   
   # Deploy validators (example - check actual commands in SMOM-DBIS-138 project)
   # make deploy-validators
   ```

4. **SSH into each validator VM** and configure:
   - Install Hyperledger Besu
   - Configure validator keys
   - Set up genesis configuration
   - Start validator service
   - Verify validator is syncing

**Reference**: See `~/projects/smom-dbis-138/README.md` and `~/projects/smom-dbis-138/runbooks/` for detailed instructions.

#### 3.2 Sentries (4 VMs)

**Get VM IPs:**
```bash
for i in 01 02 03 04; do
  echo "Sentry $i:"
  kubectl get proxmoxvm smom-sentry-$i -n default -o jsonpath='{.status.ipAddress}'
  echo
done
```

**Configuration Steps:**
1. SSH into each sentry VM
2. Install SMOM-DBIS-138 sentry software
3. Configure sentry connections to validators
4. Start sentry service
5. Verify sentry is connected

#### 3.3 RPC Nodes (4 VMs)

**Get VM IPs:**
```bash
for i in 01 02 03 04; do
  echo "RPC Node $i:"
  kubectl get proxmoxvm smom-rpc-node-$i -n default -o jsonpath='{.status.ipAddress}'
  echo
done
```

**Configuration Steps:**
1. SSH into each RPC node VM
2. Install SMOM-DBIS-138 RPC node software
3. Configure RPC endpoints
4. Start RPC service
5. Verify RPC is accessible

#### 3.4 Services VM

**Get VM IP:**
```bash
kubectl get proxmoxvm smom-services -n default -o jsonpath='{.status.ipAddress}'
```

**Configuration Steps:**
1. SSH into services VM
2. Install SMOM-DBIS-138 services
3. Configure service endpoints
4. Start services
5. Verify services are running

#### 3.5 Blockscout VM

**Get VM IP:**
```bash
kubectl get proxmoxvm smom-blockscout -n default -o jsonpath='{.status.ipAddress}'
```

**Configuration Steps:**
1. SSH into blockscout VM
2. Install Blockscout explorer
3. Configure database connection
4. Start Blockscout service
5. Verify explorer is accessible

#### 3.6 Monitoring VM

**Get VM IP:**
```bash
kubectl get proxmoxvm smom-monitoring -n default -o jsonpath='{.status.ipAddress}'
```

**Configuration Steps:**
1. SSH into monitoring VM
2. Install monitoring stack (Prometheus, Grafana)
3. Configure metrics collection
4. Start monitoring services
5. Verify dashboards are accessible

#### 3.7 Management VM

**Get VM IP:**
```bash
kubectl get proxmoxvm smom-management -n default -o jsonpath='{.status.ipAddress}'
```

**Configuration Steps:**
1. SSH into management VM
2. Install management tools
3. Configure access controls
4. Set up backup scripts
5. Verify management tools are working

---

### 4. Verify Connectivity

#### 4.1 Internal Network Connectivity

```bash
# Test connectivity between VMs
# From any VM, test connection to others
ping <other-vm-ip>
curl http://<other-vm-ip>:<port>
```

#### 4.2 External Access

```bash
# Test through Nginx Proxy
curl https://your-domain.com

# Test through Cloudflare Tunnel
curl https://smom-api.sankofa.nexus
```

#### 4.3 Service Health Checks

```bash
# Check validator status
curl http://<validator-ip>:<port>/health

# Check RPC node
curl -X POST http://<rpc-node-ip>:8545 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}'
```

---

## Useful Commands

### Check All VM Status
```bash
kubectl get proxmoxvm -A --sort-by=.metadata.name
```

### Get VM Details
```bash
kubectl describe proxmoxvm <vm-name> -n default
```

### Check Controller Logs
```bash
kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --tail=100 -f
```

### Verify Deployment
```bash
./scripts/verify-deployment.sh
```

### Configure Infrastructure
```bash
./scripts/configure-nginx-proxy.sh
./scripts/configure-cloudflare-tunnel.sh
```

---

## SMOM-DBIS-138 Project Integration

### Quick Integration

1. **Get VM IPs:**
   ```bash
   cd ~/projects/Sankofa
   ./scripts/get-smom-vm-ips.sh
   ```

2. **Switch to SMOM-DBIS-138 project:**
   ```bash
   cd ~/projects/smom-dbis-138
   ```

3. **Use SMOM-DBIS-138 scripts:**
   ```bash
   # Check available commands
   make help
   
   # Follow SMOM-DBIS-138 deployment guide
   cat README.md
   ```

See `docs/smom-dbis-138-project-integration.md` for detailed integration guide.

## Documentation Reference

### Infrastructure
- **Nginx Proxy**: `docs/configs/nginx/README.md`
- **Cloudflare Tunnel**: `docs/configs/cloudflare/README.md`
- **Tunnel Config**: `docs/configs/cloudflare/tunnel-config.yaml`

### Deployment
- **Deployment Status**: `docs/smom-dbis-138-deployment-status.md`
- **Production Ready**: `docs/PRODUCTION_DEPLOYMENT_READY.md`
- **Project Integration**: `docs/smom-dbis-138-project-integration.md`

### SMOM-DBIS-138 Project
- **Project Location**: `~/projects/smom-dbis-138`
- **Main README**: `~/projects/smom-dbis-138/README.md`
- **Runbooks**: `~/projects/smom-dbis-138/runbooks/`

### Scripts
- **Quota Check**: `scripts/check-proxmox-quota.sh`
- **Verify Deployment**: `scripts/verify-deployment.sh`
- **Configure Nginx**: `scripts/configure-nginx-proxy.sh`
- **Configure Tunnel**: `scripts/configure-cloudflare-tunnel.sh`

---

## Troubleshooting

### VMs Not Starting
1. Check controller logs for errors
2. Verify Proxmox connectivity
3. Check resource availability
4. Review VM configurations

### Cannot SSH to VMs
1. Wait for VMs to receive IP addresses
2. Check network configuration
3. Verify firewall rules
4. Check cloud-init logs

### Services Not Accessible
1. Verify services are running on VMs
2. Check firewall rules
3. Verify Nginx/Cloudflare Tunnel configuration
4. Check DNS records

---

## Timeline Estimate

- **VM Startup**: 5-15 minutes (depending on Proxmox host)
- **Infrastructure Configuration**: 30-60 minutes
- **Application Configuration**: 2-4 hours (depending on complexity)
- **Testing & Verification**: 1-2 hours

**Total Estimated Time**: 4-7 hours

---

## Support

For issues or questions:
1. Check controller logs
2. Review VM status
3. Consult documentation
4. Check troubleshooting section

---

**Last Updated**: 2025-12-08  
**Status**: ✅ Ready for Configuration