# Deployment Comparison: LXC Containers vs Single VM

This document compares the two deployment options for SMOM-DBIS-138 Besu network nodes.

## Overview

### Deployment 1: Individual LXC Containers (VMID 1000-2502)

Each Besu node runs in its own LXC container on Proxmox VE. This is the **production-ready** deployment method.

### Deployment 2: Single VM with Docker (VMID 9000)

All Besu nodes run as Docker containers within a single VM. This is the **temporary/testing** deployment method.

---

## Deployment 1: LXC Containers (1000-2502)

### Architecture

```
┌──────────────────────────────────────────────────────────────┐
│ Proxmox VE Host                                              │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │ LXC 1000     │  │ LXC 1001     │  │ LXC 1002     │      │
│  │ validator-1  │  │ validator-2  │  │ validator-3  │      │
│  │ 8GB, 4 cores │  │ 8GB, 4 cores │  │ 8GB, 4 cores │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐                        │
│  │ LXC 1003     │  │ LXC 1004     │                        │
│  │ validator-4  │  │ validator-5  │                        │
│  │ 8GB, 4 cores │  │ 8GB, 4 cores │                        │
│  └──────────────┘  └──────────────┘                        │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │ LXC 1500     │  │ LXC 1501     │  │ LXC 1502     │      │
│  │ sentry-1     │  │ sentry-2     │  │ sentry-3     │      │
│  │ 4GB, 2 cores │  │ 4GB, 2 cores │  │ 4GB, 2 cores │      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
│                                                              │
│  ┌──────────────┐                                            │
│  │ LXC 1503     │                                            │
│  │ sentry-4     │                                            │
│  │ 4GB, 2 cores │                                            │
│  └──────────────┘                                            │
│                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐      │
│  │ LXC 2500     │  │ LXC 2501     │  │ LXC 2502     │      │
│  │ rpc-1        │  │ rpc-2        │  │ rpc-3        │      │
│  │ 16GB, 4 cores│  │ 16GB, 4 cores│  │ 16GB, 4 cores│      │
│  └──────────────┘  └──────────────┘  └──────────────┘      │
└──────────────────────────────────────────────────────────────┘
```

### VMID Allocation

| Category | VMID Range | Count | Hostnames | IP Range |
|----------|-----------|-------|-----------|----------|
| **Validators** | 1000-1004 | 5 | besu-validator-1 through besu-validator-5 | 192.168.11.100-104 |
| **Sentries** | 1500-1503 | 4 | besu-sentry-1 through besu-sentry-4 | 192.168.11.150-153 |
| **RPC Nodes** | 2500-2502 | 3 | besu-rpc-1 through besu-rpc-3 | 192.168.11.250-252 |

**Total Containers**: 12

### Resource Allocation

#### Validators (1000-1004)
- **Memory**: 8GB each (40GB total)
- **CPU Cores**: 4 each (20 cores total)
- **Disk**: 100GB each (500GB total)
- **IP Addresses**: 192.168.11.100-104
- **Purpose**: QBFT consensus nodes

#### Sentries (1500-1503)
- **Memory**: 4GB each (16GB total)
- **CPU Cores**: 2 each (8 cores total)
- **Disk**: 100GB each (400GB total)
- **IP Addresses**: 192.168.11.150-153
- **Purpose**: P2P relay and protection nodes

#### RPC Nodes (2500-2502)
- **Memory**: 16GB each (48GB total)
- **CPU Cores**: 4 each (12 cores total)
- **Disk**: 200GB each (600GB total)
- **IP Addresses**: 192.168.11.250-252
- **Purpose**: Public RPC endpoints

**Total Resources**:
- **Total Memory**: 104GB
- **Total CPU Cores**: 40 cores
- **Total Disk**: 1.5TB

### Deployment Script

```bash
cd /opt/smom-dbis-138-proxmox
sudo ./scripts/deployment/deploy-besu-nodes.sh
```

### Configuration Files

Each container has:
- `/etc/besu/config-{type}.toml` - Besu configuration
- `/opt/besu/data/` - Blockchain data directory
- `/opt/besu/keys/` - Validator keys (validators only)
- Systemd service: `besu-validator`, `besu-sentry`, or `besu-rpc`

### Service Management

```bash
# Start/stop individual container services
pct exec 1000 -- systemctl start besu-validator
pct exec 1500 -- systemctl start besu-sentry
pct exec 2500 -- systemctl start besu-rpc

# Check status
pct exec 1000 -- systemctl status besu-validator

# View logs
pct exec 1000 -- journalctl -u besu-validator -f
```

### Advantages

✅ **Resource Isolation**: Each node has dedicated resources  
✅ **Individual Scaling**: Scale nodes independently  
✅ **Production Ready**: Suitable for production deployments  
✅ **Security Isolation**: Better security boundaries  
✅ **Independent Management**: Start/stop nodes individually  
✅ **Better Monitoring**: Monitor each node separately  
✅ **Fault Isolation**: Failure of one node doesn't affect others  

### Disadvantages

❌ **Longer Deployment Time**: ~30-45 minutes  
❌ **More Complex Setup**: Requires configuration per container  
❌ **Higher Resource Usage**: More overhead per node  
❌ **More Management Overhead**: Manage 12 containers  

---

## Deployment 2: Single VM with Docker (VMID 9000)

### Architecture

```
┌──────────────────────────────────────────────────────────────┐
│ Proxmox VE Host                                              │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │ VM 9000                                                │ │
│  │ IP: 192.168.11.90                                     │ │
│  │ Memory: 32GB, CPU: 8 cores, Disk: 500GB              │ │
│  │                                                        │ │
│  │  ┌──────────────────────────────────────────────────┐ │ │
│  │  │ Docker Network                                   │ │ │
│  │  │                                                  │ │ │
│  │  │  ┌─────────┐  ┌─────────┐  ┌─────────┐         │ │ │
│  │  │  │validator│  │validator│  │validator│         │ │ │
│  │  │  │   -1    │  │   -2    │  │   -3    │         │ │ │
│  │  │  └─────────┘  └─────────┘  └─────────┘         │ │ │
│  │  │                                                  │ │ │
│  │  │  ┌─────────┐  ┌─────────┐                      │ │ │
│  │  │  │validator│  │validator│                      │ │ │
│  │  │  │   -4    │  │   -5    │                      │ │ │
│  │  │  └─────────┘  └─────────┘                      │ │ │
│  │  │                                                  │ │ │
│  │  │  ┌─────────┐  ┌─────────┐  ┌─────────┐         │ │ │
│  │  │  │ sentry  │  │ sentry  │  │ sentry  │         │ │ │
│  │  │  │   -1    │  │   -2    │  │   -3    │         │ │ │
│  │  │  └─────────┘  └─────────┘  └─────────┘         │ │ │
│  │  │                                                  │ │ │
│  │  │  ┌─────────┐                                    │ │ │
│  │  │  │ sentry  │                                    │ │ │
│  │  │  │   -4    │                                    │ │ │
│  │  │  └─────────┘                                    │ │ │
│  │  │                                                  │ │ │
│  │  │  ┌─────────┐  ┌─────────┐  ┌─────────┐         │ │ │
│  │  │  │  rpc-1  │  │  rpc-2  │  │  rpc-3  │         │ │ │
│  │  │  └─────────┘  └─────────┘  └─────────┘         │ │ │
│  │  └──────────────────────────────────────────────────┘ │ │
│  └────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
```

### VMID Allocation

| Category | VMID | Description | IP Address |
|----------|------|-------------|------------|
| **Temporary VM** | 9000 | All 12 Besu nodes in Docker | 192.168.11.90 |

**Total VMs**: 1  
**Total Docker Containers**: 12 (5 validators + 4 sentries + 3 RPC)

### Resource Allocation

- **VM Memory**: 32GB (shared by all containers)
- **VM CPU Cores**: 8 cores (shared by all containers)
- **VM Disk**: 500GB (shared by all containers)
- **VM IP Address**: 192.168.11.90
- **OS**: Ubuntu 22.04

#### Container Resource Allocation (within VM)

##### Validators (5 containers)
- **Memory**: 4GB each (20GB total)
- **P2P Ports**: 30303-30307
- **Metrics Ports**: 9545-9549

##### Sentries (4 containers)
- **Memory**: 2GB each (8GB total)
- **P2P Ports**: 30308-30311
- **Metrics Ports**: 9550-9553

##### RPC Nodes (3 containers)
- **Memory**: 8GB each (24GB total)
- **HTTP RPC Ports**: 8545, 8547, 8549
- **WS RPC Ports**: 8546, 8548, 8550
- **P2P Ports**: 30312-30314
- **Metrics Ports**: 9554-9556

**Total Container Memory**: 52GB (with some headroom in 32GB VM)

### Deployment Script

```bash
cd /opt/smom-dbis-138-proxmox
sudo ./scripts/deployment/deploy-besu-temp-vm-complete.sh /opt/smom-dbis-138
```

Or step-by-step:

```bash
# Step 1: Create VM
sudo ./scripts/deployment/deploy-besu-temp-vm.sh

# Step 2: SSH into VM and setup
ssh root@192.168.11.90
cd /opt/smom-dbis-138-proxmox
./scripts/deployment/setup-docker-besu.sh /opt/smom-dbis-138

# Step 3: Start containers
cd /opt/besu
docker compose up -d
```

### Configuration Files

All containers share:
- `/opt/besu/docker-compose.yml` - Docker Compose configuration
- `/opt/besu/validators/validator-{N}/config/config-validator.toml` - Validator configs
- `/opt/besu/sentries/sentry-{N}/config/config-sentry.toml` - Sentry configs
- `/opt/besu/rpc/rpc-{N}/config/config-rpc.toml` - RPC configs
- `/opt/besu/shared/genesis/` - Shared genesis.json
- `/opt/besu/shared/permissions/` - Shared permissions files

### Service Management

```bash
# SSH into VM
ssh root@192.168.11.90

# Start/stop all containers
cd /opt/besu
docker compose up -d
docker compose stop
docker compose restart

# Start/stop specific container
docker compose start besu-validator-1
docker compose stop besu-validator-1

# View logs
docker compose logs -f
docker compose logs -f besu-validator-1

# Check status
docker compose ps
docker stats
```

### RPC Endpoints

- **RPC-1**: `http://192.168.11.90:8545`
- **RPC-2**: `http://192.168.11.90:8547`
- **RPC-3**: `http://192.168.11.90:8549`

### Metrics Endpoints

- **Validators**: `http://192.168.11.90:9545-9549/metrics`
- **Sentries**: `http://192.168.11.90:9550-9553/metrics`
- **RPC Nodes**: `http://192.168.11.90:9554-9556/metrics`

### Advantages

✅ **Faster Deployment**: ~15-30 minutes  
✅ **Easier Initial Testing**: Single point of management  
✅ **Simplified Troubleshooting**: All nodes in one place  
✅ **Lower Resource Overhead**: Shared VM resources  
✅ **Easy Migration Path**: Can migrate to LXC later  
✅ **Single Management Point**: Manage all nodes together  

### Disadvantages

❌ **Resource Sharing**: All nodes share VM resources  
❌ **Less Isolation**: No resource boundaries between nodes  
❌ **Not Production Ready**: Temporary/testing solution  
❌ **Single Point of Failure**: VM failure affects all nodes  
❌ **Resource Contention**: Nodes compete for resources  

---

## Side-by-Side Comparison

| Feature | LXC Containers (1000-2502) | Single VM (9000) |
|---------|---------------------------|------------------|
| **VMID Range** | 1000-2502 (12 containers) | 9000 (1 VM) |
| **Architecture** | Individual LXC containers | Docker containers in single VM |
| **Deployment Time** | 30-45 minutes | 15-30 minutes |
| **Total Memory** | 104GB (dedicated) | 32GB (shared) |
| **Total CPU Cores** | 40 cores (dedicated) | 8 cores (shared) |
| **Total Disk** | 1.5TB (distributed) | 500GB (shared) |
| **Resource Isolation** | High (per container) | Low (shared VM) |
| **Scalability** | Individual per node | Limited (VM limits) |
| **Management** | Per container (12 instances) | Single VM (1 instance) |
| **Production Ready** | ✅ Yes | ❌ No (temporary) |
| **Security Isolation** | High | Medium |
| **Fault Isolation** | High (independent failures) | Low (VM is SPOF) |
| **Monitoring** | Per container | Per container (within VM) |
| **Migration Path** | N/A (final state) | ✅ To LXC available |

---

## Network Configuration Comparison

### LXC Containers

| Node Type | VMID Range | IP Range | VLAN |
|-----------|-----------|----------|------|
| Validators | 1000-1004 | 192.168.11.100-104 | 100 |
| Sentries | 1500-1503 | 192.168.11.150-153 | 101 |
| RPC Nodes | 2500-2502 | 192.168.11.250-252 | 102 |

**Network Isolation**: Separate VLANs per node type

### Single VM

| Node Type | Container Count | Port Mappings | Network |
|-----------|----------------|---------------|---------|
| Validators | 5 | P2P: 30303-30307, Metrics: 9545-9549 | Docker bridge |
| Sentries | 4 | P2P: 30308-30311, Metrics: 9550-9553 | Docker bridge |
| RPC Nodes | 3 | HTTP: 8545/8547/8549, WS: 8546/8548/8550 | Docker bridge |

**Network Isolation**: Docker bridge network, port-based separation

---

## When to Use Each Deployment

### Use LXC Containers (1000-2502) When:

✅ Production deployment  
✅ Need resource isolation  
✅ Individual node scaling required  
✅ Long-term deployment  
✅ Maximum security needed  
✅ Need independent node management  
✅ Want better fault isolation  

### Use Single VM (9000) When:

✅ Quick testing and validation  
✅ Development environment  
✅ Proof of concept  
✅ Limited initial resources  
✅ Planning to migrate to LXC later  
✅ Need faster initial deployment  

---

## Migration Path

### From VM (9000) to LXC Containers (1000-2502)

```bash
# 1. Deploy LXC containers
cd /opt/smom-dbis-138-proxmox
sudo ./scripts/deployment/deploy-besu-nodes.sh

# 2. Migrate data from VM to LXC
sudo ./scripts/migration/migrate-vm-to-lxc.sh /opt/smom-dbis-138

# 3. Verify and start services in LXC containers
for vmid in 1000 1001 1002 1003 1004; do
    pct exec $vmid -- systemctl start besu-validator
done

for vmid in 1500 1501 1502 1503; do
    pct exec $vmid -- systemctl start besu-sentry
done

for vmid in 2500 2501 2502; do
    pct exec $vmid -- systemctl start besu-rpc
done

# 4. Validate deployment
./scripts/validation/validate-deployment-comprehensive.sh

# 5. Shut down temporary VM
qm stop 9000
```

---

## Deployment Scripts Reference

### LXC Containers Deployment

**Main Script**: `scripts/deployment/deploy-besu-nodes.sh`

**Key Functions**:
- Creates 12 LXC containers (5 validators, 4 sentries, 3 RPC)
- Configures network (static IPs, VLANs)
- Installs Besu in each container
- Sets up systemd services
- Generates inventory file

### Single VM Deployment

**Main Scripts**:
- `scripts/deployment/deploy-besu-temp-vm.sh` - Creates VM only
- `scripts/deployment/deploy-besu-temp-vm-complete.sh` - Complete deployment
- `scripts/deployment/setup-docker-besu.sh` - Sets up Docker and configs

**Key Functions**:
- Creates single VM (VMID 9000)
- Installs Docker and Docker Compose
- Copies configuration files
- Sets up Docker Compose file
- Starts all 12 containers

---

## Validation and Monitoring

### LXC Containers

```bash
# Validate all containers
./scripts/validation/validate-deployment-comprehensive.sh

# Check individual container status
pct exec 1000 -- systemctl status besu-validator
pct exec 1500 -- systemctl status besu-sentry
pct exec 2500 -- systemctl status besu-rpc

# View logs
pct exec 1000 -- journalctl -u besu-validator -f
```

### Single VM

```bash
# Validate VM deployment
./scripts/validation/validate-besu-temp-vm.sh

# Check container status (from within VM)
ssh root@192.168.11.90
docker compose ps
docker stats

# View logs
docker compose logs -f
```

---

## Related Documentation

- [Temporary VM Deployment Guide](TEMP_VM_DEPLOYMENT.md)
- [LXC Container Deployment Guide](DEPLOYMENT_STEPS_COMPLETE.md)
- [Deployment Options](DEPLOYMENT_OPTIONS.md)
- [Migration Guide](MIGRATION.md)
- [Troubleshooting Guide](TROUBLESHOOTING.md)

---

**Last Updated**: $(date)

