# ChainID 138 Besu Node Configuration Guide

**Purpose**: Configure all Besu nodes for ChainID 138 with proper peer discovery, permissioning, and access control.

**Scope**: All Besu nodes including new containers (1504: besu-sentry-5, 2503: besu-rpc-4)

---

## Overview

This guide covers the configuration of Besu nodes for ChainID 138, including:

1. **Static Nodes Configuration** (`static-nodes.json`) - Hard-pinned peer list
2. **Permissioned Nodes Configuration** (`permissioned-nodes.json`) - Allowlist for network access
3. **Discovery Settings** - Disabled for RPC nodes that report chainID 0x1 to MetaMask (wallet compatibility feature)
4. **Access Control** - Separation of access for Ali, Luis, and Putu

---

## Node Allocation

### VMID / Container Allocation

| VMID | Hostname / Container | Role                         | ChainID | Access | Identity | JWT Auth |
|------|----------------------|------------------------------|---------|--------|----------|----------|
| 1504 | `besu-sentry-5`      | Besu Sentry Node             | 138     | Ali (Full) | N/A | ✅ Required |
| 2503 | `besu-rpc-4`         | Besu RPC Node (Permissioned) | 138     | Ali (Full) | 0x8a | ✅ Required |
| 2504 | `besu-rpc-4`         | Besu RPC Node (Permissioned) | 138     | Ali (Full) | 0x1 | ✅ Required |
| 2505 | `besu-rpc-luis`      | Besu RPC Node (Permissioned) | 138     | Luis (RPC-only) | 0x8a | ✅ Required |
| 2506 | `besu-rpc-luis`      | Besu RPC Node (Permissioned) | 138     | Luis (RPC-only) | 0x1 | ✅ Required |
| 2507 | `besu-rpc-putu`      | Besu RPC Node (Permissioned) | 138     | Putu (RPC-only) | 0x8a | ✅ Required |
| 2508 | `besu-rpc-putu`      | Besu RPC Node (Permissioned) | 138     | Putu (RPC-only) | 0x1 | ✅ Required |
| 6201 | `firefly-2`          | Hyperledger Firefly Node     | 138     | Ali (Full) | N/A | ✅ Required |

### RPC Node Permissioned Identities

- **VMID 2503** (`besu-rpc-4`): Ali's container with identity `0x8a`
- **VMID 2504** (`besu-rpc-4`): Ali's container with identity `0x1`
- **VMID 2505** (`besu-rpc-luis`): Luis's container with identity `0x8a`
- **VMID 2506** (`besu-rpc-luis`): Luis's container with identity `0x1`
- **VMID 2507** (`besu-rpc-putu`): Putu's container with identity `0x8a`
- **VMID 2508** (`besu-rpc-putu`): Putu's container with identity `0x1`

---

## Access Model

### Ali (Dedicated Physical Proxmox Host)

- **Full root access** to entire Proxmox host
- **Full access** to all ChainID 138 components:
  - Besu Sentry Node (1504)
  - RPC Node (2503) - both `0x8a` and `0x1` identities
  - Hyperledger Firefly (6201)
- Independent networking, keys, and firewall rules
- No shared authentication with other operators

### Luis (RPC-Only Access)

- **Limited access** to dedicated RPC containers (VMIDs 2505, 2506)
- **Permissioned identity-level usage**: `0x8a` (2505) and `0x1` (2506)
- **JWT authentication required** for all access
- **No access** to:
  - Besu Sentry nodes
  - Firefly nodes
  - Ali's RPC nodes (2503, 2504)
  - Putu's RPC nodes (2507, 2508)
  - Proxmox infrastructure
- Access via reverse proxy / firewall-restricted RPC ports

### Putu (RPC-Only Access)

- **Limited access** to dedicated RPC containers (VMIDs 2507, 2508)
- **Permissioned identity-level usage**: `0x8a` (2507) and `0x1` (2508)
- **JWT authentication required** for all access
- **No access** to:
  - Besu Sentry nodes
  - Firefly nodes
  - Ali's RPC nodes (2503, 2504)
  - Luis's RPC nodes (2505, 2506)
  - Proxmox infrastructure
- Access via reverse proxy / firewall-restricted RPC ports

---

## Configuration Files

### File Locations

On each Besu VM/container:

```
/var/lib/besu/static-nodes.json
/var/lib/besu/permissions/permissioned-nodes.json
```

Alternative paths (also supported):
```
/genesis/static-nodes.json
/permissions/permissioned-nodes.json
```

### File Format

#### `static-nodes.json`

```json
[
  "enode://<PUBKEY_A>@<IP_A>:30303",
  "enode://<PUBKEY_B>@<IP_B>:30303",
  "enode://<PUBKEY_C>@<IP_C>:30303"
]
```

**Operational Rule**: Every Besu VM in ChainID 138 should have the **same** `static-nodes.json` list, including:
- All validator nodes (1000-1004)
- All sentry nodes (1500-1504)
- All RPC nodes (2500-2508)

#### `permissioned-nodes.json`

Same format as `static-nodes.json`. Must include **every Besu node** allowed to join ChainID 138.

---

## Discovery Configuration

### Discovery Settings by Node Type

| Node Type | Discovery | Notes |
|----------|-----------|-------|
| Validators (1000-1004) | Enabled | Can discover peers but must respect permissioning |
| Sentries (1500-1504) | Enabled | Can discover peers but must respect permissioning |
| RPC Core (2500) | **Disabled** | Strict local/permissioned control |
| RPC Permissioned (2501) | Enabled | Permissioned access |
| RPC Public (2502) | Enabled | Public access |
| RPC 4 (2503) | **Disabled** | Reports chainID 0x1 to MetaMask for wallet compatibility |
| RPC 5-8 (2504-2508) | **Disabled** | Reports chainID 0x1 to MetaMask for wallet compatibility |

### Why Disable Discovery for RPC Nodes (2503-2508)?

These RPC nodes are **intentionally configured** to report `chainID = 0x1` (Ethereum mainnet) to MetaMask wallets for compatibility with regulated financial entities. This is a **wallet compatibility feature** that works around MetaMask's technical limitations.

**Important:** While the nodes report chainID 0x1 to wallets, they are actually connected to ChainID 138 (the private network). Discovery is disabled to:
- Prevent actual connection to Ethereum mainnet
- Ensure nodes only connect via `static-nodes.json` and `permissioned-nodes.json`
- Keep nodes attached to ChainID 138 network topology
- Allow MetaMask to work with the private network while thinking it's mainnet

**How it works:**
1. Node runs on ChainID 138 (private network)
2. Node reports chainID 0x1 to MetaMask (wallet compatibility)
3. Discovery disabled → node stays on ChainID 138 topology
4. MetaMask works with private network while thinking it's mainnet

---

## Deployment Process

### Automated Deployment

Use the provided scripts for automated configuration:

#### 1. Main Configuration Script

```bash
# Configure all Besu nodes for ChainID 138
./scripts/configure-besu-chain138-nodes.sh
```

This script:
1. Collects enodes from all Besu nodes
2. Generates `static-nodes.json` and `permissioned-nodes.json`
3. Deploys configurations to all containers
4. Configures discovery settings
5. Restarts Besu services

#### 2. Quick Setup for New Containers

```bash
# Setup new containers (1504, 2503)
./scripts/setup-new-chain138-containers.sh
```

### Manual Deployment Steps

If you need to deploy manually:

#### Step 1: Collect Enodes

```bash
# Extract enode from a node
pct exec <VMID> -- /opt/besu/bin/besu public-key export \
  --node-private-key-file=/var/lib/besu/nodekey \
  --format=enode
```

Or via RPC (if ADMIN API enabled):
```bash
curl -X POST http://<NODE_IP>:8545 \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"admin_nodeInfo","params":[],"id":1}'
```

#### Step 2: Generate Configuration Files

Create `static-nodes.json` and `permissioned-nodes.json` with all enodes.

#### Step 3: Deploy to Containers

```bash
# Copy files to container
pct push <VMID> static-nodes.json /var/lib/besu/static-nodes.json
pct push <VMID> permissioned-nodes.json /var/lib/besu/permissions/permissioned-nodes.json

# Set ownership
pct exec <VMID> -- chown -R besu:besu /var/lib/besu
pct exec <VMID> -- chmod 644 /var/lib/besu/static-nodes.json
pct exec <VMID> -- chmod 644 /var/lib/besu/permissions/permissioned-nodes.json
```

#### Step 4: Update Besu Configuration

Edit `/etc/besu/config*.toml`:

```toml
# Static nodes
static-nodes-file="/var/lib/besu/static-nodes.json"

# Permissioning
permissions-nodes-config-file-enabled=true
permissions-nodes-config-file="/var/lib/besu/permissions/permissioned-nodes.json"

# Discovery (disable for RPC nodes showing chainID 0x1)
discovery-enabled=false  # For 2503
```

#### Step 5: Restart Besu Service

```bash
pct exec <VMID> -- systemctl restart besu*.service
```

---

## Verification

### Check Peer Connections

```bash
# Get peer count
curl -X POST http://<RPC_IP>:8545 \
  -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","method":"net_peerCount","params":[],"id":1}'

# Get peer list (if ADMIN API enabled)
curl -X POST http://<RPC_IP>:8545 \
  -H 'Content-Type: application/json' \
  --data '{"jsonrpc":"2.0","method":"admin_peers","params":[],"id":1}'
```

### Check Configuration Files

```bash
# Verify files exist
pct exec <VMID> -- ls -la /var/lib/besu/static-nodes.json
pct exec <VMID> -- ls -la /var/lib/besu/permissions/permissioned-nodes.json

# Verify content
pct exec <VMID> -- cat /var/lib/besu/static-nodes.json
```

### Check Discovery Setting

```bash
# For RPC node 2503, verify discovery is disabled
pct exec 2503 -- grep discovery-enabled /etc/besu/*.toml
```

### Check Service Status

```bash
# Check Besu service
pct exec <VMID> -- systemctl status besu*.service

# Check logs
pct exec <VMID> -- journalctl -u besu*.service -n 50
```

---

## Troubleshooting

### Issue: Node Not Connecting to Peers

1. **Check static-nodes.json exists and is valid**
   ```bash
   pct exec <VMID> -- cat /var/lib/besu/static-nodes.json | jq .
   ```

2. **Check permissioned-nodes.json includes the node**
   ```bash
   pct exec <VMID> -- grep -i <NODE_ENODE> /var/lib/besu/permissions/permissioned-nodes.json
   ```

3. **Check network connectivity**
   ```bash
   pct exec <VMID> -- ping <PEER_IP>
   ```

4. **Check firewall rules** (port 30303 must be open)

### Issue: RPC Node Showing chainID 0x1

**Solution**: Disable discovery on the RPC node:

```bash
# Edit config file
pct exec 2503 -- sed -i 's/^discovery-enabled=.*/discovery-enabled=false/' /etc/besu/config-rpc-4.toml

# Restart service
pct exec 2503 -- systemctl restart besu*.service
```

### Issue: Permission Denied Errors

1. **Check file ownership**
   ```bash
   pct exec <VMID> -- ls -la /var/lib/besu/static-nodes.json
   pct exec <VMID> -- chown besu:besu /var/lib/besu/static-nodes.json
   ```

2. **Check file permissions**
   ```bash
   pct exec <VMID> -- chmod 644 /var/lib/besu/static-nodes.json
   ```

---

## Configuration Templates

### RPC Node 4 (2503) - Discovery Disabled

See: `smom-dbis-138/config/config-rpc-4.toml`

Key settings:
- `discovery-enabled=false`
- `static-nodes-file="/var/lib/besu/static-nodes.json"`
- `permissions-nodes-config-file="/var/lib/besu/permissions/permissioned-nodes.json"`

### Sentry Node 5 (1504)

Uses standard sentry configuration with:
- `discovery-enabled=true` (can discover but respects permissioning)
- Same static-nodes.json and permissioned-nodes.json as all nodes

---

## Maintenance

### Adding a New Node

1. Extract enode from new node
2. Add enode to `static-nodes.json` on **all existing nodes**
3. Add enode to `permissioned-nodes.json` on **all existing nodes**
4. Deploy updated files to all nodes
5. Restart Besu services

### Removing a Node

1. Remove enode from `static-nodes.json` on **all remaining nodes**
2. Remove enode from `permissioned-nodes.json` on **all remaining nodes**
3. Deploy updated files to all nodes
4. Restart Besu services

---

## Security Considerations

1. **File Permissions**: Ensure `static-nodes.json` and `permissioned-nodes.json` are readable by Besu user but not world-writable
2. **Network Security**: Use firewall rules to restrict P2P port (30303) access
3. **Access Control**: Implement reverse proxy / authentication for RPC access (Luis/Putu)
4. **Key Management**: Keep node keys secure, never expose private keys

---

## Related Documentation

- [Besu Allowlist Runbook](../docs/06-besu/BESU_ALLOWLIST_RUNBOOK.md)
- [RPC Node Configuration](../docs/05-network/RPC_2500_CONFIGURATION_SUMMARY.md)
- [Network Architecture](../smom-dbis-138/docs/architecture/NETWORK.md)

---

## Quick Reference

### All Besu Nodes for ChainID 138

- **Validators**: 1000-1004 (5 nodes)
- **Sentries**: 1500-1504 (5 nodes, including new 1504)
- **RPC Nodes**: 2500-2503 (4 nodes, including new 2503)

### Configuration Files Location

- `static-nodes.json`: `/var/lib/besu/static-nodes.json`
- `permissioned-nodes.json`: `/var/lib/besu/permissions/permissioned-nodes.json`

### Discovery Settings

- **Disabled**: 2500 (core), 2503-2508 (RPC nodes reporting chainID 0x1 to MetaMask for wallet compatibility)
- **Enabled**: All other nodes

### Scripts

- Main config: `scripts/configure-besu-chain138-nodes.sh`
- New containers: `scripts/setup-new-chain138-containers.sh`

