proxmox/docs/archive/SYSTEMD_SERVICE_UPDATE_PROCESS.md

Systemd Service Update Process

Overview

During deployment, the RPC node configuration files are copied to containers, and the systemd service files are automatically updated to reference the correct config file for each node type.

---

Automatic Service File Updates

When Does It Happen?

The copy-besu-config-with-nodes.sh script automatically updates systemd service files when copying RPC node configurations.

How It Works

1. Initial Service Creation: The besu-rpc-install.sh script creates a service file with a default config path:

   ExecStart=/opt/besu/bin/besu \
       --config-file=$BESU_CONFIG/config-rpc-public.toml
   

2. Service File Update: When copying config files, copy-besu-config-with-nodes.sh updates the service file:

   # For VMID 2500 (Core RPC)
   pct exec 2500 -- sed -i "s|--config-file=\$BESU_CONFIG/config-rpc-public.toml|--config-file=\$BESU_CONFIG/config-rpc-core.toml|g" /etc/systemd/system/besu-rpc.service
   pct exec 2500 -- systemctl daemon-reload
   
   # For VMID 2501 (Permissioned RPC)
   pct exec 2501 -- sed -i "s|--config-file=\$BESU_CONFIG/config-rpc-public.toml|--config-file=\$BESU_CONFIG/config-rpc-perm.toml|g" /etc/systemd/system/besu-rpc.service
   pct exec 2501 -- systemctl daemon-reload
   
   # For VMID 2502 (Public RPC) - no change needed
   # Service already references config-rpc-public.toml
   

3. Daemon Reload: After updating, systemctl daemon-reload is called to pick up changes.

---

Service File Locations

All RPC nodes use the same service file location:

• Path: /etc/systemd/system/besu-rpc.service
• Service Name: besu-rpc.service

---

Service File Contents

After Installation (Before Update)

[Unit]
Description=Hyperledger Besu RPC Node
After=network.target
Wants=network-online.target

[Service]
Type=simple
User=besu
Group=besu
WorkingDirectory=/opt/besu

Environment="BESU_OPTS=-Xmx8g -Xms8g"
Environment="JAVA_OPTS=-XX:+UseG1GC -XX:MaxGCPauseMillis=200"

ExecStart=/opt/besu/bin/besu \
    --config-file=$BESU_CONFIG/config-rpc-public.toml

Restart=always
RestartSec=10

LimitNOFILE=65536
LimitNPROC=32768

NoNewPrivileges=true
PrivateTmp=true

StandardOutput=journal
StandardError=journal
SyslogIdentifier=besu-rpc

[Install]
WantedBy=multi-user.target

After Update (VMID 2500 - Core RPC)

# ... (same as above) ...

ExecStart=/opt/besu/bin/besu \
    --config-file=$BESU_CONFIG/config-rpc-core.toml

# ... (rest same as above) ...

After Update (VMID 2501 - Permissioned RPC)

# ... (same as above) ...

ExecStart=/opt/besu/bin/besu \
    --config-file=$BESU_CONFIG/config-rpc-perm.toml

# ... (rest same as above) ...

---

Manual Service File Updates

If you need to manually update a service file:

1. Edit the Service File

# Access the container
pct exec <VMID> -- bash

# Edit the service file
nano /etc/systemd/system/besu-rpc.service
# Or use sed:
sed -i "s|--config-file=\$BESU_CONFIG/config-rpc-public.toml|--config-file=\$BESU_CONFIG/config-rpc-{type}.toml|g" /etc/systemd/system/besu-rpc.service

2. Reload Systemd Daemon

systemctl daemon-reload

3. Restart the Service

systemctl restart besu-rpc.service

4. Verify

systemctl status besu-rpc.service
journalctl -u besu-rpc -n 50

---

Verification

Check Service File Content

# From Proxmox host
pct exec <VMID> -- cat /etc/systemd/system/besu-rpc.service | grep "config-file"

Expected outputs:

• VMID 2500: --config-file=$BESU_CONFIG/config-rpc-core.toml
• VMID 2501: --config-file=$BESU_CONFIG/config-rpc-perm.toml
• VMID 2502: --config-file=$BESU_CONFIG/config-rpc-public.toml

Check Service Status

# From Proxmox host
pct exec <VMID> -- systemctl status besu-rpc.service

Check Service Logs

# From Proxmox host
pct exec <VMID> -- journalctl -u besu-rpc.service -f

---

Troubleshooting

Service File Not Updated

Symptom: Service still references config-rpc-public.toml on Core or Permissioned RPC nodes.

Solution:

1. Verify config file exists: pct exec <VMID> -- ls -la /etc/besu/config-rpc-{type}.toml
2. Manually update service file (see "Manual Service File Updates" above)
3. Restart service: pct exec <VMID> -- systemctl restart besu-rpc.service

Service Fails to Start After Update

Symptom: systemctl status besu-rpc.service shows failed state.

Possible Causes:

1. Config file doesn't exist
2. Config file has syntax errors
3. Permissions issue

Solution:

1. Check logs: pct exec <VMID> -- journalctl -u besu-rpc.service -n 100
2. Verify config file exists and is readable
3. Check config file syntax: pct exec <VMID> -- besu --config-file=/etc/besu/config-rpc-{type}.toml --data-path=/tmp/test --genesis-file=/etc/besu/genesis.json --help (will validate syntax)

Daemon Reload Not Applied

Symptom: Changes to service file not taking effect.

Solution:

1. Ensure systemctl daemon-reload was run
2. Restart service: systemctl restart besu-rpc.service
3. Verify service file is correct: cat /etc/systemd/system/besu-rpc.service | grep "config-file"

---

Script Implementation

The service update logic in copy-besu-config-with-nodes.sh:

# Update systemd service file to use the correct config file
log_info "Updating systemd service to use $config_filename for RPC node $vmid"
pct exec "$vmid" -- sed -i "s|--config-file=\$BESU_CONFIG/config-rpc-public.toml|--config-file=\$BESU_CONFIG/$config_filename|g" /etc/systemd/system/besu-rpc.service 2>/dev/null || {
    log_warn "Failed to update systemd service file for $vmid (may need manual update)"
}
# Reload systemd daemon to pick up changes
pct exec "$vmid" -- systemctl daemon-reload 2>/dev/null || true

---

Best Practices

1. Always reload daemon after editing service files
2. Verify config file exists before updating service file
3. Check service status after updates
4. Monitor logs for configuration errors
5. Use validation scripts to verify correct config files are deployed

---

Related Documentation

• RPC Node Types: docs/RPC_NODE_TYPES_ARCHITECTURE.md
• RPC Type Comparison: docs/RPC_TYPE_COMPARISON.md
• Copy Config Script: scripts/copy-besu-config-with-nodes.sh
• Validation: scripts/validation/validate-deployment-comprehensive.sh