d-bis/proxmox

Fork 0

Files

defiQUG b45c2006be Refactor code for improved readability and performance

2025-12-21 22:32:09 -08:00

20 KiB

Raw Permalink Blame History

Complete Services Deployment List

Complete inventory of all services and components to be deployed for SMOM-DBIS-138 blockchain network.

📊 Services Overview

Category	Service	Status	Priority	Containers
Blockchain Core	Besu Validators	✅ Ready	P0 - Critical	4+
Blockchain Core	Besu Sentries	✅ Ready	P0 - Critical	3-5
Blockchain Core	Besu RPC Nodes	✅ Ready	P0 - Critical	3-5
Hyperledger	Firefly	✅ Ready	P1 - High	1
Hyperledger	Cacti	✅ Ready	P1 - High	1
Hyperledger	Fabric	✅ Ready	P2 - Medium	1
Hyperledger	Indy	✅ Ready	P2 - Medium	1
Oracle	Oracle Publisher	⏳ Pending	P1 - High	1-2
Cross-Chain	CCIP Monitor	⏳ Pending	P1 - High	1
Automation	Price Feed Keeper	⏳ Pending	P1 - High	1
Tokenization	Financial Tokenization	⏳ Pending	P2 - Medium	1
Explorer	Blockscout	⏳ Pending	P2 - Medium	1
Monitoring	Prometheus	⏳ Pending	P1 - High	1
Monitoring	Grafana	⏳ Pending	P1 - High	1
Monitoring	Loki	⏳ Pending	P2 - Medium	1
Monitoring	Alertmanager	⏳ Pending	P2 - Medium	1
Monitoring	Jaeger	⏳ Pending	P3 - Low	1
Monitoring	Node Exporter	⏳ Pending	P2 - Medium	Multiple

Total Containers: ~30-35 containers

1. Blockchain Core Services

1.1 Besu Validator Nodes ✅

Priority: P0 - Critical
Containers: 4+ (recommended: 4-6)
Status: ✅ Deployment scripts ready

Description:
QBFT consensus nodes that participate in block production and validation.

Resources per Container:

CPU: 4 cores
RAM: 8GB
Disk: 100GB+
Network: Private VLAN (VLAN 100)

Ports:

30303 - P2P TCP/UDP
9545 - Metrics

Configuration:

RPC: Disabled (security best practice)
Consensus: QBFT
Network ID: 138
Requires validator keys

VMID Range: 1000-1004 (5 nodes)

Deployment: ./scripts/deployment/deploy-besu-nodes.sh (validators)

1.2 Besu Sentry Nodes ✅

Priority: P0 - Critical
Containers: 3-5 (recommended: 3)
Status: ✅ Deployment scripts ready

Description:
Public P2P relay nodes that protect validators from direct internet exposure.

Resources per Container:

CPU: 2 cores
RAM: 4GB
Disk: 100GB+
Network: Public VLAN (VLAN 101)

Ports:

30303 - P2P TCP/UDP (public)
8545 - Internal RPC
9545 - Metrics

Configuration:

RPC: Internal only
Network ID: 138
Peered with validators and other nodes

VMID Range: 1500-1503

Deployment: ./scripts/deployment/deploy-besu-nodes.sh (sentries)

1.3 Besu RPC Nodes ✅

Priority: P0 - Critical
Containers: 3-5 (recommended: 3)
Status: ✅ Deployment scripts ready

Description:
Public-facing RPC endpoints for dApps and users to interact with the blockchain.

Resources per Container:

CPU: 4 cores
RAM: 16GB
Disk: 200GB+
Network: Public VLAN (VLAN 102)

Ports:

8545 - RPC HTTP (public)
8546 - WebSocket (public)
9545 - Metrics
30303 - P2P (for sync)

Configuration:

RPC APIs: ETH, NET, WEB3 (read-only)
CORS: Enabled
Rate limiting: Recommended (via reverse proxy)
Network ID: 138

VMID Range: 2500-2502 (3 nodes)

Deployment: ./scripts/deployment/deploy-besu-nodes.sh (RPC)

Load Balancing: Recommended (nginx/HAProxy)

2. Oracle & Price Feed Services

2.1 Oracle Publisher Service ⏳

Priority: P1 - High
Containers: 1-2 (recommended: 2 for HA)
Status: ⏳ Deployment script pending

Description:
Aggregates price data from multiple sources and publishes updates to on-chain oracle contracts.

Technology: Python 3.11
Source: services/oracle-publisher/

Resources per Container:

CPU: 2 cores
RAM: 2GB
Disk: 20GB
Network: Services VLAN (VLAN 103)

Ports:

8000 - Metrics/Health endpoint

Dependencies:

Web3.py
Prometheus client
OpenTelemetry (optional)

Configuration:

RPC endpoint (Chain-138)
Oracle contract address
Data source URLs
Update interval
Private key (for transactions)

VMID Range: 130-139

Features:

Multi-source data aggregation
Median price calculation
Deviation detection
Heartbeat updates
Prometheus metrics

2.2 Price Feed Keeper Service ⏳

Priority: P1 - High
Containers: 1
Status: ⏳ Deployment script pending

Description:
Automated keeper service that triggers price feed updates on schedule or when conditions are met.

Technology: Node.js
Source: scripts/reserve/keeper-service.js

Resources per Container:

CPU: 1 core
RAM: 1GB
Disk: 10GB
Network: Services VLAN (VLAN 103)

Ports:

3000 - Health/Status endpoint

Integration Options:

Standalone service (systemd)
Chainlink Keepers
Gelato Network

Configuration:

Keeper contract address
Update interval
Private key
RPC endpoint

VMID Range: 3500+ (Services)

Alternatives: Can use Chainlink/Gelato instead of dedicated container

3. Cross-Chain Services

3.1 CCIP Monitor Service ⏳

Priority: P1 - High
Containers: 1
Status: ⏳ Deployment script pending

Description:
Monitors Chainlink CCIP message flow, tracks latency, fees, and alerts on failures.

Technology: Python 3.11
Source: services/ccip-monitor/

Resources per Container:

CPU: 2 cores
RAM: 2GB
Disk: 20GB
Network: Services VLAN (VLAN 103)

Ports:

8000 - Metrics endpoint

Features:

Message tracking across chains
Latency monitoring
Fee tracking
Error detection and alerting
Prometheus metrics

VMID Range: 3500+ (Services)

4. Hyperledger Services

4.1 Hyperledger Firefly ✅

Priority: P1 - High
Containers: 1
Status: ✅ Deployment scripts ready

Description:
Web3 gateway for enterprise blockchain applications. Provides APIs for tokenization, data sharing, and multi-party workflows.

Technology: Docker (hyperledger/firefly)
Source: install/firefly-install.sh

Resources per Container:

CPU: 2 cores
RAM: 4GB
Disk: 50GB
Network: Services VLAN (VLAN 103)

Ports:

5000 - Firefly API
5001 - Metrics
4001 - IPFS Swarm
8080 - IPFS Gateway
5001 - IPFS API

Dependencies:

PostgreSQL database (included)
IPFS node (included)
Besu RPC endpoint

Configuration:

Chain ID: 138
Besu RPC URL
Database credentials
Node name

VMID Range: 150-159

Features:

Multi-party workflows
Tokenization APIs
Data sharing and privacy
Event streaming
Multi-blockchain support

Deployment: ./scripts/deployment/deploy-hyperledger-services.sh (firefly)

4.2 Hyperledger Cacti ✅

Priority: P1 - High
Containers: 1
Status: ✅ Deployment scripts ready

Description:
Blockchain integration platform that enables connecting different blockchain networks and executing cross-chain transactions.

Technology: Docker (cactus-cmd-api-server, cactus-plugin-ledger-connector-besu)
Source: install/cacti-install.sh

Resources per Container:

CPU: 2 cores
RAM: 4GB
Disk: 50GB
Network: Services VLAN (VLAN 103)

Ports:

4000 - Cactus API
4001 - WebSocket
4100 - Besu Connector

Dependencies:

Besu RPC endpoint

Configuration:

Cactus API settings
Besu connector configuration
Chain ID: 138

VMID Range: 150-159

Features:

Cross-chain integration
Plugin-based connectors
API server for orchestration
Besu ledger connector
Multi-ledger support

Deployment: ./scripts/deployment/deploy-hyperledger-services.sh (cacti)

4.3 Hyperledger Fabric ✅

Priority: P2 - Medium
Containers: 1
Status: ✅ Deployment scripts ready

Description:
Permissioned blockchain framework for enterprise applications. Supports smart contracts (chaincode), channels, and private data collections.

Technology: Docker, Fabric binaries
Source: install/fabric-install.sh

Resources per Container:

CPU: 4 cores
RAM: 8GB
Disk: 100GB+
Network: Services VLAN (VLAN 103)

Ports:

7050 - Orderer
7051 - Peer (various)
7054 - CA (various)
8080 - CouchDB (optional)

Dependencies:

Docker and Docker Compose
Fabric binaries (cryptogen, configtxgen, etc.)

Configuration:

Network topology
Channel configuration
Organization MSPs
Genesis block
Chaincode deployment

VMID Range: 150-159

Features:

Permissioned network
Channel-based privacy
Chaincode (smart contracts)
Private data collections
Identity management (MSP)
Consensus (Raft, etc.)

Deployment: ./scripts/deployment/deploy-hyperledger-services.sh (fabric)

Note: Requires manual network configuration and crypto material generation

4.4 Hyperledger Indy ✅

Priority: P2 - Medium
Containers: 1
Status: ✅ Deployment scripts ready

Description:
Distributed ledger purpose-built for self-sovereign identity. Enables verifiable credentials and decentralized identity management.

Technology: Docker (indy-node), Python (indy-sdk)
Source: install/indy-install.sh

Resources per Container:

CPU: 4 cores
RAM: 8GB
Disk: 100GB+
Network: Services VLAN (VLAN 103)

Ports:

9701-9708 - Node and client ports (4 nodes)

Dependencies:

Docker and Docker Compose
Python 3 with indy libraries

Configuration:

Pool name
Node configuration
Genesis transactions
Node keys and certificates

VMID Range: 150-159

Features:

Self-sovereign identity
Verifiable credentials
Decentralized identifiers (DIDs)
Credential schemas and definitions
Revocation registries
Plenum consensus

Deployment: ./scripts/deployment/deploy-hyperledger-services.sh (indy)

Note: Runs 4 Indy nodes in one container for consensus (minimum required)

5. Financial Services

5.1 Financial Tokenization Service ⏳

Priority: P2 - Medium
Containers: 1
Status: ⏳ Deployment script pending

Description:
Service for tokenizing financial instruments and managing tokenized assets.

Technology: Python/Node.js
Source: services/financial-tokenization/

Resources per Container:

CPU: 2 cores
RAM: 2GB
Disk: 20GB
Network: Services VLAN (VLAN 103)

VMID Range: 3500+ (Services)

6. Blockchain Explorer

5.1 Blockscout Explorer ⏳

Priority: P2 - Medium
Containers: 1
Status: ⏳ Deployment script pending

Description:
Blockchain explorer with transaction history, contract verification, and token tracking.

Technology: Elixir/Phoenix + PostgreSQL
Source: Blockscout Docker image

Resources per Container:

CPU: 4 cores
RAM: 8GB
Disk: 100GB+
Network: Public VLAN (VLAN 102)

Ports:

4000 - Web UI (HTTP)
5432 - PostgreSQL (internal)

Dependencies:

PostgreSQL database
RPC endpoint access

VMID Range: 130-139

7. Monitoring Stack

6.1 Prometheus ⏳

Priority: P1 - High
Containers: 1
Status: ⏳ Deployment script pending

Description:
Metrics collection and storage for all services.

Resources per Container:

CPU: 2 cores
RAM: 4GB
Disk: 50GB+ (metrics retention)
Network: Monitoring VLAN (VLAN 104)

Ports:

9090 - Web UI
9093 - Alertmanager (internal)

Features:

Scrape configs for all services
Retention: 30-90 days
Alert rules for critical metrics

VMID Range: 130-139

6.2 Grafana ⏳

Priority: P1 - High
Containers: 1
Status: ⏳ Deployment script pending

Description:
Dashboards and visualization for metrics.

Resources per Container:

CPU: 2 cores
RAM: 2GB
Disk: 10GB
Network: Monitoring VLAN (VLAN 104)

Ports:

3000 - Web UI

Dashboards:

Besu node metrics
CCIP message tracking
Oracle price feeds
Network health
System resources

VMID Range: 130-139

6.3 Loki ⏳

Priority: P2 - Medium
Containers: 1
Status: ⏳ Deployment script pending

Description:
Centralized log aggregation.

Resources per Container:

CPU: 2 cores
RAM: 4GB
Disk: 50GB+ (log retention)
Network: Monitoring VLAN (VLAN 104)

Ports:

3100 - API

VMID Range: 130-139

6.4 Alertmanager ⏳

Priority: P2 - Medium
Containers: 1
Status: ⏳ Deployment script pending

Description:
Alert routing and notification management.

Resources per Container:

CPU: 1 core
RAM: 1GB
Disk: 5GB
Network: Monitoring VLAN (VLAN 104)

Ports:

9093 - API

Integrations:

Email
Slack
PagerDuty
Webhooks

VMID Range: 130-139

6.5 Jaeger (Optional) ⏳

Priority: P3 - Low
Containers: 1
Status: ⏳ Deployment script pending

Description:
Distributed tracing for debugging and performance analysis.

Resources per Container:

CPU: 2 cores
RAM: 2GB
Disk: 20GB
Network: Monitoring VLAN (VLAN 104)

Ports:

16686 - Web UI
14268 - API

VMID Range: 130-139

6.6 Node Exporter ⏳

Priority: P2 - Medium
Containers: 1 per host (or shared)
Status: ⏳ Deployment script pending

Description:
Exports host/container metrics for Prometheus.

Resources per Container:

CPU: 0.5 cores
RAM: 256MB
Disk: 1GB

Ports:

9100 - Metrics

Deployment: Can run on Proxmox host or in containers

7. Supporting Infrastructure

7.1 Reverse Proxy / Load Balancer (Optional)

Priority: P2 - Medium
Technology: Nginx or HAProxy

Purpose:

Load balance RPC requests
SSL/TLS termination
Rate limiting
DDoS protection

Can run: On Proxmox host or separate container

7.2 DNS Service (External)

Priority: P1 - High
Provider: Cloudflare (configured in .env)

Endpoints:

rpc.d-bis.org → RPC nodes
explorer.d-bis.org → Blockscout
rpc2.d-bis.org → Secondary RPC

Deployment Summary

Phase 1: Core Blockchain (P0) ✅

Besu Validators (4+ nodes)
Besu Sentries (3-5 nodes)
Besu RPC Nodes (3-5 nodes)

Status: ✅ Ready for deployment

Phase 2: Hyperledger Services (P1-P2) ✅

Hyperledger Firefly
Hyperledger Cacti
Hyperledger Fabric
Hyperledger Indy

Status: ✅ Ready for deployment

Phase 3: Oracle & Automation (P1)

Oracle Publisher Service
Price Feed Keeper Service
CCIP Monitor Service

Status: ⏳ Pending deployment scripts

Phase 4: Monitoring (P1-P2)

Prometheus
Grafana
Loki
Alertmanager
Node Exporter

Status: ⏳ Pending deployment scripts

Phase 5: Explorer & Additional Services (P2)

Blockscout Explorer
Financial Tokenization Service
Jaeger (optional)

Status: ⏳ Pending deployment scripts

Resource Requirements Summary

Minimum Resources

Total CPU: ~50 cores
Total RAM: ~120GB
Total Disk: ~1.5TB

Recommended Resources

Total CPU: ~80 cores
Total RAM: ~200GB
Total Disk: ~2TB

Per Container Average

CPU: 2-4 cores
RAM: 2-8GB
Disk: 20-200GB

Network Architecture

┌─────────────────────────────────────────────────────────────┐
│ VLAN 100 - Validators (Private)                            │
│   - 4+ validator nodes                                      │
│   - No public IPs                                           │
│   - Peered with sentries only                              │
└─────────────────────────────────────────────────────────────┘
                           ↓
┌─────────────────────────────────────────────────────────────┐
│ VLAN 101 - Sentries (Public P2P)                           │
│   - 3-5 sentry nodes                                        │
│   - Public P2P ports                                        │
│   - Internal RPC                                            │
└─────────────────────────────────────────────────────────────┘
                           ↓
┌─────────────────────────────────────────────────────────────┐
│ VLAN 102 - RPC & Explorer (Public)                         │
│   - 3-5 RPC nodes                                           │
│   - 1 Blockscout explorer                                   │
│   - Public HTTP/WebSocket                                   │
└─────────────────────────────────────────────────────────────┘
                           ↓
┌─────────────────────────────────────────────────────────────┐
│ VLAN 103 - Services (Internal)                             │
│   - Oracle Publisher                                        │
│   - CCIP Monitor                                            │
│   - Price Feed Keeper                                       │
│   - Financial Tokenization                                  │
└─────────────────────────────────────────────────────────────┘
                           ↓
┌─────────────────────────────────────────────────────────────┐
│ VLAN 104 - Monitoring (Internal)                           │
│   - Prometheus                                              │
│   - Grafana                                                 │
│   - Loki                                                    │
│   - Alertmanager                                            │
│   - Jaeger (optional)                                       │
└─────────────────────────────────────────────────────────────┘

Next Steps

✅ Phase 1 Complete: Besu nodes deployment scripts ready
⏳ Phase 2: Create deployment scripts for Oracle & Automation services
⏳ Phase 3: Create deployment scripts for Monitoring stack
⏳ Phase 4: Create deployment scripts for Explorer & Additional services

Notes

Priority P0: Essential for blockchain operation
Priority P1: High importance for production
Priority P2: Important but can be deployed later
Priority P3: Optional/optimization
All services can be scaled horizontally (except validators which have fixed count)
Resource allocations are recommendations and can be adjusted based on load
Some services can share containers if resources are limited

20 KiB Raw Permalink Blame History

Complete Services Deployment List

📊 Services Overview

1. Blockchain Core Services

1.1 Besu Validator Nodes ✅

1.2 Besu Sentry Nodes ✅

1.3 Besu RPC Nodes ✅

2. Oracle & Price Feed Services

2.1 Oracle Publisher Service ⏳

2.2 Price Feed Keeper Service ⏳

3. Cross-Chain Services

3.1 CCIP Monitor Service ⏳

4. Hyperledger Services

4.1 Hyperledger Firefly ✅

4.2 Hyperledger Cacti ✅

4.3 Hyperledger Fabric ✅

4.4 Hyperledger Indy ✅

5. Financial Services

5.1 Financial Tokenization Service ⏳

6. Blockchain Explorer

5.1 Blockscout Explorer ⏳

7. Monitoring Stack

6.1 Prometheus ⏳

6.2 Grafana ⏳

6.3 Loki ⏳

6.4 Alertmanager ⏳

6.5 Jaeger (Optional) ⏳

6.6 Node Exporter ⏳

7. Supporting Infrastructure

7.1 Reverse Proxy / Load Balancer (Optional)

7.2 DNS Service (External)

Deployment Summary

Phase 1: Core Blockchain (P0) ✅

Phase 2: Hyperledger Services (P1-P2) ✅

Phase 3: Oracle & Automation (P1)

Phase 4: Monitoring (P1-P2)

Phase 5: Explorer & Additional Services (P2)

Resource Requirements Summary

Minimum Resources

Recommended Resources

Per Container Average

Network Architecture

Next Steps

Notes

20 KiB

Raw Permalink Blame History