loc_az_hci/docs/architecture/overview.md

Architecture Overview

System Architecture

This document describes the complete architecture of the Proxmox VE → Azure Arc → Hybrid Cloud Stack implementation.

High-Level Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Azure Portal                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐         │
│  │ Azure Arc    │  │ Azure Policy │  │ Azure Monitor │         │
│  │ Servers      │  │              │  │              │         │
│  └──────────────┘  └──────────────┘  └──────────────┘         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐         │
│  │ Arc K8s      │  │ GitOps       │  │ Defender     │         │
│  │              │  │ (Flux)       │  │ for Cloud    │         │
│  └──────────────┘  └──────────────┘  └──────────────┘         │
└─────────────────────────────────────────────────────────────────┘
                              │
                              │ Azure Arc Connection
                              │
┌─────────────────────────────────────────────────────────────────┐
│                    On-Premises Infrastructure                    │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐ │
│  │              Proxmox VE Cluster (2 Nodes)                 │ │
│  │  ┌──────────────┐              ┌──────────────┐          │ │
│  │  │  PVE Node 1  │◄────────────►│  PVE Node 2  │          │ │
│  │  │              │   Cluster     │              │          │ │
│  │  │  Azure Arc   │   Network     │  Azure Arc   │          │ │
│  │  │  Agent       │               │  Agent       │          │ │
│  │  └──────────────┘               └──────────────┘          │ │
│  │         │                           │                     │ │
│  │         └───────────┬───────────────┘                     │ │
│  │                     │                                     │ │
│  │              ┌──────▼──────┐                              │ │
│  │              │ NFS Storage │                              │ │
│  │              │  (Shared)   │                              │ │
│  │              └─────────────┘                              │ │
│  └──────────────────────────────────────────────────────────┘ │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐ │
│  │                    Proxmox VMs                             │ │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │ │
│  │  │   K3s VM     │  │  Git Server  │  │  Other VMs   │    │ │
│  │  │              │  │  (Gitea/     │  │              │    │ │
│  │  │  Azure Arc   │  │  GitLab)     │  │  Azure Arc   │    │ │
│  │  │  K8s         │  │              │  │  Agents      │    │ │
│  │  │  Resource    │  │              │  │              │    │ │
│  │  │  Bridge      │  │              │  │              │    │ │
│  │  └──────────────┘  └──────────────┘  └──────────────┘    │ │
│  └──────────────────────────────────────────────────────────┘ │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐ │
│  │              Kubernetes Cluster (K3s)                     │ │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │ │
│  │  │  Ingress     │  │  Cert-       │  │  GitOps      │    │ │
│  │  │  Controller  │  │  Manager     │  │  (Flux)      │    │ │
│  │  └──────────────┘  └──────────────┘  └──────────────┘    │ │
│  │                                                             │ │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │ │
│  │  │   Besu       │  │  Firefly     │  │  Chainlink   │    │ │
│  │  │  (Ethereum)  │  │  (Middleware)│  │  CCIP        │    │ │
│  │  └──────────────┘  └──────────────┘  └──────────────┘    │ │
│  │                                                             │ │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐    │ │
│  │  │ Blockscout   │  │   Cacti      │  │  NGINX       │    │ │
│  │  │ (Explorer)   │  │ (Monitoring) │  │  Proxy      │    │ │
│  │  └──────────────┘  └──────────────┘  └──────────────┘    │ │
│  └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Component Details

1. Proxmox VE Cluster

Purpose: Hypervisor layer providing virtualization and high availability

Components:

• 2 Proxmox nodes in cluster configuration
• Shared NFS storage for VM data
• Linux bridge networking (vmbr0)
• Corosync for cluster communication

Features:

• High availability (HA) for VMs
• Live migration between nodes
• Centralized management via web UI
• Azure Arc integration for portal visibility

2. Azure Arc Integration

Purpose: Extend Azure management capabilities to on-premises infrastructure

Components:

• Azure Connected Machine Agent: Installed on Proxmox hosts and VMs
• Azure Arc Kubernetes: K3s cluster onboarded to Azure Arc
• Resource Bridge: Custom Kubernetes-based bridge for VM lifecycle control
• GitOps Extension: Flux-based GitOps for declarative deployments

Capabilities:

• VM visibility in Azure Portal
• Azure Policy enforcement
• Azure Update Manager
• Defender for Cloud
• Azure Monitor integration
• GitOps-based deployments

3. Kubernetes (K3s)

Purpose: Container orchestration platform for HC Stack services

Components:

• K3s lightweight Kubernetes distribution
• NGINX Ingress Controller
• Cert-Manager for TLS certificates
• Flux GitOps operator

Namespaces:

• hc-stack: Core infrastructure
• blockchain: Blockchain services (Besu, Firefly, Chainlink)
• monitoring: Monitoring tools (Cacti)
• ingress-nginx: Ingress controller
• cert-manager: Certificate management

4. Hybrid Cloud Stack Services

Hyperledger Besu

• Ethereum client for blockchain operations
• RPC endpoints (HTTP/WebSocket)
• P2P networking
• Metrics and monitoring

Hyperledger Firefly

• Blockchain middleware and API layer
• Multi-party system support
• Token and asset management
• Event streaming

Chainlink CCIP

• Cross-chain interoperability protocol
• Oracle services
• Secure cross-chain messaging

Blockscout

• Blockchain explorer
• Transaction and block visualization
• Contract verification
• Analytics dashboard

Cacti

• Network monitoring and graphing
• Performance metrics
• Alerting capabilities

NGINX Proxy

• Reverse proxy for all services
• Load balancing
• SSL termination

5. Private Git/DevOps

Options:

• Gitea: Lightweight Git server (recommended for small deployments)
• GitLab CE: Full-featured DevOps platform
• Azure DevOps: Self-hosted agents for Azure DevOps pipelines

Purpose:

• Version control for infrastructure and applications
• CI/CD pipeline execution
• GitOps repository for Kubernetes deployments

Data Flow

1. Infrastructure Management:

   • Terraform → Proxmox API → VM Creation
   • Azure Arc Agent → Azure Portal → Visibility & Management

2. Application Deployment:

   • Git Repository → Flux GitOps → Kubernetes API → Pod Deployment
   • Azure Arc GitOps → Flux → Kubernetes → Application Updates

3. Monitoring & Observability:

   • Services → Metrics → Azure Monitor / Cacti
   • Logs → Azure Log Analytics / Local Storage

Security Architecture

• Network Isolation: Separate networks for management, storage, and application traffic
• Azure Arc Security: Managed identities and RBAC
• Kubernetes Security: RBAC, network policies, pod security policies
• TLS/SSL: Cert-Manager for automatic certificate management
• Secrets Management: Kubernetes secrets (consider Azure Key Vault integration)

High Availability

• Proxmox Cluster: 2-node cluster with shared storage
• VM HA: Automatic failover for VMs
• Kubernetes: Multiple replicas for stateless services
• Storage: NFS shared storage for persistent data
• Load Balancing: NGINX Ingress for service distribution

Scalability

• Horizontal Scaling: Add more Proxmox nodes to cluster
• Kubernetes Scaling: Add worker nodes to K3s cluster
• Application Scaling: Kubernetes HPA for automatic scaling
• Storage Scaling: Expand NFS storage as needed

Integration Points

1. Azure Portal: Full visibility and management
2. Git Repository: Source of truth for infrastructure and applications
3. Kubernetes API: Application deployment and management
4. Proxmox API: VM lifecycle management
5. Monitoring Systems: Metrics and alerting