Sankofa/docs/proxmox/INTER_INSTANCE_CONNECTIVITY.md

Proxmox Inter-Instance Connectivity

Last Updated: 2024-12-19

Overview

This document describes the connectivity status between the two Proxmox instances and how to verify they can see each other.

Instance Configuration

• Instance 1: ML110-01

  • IP: 192.168.11.10
  • FQDN: ml110-01.sankofa.nexus
  • Site: us-sfvalley

• Instance 2: R630-01

  • IP: 192.168.11.11
  • FQDN: r630-01.sankofa.nexus
  • Site: us-sfvalley-2

Network Configuration

Both instances are on the same network:

• Subnet: 192.168.11.0/24
• Network Type: Private LAN
• Gateway: (to be determined)

Connectivity Test Results

From External Machine (This Machine)

✅ Both instances are reachable:

• ML110-01 → R630-01: ✅ Reachable (HTTP 401 - auth required)
• R630-01 → ML110-01: ✅ Reachable (HTTP 401 - auth required)
• Both respond to API calls with authentication ✅

API-Based Tests

Using Proxmox API tokens:

1. ML110-01 can reach R630-01 API: ✅

   • Version: 9.1.1
   • Authentication: Successful

2. R630-01 can reach ML110-01 API: ✅

   • Version: 9.1.1
   • Authentication: Successful

Cluster Status

Check cluster membership:

# From ML110-01
curl -k -H "Authorization: PVEAPIToken <token>" \
  https://192.168.11.10:8006/api2/json/cluster/status

# From R630-01
curl -k -H "Authorization: PVEAPIToken <token>" \
  https://192.168.11.11:8006/api2/json/cluster/status

Verification Methods

Method 1: API-Based (From External Machine)

source .env

# Test ML110-01 → R630-01
curl -k -H "Authorization: PVEAPIToken ${PROXMOX_TOKEN_R630_01}" \
  https://192.168.11.11:8006/api2/json/version

# Test R630-01 → ML110-01
curl -k -H "Authorization: PVEAPIToken ${PROXMOX_TOKEN_ML110_01}" \
  https://192.168.11.10:8006/api2/json/version

Method 2: SSH-Based (From Within Instances)

Requires SSH access to instances:

# SSH into ML110-01
ssh root@192.168.11.10

# Test connectivity to R630-01
curl -k https://192.168.11.11:8006/api2/json/version
ping -c 3 192.168.11.11

# SSH into R630-01
ssh root@192.168.11.11

# Test connectivity to ML110-01
curl -k https://192.168.11.10:8006/api2/json/version
ping -c 3 192.168.11.10

Method 3: Using Proxmox Web UI

1. Log in to ML110-01 web UI
2. Go to: Datacenter → Cluster
3. Check if R630-01 appears in cluster members
4. Repeat from R630-01

Cluster Requirements

If you want to cluster these instances:

Network Requirements

• ✅ Same subnet (192.168.11.0/24) - Met
• ✅ Network connectivity - Confirmed
• ⚠️ Firewall rules for clustering ports

Firewall Ports for Clustering

Required ports:

• 8006: Proxmox API (HTTPS)
• 5404-5405: Corosync (cluster communication)
• 22: SSH (for cluster operations)
• 3128: Spice proxy (optional)

Cluster Setup Steps

1. Prepare first node (ML110-01):

   # On ML110-01
   pvecm create <cluster-name>
   

2. Add second node (R630-01):

   # On R630-01
   pvecm add <first-node-ip> -link0 <first-node-ip>
   

3. Verify cluster:

   # On either node
   pvecm status
   pvecm nodes
   

Current Status

✅ Confirmed

• Both instances are on the same network
• Both instances are reachable via IP and FQDN
• API connectivity works from external machine
• Both instances respond to authenticated API calls

⚠️ To Verify

• Direct connectivity from within instances (requires SSH)
• Cluster membership status
• Firewall configuration for clustering
• Corosync port accessibility

Troubleshooting

Instances Can't See Each Other

1. Check network connectivity:

   ping <other-instance-ip>
   

2. Check firewall rules:

   # On Proxmox
   iptables -L -n | grep <other-instance-ip>
   pve-firewall status
   

3. Check routing:

   ip route show
   route -n
   

Cluster Issues

1. Check corosync:

   systemctl status corosync
   corosync-cmapctl | grep members
   

2. Check quorum:

   pvecm status
   pvecm expected 2  # For 2-node cluster
   

Related Documentation

• Deployment Readiness
• Network Configuration
• Cluster Setup Guide