d-bis/proxmox
4
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
Files
3f76bc950779f8c1a4403abddb952cdbce5fa420
proxmox/docs/04-configuration/R630-02_STATUS_REPORT.md
defiQUG fbda1b4beb
Some checks failed
Deploy to Phoenix / deploy (push) Has been cancelled
Details
docs: Ledger Live integration, contract deploy learnings, NEXT_STEPS updates 
- ADD_CHAIN138_TO_LEDGER_LIVE: Ledger form done; public code review repo bis-innovations/LedgerLive; init/push commands
- CONTRACT_DEPLOYMENT_RUNBOOK: Chain 138 gas price 1 gwei, 36-addr check, TransactionMirror workaround
- CONTRACT_*: AddressMapper, MirrorManager deployed 2026-02-12; 36-address on-chain check
- NEXT_STEPS_FOR_YOU: Ledger done; steps completable now (no LAN); run-completable-tasks-from-anywhere
- MASTER_INDEX, OPERATOR_OPTIONAL, SMART_CONTRACTS_INVENTORY_SIMPLE: updates
- LEDGER_BLOCKCHAIN_INTEGRATION_COMPLETE: bis-innovations/LedgerLive reference

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-12 15:46:57 -08:00

6.5 KiB
Raw Blame History

r630-02 Status Report

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation

Date: 2026-01-15
Host: r630-02 (192.168.11.12)
Status: ✅ OPERATIONAL - Some Concerns

System Information

Host Details

  • Hostname: r630-02
  • IP Address: 192.168.11.12/24
  • OS: Proxmox VE 9.1.4
  • Kernel: 6.17.4-1-pve
  • Uptime: 1 day, 6 hours, 40 minutes
  • Architecture: x86_64

Resource Usage

  • Memory: 251GB total, 8.6GB used, 235GB free, 243GB available
  • CPU: Intel Xeon E5-2660 v4 @ 2.00GHz (2 sockets, 28 cores, 56 threads)
  • CPU Usage: 4.9% user, 5.5% system, 89.2% idle
  • Load Average: ⚠️ 12.03, 11.65, 11.51 (High - concerning)
  • Swap: 0B (no swap configured)

Proxmox Status

Services

  • pve-cluster: ✅ Active (running) - 1 day 6h uptime
  • pvedaemon: ✅ Active (running) - 1 day 6h uptime
  • pveproxy: ✅ Active (running)

Version

  • Proxmox VE: 9.1.4/5ac30304265fbd8e
  • Kernel: 6.17.4-1-pve

VMs and Containers

  • Containers: 4 total
    • ✅ 5000 - blockscout-1 (running)
    • ✅ 6200 - firefly-1 (running)
    • ⏸️ 6201 - firefly-ali-1 (stopped)
    • ✅ 7811 - mim-api-1 (running)
  • VMs: 0

Network Configuration

Interfaces

  • Primary: vmbr0 (bridge)
    • IP: 192.168.11.12/24
    • Gateway: 192.168.11.1
    • Bridge Port: nic2
  • Physical: nic2 (UP, connected to vmbr0)
  • Other NICs: nic0, nic1, nic3 (DOWN)

VLAN Configuration

  • Bridge: vmbr0 (VLAN-aware)
  • Native VLAN: 1 (untagged)
  • Container VLANs: All on VLAN 1 (untagged)
  • Note: ⚠️ Containers are not using VLAN tags - all on native VLAN

Routing

  • Default Gateway: 192.168.11.1 (VLAN 11)
  • Local Network: 192.168.11.0/24

Cluster Status

Cluster Membership

  • Cluster Name: h
  • Status: ✅ Quorate (3/3 nodes)
  • Nodes:
    • Node 1: ml110 (192.168.11.10)
    • Node 2: r630-01 (192.168.11.11)
    • Node 3: r630-02 (192.168.11.12) - Local

Quorum

  • Expected Votes: 3
  • Total Votes: 3
  • Quorum: 2
  • Status: ✅ Quorate

Storage Status

Storage Pools

Name Type Status Total Used Available Usage %
local dir ✅ Active 220GB 7.3GB 212GB 3.31%
thin1-r630-02 lvmthin ✅ Active 226GB 200GB 26GB ⚠️ 88.51%
thin2 lvmthin ✅ Active 226GB 200GB 26GB ⚠️ 88.33%
thin3 lvmthin ✅ Active 226GB 0GB 226GB 0.00%
thin4 lvmthin ✅ Active 226GB 29GB 197GB 12.69%
thin5 lvmthin ✅ Active 226GB 0GB 226GB 0.00%
thin6 lvmthin ✅ Active 226GB 0GB 226GB 0.00%

Concerns:

  • ⚠️ thin1-r630-02: 88.51% full (26GB free)
  • ⚠️ thin2: 88.33% full (26GB free)
  • ⚠️ Consider cleanup or expansion

System Logs

Recent Activity

  • ✅ Cluster filesystem verification: Successful (hourly)
  • ✅ Container operations: Normal (start, stop, backup)
  • ✅ System updates: Regular apt updates

Errors/Warnings

⚠️ CRITICAL: Memory OOM (Out of Memory) Kills

Multiple containers have been killed due to memory exhaustion:

Date Process Container Issue
Jan 14 01:39 npm exec func s UID:100000 OOM killed (708MB)
Jan 14 07:42 systemd-journal UID:100000 OOM killed (39MB)
Jan 14 07:42 npm exec func s UID:100000 OOM killed (633MB)
Jan 14 09:37 apt-get UID:100000 OOM killed (88MB)
Jan 14 11:10 node UID:100000 OOM killed (331MB)
Jan 14 13:01 python3 UID:100000 OOM killed (38MB)
Jan 14 16:06 npm exec func s UID:100000 OOM killed (633MB)
Jan 14 16:40 systemd-journal UID:100000 OOM killed (31MB)
Jan 14 16:48 networkd-dispat UID:100000 OOM killed (29MB)

Analysis:

  • Multiple containers experiencing OOM kills
  • Pattern suggests memory limits may be too restrictive
  • Some containers (npm/node processes) consuming significant memory
  • Systemd-journal and networkd-dispat also being killed

Recommendation:

  • Review container memory limits
  • Increase memory allocation for affected containers
  • Monitor memory usage patterns
  • Consider adding swap (currently 0B)

Other Issues:

  • ⚠️ Subscription check failed: DNS resolution issue (non-critical)

Firewall Status

Proxmox Firewall

  • Status: ✅ Enabled/Running
  • Configuration: Active

Firewall Rules

  • ✅ Rule: Allow Default Network (192.168.0.0/24)
    • Direction: IN
    • Action: ACCEPT
    • Source: 192.168.0.0/24
    • Logging: Disabled

Note: Firewall is properly configured to allow access from Default network.

Summary

✅ Operational Status

System: ✅ OPERATIONAL

  • Proxmox services running
  • Cluster quorate (3/3 nodes)
  • Network configured correctly
  • Firewall enabled and configured

⚠️ Concerns

  1. High Load Average:

    • Load: 12.03, 11.65, 11.51
    • With 56 CPU threads, this is moderate but worth monitoring
    • May indicate CPU-intensive workloads

  2. Memory OOM Kills:

    • Multiple containers killed due to memory exhaustion
    • Pattern suggests memory limits may be too restrictive
    • Need to review and adjust container memory allocations

  3. Storage Usage:

    • thin1-r630-02: 88.51% full (26GB free)
    • thin2: 88.33% full (26GB free)
    • Consider cleanup or expansion

  4. Container VLAN Assignment:

    • All containers on native VLAN (untagged)
    • Not utilizing VLAN plan yet
    • Ready for VLAN migration

📋 Recommendations

  1. Immediate:

    • Review container memory limits
    • Increase memory for containers experiencing OOM kills
    • Monitor load average trends

  2. Short-term:

    • Clean up or expand thin1-r630-02 and thin2 storage pools
    • Migrate containers to appropriate VLANs
    • Add swap space (currently 0B)

  3. Long-term:

    • Optimize container resource allocation
    • Plan storage expansion
    • Complete VLAN migration

🎯 Overall Assessment

Status: ✅ OPERATIONAL WITH CONCERNS

The system is functional and stable, but requires attention to:

  • Memory management (OOM kills)
  • Storage capacity (thin pools nearly full)
  • Load monitoring (high but manageable)

Ready for:

  • ✅ Normal operations
  • ✅ Container management
  • ✅ VLAN migration (when ready)
  • ⚠️ Memory optimization needed

Last Updated: 2026-01-15
Last Reviewed: 2026-01-15
Actions Taken: Memory limits adjusted, status re-checked

Reference in New Issue View Git Blame Copy Permalink