proxmox/docs/04-configuration/R630-02_STORAGE_REVIEW.md

r630-02 Storage Review and Recommendations

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation

---

Date: 2026-01-15
Host: r630-02 (192.168.11.12)
Status: ✅ REVIEW COMPLETE

---

Executive Summary

Total Storage: ~1.85TB (8 x 232GB disks)
Critical Issues: 2 pools at 88%+ capacity, 1 container at 94.8% full
Immediate Actions Required: Container 7811 disk expansion, thin2 cleanup
Potential Space Recovery: ~300GB from duplicate volumes and old backups

---

Storage Architecture

Physical Disks

Disk	Size	Type	Usage	Status
sda	232.9GB	ZFS	System/Boot	Active
sdb	232.9GB	ZFS	System/Boot	Active
sdc	232.9GB	LVM	thin2 pool	88.33% full
sdd	232.9GB	LVM	thin3 pool	0% (empty)
sde	232.9GB	LVM	thin4 pool	12.69% used
sdf	232.9GB	LVM	thin5 pool	0% (empty)
sdg	232.9GB	LVM	thin6 pool	0% (empty)
sdh	232.9GB	LVM	thin1 pool	88.51% full

Total Raw Capacity: ~1.86TB
Total Usable (LVM Thin Pools): ~1.36TB (6 x 226GB pools)

---

Storage Pools Status

Active Pools

Pool	Type	Status	Total	Used	Available	Usage %	Priority
thin1-r630-02	lvmthin	Active	226GB	200GB	26GB	88.51%	🔴 Critical
thin2	lvmthin	Active	226GB	200GB	27GB	88.33%	🔴 Critical
thin4	lvmthin	Active	226GB	29GB	197GB	12.69%	🟢 Healthy
thin3	lvmthin	Active	226GB	0GB	226GB	0.00%	🟢 Available
thin5	lvmthin	Active	226GB	0GB	226GB	0.00%	🟢 Available
thin6	lvmthin	Active	226GB	0GB	226GB	0.00%	🟢 Available
local	dir	Active	220GB	7.3GB	213GB	3.31%	🟢 Healthy

Inactive/Problematic Pools

Pool	Type	Status	Issue
data	lvmthin	Inactive	No logical volume found
thin1	lvmthin	Inactive	No logical volume found
local-lvm	lvmthin	Disabled	Not configured

---

Container Disk Usage

Current Allocation

Container	Name	Pool	Allocated	Used	Available	Usage %	Status
5000	blockscout-1	thin2	200GB	125.2GB	60.6GB	63.9%	🟡 Moderate
6200	firefly-1	thin2	50GB	3.3GB	43.1GB	6.7%	🟢 Healthy
6201	firefly-ali-1	thin2	50GB	2.8GB	43.6GB	5.7%	🟢 Healthy
7811	mim-api-1	thin4	30GB	27.8GB	0GB	94.8%	🔴 CRITICAL

Critical Issues

1. Container 7811 (mim-api-1): 94.8% full, 0GB available - IMMEDIATE ACTION REQUIRED
2. thin2 Pool: 88.33% full with 3 containers - NEEDS CLEANUP
3. thin1-r630-02 Pool: 88.51% full - NEEDS INVESTIGATION

---

Duplicate Volumes Issue

Problem Identified

Duplicate volumes exist on both thin1 and thin2:

Container	thin1 Volume	thin2 Volume	Status
5000	vm-5000-disk-0 (200GB)	vm-5000-disk-0 (200GB)	⚠️ Duplicate
6200	vm-6200-disk-0 (50GB)	vm-6200-disk-0 (50GB)	⚠️ Duplicate
6201	vm-6201-disk-0 (50GB)	vm-6201-disk-0 (50GB)	⚠️ Duplicate

Total Duplicate Space: ~300GB (200GB + 50GB + 50GB)

Analysis:

• Containers are currently using thin2 volumes (confirmed by pct config)
• thin1 volumes appear to be orphaned/unused
• This explains why thin1-r630-02 shows 88.51% usage despite no active containers

---

Snapshots and Backups

Snapshots

Current Snapshots: None (all containers show only "current" snapshot)

Recommendation: Consider creating snapshots before major changes.

Backup Files

Location: /var/lib/vz/dump
Total Size: 7.2GB
Count: 13 backup files
Date Range: January 3-7, 2026

Largest Backup Files:

• vzdump-lxc-103-2026_01_07-12_38_26.tar.gz (1.4GB)
• vzdump-lxc-7811-2026_01_03-14_57_45.tar.gz (1.3GB)
• vzdump-lxc-130-2026_01_07-12_46_28.tar.gz (1.1GB)
• vzdump-lxc-7810-2026_01_03-14_55_23.tar.gz (781MB)
• vzdump-lxc-7810-2026_01_03-14_03_16.tar.gz (781MB)

Note: These backups are from containers that may no longer exist (7800, 7801, 7802, 7810, 103, 104, 105, 130).

---

Detailed Recommendations

🔴 IMMEDIATE ACTIONS (Critical)

1. Expand Container 7811 Disk (URGENT)

Issue: Container 7811 is 94.8% full with 0GB available.

Action:

# Expand container 7811 disk from 30GB to 50GB
ssh root@192.168.11.12 "pct resize 7811 rootfs +20G"

Impact: Prevents container from running out of space and crashing.

Priority: CRITICAL - Do immediately

---

2. Investigate and Clean Up Duplicate Volumes on thin1

Issue: ~300GB of duplicate volumes on thin1-r630-02 pool.

Steps:

1. Verify thin1 volumes are not in use:

ssh root@192.168.11.12 "lvs -o lv_name,vg_name,attr | grep -E 'vm-5000|vm-6200|vm-6201' | grep thin1"

2. Check if any containers reference thin1:

ssh root@192.168.11.12 "for CTID in 5000 6200 6201; do echo \"Container \$CTID:\"; pct config \$CTID | grep rootfs; done"

3. If confirmed unused, remove duplicate volumes:

# WARNING: Only if confirmed unused!
ssh root@192.168.11.12 "lvremove /dev/thin1/vm-5000-disk-0"
ssh root@192.168.11.12 "lvremove /dev/thin1/vm-6200-disk-0"
ssh root@192.168.11.12 "lvremove /dev/thin1/vm-6201-disk-0"

Potential Space Recovery: ~300GB on thin1-r630-02

Priority: HIGH - Frees significant space

---

🟡 SHORT-TERM ACTIONS (Within 1 Week)

3. Review and Clean Up Old Backup Files

Issue: 7.2GB of backup files from January 3-7, 2026, possibly from deleted containers.

Action:

1. Identify backups for non-existent containers:

ssh root@192.168.11.12 "pct list | awk '{print \$1}' > /tmp/active_containers.txt"
ssh root@192.168.11.12 "for file in /var/lib/vz/dump/*.tar.gz; do CTID=\$(echo \$file | grep -oP 'lxc-\K[0-9]+'); if ! grep -q \"^\$CTID\$\" /tmp/active_containers.txt; then echo \"Orphaned: \$file\"; fi; done"

2. Move old backups to archive or delete:

# Archive backups older than 30 days
ssh root@192.168.11.12 "find /var/lib/vz/dump -name '*.tar.gz' -mtime +30 -exec mv {} /var/lib/vz/archive/ \;"

Potential Space Recovery: ~7GB on local storage

Priority: MEDIUM - Good housekeeping

---

4. Optimize thin2 Pool Usage

Issue: thin2 is 88.33% full with 3 containers.

Options:

Option A: Move containers to empty pools

• Move container 6200 or 6201 to thin3, thin5, or thin6
• Reduces thin2 usage to ~70%

Option B: Expand thin2 pool (if possible)

• Check if physical disk can be expanded
• Requires additional storage hardware

Option C: Clean up container 5000

• Container 5000 uses 125.2GB of 200GB
• Review if data can be cleaned up or archived

Recommended: Option A - Move container 6201 (stopped) to thin3

Priority: MEDIUM - Prevents future issues

---

🟢 LONG-TERM ACTIONS (Within 1 Month)

5. Implement Backup Retention Policy

Action:

• Configure automated backup cleanup
• Keep only last 7 days of daily backups
• Keep weekly backups for 4 weeks
• Keep monthly backups for 6 months

Implementation:

# Add to cron or Proxmox backup job
find /var/lib/vz/dump -name '*.tar.gz' -mtime +30 -delete

Priority: LOW - Prevents future accumulation

---

6. Monitor Storage Usage

Action:

• Set up alerts for pools >80% full
• Monitor container disk usage weekly
• Review storage allocation quarterly

Priority: LOW - Preventive maintenance

---

7. Plan for Storage Expansion

Current State:

• 2 pools at 88%+ capacity
• 1 container at 94.8% full
• 3 empty pools available (thin3, thin5, thin6)

Recommendation:

• Utilize empty pools for new containers
• Consider storage expansion if growth continues
• Plan for 6-12 month capacity

Priority: LOW - Strategic planning

---

Space Recovery Summary

Action	Potential Recovery	Pool	Priority
Remove duplicate thin1 volumes	~300GB	thin1-r630-02	🔴 High
Clean up old backups	~7GB	local	🟡 Medium
Move containers to empty pools	N/A (redistribution)	thin2 → thin3/5/6	🟡 Medium
Total Potential Recovery	~307GB

---

Action Plan Summary

Immediate (Today)

1. ✅ Expand container 7811 disk to 50GB
2. ✅ Verify and remove duplicate thin1 volumes

Short-term (This Week)

3. Review and archive old backup files
4. Move container 6201 to thin3 pool

Long-term (This Month)

5. Implement backup retention policy
6. Set up storage monitoring
7. Plan for future expansion

---

Risk Assessment

Low Risk Actions

• ✅ Expanding container 7811 disk (non-destructive)
• ✅ Moving stopped container 6201 to thin3
• ✅ Cleaning up old backup files

Medium Risk Actions

• ⚠️ Removing duplicate thin1 volumes (verify first!)
• ⚠️ Moving running containers (requires downtime)

High Risk Actions

• ❌ None identified

---

Storage Utilization Summary

Current Utilization:

• Used: ~430GB (23% of total)
• Available: ~1.43TB (77% of total)
• Critical Pools: 2 (thin1-r630-02, thin2)
• Empty Pools: 3 (thin3, thin5, thin6)

After Recommended Actions:

• Used: ~130GB (7% of total)
• Available: ~1.73TB (93% of total)
• Critical Pools: 0
• Empty Pools: 3 (thin3, thin5, thin6)

---

Last Updated: 2026-01-15