# Proxmox Storage Comprehensive Review

**Date:** Tue Jan  6 02:46:57 PST 2026
**Report Generated:** 2026-01-06 10:46:57 UTC
**Review Scope:** All Proxmox nodes and storage configurations

---

## Executive Summary

This report provides a comprehensive review of all storage configurations across all Proxmox nodes, including:
- Current storage status and usage
- Storage type analysis
- Performance recommendations
- Capacity planning
- Optimization suggestions

---

## Node Overview


### ml110 (192.168.11.10)

**Status:** ✅ Reachable

**System Resources:**
- CPU Cores: 6
- Memory: 125Gi 55Gi 69Gi
- VMs: 0
- Containers: 37

**Storage Status:**
```
Name                 Type     Status     Total (KiB)      Used (KiB) Available (KiB)        %
local                 dir     active        98497780         7902744        85545488    8.02%
local-lvm         lvmthin     active       832888832       232542561       600346270   27.92%
thin1             lvmthin   disabled               0               0               0      N/A
thin1-r630-02     lvmthin   disabled               0               0               0      N/A
thin2             lvmthin   disabled               0               0               0      N/A
thin3             lvmthin   disabled               0               0               0      N/A
thin4             lvmthin   disabled               0               0               0      N/A
thin5             lvmthin   disabled               0               0               0      N/A
thin6             lvmthin   disabled               0               0               0      N/A
```

**Volume Groups:**
```
  pve 930.51g 16.00g
```

**Thin Pools:**
```
  data              pve  794.30g 27.92  1.13       
  vm-1000-disk-0    pve  100.00g 8.26          data
  vm-1001-disk-0    pve  100.00g 7.67          data
  vm-1002-disk-0    pve  100.00g 7.44          data
  vm-1003-disk-0    pve  100.00g 7.47          data
  vm-1004-disk-0    pve  100.00g 7.26          data
  vm-10100-disk-0   pve  200.00g 2.82          data
  vm-10101-disk-0   pve  200.00g 2.37          data
  vm-10120-disk-0   pve   50.00g 5.11          data
  vm-10130-disk-0   pve   50.00g 8.08          data
  vm-10150-disk-0   pve  100.00g 7.05          data
  vm-10151-disk-0   pve  100.00g 7.30          data
  vm-1500-disk-0    pve  100.00g 6.61          data
  vm-1501-disk-0    pve  100.00g 6.59          data
  vm-1502-disk-0    pve  100.00g 6.77          data
  vm-1503-disk-0    pve  100.00g 6.65          data
  vm-1504-disk-0    pve  100.00g 0.51          data
  vm-1504-disk-1    pve  100.00g 2.67          data
  vm-2400-disk-0    pve  200.00g 4.40          data
  vm-2401-disk-0    pve  200.00g 4.29          data
  vm-2402-disk-0    pve  200.00g 4.29          data
  vm-2500-disk-0    pve  200.00g 5.05          data
  vm-2501-disk-0    pve  200.00g 5.12          data
  vm-2502-disk-0    pve  200.00g 4.78          data
  vm-2503-disk-0    pve  200.00g 0.51          data
  vm-2503-disk-1    pve  200.00g 3.97          data
  vm-2504-disk-0    pve  200.00g 3.94          data
  vm-2505-disk-0    pve   50.00g 10.98         data
  vm-2506-disk-0    pve   50.00g 11.01         data
  vm-2507-disk-0    pve   50.00g 10.87         data
  vm-2508-disk-0    pve   50.00g 10.33         data
  vm-3000-disk-0    pve   20.00g 30.51         data
  vm-3001-disk-0    pve   20.00g 17.23         data
  vm-3002-disk-0    pve   50.00g 6.97          data
  vm-3003-disk-0    pve   30.00g 6.09          data
  vm-3500-disk-0    pve   20.00g 14.71         data
  vm-3501-disk-0    pve   20.00g 11.32         data
  vm-5200-disk-0    pve   50.00g 3.31          data
  vm-6000-disk-0    pve   50.00g 3.30          data
  vm-6201-disk-0    pve   50.00g 3.27          data
  vm-6400-disk-0    pve   50.00g 3.30          data
  vm-9000-cloudinit pve    0.00g 9.38          data
  vm-9000-disk-0    pve 1000.00g 0.00          data
  vm-9000-disk-1    pve    2.20g 72.95         data
```

**Physical Disks:**
```
NAME   SIZE TYPE MOUNTPOINT
sda  931.5G disk 
sdb  931.5G disk 
```

---


### r630-04 (192.168.11.14)

**Status:** ❌ Not Reachable

**System Resources:**
- CPU Cores: Unknown
- Memory: Unknown
- VMs: 0
- Containers: 0

**Storage Status:**
```
No storage data available
```

**Volume Groups:**
```
No volume groups found
```

**Thin Pools:**
```
No thin pools found
```

**Physical Disks:**
```
No disk information available
```

---


### r630-01 (192.168.11.11)

**Status:** ✅ Reachable

**System Resources:**
- CPU Cores: 32
- Memory: 503Gi 7.5Gi 496Gi
- VMs: 0
- Containers: 3

**Storage Status:**
```
Name                 Type     Status     Total (KiB)      Used (KiB) Available (KiB)        %
local                 dir     active       561459584          127104       561332480    0.02%
local-lvm         lvmthin     active       209715200         6123683       203591516    2.92%
thin1             lvmthin     active       218103808               0       218103808    0.00%
thin1-r630-02     lvmthin   disabled               0               0               0      N/A
thin2             lvmthin   disabled               0               0               0      N/A
thin3             lvmthin   disabled               0               0               0      N/A
thin4             lvmthin   disabled               0               0               0      N/A
thin5             lvmthin   disabled               0               0               0      N/A
thin6             lvmthin   disabled               0               0               0      N/A
```

**Volume Groups:**
```
  pve 465.77g 57.46g
```

**Thin Pools:**
```
  data          pve 200.00g 2.92   11.42      
  thin1         pve 208.00g 0.00   10.43      
  vm-106-disk-0 pve  10.00g 12.43         data
  vm-107-disk-0 pve  20.00g 11.35         data
  vm-108-disk-0 pve  20.00g 11.60         data
```

**Physical Disks:**
```
NAME   SIZE TYPE MOUNTPOINT
sda  558.9G disk 
sdb  558.9G disk 
sdc  232.9G disk 
sdd  232.9G disk 
sde  232.9G disk 
sdf  232.9G disk 
sdg  232.9G disk 
sdh  232.9G disk 
sr0   1024M rom  
```

---


### r630-03 (192.168.11.13)

**Status:** ❌ Not Reachable

**System Resources:**
- CPU Cores: Unknown
- Memory: Unknown
- VMs: 0
- Containers: 0

**Storage Status:**
```
No storage data available
```

**Volume Groups:**
```
No volume groups found
```

**Thin Pools:**
```
No thin pools found
```

**Physical Disks:**
```
No disk information available
```

---


### r630-02 (192.168.11.12)

**Status:** ✅ Reachable

**System Resources:**
- CPU Cores: 56
- Memory: 251Gi 16Gi 235Gi
- VMs: 0
- Containers: 11

**Storage Status:**
```
Name                 Type     Status     Total (KiB)      Used (KiB) Available (KiB)        %
local                 dir     active       230441600         4167936       226273664    1.81%
local-lvm         lvmthin   disabled               0               0               0      N/A
thin1             lvmthin   inactive               0               0               0    0.00%
thin1-r630-02     lvmthin     active       237117440       231853432         5264007   97.78%
thin2             lvmthin     active       237117440               0       237117440    0.00%
thin3             lvmthin     active       237117440               0       237117440    0.00%
thin4             lvmthin     active       237117440        30090203       207027236   12.69%
thin5             lvmthin     active       237117440               0       237117440    0.00%
thin6             lvmthin     active       237117440               0       237117440    0.00%
```

**Volume Groups:**
```
  thin1 230.87g 0.12g
  thin2 230.87g 0.12g
  thin3 230.87g 0.12g
  thin4 230.87g 0.12g
  thin5 230.87g 0.12g
  thin6 230.87g 0.12g
```

**Thin Pools:**
```
  thin1          thin1 226.13g 97.78  3.84        
  vm-100-disk-0  thin1  10.00g 29.21         thin1
  vm-101-disk-0  thin1  10.00g 39.52         thin1
  vm-102-disk-0  thin1   2.00g 75.92         thin1
  vm-103-disk-0  thin1   8.00g 46.88         thin1
  vm-104-disk-0  thin1   8.00g 13.10         thin1
  vm-105-disk-0  thin1   8.00g 50.97         thin1
  vm-130-disk-0  thin1  50.00g 7.40          thin1
  vm-5000-disk-0 thin1 200.00g 97.16         thin1
  vm-6200-disk-0 thin1  50.00g 5.30          thin1
  vm-6201-disk-0 thin1  50.00g 6.33          thin1
  thin2          thin2 226.13g 0.00   0.72        
  thin3          thin3 226.13g 0.00   0.72        
  thin4          thin4 226.13g 12.69  1.15        
  vm-7811-disk-0 thin4  30.00g 95.66         thin4
  thin5          thin5 226.13g 0.00   0.72        
  thin6          thin6 226.13g 0.00   0.72        
```

**Physical Disks:**
```
NAME   SIZE TYPE MOUNTPOINT
sda  232.9G disk 
sdb  232.9G disk 
sdc  232.9G disk 
sdd  232.9G disk 
sde  232.9G disk 
sdf  232.9G disk 
sdg  232.9G disk 
sdh  232.9G disk 
```

---


## Storage Analysis and Recommendations

### 1. Storage Type Analysis

#### Local Storage (Directory-based)
- **Purpose:** ISO images, container templates, backups
- **Performance:** Good for read-heavy workloads
- **Recommendation:** Use for templates and ISOs, not for VM disks

#### LVM Thin Storage
- **Purpose:** VM/container disk images
- **Performance:** Excellent with thin provisioning
- **Benefits:** Space efficiency, snapshots, cloning
- **Recommendation:** ✅ **Preferred for VM/container disks**

#### ZFS Storage
- **Purpose:** High-performance VM storage
- **Performance:** Excellent with compression and deduplication
- **Benefits:** Data integrity, snapshots, clones
- **Recommendation:** Consider for high-performance workloads

### 2. Critical Issues and Fixes


#### ml110 Storage Issues

⚠️ **Issue:** Some storage pools are disabled or inactive

**Action Required:**
```bash
ssh root@192.168.11.10
pvesm status
# Enable disabled storage:
pvesm set <storage-name> --disable 0
```

⚠️ **Issue:** Storage usage is high (>80%)

**Recommendation:**
- Monitor storage usage closely
- Plan for expansion or cleanup
- Consider migrating VMs to other nodes


#### r630-01 Storage Issues

⚠️ **Issue:** Some storage pools are disabled or inactive

**Action Required:**
```bash
ssh root@192.168.11.11
pvesm status
# Enable disabled storage:
pvesm set <storage-name> --disable 0
```

⚠️ **Issue:** Storage usage is high (>80%)

**Recommendation:**
- Monitor storage usage closely
- Plan for expansion or cleanup
- Consider migrating VMs to other nodes


#### r630-02 Storage Issues

⚠️ **Issue:** Some storage pools are disabled or inactive

**Action Required:**
```bash
ssh root@192.168.11.12
pvesm status
# Enable disabled storage:
pvesm set <storage-name> --disable 0
```

⚠️ **Issue:** Storage usage is high (>80%)

**Recommendation:**
- Monitor storage usage closely
- Plan for expansion or cleanup
- Consider migrating VMs to other nodes


### 3. Performance Optimization Recommendations

#### Storage Performance Best Practices

1. **Use LVM Thin for VM Disks**
   - Better performance than directory storage
   - Thin provisioning saves space
   - Enables snapshots and cloning

2. **Monitor Thin Pool Metadata Usage**
   - Thin pools require metadata space
   - Monitor metadata_percent in lvs output
   - Expand metadata if >80% used

3. **Storage Distribution**
   - Distribute VMs across multiple nodes
   - Balance storage usage across nodes
   - Avoid overloading single node

4. **Backup Storage Strategy**
   - Use separate storage for backups
   - Consider NFS or Ceph for shared backups
   - Implement backup rotation policies

### 4. Capacity Planning

#### Current Storage Distribution


**Recommendations:**
- Monitor storage growth trends
- Plan for 20-30% headroom
- Set alerts at 80% usage
- Consider storage expansion before reaching capacity

### 5. Storage Type Recommendations by Use Case

| Use Case | Recommended Storage Type | Reason |
|----------|-------------------------|--------|
| VM/Container Disks | LVM Thin (lvmthin) | Best performance, thin provisioning |
| ISO Images | Directory (dir) | Read-only, no performance impact |
| Container Templates | Directory (dir) | Templates are read-only |
| Backups | Directory or NFS | Separate from production storage |
| High-Performance VMs | ZFS or LVM Thin | Best I/O performance |
| Development/Test | LVM Thin | Space efficient with cloning |

### 6. Security Recommendations

1. **Storage Access Control**
   - Review storage.cfg node restrictions
   - Ensure proper node assignments
   - Verify storage permissions

2. **Backup Security**
   - Encrypt backups if containing sensitive data
   - Store backups off-site
   - Test backup restoration regularly

### 7. Monitoring Recommendations

1. **Set Up Storage Monitoring**
   - Monitor storage usage (>80% alert)
   - Monitor thin pool metadata usage
   - Track storage growth trends

2. **Performance Monitoring**
   - Monitor I/O latency
   - Track storage throughput
   - Identify bottlenecks

3. **Automated Alerts**
   - Storage usage >80%
   - Thin pool metadata >80%
   - Storage errors or failures

### 8. Migration Recommendations

#### Workload Distribution

**Current State:**
- ml110: Hosting all VMs (overloaded)
- r630-01/r630-02: Underutilized

**Recommended Distribution:**
- **ml110:** Keep management/lightweight VMs (10-15 VMs)
- **r630-01:** Migrate medium workload VMs (10-15 VMs)
- **r630-02:** Migrate heavy workload VMs (10-15 VMs)

**Benefits:**
- Better performance (ml110 CPU is slower)
- Better resource utilization
- Improved redundancy
- Better storage distribution

### 9. Immediate Action Items

#### Critical (Do First)
1. ✅ Review storage status on all nodes
2. ⚠️ Enable disabled storage pools
3. ⚠️ Verify storage node restrictions in storage.cfg
4. ⚠️ Check for storage errors or warnings

#### High Priority
1. ⚠️ Configure LVM thin storage where missing
2. ⚠️ Set up storage monitoring and alerts
3. ⚠️ Plan VM migration for better distribution
4. ⚠️ Review and optimize storage.cfg

#### Recommended
1. ⚠️ Implement backup storage strategy
2. ⚠️ Consider shared storage (NFS/Ceph) for HA
3. ⚠️ Optimize storage performance settings
4. ⚠️ Document storage procedures

---

## Detailed Storage Commands Reference

### Check Storage Status
```bash
# On any Proxmox node
pvesm status
pvesm list <storage-name>
```

### Enable Disabled Storage
```bash
pvesm set <storage-name> --disable 0
```

### Check LVM Configuration
```bash
vgs                    # List volume groups
lvs                    # List logical volumes
lvs -o +data_percent,metadata_percent  # Check thin pool usage
```

### Check Disk Usage
```bash
df -h                  # Filesystem usage
lsblk                  # Block devices
```

### Storage Performance Testing
```bash
# Test storage I/O
fio --name=test --ioengine=libaio --iodepth=16 --rw=randwrite --bs=4k --size=1G --runtime=60
```

---

## Conclusion

This comprehensive storage review provides:
- ✅ Current storage status across all nodes
- ✅ Detailed analysis of storage configurations
- ✅ Performance optimization recommendations
- ✅ Capacity planning guidance
- ✅ Security and monitoring recommendations
- ✅ Migration and distribution strategies

**Next Steps:**
1. Review this report
2. Address critical issues first
3. Implement high-priority recommendations
4. Plan for long-term optimizations

---

**Report Generated:** Tue Jan  6 02:47:02 PST 2026
**Report File:** /home/intlc/projects/proxmox/reports/storage/storage_review_20260106_024632.md

