Files
Sankofa/docs/guides/FORCE_UNLOCK_INSTRUCTIONS.md
defiQUG fe0365757a Update documentation structure and enhance .gitignore
- Added generated index files and report directories to .gitignore to prevent unnecessary tracking of transient files.
- Updated README links to reflect new documentation paths for better navigation.
- Improved documentation organization by ensuring all links point to the correct locations, enhancing user experience and accessibility.
2025-12-12 21:18:55 -08:00

2.5 KiB

Force Unlock VM Instructions

Date: 2025-12-09
Issue: qm unlock 100 is timing out


Problem

The qm unlock command is timing out, which indicates:

  • A stuck process is holding the lock
  • The lock file is corrupted or in an invalid state
  • Another operation is blocking the unlock

Solution: Force Unlock

On Proxmox Node (root@ml110-01):

# Copy the script to the Proxmox node
# Or run commands manually (see Option 2)

# Run the script
bash force-unlock-vm-proxmox.sh 100

Option 2: Manual Commands

On Proxmox Node (root@ml110-01):

# 1. Check for stuck processes
ps aux | grep -E 'qm|qemu' | grep 100

# 2. Check lock file
ls -la /var/lock/qemu-server/lock-100.conf
cat /var/lock/qemu-server/lock-100.conf 2>/dev/null

# 3. Kill stuck processes (if found)
pkill -9 -f 'qm.*100'
pkill -9 -f 'qemu.*100'

# 4. Wait a moment
sleep 2

# 5. Force remove lock file
rm -f /var/lock/qemu-server/lock-100.conf

# 6. Verify lock is gone
ls -la /var/lock/qemu-server/lock-100.conf
# Should show: No such file or directory

# 7. Check VM status
qm status 100

# 8. Try unlock again (should work now)
qm unlock 100

If Lock Persists

Check for Other Issues

# Check if VM is in a transitional state
qm status 100

# Check VM configuration
qm config 100

# Check for other locks
ls -la /var/lock/qemu-server/lock-*.conf

# Check system resources
df -h
free -h

Nuclear Option: Restart Proxmox Services

⚠️ WARNING: This will affect all VMs on the node

# Only if absolutely necessary
systemctl restart pve-cluster
systemctl restart pvedaemon

After Successful Unlock

  1. Monitor VM Status:

    qm status 100
    
  2. Check Provider Logs (from Kubernetes):

    kubectl logs -n crossplane-system -l app=crossplane-provider-proxmox --tail=50 -f
    
  3. Watch VM Resource:

    kubectl get proxmoxvm basic-vm-001 -w
    
  4. Expected Outcome:

    • Provider will retry within 1 minute
    • VM configuration will complete
    • VM will boot successfully

Prevention

To prevent this issue in the future:

  1. Ensure proper VM shutdown before operations
  2. Wait for operations to complete before starting new ones
  3. Monitor for stuck processes regularly
  4. Implement lock timeout handling in provider code (already added)

Last Updated: 2025-12-09
Status: ⚠️ MANUAL FORCE UNLOCK REQUIRED