proxmox/docs/04-configuration/PROXMOX_ACME_CLOUDFLARE_PLAN.md

# Proxmox VE ACME Certificate Management Plan - Cloudflare Integration

**Date:** 2025-01-20  
**Status:** 📋 Planning Document  
**Purpose:** Comprehensive plan for SSL/TLS certificate management using ACME with Cloudflare

---

## Executive Summary

This document provides a comprehensive plan for implementing ACME (Automatic Certificate Management Environment) certificate management in Proxmox VE using Cloudflare as the DNS provider. This ensures proper security for all domains and services across hardware installations and VMs.

---

## Current Infrastructure

### Proxmox Nodes
- **ml110** (192.168.11.10) - Cluster master
- **r630-01** (192.168.11.11)
- **r630-02** (192.168.11.12)

### Services Requiring Certificates
- Proxmox VE Web UI (HTTPS on port 8006)
- VM/Container web services
- API endpoints
- Reverse proxy services (nginx, Cloudflare Tunnel)

---

## ACME Overview

**ACME (Automatic Certificate Management Environment):**
- Standard protocol for automated certificate management
- Proxmox VE has built-in ACME plugin
- Supports Let's Encrypt and other ACME-compliant CAs
- Automatic renewal before expiration

**Benefits:**
- ✅ Automated certificate provisioning
- ✅ Automatic renewal
- ✅ No manual intervention required
- ✅ Free certificates (Let's Encrypt)
- ✅ Secure by default

---

## Cloudflare Integration Options

### Option 1: Cloudflare API Token (Recommended)

**Method:** DNS-01 Challenge using Cloudflare API
- Most secure method
- Uses API tokens with minimal permissions
- Works for any domain in Cloudflare account
- Recommended for production

### Option 2: Cloudflare Global API Key

**Method:** DNS-01 Challenge using Global API Key
- Less secure (full account access)
- Easier initial setup
- Not recommended for production

### Option 3: HTTP-01 Challenge (Limited)

**Method:** HTTP-01 Challenge
- Requires public HTTP access
- Not suitable for internal-only services
- Limited applicability

---

## Implementation Plan

### Phase 1: Prerequisites and Preparation

#### 1.1 Cloudflare API Setup

**Requirements:**
- Cloudflare account with domains
- API token with DNS edit permissions
- Domain list inventory

**Steps:**
1. Create Cloudflare API token
   - Scope: Zone → DNS → Edit
   - Zone Resources: All zones (or specific zones)
   - Token expiration: Set appropriate expiration

2. Document domains requiring certificates
   - Proxmox node FQDNs (if configured)
   - VM/container service domains
   - API endpoint domains

3. Verify DNS management
   - Confirm Cloudflare manages DNS for all domains
   - Verify DNS records are accessible

#### 1.2 Proxmox VE Preparation

**Requirements:**
- Proxmox VE 7.0+ (ACME plugin included)
- Root or admin access to all nodes
- Network connectivity to ACME servers

**Steps:**
1. Verify ACME plugin availability
   ```bash
   pveversion
   # Should show version 7.0+
   ```

2. Check DNS resolution
   - Verify domains resolve correctly
   - Test external DNS queries

3. Prepare certificate storage
   - Review `/etc/pve/priv/acme/` directory
   - Plan certificate organization

---

### Phase 2: ACME Account Configuration

#### 2.1 Create ACME Account

**Location:** Proxmox Web UI → Datacenter → ACME

**Steps:**
1. Navigate to ACME settings
2. Add ACME account
3. Choose ACME directory:
   - **Let's Encrypt Production:** `https://acme-v02.api.letsencrypt.org/directory`
   - **Let's Encrypt Staging:** `https://acme-staging-v02.api.letsencrypt.org/directory` (for testing)

4. Configure account:
   - Email: Your contact email
   - Accept Terms of Service

5. Test with staging directory first
6. Switch to production after verification

#### 2.2 Configure Cloudflare DNS Plugin

**Method:** DNS-01 Challenge with Cloudflare API Token

**Configuration:**
1. In ACME account settings, select "DNS Plugin"
2. Choose plugin: **cloudflare**
3. Configure credentials:
   - **API Token:** Your Cloudflare API token
   - **Alternative:** Global API Key + Email (less secure)

**Security Best Practices:**
- ✅ Use API Token (not Global API Key)
- ✅ Limit token permissions to DNS edit only
- ✅ Use zone-specific tokens when possible
- ✅ Store tokens securely (consider secrets management)

---

### Phase 3: Certificate Configuration

#### 3.1 Proxmox Node Certificates

**Purpose:** Secure Proxmox VE Web UI

**Configuration:**
1. Navigate to: Node → System → Certificates
2. Select "ACME" tab
3. Add certificate:
   - **Name:** Descriptive name (e.g., "ml110-cert")
   - **Domain:** Node FQDN (e.g., `ml110.example.com`)
   - **ACME Account:** Select configured account
   - **DNS Plugin:** Select Cloudflare plugin
   - **Challenge Type:** DNS-01

4. Generate certificate
5. Apply to node
6. Repeat for all nodes

**Domains:**
- `ml110.yourdomain.com` (if configured)
- `r630-01.yourdomain.com` (if configured)
- `r630-02.yourdomain.com` (if configured)
- Or use IP-based access with self-signed (current)

#### 3.2 VM/Container Service Certificates

**Purpose:** Secure services running in VMs/containers

**Options:**

**Option A: Individual Certificates per Service**
- Generate separate certificate for each service domain
- Most granular control
- Suitable for: Multiple domains, different security requirements

**Option B: Wildcard Certificates**
- Generate `*.yourdomain.com` certificate
- Single certificate for all subdomains
- Suitable for: Many subdomains, simplified management

**Option C: Multi-Domain Certificates**
- Single certificate with multiple SANs
- Balance between granularity and simplicity
- Suitable for: Related services, limited domains

**Recommendation:** Start with individual certificates, consider wildcard for subdomains.

---

### Phase 4: Domain-Specific Certificate Plan

#### 4.1 Inventory All Domains

**Required Information:**
- Domain name
- Purpose/service
- VM/container hosting
- Current certificate status
- Certificate type needed

**Example Inventory:**
```
Domain                    | Service          | VM/Container | Type
-------------------------|------------------|--------------|----------
proxmox.yourdomain.com   | Proxmox UI       | ml110        | Individual
api.yourdomain.com       | API Gateway      | VM 100       | Individual
*.yourdomain.com         | All subdomains   | Multiple     | Wildcard
```

#### 4.2 Certificate Assignment Strategy

**Tier 1: Critical Infrastructure**
- Proxmox nodes (if using FQDNs)
- Core services
- API endpoints
- Individual certificates with short renewal periods

**Tier 2: Application Services**
- Web applications
- Services with public access
- Individual or multi-domain certificates

**Tier 3: Internal Services**
- Development environments
- Internal-only services
- Wildcard or self-signed (with proper internal CA)

---

### Phase 5: Implementation Steps

#### 5.1 Initial Setup (One-Time)

1. **Create Cloudflare API Token**
   ```bash
   # Via Cloudflare Dashboard:
   # My Profile → API Tokens → Create Token
   # Template: Edit zone DNS
   # Permissions: Zone → DNS → Edit
   # Zone Resources: All zones or specific zones
   ```

2. **Configure ACME Account in Proxmox**
   - Use Proxmox Web UI or CLI
   - Add account with Cloudflare plugin
   - Test with staging environment first

3. **Verify DNS Resolution**
   ```bash
   # Test domain resolution
   dig yourdomain.com +short
   nslookup yourdomain.com
   ```

#### 5.2 Certificate Generation (Per Domain)

**Via Proxmox Web UI:**
1. Navigate to ACME settings
2. Add certificate
3. Configure domain and plugin
4. Generate certificate
5. Apply to service

**Via CLI (Alternative):**
```bash
# Add ACME account
pvesh create /cluster/acme/account --directory-url https://acme-v02.api.letsencrypt.org/directory --contact email@example.com

# Register account
pvesh create /cluster/acme/account/test-account/register

# Generate certificate
pvesh create /cluster/acme/certificate --account test-account --domain yourdomain.com --dns cloudflare --plugin cloudflare --api-token YOUR_TOKEN
```

#### 5.3 Certificate Application

**For Proxmox Nodes:**
- Apply via Web UI: Node → System → Certificates
- Automatically updates web interface
- Requires service restart

**For VM/Container Services:**
- Copy certificate files to VM/container
- Configure service to use certificate
- Update service configuration
- Restart service

**Certificate File Locations:**
- Certificate: `/etc/pve/nodes/<node>/pve-ssl.pem`
- Private Key: `/etc/pve/nodes/<node>/pve-ssl.key`
- Full Chain: Combined certificate + chain

---

### Phase 6: Certificate Renewal and Maintenance

#### 6.1 Automatic Renewal

**Proxmox VE Automatic Renewal:**
- Built-in renewal mechanism
- Runs automatically before expiration
- Typically renews 30 days before expiry
- No manual intervention required

**Verification:**
- Monitor certificate expiration dates
- Check renewal logs
- Set up monitoring/alerting

#### 6.2 Monitoring and Alerts

**Monitoring Points:**
- Certificate expiration dates
- Renewal success/failure
- Service availability after renewal
- DNS challenge success rate

**Alerting Options:**
- Proxmox VE logs
- External monitoring tools
- Email notifications (configured in ACME account)

#### 6.3 Backup and Recovery

**Certificate Backup:**
- Backup `/etc/pve/priv/acme/` directory
- Backup certificate files
- Store API tokens securely
- Document certificate configuration

**Recovery Procedures:**
- Restore certificates from backup
- Re-generate if needed
- Update service configurations

---

## Security Best Practices

### 1. API Token Security

**Recommendations:**
- ✅ Use API Tokens (not Global API Key)
- ✅ Minimal required permissions
- ✅ Zone-specific tokens when possible
- ✅ Token rotation schedule
- ✅ Secure storage (encrypted, access-controlled)

### 2. Certificate Security

**Recommendations:**
- ✅ Use strong key sizes (RSA 2048+ or ECDSA P-256+)
- ✅ Enable HSTS where applicable
- ✅ Use TLS 1.2+ only
- ✅ Proper certificate chain validation
- ✅ Secure private key storage

### 3. Access Control

**Recommendations:**
- ✅ Limit ACME account access
- ✅ Role-based access control
- ✅ Audit certificate operations
- ✅ Secure credential storage

### 4. Network Security

**Recommendations:**
- ✅ Firewall rules for ACME endpoints
- ✅ DNS security (DNSSEC)
- ✅ Monitor for certificate abuse
- ✅ Rate limiting awareness

---

## Domain Inventory Template

```markdown
## Domain Certificate Inventory

### Proxmox Nodes
| Node    | Domain (if configured) | Certificate Type | Status |
|---------|------------------------|------------------|--------|
| ml110   | ml110.yourdomain.com   | Individual       | ⏳ Pending |
| r630-01 | r630-01.yourdomain.com | Individual       | ⏳ Pending |
| r630-02 | r630-02.yourdomain.com | Individual       | ⏳ Pending |

### VM/Container Services
| VMID | Service        | Domain              | Certificate Type | Status |
|------|----------------|---------------------|------------------|--------|
| 100  | Mail Gateway   | mail.yourdomain.com | Individual       | ⏳ Pending |
| 104  | Gitea          | git.yourdomain.com  | Individual       | ⏳ Pending |
| ...  | ...            | ...                 | ...              | ...    |

### Wildcard Certificates
| Domain Pattern      | Purpose          | Status |
|---------------------|------------------|--------|
| *.yourdomain.com    | All subdomains   | ⏳ Pending |
| *.api.yourdomain.com| API subdomains   | ⏳ Pending |
```

---

## Implementation Checklist

### Pre-Implementation
- [ ] Inventory all domains requiring certificates
- [ ] Create Cloudflare API token
- [ ] Document current certificate status
- [ ] Plan certificate assignment strategy
- [ ] Test with staging environment

### Implementation
- [ ] Configure ACME account in Proxmox
- [ ] Configure Cloudflare DNS plugin
- [ ] Generate test certificate (staging)
- [ ] Verify certificate generation works
- [ ] Switch to production ACME directory
- [ ] Generate production certificates
- [ ] Apply certificates to services
- [ ] Verify services work with new certificates

### Post-Implementation
- [ ] Monitor certificate expiration
- [ ] Verify automatic renewal works
- [ ] Set up monitoring/alerting
- [ ] Document certificate locations
- [ ] Create backup procedures
- [ ] Train team on certificate management

---

## Troubleshooting

### Common Issues

**1. DNS Challenge Fails**
- Verify API token permissions
- Check DNS propagation
- Verify domain is in Cloudflare account
- Check token expiration

**2. Certificate Generation Fails**
- Check ACME account status
- Verify domain ownership
- Check rate limits (Let's Encrypt)
- Review logs: `/var/log/pveproxy/access.log`

**3. Certificate Renewal Fails**
- Check automatic renewal configuration
- Verify DNS plugin still works
- Check API token validity
- Review renewal logs

**4. Service Not Using New Certificate**
- Verify certificate is applied to node
- Check service configuration
- Restart service
- Verify certificate file locations

---

## Alternative: External Certificate Management

If Proxmox ACME doesn't meet requirements:

### Option: Certbot with Cloudflare Plugin
- Install certbot on VM/container
- Use certbot-dns-cloudflare plugin
- Manual or automated renewal
- More control, more complexity

### Option: External ACME Client
- Use external ACME client (acme.sh, cert-manager)
- Generate certificates externally
- Copy to Proxmox/VMs
- More flexibility, manual integration

---

## Next Steps

1. **Complete domain inventory**
2. **Create Cloudflare API token**
3. **Configure ACME account (staging)**
4. **Test certificate generation**
5. **Switch to production**
6. **Generate certificates for all domains**
7. **Apply and verify**
8. **Monitor and maintain**

---

## Related Documentation

- [Proxmox VE ACME Documentation](https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_certificate_management)
- [Cloudflare API Token Guide](https://developers.cloudflare.com/api/tokens/)
- [Let's Encrypt Documentation](https://letsencrypt.org/docs/)
- Domain Structure: `docs/02-architecture/DOMAIN_STRUCTURE.md`
- Cloudflare API Setup: `CLOUDFLARE_API_SETUP.md`

---

**Last Updated:** 2025-01-20  
**Status:** 📋 Planning Document  
**Next Review:** After implementation