proxmox/docs/04-configuration/ALL_NEXT_STEPS.md

All Next Steps - Comprehensive List

Last Updated: 2026-01-31
Document Version: 1.0
Status: Active Documentation

---

Date: 2026-01-19
Status: Complete Inventory of Remaining Tasks
Purpose: Comprehensive list of all next steps, organized by priority and category

---

🎯 Priority 1: Critical/Blocking Tasks

1. Sankofa Services Deployment & Cutover

Status: ⚠️ BLOCKING - Sankofa domains currently route to wrong backend
Priority: 🔴 Critical
Reference: docs/04-configuration/SANKOFA_CUTOVER_PLAN.md

Pre-Deployment Tasks:

• Deploy Sankofa services on Proxmox VMs
• Assign VMIDs to Sankofa services
• Assign IP addresses to Sankofa services
• Document VMIDs, IPs, and ports in cutover plan
• Verify services are running and healthy
• Test health endpoints

Cutover Tasks:

• Update NPMplus proxy hosts for 5 Sankofa domains:
  • sankofa.nexus (Proxy Host ID: 21)
  • www.sankofa.nexus (Proxy Host ID: 22)
  • phoenix.sankofa.nexus (Proxy Host ID: 23)
  • www.phoenix.sankofa.nexus (Proxy Host ID: 24)
  • the-order.sankofa.nexus (Proxy Host ID: 25)
• Update backend targets from 192.168.11.140:80 to actual Sankofa service IPs/ports
• Verify SSL certificates still work after cutover
• Test all 5 domains end-to-end
• Update documentation with actual values (replace TBD placeholders)

Commands:

# After Sankofa services are deployed, update NPMplus routing
# See: docs/04-configuration/SANKOFA_CUTOVER_PLAN.md for detailed steps

---

2. Resolve TBD Nginx Config Paths

Status: ⚠️ BLOCKS VERIFICATION - Verification script skips these VMs
Priority: 🟡 Important
Reference: scripts/verify/verify-backend-vms.sh (lines 35-36)

VMID 10130 (dbis-frontend):

• SSH to VM: ssh root@192.168.11.130
• Find nginx config: find /etc/nginx -name "*dbis*" -o -name "*admin*"
• Verify config path (likely /etc/nginx/sites-available/dbis-frontend or /etc/nginx/sites-available/dbis-admin)
• Update script with actual path
• Verify config exists and is enabled

VMID 2400 (thirdweb-rpc-1):

• SSH to VM: ssh root@192.168.11.240
• Find nginx config: find /etc/nginx -name "*rpc*" -o -name "*thirdweb*"
• Verify config path (likely /etc/nginx/sites-available/thirdweb-rpc or /etc/nginx/sites-available/rpc)
• Update script with actual path
• Verify config exists and is enabled

Impact: Verification script will skip nginx config verification for these VMs until resolved.

---

🎯 Priority 2: Important Enhancements

3. Create NPMplus Backup Script

Status: ⚠️ MISSING - Referenced in documentation but not created
Priority: 🟡 Important
Reference: docs/04-configuration/NPMPLUS_BACKUP_RESTORE.md

Required Functionality:

• Automated backup of NPMplus database (/data/database.sqlite)
• Export of proxy hosts via API
• Export of certificates via API
• Certificate file backup from disk
• Compression and timestamping
• Configurable backup destination
• Backup verification
• Retention policy

Location: scripts/verify/backup-npmplus.sh

Reference: See NPMPLUS_BACKUP_RESTORE.md for detailed requirements.

---

4. Enhance Source of Truth Generation

Status: ⚠️ PARTIAL - Needs file dependency validation
Priority: 🟡 Important
Reference: scripts/verify/generate-source-of-truth.sh

Tasks:

• Add file existence checks before parsing JSON
• Provide clear error messages if dependencies are missing
• Add option to generate partial source-of-truth if some verifications haven't run
• Validate JSON structure before writing
• Add schema validation

Impact: Prevents errors when verification scripts haven't run first.

---

5. Security Hardening (Non-Breaking)

Status: ⚠️ RECOMMENDED - Security improvements
Priority: 🟡 Important
Reference: docs/04-configuration/INGRESS_RISKS_AND_HARDENING.md

Rate Limiting:

• Configure rate limiting for RPC endpoints (especially public RPC)
• Set rate limits per IP or per domain
• Configure in NPMplus or Nginx

Monitoring & Alerting:

• Set up log aggregation for NPMplus access logs
• Configure alerts for unusual traffic patterns
• Detect DDoS attempts early
• Set up certificate expiration monitoring

Cloudflare Access (Optional):

• Add authentication layer for dbis-admin.d-bis.org
• Add authentication layer for secure.d-bis.org
• Configure Cloudflare Access rules

Note: These can be implemented without breaking production.

---

🎯 Priority 3: Documentation & Quality of Life

6. Documentation Improvements

Status: ⚠️ RECOMMENDED - Documentation clarity
Priority: 🟢 Nice to Have

Update Placeholder References:

• Add clear notes in INGRESS_VERIFICATION_RUNBOOK.md that password examples should use .env file
• Add clear notes in NPMPLUS_BACKUP_RESTORE.md about .env file usage
• Add clear notes in SANKOFA_CUTOVER_PLAN.md about .env file usage
• Ensure all documentation references .env.example for required variables

Impact: Prevents confusion about where to get credentials.

---

7. HA Monitoring Enhancements

Status: ⚠️ OPTIONAL - Enhance existing monitoring
Priority: 🟢 Nice to Have

Tasks:

• Add email/webhook alerts to monitor-ha-status.sh
• Set up alerting for certificate expiration
• Add performance metrics collection
• Create dashboard for HA status
• Set up automated failover testing

Reference: scripts/npmplus/monitor-ha-status.sh

---

8. Verification Script Enhancements

Status: ⚠️ OPTIONAL - Improve verification coverage
Priority: 🟢 Nice to Have

Tasks:

• Add WebSocket connection testing to end-to-end verification
• Add certificate expiration date checking
• Add response time metrics
• Add automated comparison with source of truth
• Create verification report with pass/fail summary

Reference: scripts/verify/verify-end-to-end-routing.sh

---

🎯 Priority 4: Future Enhancements

9. Active-Active HA Upgrade

Status: ⚠️ FUTURE - Current HA is Active-Passive
Priority: 🔵 Future Consideration
Reference: docs/04-configuration/NPMPLUS_HA_SETUP_GUIDE.md

Current: Active-Passive with Keepalived
Future: Active-Active with load balancing

Tasks (when needed):

• Evaluate load balancing solution (HAProxy, Nginx, etc.)
• Design Active-Active architecture
• Plan shared database solution
• Test Active-Active failover
• Document migration path

Note: Current Active-Passive setup is sufficient for most use cases.

---

10. Cloudflare Proxy/WAF Evaluation

Status: ⚠️ FUTURE - Currently DNS-only by design
Priority: 🔵 Future Consideration
Reference: docs/04-configuration/INGRESS_RISKS_AND_HARDENING.md

Current: DNS-only mode (intentional for direct SSL termination)
Future: Evaluate enabling Cloudflare proxy/WAF

Considerations:

• Would require changing SSL termination approach
• Would require certificate changes
• Would provide DDoS protection and WAF
• Would add CDN caching

Note: Current DNS-only mode is intentional and working well.

---

📋 Quick Reference: Task Summary

Priority	Task	Status	Estimated Effort
🔴 Critical	Sankofa Services Deployment & Cutover	⚠️ Pending	2-4 hours
🟡 Important	Resolve TBD Nginx Config Paths	⚠️ Pending	30 minutes
🟡 Important	Create NPMplus Backup Script	⚠️ Pending	1-2 hours
🟡 Important	Enhance Source of Truth Generation	⚠️ Pending	1 hour
🟡 Important	Security Hardening	⚠️ Pending	2-4 hours
🟢 Nice to Have	Documentation Improvements	⚠️ Pending	30 minutes
🟢 Nice to Have	HA Monitoring Enhancements	⚠️ Pending	2-3 hours
🟢 Nice to Have	Verification Script Enhancements	⚠️ Pending	2-3 hours
🔵 Future	Active-Active HA Upgrade	⚠️ Future	TBD
🔵 Future	Cloudflare Proxy/WAF Evaluation	⚠️ Future	TBD

---

🚀 Immediate Action Items (This Week)

1. Deploy Sankofa Services (if not already deployed)

   • This is blocking the cutover
   • All 5 domains are currently misrouted

2. Resolve Nginx Config Paths

   • Quick wins - just need to find the actual paths
   • Unblocks verification script

3. Create Backup Script

   • Important for disaster recovery
   • Referenced in documentation but missing

---

📝 Notes

• HA Setup: ✅ Complete - All HA infrastructure is operational
• Verification Scripts: ✅ Complete - All scripts created and working
• Documentation: ✅ Complete - Comprehensive documentation available
• Sankofa Cutover: ⚠️ Pending - Waiting for Sankofa services deployment
• TBD Values: ⚠️ Expected - Will be resolved as services are deployed

---

🔗 Related Documentation

• Sankofa Cutover: docs/04-configuration/SANKOFA_CUTOVER_PLAN.md
• Gaps & TODOs: docs/04-configuration/VERIFICATION_GAPS_AND_TODOS.md
• Risks & Hardening: docs/04-configuration/INGRESS_RISKS_AND_HARDENING.md
• HA Setup: docs/04-configuration/NPMPLUS_HA_SETUP_GUIDE.md
• Backup/Restore: docs/04-configuration/NPMPLUS_BACKUP_RESTORE.md

---

Last Updated: 2026-01-19
Status: Complete inventory of all next steps