Sankofa/docs/runbooks/DATA_RETENTION_POLICY.md

Data Retention Policy

Overview

This document defines data retention policies for the Sankofa Phoenix platform to ensure compliance with regulatory requirements and optimize storage costs.

Retention Periods

Application Data

User Data

• Active Users: Retained indefinitely while account is active
• Inactive Users: Retained for 7 years after last login
• Deleted Users: Soft delete for 90 days, then permanent deletion
• User Activity Logs: 2 years

Tenant Data

• Active Tenants: Retained indefinitely while tenant is active
• Suspended Tenants: Retained for 1 year after suspension
• Deleted Tenants: Soft delete for 90 days, then permanent deletion

Resource Data

• Active Resources: Retained indefinitely
• Deleted Resources: Retained for 90 days for recovery purposes
• Resource History: 1 year

Audit and Compliance Data

Audit Logs

• Security Events: 7 years (compliance requirement)
• Authentication Logs: 2 years
• Authorization Logs: 2 years
• Data Access Logs: 2 years
• Administrative Actions: 7 years

Compliance Data

• STIG Compliance Reports: 7 years
• RMF Documentation: 7 years
• Incident Reports: 7 years
• Risk Assessments: 7 years

Operational Data

Application Logs

• Application Logs (Loki): 30 days
• Access Logs: 90 days
• Error Logs: 90 days
• Performance Logs: 30 days

Metrics

• Prometheus Metrics: 30 days (raw)
• Aggregated Metrics: 1 year
• Custom Metrics: 90 days

Backups

• Database Backups: 7 days (daily), 4 weeks (weekly), 12 months (monthly)
• Configuration Backups: 90 days
• Disaster Recovery Backups: 7 years

Blockchain Data

Transaction History

• All Transactions: Retained indefinitely (immutable)
• Transaction Logs: 7 years

Smart Contract Data

• Contract State: Retained indefinitely
• Contract Events: 7 years

Data Deletion Procedures

Automated Deletion

Scheduled Cleanup Jobs

# Run daily cleanup job
kubectl create cronjob cleanup-old-data \
  --image=postgres:14-alpine \
  --schedule="0 3 * * *" \
  --restart=OnFailure \
  -- /bin/bash -c "psql $DATABASE_URL -f /scripts/cleanup-old-data.sql"

Cleanup Scripts

• User Data Cleanup: Runs monthly, deletes users inactive > 7 years
• Log Cleanup: Runs daily, deletes logs older than retention period
• Backup Cleanup: Runs daily, deletes backups older than retention period

Manual Deletion

User-Requested Deletion

1. User submits deletion request
2. Account marked for deletion
3. 30-day grace period for account recovery
4. Data anonymized after grace period
5. Permanent deletion after 90 days

Administrative Deletion

1. Admin initiates deletion
2. Approval required for sensitive data
3. Data exported for compliance (if required)
4. Data deleted according to retention policy

Compliance Requirements

GDPR (General Data Protection Regulation)

• Right to Erasure: Users can request data deletion
• Data Portability: Users can export their data
• Retention Limitation: Data retained only as long as necessary

SOX (Sarbanes-Oxley Act)

• Financial Records: 7 years retention
• Audit Trails: 7 years retention

HIPAA (Health Insurance Portability and Accountability Act)

• PHI Data: 6 years minimum retention
• Access Logs: 6 years minimum retention

DoD/MilSpec Compliance

• Security Logs: 7 years retention
• Audit Trails: 7 years retention
• Compliance Reports: 7 years retention

Implementation

Database Retention

Automated Cleanup Queries

-- Delete inactive users (7 years)
DELETE FROM users 
WHERE last_login < NOW() - INTERVAL '7 years'
  AND status = 'INACTIVE';

-- Delete old audit logs (after 2 years, archive first)
INSERT INTO audit_logs_archive 
SELECT * FROM audit_logs 
WHERE created_at < NOW() - INTERVAL '2 years';

DELETE FROM audit_logs 
WHERE created_at < NOW() - INTERVAL '2 years';

Log Retention

Loki Retention Configuration

# gitops/apps/monitoring/loki-config.yaml
retention_period: 30d
retention_stream:
  - selector: '{job="api"}'
    period: 90d
  - selector: '{job="portal"}'
    period: 90d

Prometheus Retention Configuration

# gitops/apps/monitoring/prometheus-config.yaml
retention: 30d
retentionSize: 50GB

Backup Retention

Backup Cleanup Script

# Delete backups older than retention period
find /backups/postgres -name "*.sql.gz" -mtime +7 -delete
find /backups/postgres -name "*.sql.gz" -mtime +30 -delete  # Weekly backups
find /backups/postgres -name "*.sql.gz" -mtime +365 -delete  # Monthly backups

Data Archival

Long-Term Storage

Archived Data Storage

• Location: S3 Glacier or equivalent
• Format: Compressed, encrypted archives
• Retention: Per compliance requirements
• Access: On-demand restoration

Archive Process

1. Data identified for archival
2. Data compressed and encrypted
3. Data uploaded to archival storage
4. Index updated with archive location
5. Original data deleted after verification

Monitoring and Compliance

Retention Policy Compliance

Automated Checks

• Daily verification of retention policies
• Alert on data older than retention period
• Report on data deletion activities

Compliance Reporting

• Monthly retention compliance report
• Quarterly audit of data retention
• Annual compliance review

Exceptions and Extensions

Legal Hold

• Data subject to legal hold cannot be deleted
• Legal hold overrides retention policy
• Legal hold must be documented
• Data released after legal hold lifted

Business Requirements

• Extended retention for business-critical data
• Approval required for extensions
• Extensions documented and reviewed annually

Contact

For questions about data retention:

• Data Protection Officer: dpo@sankofa.nexus
• Compliance Team: compliance@sankofa.nexus
• Legal Team: legal@sankofa.nexus