dbis_docs/00_document_control/processes/Link_Verification_Automation.md

# LINK VERIFICATION AUTOMATION
## Procedures for Automated Link Verification

---

## DOCUMENT METADATA

**Document Number:** DBIS-DC-LVA-001  
**Version:** 1.0  
**Date:** 2024-12-08  
**Classification:** UNCLASSIFIED  
**Authority:** DBIS Executive Directorate  
**Approved By:** [See signature block - requires SCC approval]  
**Effective Date:** 2024-12-08  
**Distribution:** Distribution Statement A - Public Release Unlimited

**Change Log:**
- 2024-12-08 - Version 1.0 - Initial Release

---

## EXECUTIVE SUMMARY

This document provides procedures for setting up and using automated link verification to maintain 100% link integrity in the DBIS documentation corpus. It covers script usage, scheduling, reporting, and maintenance.

**Purpose:** Ensure link integrity is maintained through automated verification.

**Current Status:** ✅ 100.00% link verification success rate (1268/1268 links valid)

---

## AVAILABLE VERIFICATION SCRIPTS

### Scripts Available

**Location:** `scripts/` directory

1. **verify_cross_references.sh** - Comprehensive bash script
   - Full cross-reference verification
   - Detailed reporting
   - Color-coded output
   - Broken link detection

2. **verify_cross_references_simple.sh** - Simplified bash script
   - Quick verification
   - Basic reporting
   - Fast execution

3. **verify_links.py** - Python script
   - Advanced link verification
   - Detailed analysis
   - Report generation
   - Anchor verification

### Script Selection Guide

**Use verify_cross_references.sh when:**
- Running comprehensive verification
- Need detailed reporting
- Preparing for releases
- After major structural changes

**Use verify_cross_references_simple.sh when:**
- Quick verification needed
- Daily checks
- Pre-commit verification
- Fast feedback needed

**Use verify_links.py when:**
- Need advanced analysis
- Anchor verification required
- Detailed statistics needed
- Integration with other tools

---

## MANUAL VERIFICATION PROCEDURES

### Running Verification Manually

**Basic Verification:**
```bash
cd /home/intlc/projects/dbis_docs
./scripts/verify_cross_references.sh
```

**Simple Verification:**
```bash
cd /home/intlc/projects/dbis_docs
./scripts/verify_cross_references_simple.sh
```

**Python Verification:**
```bash
cd /home/intlc/projects/dbis_docs
python3 scripts/verify_links.py
```

### Verification Workflow

**Before Committing Changes:**
1. Run verification script
2. Review results
3. Fix any broken links
4. Re-run verification
5. Commit when 100% success

**After File Moves:**
1. Run verification script
2. Identify broken links
3. Update all broken links
4. Re-run verification
5. Verify 100% success

**After Structural Changes:**
1. Run comprehensive verification
2. Review detailed report
3. Fix all broken links
4. Update cross-references
5. Re-run verification
6. Document changes

---

## AUTOMATED VERIFICATION SETUP

### Scheduled Verification (Cron Job)

**Daily Verification:**
```bash
# Add to crontab (crontab -e)
# Run daily at 2 AM
0 2 * * * cd /home/intlc/projects/dbis_docs && ./scripts/verify_cross_references_simple.sh >> /var/log/dbis_link_verification.log 2>&1
```

**Weekly Comprehensive Verification:**
```bash
# Run weekly on Sunday at 3 AM
0 3 * * 0 cd /home/intlc/projects/dbis_docs && ./scripts/verify_cross_references.sh >> /var/log/dbis_link_verification_weekly.log 2>&1
```

**Post-Commit Hook (Git):**
```bash
# .git/hooks/post-commit
#!/bin/bash
cd /home/intlc/projects/dbis_docs
./scripts/verify_cross_references_simple.sh
if [ $? -ne 0 ]; then
    echo "Warning: Link verification found issues. Please review."
fi
```

### Automated Verification Procedures

**Setup Steps:**
1. **Review Scripts:**
   - [ ] Review verify_cross_references.sh
   - [ ] Review verify_cross_references_simple.sh
   - [ ] Review verify_links.py
   - [ ] Test scripts manually

2. **Configure Logging:**
   - [ ] Create log directory: `logs/`
   - [ ] Set up log rotation
   - [ ] Configure log retention (30 days recommended)

3. **Set Up Scheduling:**
   - [ ] Configure daily verification (cron)
   - [ ] Configure weekly comprehensive verification
   - [ ] Set up email alerts for failures (optional)

4. **Create Alert System:**
   - [ ] Set up email alerts for broken links
   - [ ] Configure notification thresholds
   - [ ] Test alert system

5. **Document Procedures:**
   - [ ] Document cron job setup
   - [ ] Document alert configuration
   - [ ] Document maintenance procedures

### Verification Report Handling

**Report Storage:**
- Daily reports: `logs/daily_verification_YYYY-MM-DD.log`
- Weekly reports: `logs/weekly_verification_YYYY-MM-DD.log`
- Comprehensive reports: `project_management/reports/CROSS_REFERENCE_VERIFICATION_REPORT.md`

**Report Review:**
- Review daily reports weekly
- Review weekly reports monthly
- Review comprehensive reports quarterly
- Address issues promptly

---

## VERIFICATION WORKFLOW

### Standard Verification Workflow

```
1. Run Verification
   ├─ Select appropriate script
   ├─ Execute verification
   └─ Review results

2. Analyze Results
   ├─ Check success rate
   ├─ Identify broken links
   ├─ Categorize issues
   └─ Prioritize fixes

3. Fix Issues
   ├─ Update broken links
   ├─ Fix file paths
   ├─ Update cross-references
   └─ Verify fixes

4. Re-Verify
   ├─ Run verification again
   ├─ Confirm 100% success
   └─ Document completion

5. Report
   ├─ Update verification report
   ├─ Document fixes made
   └─ Update change log
```

### Verification Decision Tree

```
When to run verification?
├─ Before commit → Simple verification
├─ After file moves → Comprehensive verification
├─ After structural changes → Comprehensive verification
├─ Scheduled (daily) → Simple verification
├─ Scheduled (weekly) → Comprehensive verification
└─ Before release → Comprehensive verification

What script to use?
├─ Quick check → verify_cross_references_simple.sh
├─ Comprehensive check → verify_cross_references.sh
└─ Advanced analysis → verify_links.py

Issues found?
├─ Broken links → Fix immediately
├─ Missing files → Verify file exists, update path
└─ Invalid anchors → Fix anchor references
```

---

## VERIFICATION MAINTENANCE

### Script Maintenance

**Regular Maintenance:**
- [ ] Review scripts quarterly
- [ ] Update scripts for new directory structure
- [ ] Test scripts after structural changes
- [ ] Document script changes

**Script Updates:**
- Update scripts when directory structure changes
- Update scripts when new file types added
- Update scripts for new link patterns
- Test updates before deployment

### Verification Metrics

**Track Metrics:**
- Total links verified
- Success rate
- Broken links found
- Fix time
- Verification frequency

**Metrics Dashboard:**
- Monthly verification summary
- Success rate trends
- Issue resolution time
- Verification coverage

---

## VERIFICATION CHECKLIST

### Pre-Commit Verification

- [ ] Run simple verification
- [ ] Review results
- [ ] Fix any broken links
- [ ] Re-verify
- [ ] Commit when clean

### Post-Move Verification

- [ ] Run comprehensive verification
- [ ] Review detailed report
- [ ] Fix all broken links
- [ ] Update all references
- [ ] Re-verify
- [ ] Document fixes

### Scheduled Verification

- [ ] Daily: Run simple verification
- [ ] Weekly: Run comprehensive verification
- [ ] Monthly: Review verification reports
- [ ] Quarterly: Review verification procedures

---

## TROUBLESHOOTING

### Common Issues

**Issue: Script not executable**
```bash
Solution: chmod +x scripts/verify_cross_references.sh
```

**Issue: Path resolution errors**
```bash
Solution: Run from project root directory
```

**Issue: False positives (example links)**
```bash
Solution: Scripts should skip example/placeholder links
Verify script configuration
```

**Issue: Relative path issues**
```bash
Solution: Ensure scripts handle relative paths correctly
Check script path resolution logic
```

### Verification Best Practices

1. **Always run from project root**
2. **Review results before fixing**
3. **Fix all issues before committing**
4. **Document verification results**
5. **Update scripts when structure changes**

---

## RELATED DOCUMENTS

- [Maintenance Schedule](Maintenance_Schedule.md) - Review schedule includes link verification
- [File Placement Guidelines](File_Placement_Guidelines.md) - File organization affects links
- [Change Management Process](Change_Management_Process.md) - Change procedures
- [Update Trigger Procedures](Update_Trigger_Procedures.md) - Update triggers
- [scripts/verify_cross_references.sh](../../scripts/verify_cross_references.sh) - Verification script
- [scripts/verify_links.py](../../scripts/verify_links.py) - Python verification script

---

**END OF LINK VERIFICATION AUTOMATION**