150 lines
3.7 KiB
Markdown
150 lines
3.7 KiB
Markdown
|
# Validation Guide
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
This guide provides procedures for validating the DeFi Oracle Meta Mainnet (ChainID 138) deployment.
|
||
|
|
|
||
|
|
## Validation Scripts
|
||
|
|
|
||
|
|
All validation scripts are located in `scripts/validation/`:
|
||
|
|
|
||
|
|
- `validate-genesis.sh` - Validate genesis file
|
||
|
|
- `validate-deployment.sh` - Validate deployments
|
||
|
|
- `validate-network-policies.sh` - Validate Network Policies
|
||
|
|
- `validate-rbac.sh` - Validate RBAC configuration
|
||
|
|
- `validate-hpa.sh` - Validate HPA configuration
|
||
|
|
- `validate-monitoring.sh` - Validate monitoring setup
|
||
|
|
- `security-scan.sh` - Run security scans
|
||
|
|
- `load-test.sh` - Run load tests
|
||
|
|
- `disaster-recovery-test.sh` - Test disaster recovery
|
||
|
|
- `run-all-validations.sh` - Run all validations
|
||
|
|
|
||
|
|
## Running Validations
|
||
|
|
|
||
|
|
### Individual Validations
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Validate genesis file
|
||
|
|
./scripts/validation/validate-genesis.sh
|
||
|
|
|
||
|
|
# Validate deployments
|
||
|
|
./scripts/validation/validate-deployment.sh
|
||
|
|
|
||
|
|
# Validate Network Policies
|
||
|
|
./scripts/validation/validate-network-policies.sh
|
||
|
|
|
||
|
|
# Validate RBAC
|
||
|
|
./scripts/validation/validate-rbac.sh
|
||
|
|
|
||
|
|
# Validate HPA
|
||
|
|
./scripts/validation/validate-hpa.sh
|
||
|
|
|
||
|
|
# Validate monitoring
|
||
|
|
./scripts/validation/validate-monitoring.sh
|
||
|
|
|
||
|
|
# Run security scans
|
||
|
|
./scripts/validation/security-scan.sh
|
||
|
|
|
||
|
|
# Run load tests
|
||
|
|
./scripts/validation/load-test.sh
|
||
|
|
|
||
|
|
# Test disaster recovery
|
||
|
|
./scripts/validation/disaster-recovery-test.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
### Run All Validations
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Run all validations
|
||
|
|
./scripts/validation/run-all-validations.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
## Validation Requirements
|
||
|
|
|
||
|
|
### Prerequisites
|
||
|
|
|
||
|
|
- `kubectl` - Kubernetes CLI
|
||
|
|
- `jq` - JSON processor
|
||
|
|
- `curl` - HTTP client
|
||
|
|
- `bash` - Shell
|
||
|
|
|
||
|
|
### Optional Tools
|
||
|
|
|
||
|
|
- `trivy` - Container image scanner
|
||
|
|
- `slither` - Smart contract scanner
|
||
|
|
- `k6` - Load testing tool
|
||
|
|
- `checkov` - Terraform security scanner
|
||
|
|
- `kube-score` - Kubernetes manifest scanner
|
||
|
|
- `safety` - Python dependency scanner
|
||
|
|
- `promtool` - Prometheus rule validator
|
||
|
|
- `besu` - Besu CLI for genesis validation
|
||
|
|
|
||
|
|
## Validation Checklist
|
||
|
|
|
||
|
|
### Pre-Deployment
|
||
|
|
|
||
|
|
- [ ] Genesis file validated
|
||
|
|
- [ ] Terraform configuration validated
|
||
|
|
- [ ] Kubernetes manifests validated
|
||
|
|
- [ ] Security scans completed
|
||
|
|
- [ ] Smart contracts tested
|
||
|
|
|
||
|
|
### Post-Deployment
|
||
|
|
|
||
|
|
- [ ] All pods running
|
||
|
|
- [ ] Health checks working
|
||
|
|
- [ ] Network Policies applied
|
||
|
|
- [ ] RBAC configured
|
||
|
|
- [ ] HPA working
|
||
|
|
- [ ] Monitoring collecting metrics
|
||
|
|
- [ ] Alerts configured
|
||
|
|
- [ ] Load tests passed
|
||
|
|
- [ ] Disaster recovery tested
|
||
|
|
|
||
|
|
## Validation Results
|
||
|
|
|
||
|
|
Validation results are saved in `/tmp/` directory:
|
||
|
|
|
||
|
|
- `/tmp/trivy-scan-*.log` - Container scan results
|
||
|
|
- `/tmp/slither-scan-*.log` - Smart contract scan results
|
||
|
|
- `/tmp/foundry-tests.log` - Foundry test results
|
||
|
|
- `/tmp/k6-load-test.log` - Load test results
|
||
|
|
- `/tmp/checkov-scan.log` - Terraform scan results
|
||
|
|
- `/tmp/kube-score-scan.log` - Kubernetes scan results
|
||
|
|
- `/tmp/disaster-recovery-test-results.txt` - Disaster recovery test results
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### Validation Failures
|
||
|
|
|
||
|
|
1. Check logs in `/tmp/` directory
|
||
|
|
2. Verify prerequisites are installed
|
||
|
|
3. Check cluster connectivity
|
||
|
|
4. Verify namespace exists
|
||
|
|
5. Check pod status
|
||
|
|
|
||
|
|
### Common Issues
|
||
|
|
|
||
|
|
- **Pods not ready**: Wait for pods to be ready, check resource limits
|
||
|
|
- **Network Policies blocking**: Verify Network Policies are correctly configured
|
||
|
|
- **RBAC permissions**: Verify Service Accounts have correct permissions
|
||
|
|
- **Metrics not collecting**: Verify metrics endpoints are accessible
|
||
|
|
- **Load tests failing**: Check RPC endpoint availability, verify rate limiting
|
||
|
|
|
||
|
|
## Next Steps
|
||
|
|
|
||
|
|
After validation:
|
||
|
|
|
||
|
|
1. Review validation results
|
||
|
|
2. Fix any issues found
|
||
|
|
3. Re-run validations
|
||
|
|
4. Document results
|
||
|
|
5. Proceed with production deployment
|
||
|
|
|
||
|
|
## References
|
||
|
|
|
||
|
|
- [Deployment Guide](DEPLOYMENT.md)
|
||
|
|
- [Troubleshooting Guide](TROUBLESHOOTING.md)
|
||
|
|
- [Runbooks](../runbooks/)
|
||
|
|
|