smom-dbis-138/runbooks/oracle-operations.md

# Oracle Operations Runbook

## Overview

This runbook provides procedures for operating and maintaining the oracle system.

## Oracle Update Procedures

### Manual Oracle Update

1. **Check current oracle state:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "latestRoundData()" --rpc-url $RPC_URL
   ```

2. **Update oracle:**
   ```bash
   cast send $AGGREGATOR_ADDRESS "updateAnswer(uint256)" $PRICE \
     --rpc-url $RPC_URL \
     --private-key $PRIVATE_KEY
   ```

3. **Verify update:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "latestAnswer()" --rpc-url $RPC_URL
   ```

### Automated Oracle Updates

Oracle updates are automated via the oracle publisher service:

```bash
# Check oracle publisher status
kubectl get pods -n besu-network -l app=oracle-publisher

# View logs
kubectl logs -n besu-network -l app=oracle-publisher --tail=100

# Restart if needed
kubectl rollout restart deployment/oracle-publisher -n besu-network
```

## Oracle Troubleshooting

### Oracle Not Updating

1. **Check transmitter status:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "isTransmitter(address)" $TRANSMITTER_ADDRESS --rpc-url $RPC_URL
   ```

2. **Check oracle publisher:**
   ```bash
   kubectl logs -n besu-network -l app=oracle-publisher
   ```

3. **Check heartbeat:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "heartbeat()" --rpc-url $RPC_URL
   ```

### Oracle Price Deviation

1. **Check deviation threshold:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "deviationThreshold()" --rpc-url $RPC_URL
   ```

2. **Review price sources:**
   - Check data source health
   - Verify aggregation logic
   - Review recent updates

## Oracle Recovery Procedures

### Recover from Stale Oracle

1. **Identify stale oracle:**
   ```bash
   cast call $AGGREGATOR_ADDRESS "latestRoundData()" --rpc-url $RPC_URL
   ```

2. **Force update:**
   ```bash
   cast send $AGGREGATOR_ADDRESS "updateAnswer(uint256)" $PRICE \
     --rpc-url $RPC_URL \
     --private-key $PRIVATE_KEY
   ```

### Recover from Failed Transmitter

1. **Remove failed transmitter:**
   ```bash
   cast send $AGGREGATOR_ADDRESS "removeTransmitter(address)" $TRANSMITTER \
     --rpc-url $RPC_URL \
     --private-key $ADMIN_KEY
   ```

2. **Add new transmitter:**
   ```bash
   cast send $AGGREGATOR_ADDRESS "addTransmitter(address)" $NEW_TRANSMITTER \
     --rpc-url $RPC_URL \
     --private-key $ADMIN_KEY
   ```

## Monitoring

### Key Metrics

- Oracle update frequency
- Price deviation
- Transmitter availability
- Update success rate

### Alerts

- Oracle update stale (> 5 minutes)
- Price deviation > 5%
- Transmitter unavailable
- Update failures

## Best Practices

1. **Regular Monitoring**: Check oracle status daily
2. **Backup Transmitters**: Maintain multiple transmitters
3. **Price Source Diversity**: Use multiple data sources
4. **Automated Updates**: Rely on oracle publisher service
5. **Incident Response**: Follow incident response procedures