smom-dbis-138/docs/monitoring/MONITORING_SETUP.md

# Monitoring and Alerting Setup

**Date**: 2025-12-11
**Status**: Configuration Guide

---

## 📋 Overview

This document provides guidance for setting up monitoring and alerting for the deployed MainnetTether and TransactionMirror contracts.

---

## 🔍 Monitoring Targets

### MainnetTether Contract
- **Address**: `0x15DF1D5BFDD8Aa4b380445D4e3E9B38d34283619`
- **Explorer**: https://etherscan.io/address/0x15DF1D5BFDD8Aa4b380445D4e3E9B38d34283619

### TransactionMirror Contract
- **Address**: `0x4CF42c4F1dBa748601b8938be3E7ABD732E87cE9`
- **Explorer**: https://etherscan.io/address/0x4CF42c4F1dBa748601b8938be3E7ABD732E87cE9

---

## 📊 Key Metrics to Monitor

### Contract Health
- **Paused Status**: Monitor if contracts are paused
- **Admin Changes**: Alert on admin address changes
- **Contract Balance**: Monitor ETH balance (if applicable)

### MainnetTether Specific
- **State Proof Anchoring**: Monitor `StateProofAnchored` events
- **Anchored Block Count**: Track number of anchored blocks
- **Anchoring Frequency**: Monitor time between anchors

### TransactionMirror Specific
- **Transaction Mirroring**: Monitor `TransactionMirrored` events
- **Batch Mirroring**: Monitor `BatchTransactionsMirrored` events
- **Mirrored Transaction Count**: Track total mirrored transactions
- **Mirroring Frequency**: Monitor time between mirrors

### Operational
- **Off-Chain Service Status**: Monitor state anchor and transaction mirror services
- **RPC Connection Health**: Monitor Chain-138 and Ethereum Mainnet RPC connections
- **Gas Prices**: Monitor gas prices for cost optimization

---

## 🛠️ Monitoring Tools

### 1. Etherscan Alerts

**Setup**:
1. Go to https://etherscan.io/
2. Create account and verify email
3. Navigate to contract addresses
4. Set up alerts for:
   - Contract events
   - Admin changes
   - Large transactions

**Recommended Alerts**:
- `StateProofAnchored` events
- `TransactionMirrored` events
- `AdminChanged` events
- `Paused` / `Unpaused` events

### 2. OpenZeppelin Defender

**Setup**:
1. Go to https://defender.openzeppelin.com/
2. Create account
3. Add contracts:
   - MainnetTether
   - TransactionMirror
4. Configure monitoring:
   - Event monitoring
   - Function call monitoring
   - Admin action alerts

**Recommended Monitors**:
- Pause/unpause actions
- Admin changes
- State proof anchoring frequency
- Transaction mirroring frequency

### 3. Custom Monitoring Script

A custom monitoring script can be created using ethers.js:

```javascript
const { ethers } = require('ethers');

const MAINNET_TETHER = '0x15DF1D5BFDD8Aa4b380445D4e3E9B38d34283619';
const TRANSACTION_MIRROR = '0x4CF42c4F1dBa748601b8938be3E7ABD732E87cE9';
const RPC_URL = process.env.ETHEREUM_MAINNET_RPC;

const provider = new ethers.JsonRpcProvider(RPC_URL);

// Monitor events
const tetherContract = new ethers.Contract(MAINNET_TETHER, [...], provider);
const mirrorContract = new ethers.Contract(TRANSACTION_MIRROR, [...], provider);

// Listen for events
tetherContract.on('StateProofAnchored', (blockNumber, blockHash, stateRoot, timestamp, validatorCount) => {
    console.log(`State proof anchored: Block ${blockNumber} at ${new Date(timestamp * 1000)}`);
});

mirrorContract.on('TransactionMirrored', (txHash, from, to, value, blockNumber, timestamp) => {
    console.log(`Transaction mirrored: ${txHash} from ${from} to ${to}`);
});
```

### 4. Prometheus + Grafana

**Setup**:
1. Deploy Prometheus instance
2. Configure exporters for:
   - Ethereum node metrics
   - Contract event metrics
   - Off-chain service metrics
3. Set up Grafana dashboards

**Metrics to Export**:
- Contract event counts
- Gas usage
- Transaction success rates
- Off-chain service uptime

---

## 🚨 Alerting Rules

### Critical Alerts

1. **Contract Paused**
   - Severity: High
   - Action: Investigate immediately
   - Condition: `paused() == true`

2. **Admin Changed**
   - Severity: Critical
   - Action: Verify admin change is authorized
   - Condition: `AdminChanged` event emitted

3. **Off-Chain Service Down**
   - Severity: High
   - Action: Restart service
   - Condition: Service health check fails

4. **No State Proofs Anchored (24h)**
   - Severity: Medium
   - Action: Check Chain-138 connection
   - Condition: No `StateProofAnchored` events in 24 hours

5. **No Transactions Mirrored (24h)**
   - Severity: Medium
   - Action: Check Chain-138 connection
   - Condition: No `TransactionMirrored` events in 24 hours

### Warning Alerts

1. **High Gas Prices**
   - Severity: Low
   - Action: Consider delaying non-critical operations
   - Condition: Gas price > threshold

2. **Low Contract Balance**
   - Severity: Medium
   - Action: Fund contract if needed
   - Condition: Balance < threshold

3. **Slow Anchoring**
   - Severity: Low
   - Action: Check Chain-138 block production
   - Condition: Time since last anchor > expected interval

---

## 📝 Logging

### Recommended Log Levels

- **ERROR**: Contract errors, RPC failures, service crashes
- **WARN**: High gas prices, slow operations, service degradation
- **INFO**: Successful operations, state changes, service start/stop
- **DEBUG**: Detailed operation logs, transaction details

### Log Storage

- **Local**: Store logs in `/var/log/` or project `logs/` directory
- **Centralized**: Use services like:
  - ELK Stack (Elasticsearch, Logstash, Kibana)
  - Loki + Grafana
  - CloudWatch (AWS)
  - Azure Monitor

---

## 🔧 Health Check Endpoints

### Contract Health Check

```bash
#!/bin/bash
# Check contract health

MAINNET_TETHER="0x15DF1D5BFDD8Aa4b380445D4e3E9B38d34283619"
TRANSACTION_MIRROR="0x4CF42c4F1dBa748601b8938be3E7ABD732E87cE9"
RPC_URL="$ETHEREUM_MAINNET_RPC"

# Check if contracts are paused
TETHER_PAUSED=$(cast call $MAINNET_TETHER "paused()" --rpc-url "$RPC_URL")
MIRROR_PAUSED=$(cast call $TRANSACTION_MIRROR "paused()" --rpc-url "$RPC_URL")

if [ "$TETHER_PAUSED" != "0x0000000000000000000000000000000000000000000000000000000000000000" ]; then
    echo "⚠️  MainnetTether is paused"
    exit 1
fi

if [ "$MIRROR_PAUSED" != "0x0000000000000000000000000000000000000000000000000000000000000000" ]; then
    echo "⚠️  TransactionMirror is paused"
    exit 1
fi

echo "✅ Contracts are healthy"
```

### Off-Chain Service Health Check

```bash
#!/bin/bash
# Check off-chain service health

# Check state anchor service
if ! pgrep -f "state-anchor-service.js" > /dev/null; then
    echo "❌ State anchor service is not running"
    exit 1
fi

# Check transaction mirror service
if ! pgrep -f "transaction-mirror-service.js" > /dev/null; then
    echo "❌ Transaction mirror service is not running"
    exit 1
fi

echo "✅ Off-chain services are running"
```

---

## 📈 Dashboards

### Recommended Dashboard Views

1. **Contract Overview**
   - Contract status (paused/active)
   - Admin addresses
   - Recent events
   - Gas usage

2. **State Anchoring**
   - Anchored blocks over time
   - Anchoring frequency
   - Block numbers anchored
   - Validator counts

3. **Transaction Mirroring**
   - Mirrored transactions over time
   - Mirroring frequency
   - Batch sizes
   - Success rates

4. **Operational**
   - Off-chain service uptime
   - RPC connection status
   - Gas prices
   - Error rates

---

## 🔐 Security Monitoring

### Admin Actions
- Monitor all admin function calls
- Alert on unexpected admin changes
- Log all pause/unpause actions

### Access Control
- Monitor contract access patterns
- Alert on suspicious activity
- Track function call frequencies

### Event Analysis
- Analyze event patterns for anomalies
- Monitor for unexpected event sequences
- Track event volumes

---

## 📚 Additional Resources

- [Etherscan API Documentation](https://docs.etherscan.io/)
- [OpenZeppelin Defender Documentation](https://docs.openzeppelin.com/defender/)
- [Prometheus Documentation](https://prometheus.io/docs/)
- [Grafana Documentation](https://grafana.com/docs/)

---

**Last Updated**: 2025-12-11
**Status**: Configuration Guide