explorer-monorepo/docs/specs/multichain/multichain-indexing.md

Multi-Chain Indexing Strategy

Overview

This document specifies the strategy for indexing multiple blockchain networks with shared schema and per-chain workers.

Architecture

flowchart TB
    subgraph Chains[Blockchain Networks]
        C1[Chain 138]
        C2[Chain 1]
        C3[Chain 137]
    end
    
    subgraph Workers[Indexer Workers]
        W1[Worker: Chain 138]
        W2[Worker: Chain 1]
        W3[Worker: Chain 137]
    end
    
    subgraph DB[(Shared Database)]
        PG[(PostgreSQL<br/>chain_id partitioned)]
        ES[(Elasticsearch<br/>per-chain indices)]
    end
    
    C1 --> W1
    C2 --> W2
    C3 --> W3
    
    W1 --> PG
    W2 --> PG
    W3 --> PG
    
    W1 --> ES
    W2 --> ES
    W3 --> ES

Per-Chain Indexer Workers

Worker Architecture

Design: One indexer worker process per chain

Isolation:

• Separate processes/containers per chain
• Independent scaling per chain
• Chain-specific configuration

Shared Components:

• Database connection pool
• Message queue consumers
• Common indexing logic

Worker Configuration

workers:
  - chain_id: 138
    name: "indexer-chain-138"
    rpc_url: "http://192.168.11.221:8545"  # Internal IP for direct connection
    adapter: "evm"
    enabled: true
    
  - chain_id: 1
    name: "indexer-chain-1"
    rpc_url: "https://eth-mainnet.alchemyapi.io/..."
    adapter: "evm"
    enabled: true

Worker Scaling

Horizontal Scaling:

• Multiple workers per chain for high-throughput chains
• Partition blocks across workers (by block number range)
• Coordinate via database locks or message queue

Scaling Strategy:

• Scale workers based on chain activity
• Monitor worker lag per chain
• Auto-scale based on metrics

Shared Schema with Chain ID Partitioning

Database Partitioning

Strategy: Partition all tables by chain_id

Benefits:

• Efficient queries (partition pruning)
• Independent maintenance per chain
• Easy data isolation

Implementation: See ../database/postgres-schema.md

Index Strategy

Global Indexes: Across all chains for unified queries Chain-Specific Indexes: Within partitions for chain-specific queries

Cross-Chain Data Consistency

Consistency Model

Per-Chain Consistency: Strong consistency within each chain Cross-Chain Consistency: Eventual consistency (independent chains)

Transaction Ordering

Within Chain: Maintain strict block/transaction ordering Across Chains: No ordering guarantees (independent chains)

Worker Coordination

Block Processing Coordination

Strategy: Use database locks or message queue for coordination

Approach 1: Database Locks

• Acquire lock before processing block
• Release lock after completion
• Prevent duplicate processing

Approach 2: Message Queue

• Single consumer per chain partition
• Queue ensures ordering
• Automatic retry on failure

Checkpointing

Per-Chain Checkpoints:

• Track last processed block per chain
• Stored in database
• Enable recovery and resume

Performance Optimization

Chain-Specific Optimizations

High-Activity Chains:

• More workers
• Larger batch sizes
• Optimized processing

Low-Activity Chains:

• Fewer workers
• Longer polling intervals
• Resource-efficient processing

Resource Allocation

CPU/Memory: Allocate based on chain activity Database Connections: Pool per chain Network Bandwidth: Prioritize high-activity chains

Monitoring

Per-Chain Metrics

• Block lag (current block - last indexed)
• Processing rate (blocks/minute)
• Error rate
• Worker health

Cross-Chain Metrics

• Total chains indexed
• Total blocks indexed across all chains
• System-wide throughput

References

• Chain Adapter Interface: See chain-adapter-interface.md
• Database Schema: See ../database/postgres-schema.md
• Indexer Architecture: See ../indexing/indexer-architecture.md