explorer-monorepo/docs/BLOCKSCOUT_CRASH_FIX.md

# Fix Blockscout Container Crash

## Problem

Blockscout container starts but immediately stops (crashes). This is indicated by:
- Container shows as "Exited" after `docker start`
- Exit code is non-zero
- Container logs show errors or the process terminates

## Diagnosis

### Quick Diagnosis Commands

```bash
# From VMID 5000

# 1. Check container status and exit code
BLOCKSCOUT_CONTAINER=$(docker ps -a | grep blockscout | grep -v postgres | awk '{print $1}' | head -1)
docker inspect --format='Exit Code: {{.State.ExitCode}}' $BLOCKSCOUT_CONTAINER

# 2. Check recent logs
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | tail -50

# 3. Check for errors
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | grep -i "error\|fatal\|exception" | tail -20

# 4. Check startup command
docker inspect --format='{{.Config.Cmd}}' $BLOCKSCOUT_CONTAINER
docker inspect --format='{{.Config.Entrypoint}}' $BLOCKSCOUT_CONTAINER
```

### Automated Diagnosis

```bash
# From Proxmox host
cd /home/intlc/projects/proxmox/explorer-monorepo
./scripts/diagnose-blockscout-crash.sh
```

## Common Causes and Fixes

### 1. Missing Startup Command

**Symptom**: Container starts but exits immediately with code 0 or 1

**Fix**: Add startup command to docker-compose.yml

```bash
cd /opt/blockscout

# Check current configuration
grep -A 10 "blockscout:" docker-compose.yml

# Add startup command if missing
if ! grep -q "command:.*blockscout start" docker-compose.yml; then
    # Backup
    cp docker-compose.yml docker-compose.yml.backup
    
    # Add command after blockscout: line
    sed -i '/blockscout:/a\    command: bin/blockscout start' docker-compose.yml
    
    # Or edit manually
    # nano docker-compose.yml
    # Add: command: bin/blockscout start
fi

# Restart with new configuration
docker compose down blockscout
docker compose up -d blockscout
```

### 2. Database Connection Failed

**Symptom**: Logs show database connection errors

**Fix**: Verify database is accessible

```bash
# Check postgres container
docker ps | grep postgres

# Test database connection
docker exec blockscout-postgres psql -U blockscout -d blockscout -c "SELECT 1;"

# Check DATABASE_URL in Blockscout container
docker inspect blockscout | grep -A 5 DATABASE_URL
```

### 3. Port Conflict

**Symptom**: Port 4000 already in use

**Fix**: Check and resolve port conflict

```bash
# Check what's using port 4000
netstat -tlnp | grep 4000
# Or
lsof -i :4000

# Stop conflicting service or change Blockscout port in docker-compose.yml
```

### 4. Missing Environment Variables

**Symptom**: Logs show missing configuration errors

**Fix**: Check and set required environment variables

```bash
# Check docker-compose.yml environment section
grep -A 20 "blockscout:" /opt/blockscout/docker-compose.yml | grep -E "environment:|DATABASE|SECRET"

# Check .env file
cat /opt/blockscout/.env 2>/dev/null || echo ".env file not found"

# Required variables typically include:
# - DATABASE_URL
# - SECRET_KEY_BASE
# - ETHEREUM_JSONRPC_HTTP_URL
# - ETHEREUM_JSONRPC_WS_URL
# - CHAIN_ID
```

### 5. Resource Limits

**Symptom**: Container runs out of memory or CPU

**Fix**: Check and increase resource limits

```bash
# Check current limits
docker inspect blockscout | grep -A 5 "Memory\|Cpu"

# Check system resources
free -h
df -h

# Increase limits in docker-compose.yml if needed
```

## Complete Fix Procedure

### Step 1: Diagnose the Issue

```bash
# Check logs
BLOCKSCOUT_CONTAINER=$(docker ps -a | grep blockscout | grep -v postgres | awk '{print $1}' | head -1)
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | tail -50
```

### Step 2: Fix Based on Diagnosis

**If missing startup command:**
```bash
cd /opt/blockscout
sed -i '/blockscout:/a\    command: bin/blockscout start' docker-compose.yml
docker compose up -d blockscout
```

**If database connection issue:**
```bash
# Verify database
docker exec blockscout-postgres psql -U blockscout -d blockscout -c "SELECT 1;"

# Check DATABASE_URL
grep DATABASE_URL /opt/blockscout/docker-compose.yml
```

**If port conflict:**
```bash
# Find what's using port 4000
lsof -i :4000
# Stop it or change Blockscout port
```

### Step 3: Restart and Verify

```bash
# Restart with fixes
cd /opt/blockscout
docker compose restart blockscout
# Or
docker compose down blockscout && docker compose up -d blockscout

# Wait and check
sleep 30
docker ps | grep blockscout
docker logs blockscout 2>&1 | tail -30
```

## Manual Container Start (If Docker Compose Fails)

If docker-compose doesn't work, start manually:

```bash
# Get environment from docker-compose
cd /opt/blockscout
docker compose config | grep -A 30 "blockscout:" > /tmp/blockscout-config.txt

# Start manually with correct command
docker run -d \
  --name blockscout \
  --env-file .env \
  -p 4000:4000 \
  --link blockscout-postgres:postgres \
  blockscout/blockscout:latest \
  bin/blockscout start
```

## Verification

After applying fixes:

```bash
# 1. Check container is running
docker ps | grep blockscout

# 2. Check logs for errors
docker logs blockscout 2>&1 | tail -30

# 3. Test API
curl -s http://localhost:4000/api/v2/stats | head -20

# 4. Check process
docker exec blockscout pgrep -f "beam.smp" && echo "✅ Blockscout process running"
```

## Next Steps

Once container stays running:

1. ✅ Build static assets: `docker exec -it blockscout mix phx.digest`
2. ✅ Verify assets: `docker exec -it blockscout test -f priv/static/cache_manifest.json`
3. ✅ Test API: `curl http://localhost:4000/api/v2/stats`