explorer-monorepo/docs/BLOCKSCOUT_CRASH_FIX.md

Fix Blockscout Container Crash

Problem

Blockscout container starts but immediately stops (crashes). This is indicated by:

• Container shows as "Exited" after docker start
• Exit code is non-zero
• Container logs show errors or the process terminates

Diagnosis

Quick Diagnosis Commands

# From VMID 5000

# 1. Check container status and exit code
BLOCKSCOUT_CONTAINER=$(docker ps -a | grep blockscout | grep -v postgres | awk '{print $1}' | head -1)
docker inspect --format='Exit Code: {{.State.ExitCode}}' $BLOCKSCOUT_CONTAINER

# 2. Check recent logs
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | tail -50

# 3. Check for errors
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | grep -i "error\|fatal\|exception" | tail -20

# 4. Check startup command
docker inspect --format='{{.Config.Cmd}}' $BLOCKSCOUT_CONTAINER
docker inspect --format='{{.Config.Entrypoint}}' $BLOCKSCOUT_CONTAINER

Automated Diagnosis

# From Proxmox host
cd /home/intlc/projects/proxmox/explorer-monorepo
./scripts/diagnose-blockscout-crash.sh

Common Causes and Fixes

1. Missing Startup Command

Symptom: Container starts but exits immediately with code 0 or 1

Fix: Add startup command to docker-compose.yml

cd /opt/blockscout

# Check current configuration
grep -A 10 "blockscout:" docker-compose.yml

# Add startup command if missing
if ! grep -q "command:.*blockscout start" docker-compose.yml; then
    # Backup
    cp docker-compose.yml docker-compose.yml.backup
    
    # Add command after blockscout: line
    sed -i '/blockscout:/a\    command: bin/blockscout start' docker-compose.yml
    
    # Or edit manually
    # nano docker-compose.yml
    # Add: command: bin/blockscout start
fi

# Restart with new configuration
docker compose down blockscout
docker compose up -d blockscout

2. Database Connection Failed

Symptom: Logs show database connection errors

Fix: Verify database is accessible

# Check postgres container
docker ps | grep postgres

# Test database connection
docker exec blockscout-postgres psql -U blockscout -d blockscout -c "SELECT 1;"

# Check DATABASE_URL in Blockscout container
docker inspect blockscout | grep -A 5 DATABASE_URL

3. Port Conflict

Symptom: Port 4000 already in use

Fix: Check and resolve port conflict

# Check what's using port 4000
netstat -tlnp | grep 4000
# Or
lsof -i :4000

# Stop conflicting service or change Blockscout port in docker-compose.yml

4. Missing Environment Variables

Symptom: Logs show missing configuration errors

Fix: Check and set required environment variables

# Check docker-compose.yml environment section
grep -A 20 "blockscout:" /opt/blockscout/docker-compose.yml | grep -E "environment:|DATABASE|SECRET"

# Check .env file
cat /opt/blockscout/.env 2>/dev/null || echo ".env file not found"

# Required variables typically include:
# - DATABASE_URL
# - SECRET_KEY_BASE
# - ETHEREUM_JSONRPC_HTTP_URL
# - ETHEREUM_JSONRPC_WS_URL
# - CHAIN_ID

5. Resource Limits

Symptom: Container runs out of memory or CPU

Fix: Check and increase resource limits

# Check current limits
docker inspect blockscout | grep -A 5 "Memory\|Cpu"

# Check system resources
free -h
df -h

# Increase limits in docker-compose.yml if needed

Complete Fix Procedure

Step 1: Diagnose the Issue

# Check logs
BLOCKSCOUT_CONTAINER=$(docker ps -a | grep blockscout | grep -v postgres | awk '{print $1}' | head -1)
docker logs $BLOCKSCOUT_CONTAINER 2>&1 | tail -50

Step 2: Fix Based on Diagnosis

If missing startup command:

cd /opt/blockscout
sed -i '/blockscout:/a\    command: bin/blockscout start' docker-compose.yml
docker compose up -d blockscout

If database connection issue:

# Verify database
docker exec blockscout-postgres psql -U blockscout -d blockscout -c "SELECT 1;"

# Check DATABASE_URL
grep DATABASE_URL /opt/blockscout/docker-compose.yml

If port conflict:

# Find what's using port 4000
lsof -i :4000
# Stop it or change Blockscout port

Step 3: Restart and Verify

# Restart with fixes
cd /opt/blockscout
docker compose restart blockscout
# Or
docker compose down blockscout && docker compose up -d blockscout

# Wait and check
sleep 30
docker ps | grep blockscout
docker logs blockscout 2>&1 | tail -30

Manual Container Start (If Docker Compose Fails)

If docker-compose doesn't work, start manually:

# Get environment from docker-compose
cd /opt/blockscout
docker compose config | grep -A 30 "blockscout:" > /tmp/blockscout-config.txt

# Start manually with correct command
docker run -d \
  --name blockscout \
  --env-file .env \
  -p 4000:4000 \
  --link blockscout-postgres:postgres \
  blockscout/blockscout:latest \
  bin/blockscout start

Verification

After applying fixes:

# 1. Check container is running
docker ps | grep blockscout

# 2. Check logs for errors
docker logs blockscout 2>&1 | tail -30

# 3. Test API
curl -s http://localhost:4000/api/v2/stats | head -20

# 4. Check process
docker exec blockscout pgrep -f "beam.smp" && echo "✅ Blockscout process running"

Next Steps

Once container stays running:

1. ✅ Build static assets: docker exec -it blockscout mix phx.digest
2. ✅ Verify assets: docker exec -it blockscout test -f priv/static/cache_manifest.json
3. ✅ Test API: curl http://localhost:4000/api/v2/stats