# Virtual Banker Architecture

## Overview

The Virtual Banker is a multi-layered system that provides a digital human banking experience with full video realism, real-time voice interaction, and embeddable widget capabilities.

## System Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    Client Layer                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  Embeddable Widget (React/TypeScript)                 │  │
│  │  - Chat UI                                             │  │
│  │  - Voice Controls                                      │  │
│  │  - Avatar View                                         │  │
│  │  - WebRTC Client                                       │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Edge Layer                                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ CDN          │  │ API Gateway  │  │ WebRTC       │     │
│  │ (Widget)     │  │ (Auth/Rate)  │  │ Gateway      │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Core Services                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ Session      │  │ Orchestrator │  │ LLM Gateway  │     │
│  │ Service      │  │              │  │              │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ RAG Service  │  │ Tool/Action  │  │ Safety/      │     │
│  │              │  │ Service      │  │ Compliance   │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Media Services                            │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ ASR Service  │  │ TTS Service   │  │ Avatar       │     │
│  │ (Streaming)  │  │ (Streaming)   │  │ Renderer     │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                    Data Layer                                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐     │
│  │ PostgreSQL   │  │ Redis        │  │ Vector DB    │     │
│  │ (State)      │  │ (Cache)      │  │ (pgvector)   │     │
│  └──────────────┘  └──────────────┘  └──────────────┘     │
└─────────────────────────────────────────────────────────────┘
```

## Data Flow

### Voice Turn Flow

1. **User speaks** → Widget captures audio via microphone
2. **Audio stream** → WebRTC gateway → ASR service
3. **ASR** → Transcribes to text (partial + final)
4. **Orchestrator** → Sends transcript to LLM with context
5. **LLM** → Generates response + tool calls + emotion tags
6. **TTS** → Converts text to audio stream
7. **Avatar** → Generates visemes, expressions, gestures
8. **Widget** → Plays audio, displays captions, animates avatar

### Text Turn Flow

1. **User types** → Widget sends text message
2. **Orchestrator** → Processes message (same as step 4+ above)

## Components

### Backend Services

#### Session Service
- Creates and manages sessions
- Issues ephemeral tokens
- Loads tenant configurations
- Tracks session state

#### Conversation Orchestrator
- Maintains conversation state machine
- Routes messages to appropriate services
- Handles barge-in (interruptions)
- Synchronizes audio/video

#### LLM Gateway
- Multi-tenant prompt templates
- Function/tool calling
- Output schema enforcement
- Model routing

#### RAG Service
- Document ingestion and embedding
- Vector similarity search
- Reranking
- Citation formatting

#### Tool/Action Service
- Tool registry and execution
- Banking service integrations
- Human-in-the-loop confirmations
- Audit logging

### Frontend Widget

#### Components
- **ChatPanel**: Main chat interface
- **VoiceControls**: Push-to-talk, hands-free, volume
- **AvatarView**: Video stream display
- **Captions**: Real-time captions overlay
- **Settings**: User preferences

#### Hooks
- **useSession**: Session management
- **useConversation**: Message handling
- **useWebRTC**: WebRTC connection

### Avatar System

#### Unreal Engine
- Digital human character
- Blendshapes for visemes/expressions
- Animation blueprints
- PixelStreaming for video output

#### Render Service
- Controls Unreal instances
- Manages GPU resources
- Streams video via WebRTC

## Security

- JWT/SSO authentication
- Ephemeral session tokens
- PII redaction
- Content filtering
- Rate limiting
- Audit trails

## Accessibility

- WCAG 2.1 AA compliance
- Keyboard navigation
- Screen reader support
- Captions (always available)
- Reduced motion support
- ARIA labels

## Scalability

- Stateless services (behind load balancer)
- Redis for session caching
- PostgreSQL for persistent state
- GPU cluster for avatar rendering
- CDN for widget assets