# System Architecture

This document provides a comprehensive overview of the NowYouSeeMe holodeck environment architecture, including system design, data flow, and component interactions.

## 🏗️ High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                           NowYouSeeMe Holodeck                            │
├─────────────────────────────────────────────────────────────────────────────┤
│  📷 Camera Module    │  📡 RF Module      │  🧠 Processing Module        │
│  • OpenCV/GStreamer  │  • Intel 5300      │  • SLAM Algorithms          │
│  • Real-time capture │  • Nexmon CSI      │  • Sensor Fusion            │
│  • Calibration       │  • AoA Estimation  │  • Neural Enhancement       │
└─────────────────────┼─────────────────────┼───────────────────────────────┘
                      │                     │
                      ▼                     ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                        🎯 Core Processing Engine                          │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────────┐   │
│  │   Vision SLAM   │  │    RF SLAM      │  │    Sensor Fusion       │   │
│  │ • ORB-SLAM3     │  │ • AoA Estimation│  │ • EKF Filter           │   │
│  │ • Feature Track  │  │ • CIR Analysis  │  │ • Particle Filter      │   │
│  │ • Pose Graph     │  │ • RF Mapping    │  │ • Multi-sensor Fusion  │   │
│  └─────────────────┘  └─────────────────┘  └─────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                        🎨 Rendering & Output                              │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────────┐   │
│  │   3D Scene      │  │   NeRF Render   │  │   Export Engine        │   │
│  │ • OpenGL        │  │ • Neural Fields │  │ • Unity/Unreal         │   │
│  │ • Real-time     │  │ • Photo-real    │  │ • VR/AR Support        │   │
│  │ • Interactive   │  │ • GPU Accelerated│ │ • Projection Mapping   │   │
│  └─────────────────┘  └─────────────────┘  └─────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
```

## 🔄 Data Flow Architecture

### Primary Data Flow

```mermaid
graph TD
    A[Camera Input] --> B[Image Processing]
    C[WiFi CSI] --> D[RF Processing]
    B --> E[Feature Extraction]
    D --> F[AoA Estimation]
    E --> G[Vision SLAM]
    F --> H[RF SLAM]
    G --> I[Sensor Fusion]
    H --> I
    I --> J[Pose Estimation]
    J --> K[3D Scene Update]
    K --> L[Rendering Engine]
    L --> M[User Interface]
    
    N[Azure Cloud] --> O[GPU Computing]
    O --> P[Neural Enhancement]
    P --> L
```

### Real-time Processing Pipeline

```
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│ Camera Capture  │    │ Wi-Fi CSI Capture│    │ Calibration     │
│ (OpenCV/GStream)│    │ (Intel 5300/Nex) │    │ Store           │
└─────────┬───────┘    └────────┬─────────┘    └─────────────────┘
          │                      │
          ▼                      ▼
┌─────────────────────────────────────────────────────────────┐
│              Sensor Fusion Module                          │
│  - RF point cloud & occupancy grid                        │
│  - Vision pose graph & dense point cloud                  │
└─────────────┬───────────────────────────────┬─────────────┘
              │                               │
              ▼                               ▼
    ┌─────────────────┐            ┌─────────────────────┐
    │ Export Engine   │            │ Rendering Engine    │
    │ (Unity/UE4)     │            │ (VR/Projection Map) │
    └─────────────────┘            └─────────────────────┘
```

## 🧩 Component Architecture

### 1. Data Ingestion Layer

#### Camera Module (`src/ingestion/capture.py`)
```python
class CameraCapture:
    """Real-time camera data acquisition"""
    
    def __init__(self, config: CameraConfig):
        self.config = config
        self.cap = cv2.VideoCapture(config.device_id)
    
    def get_frame(self) -> np.ndarray:
        """Capture and return current frame"""
        ret, frame = self.cap.read()
        return frame if ret else None
    
    def calibrate(self) -> CalibrationResult:
        """Perform camera calibration"""
        # Implementation for intrinsic/extrinsic calibration
```

#### CSI Module (`src/ingestion/csi_acquirer.py`)
```python
class CSIAcquirer:
    """WiFi Channel State Information capture"""
    
    def __init__(self, config: CSIConfig):
        self.config = config
        self.interface = config.interface
    
    def capture_csi(self) -> CSIPacket:
        """Capture CSI data from WiFi interface"""
        # Implementation for CSI packet capture
```

### 2. Processing Layer

#### Vision SLAM (`src/vision_slam/`)
```cpp
class VisionSLAM {
public:
    VisionSLAM(const VisionConfig& config);
    
    // Main processing methods
    PoseResult processFrame(const cv::Mat& frame);
    std::vector<MapPoint> getMapPoints() const;
    void reset();
    
private:
    // ORB-SLAM3 integration
    std::unique_ptr<ORB_SLAM3::System> slam_system;
    // Feature tracking and pose estimation
};
```

#### RF SLAM (`src/rf_slam/`)
```cpp
class RFSLAM {
public:
    RFSLAM(const RFConfig& config);
    
    // RF processing methods
    AoAResult estimateAoA(const CSIPacket& packet);
    RFMap generateRFMap() const;
    void updateRFModel();
    
private:
    // CIR analysis and AoA estimation
    std::unique_ptr<CIRConverter> cir_converter;
    std::unique_ptr<AoAEstimator> aoa_estimator;
};
```

#### Sensor Fusion (`src/fusion/`)
```cpp
class SensorFusion {
public:
    SensorFusion(const FusionConfig& config);
    
    // Multi-sensor fusion
    FusionResult fuseData(const VisionData& vision, const RFData& rf);
    PoseResult getCurrentPose() const;
    void updateFusionModel();
    
private:
    // EKF and particle filter implementations
    std::unique_ptr<EKFFusion> ekf_fusion;
    std::unique_ptr<ParticleFilter> particle_filter;
};
```

### 3. Rendering Layer

#### 3D Scene (`src/ui/holodeck_ui.py`)
```python
class HolodeckUI(QMainWindow):
    """Main user interface for the holodeck environment"""
    
    def __init__(self):
        super().__init__()
        self.setup_ui()
        self.setup_3d_scene()
        self.setup_controls()
    
    def setup_3d_scene(self):
        """Initialize 3D OpenGL scene"""
        self.gl_widget = HolodeckGLWidget()
        self.setCentralWidget(self.gl_widget)
    
    def update_scene(self, pose_data: PoseData):
        """Update 3D scene with new pose data"""
        self.gl_widget.update_pose(pose_data)
```

#### NeRF Rendering (`src/nerf/`)
```python
class NeRFRenderer:
    """Neural Radiance Fields rendering"""
    
    def __init__(self, config: NeRFConfig):
        self.config = config
        self.model = self.load_nerf_model()
    
    def render_scene(self, pose: np.ndarray) -> np.ndarray:
        """Render photo-realistic scene from pose"""
        # GPU-accelerated NeRF rendering
```

### 4. Cloud Integration

#### Azure Integration (`src/cloud/azure_integration.cpp`)
```cpp
class AzureIntegration {
public:
    AzureIntegration(const AzureConfig& config);
    
    // Cloud GPU management
    bool provisionGPUResource(const std::string& vm_name);
    bool deployModel(const std::string& model_name);
    ComputeJob submitJob(const JobRequest& request);
    
private:
    // Azure SDK integration
    std::unique_ptr<Azure::Compute::VirtualMachines> vm_client;
    std::unique_ptr<Azure::AI::ML::Workspace> ml_workspace;
};
```

## 🔧 System Configuration

### Configuration Hierarchy

```
config/
├── camera_config.json      # Camera settings
├── csi_config.json        # WiFi CSI settings
├── slam_config.json       # SLAM parameters
├── fusion_config.json     # Sensor fusion settings
├── nerf_config.json       # NeRF rendering settings
├── azure_config.json      # Azure integration settings
└── ui_config.json         # User interface settings
```

### Configuration Example

```json
{
  "system": {
    "latency_target": 20,
    "accuracy_target": 10,
    "fps_target": 30
  },
  "camera": {
    "device_id": 0,
    "width": 1280,
    "height": 720,
    "fps": 30
  },
  "csi": {
    "interface": "wlan0",
    "channel": 6,
    "bandwidth": 20,
    "packet_rate": 100
  },
  "slam": {
    "vision_enabled": true,
    "rf_enabled": true,
    "fusion_enabled": true
  },
  "rendering": {
    "nerf_enabled": true,
    "gpu_acceleration": true,
    "quality": "high"
  }
}
```

## 🚀 Performance Architecture

### Real-time Constraints

| Component | Latency Target | Throughput | Resource Usage |
|-----------|----------------|------------|----------------|
| **Camera Capture** | <5ms | 30-60 FPS | Low CPU |
| **CSI Processing** | <10ms | 100+ pkt/s | Medium CPU |
| **Vision SLAM** | <15ms | 30 FPS | High CPU/GPU |
| **RF SLAM** | <10ms | 100 pkt/s | Medium CPU |
| **Sensor Fusion** | <5ms | 30 Hz | Medium CPU |
| **Rendering** | <10ms | 30-60 FPS | High GPU |
| **Total Pipeline** | <20ms | 30 Hz | Optimized |

### Resource Management

```python
class ResourceManager:
    """Manages system resources and performance"""
    
    def __init__(self):
        self.cpu_monitor = CPUMonitor()
        self.gpu_monitor = GPUMonitor()
        self.memory_monitor = MemoryMonitor()
    
    def optimize_performance(self):
        """Dynamically adjust settings based on resources"""
        cpu_usage = self.cpu_monitor.get_usage()
        gpu_usage = self.gpu_monitor.get_usage()
        
        if cpu_usage > 80:
            self.reduce_processing_quality()
        if gpu_usage > 90:
            self.reduce_rendering_quality()
```

## 🔒 Security Architecture

### Data Protection

```python
class SecurityManager:
    """Handles data security and privacy"""
    
    def __init__(self):
        self.encryption = AESEncryption()
        self.authentication = OAuth2Auth()
    
    def secure_data_transmission(self, data: bytes) -> bytes:
        """Encrypt data for transmission"""
        return self.encryption.encrypt(data)
    
    def authenticate_user(self, credentials: dict) -> bool:
        """Authenticate user access"""
        return self.authentication.verify(credentials)
```

### Privacy Considerations

- **Local Processing**: Sensitive data processed locally
- **Data Encryption**: All transmissions encrypted
- **User Consent**: Clear data usage policies
- **Data Retention**: Configurable retention periods

## 🔄 Scalability Architecture

### Horizontal Scaling

```yaml
# docker-compose.yml
services:
  nowyouseeme:
    image: nowyouseeme/nowyouseeme
    scale: 3  # Multiple instances
    load_balancer: true
  
  redis:
    image: redis:alpine
    # Shared state management
  
  postgres:
    image: postgres:alpine
    # Persistent data storage
```

### Vertical Scaling

```python
class ScalabilityManager:
    """Manages system scaling"""
    
    def auto_scale(self):
        """Automatically scale based on load"""
        load = self.get_system_load()
        
        if load > 80:
            self.scale_up()
        elif load < 30:
            self.scale_down()
```

## 🧪 Testing Architecture

### Test Pyramid

```
┌─────────────────────────────────────┐
│           E2E Tests                │  (10%)
│    Complete system workflows       │
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│         Integration Tests           │  (20%)
│    Component interaction tests     │
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│           Unit Tests               │  (70%)
│    Individual component tests      │
└─────────────────────────────────────┘
```

### Test Categories

```python
# Unit Tests
class TestVisionSLAM:
    def test_feature_extraction(self):
        """Test feature extraction from images"""
    
    def test_pose_estimation(self):
        """Test pose estimation accuracy"""

# Integration Tests
class TestSensorFusion:
    def test_vision_rf_fusion(self):
        """Test fusion of vision and RF data"""
    
    def test_real_time_performance(self):
        """Test real-time performance constraints"""

# End-to-End Tests
class TestHolodeckWorkflow:
    def test_complete_session(self):
        """Test complete holodeck session"""
    
    def test_calibration_workflow(self):
        """Test camera and RF calibration"""
```

## 📊 Monitoring Architecture

### Metrics Collection

```python
class MetricsCollector:
    """Collects system performance metrics"""
    
    def __init__(self):
        self.prometheus_client = PrometheusClient()
        self.grafana_client = GrafanaClient()
    
    def collect_metrics(self):
        """Collect real-time metrics"""
        metrics = {
            'latency': self.measure_latency(),
            'accuracy': self.measure_accuracy(),
            'fps': self.measure_fps(),
            'cpu_usage': self.measure_cpu(),
            'gpu_usage': self.measure_gpu(),
            'memory_usage': self.measure_memory()
        }
        self.prometheus_client.push_metrics(metrics)
```

### Alerting System

```python
class AlertManager:
    """Manages system alerts and notifications"""
    
    def check_alerts(self):
        """Check for alert conditions"""
        if self.latency > 20:
            self.send_alert("High latency detected")
        
        if self.accuracy < 10:
            self.send_alert("Low accuracy detected")
```

## 🔮 Future Architecture

### Planned Enhancements

1. **Edge Computing**: Distributed processing nodes
2. **5G Integration**: Low-latency wireless communication
3. **AI/ML Enhancement**: Advanced neural networks
4. **Quantum Computing**: Quantum-accelerated algorithms
5. **Holographic Display**: True holographic rendering

### Architecture Evolution

```
Current: Single-node processing
    ↓
Future: Distributed edge computing
    ↓
Future+: Quantum-enhanced processing
    ↓
Future++: Holographic reality
```

---

For more detailed information about specific components, see:
- [API Reference](API_REFERENCE.md) - Complete API documentation
- [Data Flow](dataflow.md) - Detailed data flow diagrams
- [Performance Guide](performance.md) - Optimization strategies
- [Security Guide](security.md) - Security considerations