d-bis/NYSM-NYD
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
NYSM-NYD/docs/API_REFERENCE.md

12 KiB
Raw Permalink Blame History

NowYouSeeMe API Reference

Overview

The NowYouSeeMe holodeck environment provides comprehensive APIs for multi-device connectivity, Azure integration, and real-time 6DOF pose estimation. This document describes the complete API surface for all components.

Table of Contents

  1. Device Management API
  2. Azure Integration API
  3. SLAM Processing API
  4. UI/UX API
  5. Configuration API
  6. Network Protocols

Device Management API

DeviceManager Class

The DeviceManager class handles discovery, connection, and management of multiple device types simultaneously.

Constructor

DeviceManager(int discoveryPort = 8080, int maxConnections = 100)

Device Types

  • WIFI_CARD - WiFi network interfaces
  • ACCESS_POINT - WiFi access points
  • CELL_PHONE - Mobile devices
  • HOTSPOT - Mobile hotspots
  • BLUETOOTH - Bluetooth devices
  • ZIGBEE - ZigBee devices
  • LORA - LoRa devices
  • CUSTOM - Custom network interfaces

Device Status

  • DISCONNECTED - Device not connected
  • CONNECTING - Connection in progress
  • CONNECTED - Successfully connected
  • STREAMING - Actively streaming data
  • ERROR - Connection error

Methods

Discovery Management
bool startDiscovery()
void stopDiscovery()
Device Connection
bool connectToDevice(const std::string& deviceId)
bool disconnectFromDevice(const std::string& deviceId)
void disconnectAll()
Data Transmission
bool sendData(const std::string& deviceId, const std::vector<uint8_t>& data)
std::vector<uint8_t> receiveData(const std::string& deviceId, int timeout_ms = 1000)
Device Information
std::vector<DeviceInfo> getConnectedDevices()
std::vector<DeviceInfo> getAllDevices()
DeviceInfo getDeviceInfo(const std::string& deviceId)
Callbacks
void setDeviceCallback(std::function<void(const DeviceInfo&)> callback)
void setDataCallback(std::function<void(const std::string&, const std::vector<uint8_t>&)> callback)
Statistics
DeviceStats getStats()

DeviceInfo Structure

struct DeviceInfo {
    std::string id;
    std::string name;
    DeviceType type;
    DeviceStatus status;
    std::string mac_address;
    std::string ip_address;
    int port;
    double signal_strength;
    std::map<std::string, std::string> capabilities;
    std::chrono::steady_clock::time_point last_seen;
    bool is_active;
};

DeviceStats Structure

struct DeviceStats {
    int total_devices;
    int connected_devices;
    int active_streams;
    double average_signal_strength;
    std::chrono::steady_clock::time_point last_update;
};

Azure Integration API

AzureIntegration Class

The AzureIntegration class provides GPU computing and AI Foundry resources through Azure services.

Constructor

AzureIntegration(const AzureConfig& config)

AzureConfig Structure

struct AzureConfig {
    std::string subscription_id;
    std::string resource_group;
    std::string location;
    std::string storage_account;
    std::string container_name;
    std::string compute_cluster;
    std::string ai_workspace;
    std::string tenant_id;
    std::string client_id;
    std::string client_secret;
};

Methods

Authentication
bool authenticate()
GPU Resource Management
bool provisionGPUResource(const std::string& vmName, const std::string& gpuType, 
                         int gpuCount, int memoryGB)
bool deprovisionGPUResource(const std::string& vmName)
std::vector<GPUResource> getAvailableGPUResources()
AI Foundry Integration
bool setupAIWorkspace(const std::string& workspaceName)
bool deployModel(const std::string& workspaceName, const std::string& modelName,
                const std::vector<uint8_t>& modelData)
Compute Job Management
std::string submitComputeJob(const std::string& jobType, const std::string& resourceId,
                            const std::vector<uint8_t>& inputData,
                            std::function<void(const std::vector<uint8_t>&)> callback)
bool cancelJob(const std::string& jobId)
std::vector<ComputeJob> getActiveJobs()
Monitoring
bool startMonitoring()
void stopMonitoring()
Callbacks
void setJobCompletionCallback(std::function<void(const std::string&, const std::vector<uint8_t>&)> callback)
void setResourceStatusCallback(std::function<void(const std::string&, const std::string&)> callback)
Statistics
AzureStats getStats()

GPUResource Structure

struct GPUResource {
    std::string vm_id;
    std::string vm_name;
    std::string gpu_type;
    int gpu_count;
    int memory_gb;
    std::string status;
    std::chrono::steady_clock::time_point last_used;
    bool is_available;
};

AIFoundryResource Structure

struct AIFoundryResource {
    std::string workspace_id;
    std::string workspace_name;
    std::string compute_target;
    std::string model_registry;
    std::vector<std::string> available_models;
    std::string status;
    int active_jobs;
    int max_jobs;
};

ComputeJob Structure

struct ComputeJob {
    std::string job_id;
    std::string job_type;
    std::string resource_id;
    std::string status;
    std::chrono::steady_clock::time_point start_time;
    std::chrono::steady_clock::time_point end_time;
    double progress;
    std::vector<uint8_t> input_data;
    std::vector<uint8_t> output_data;
    std::function<void(const std::vector<uint8_t>&)> callback;
};

SLAM Processing API

SLAMProcessor Class

The SLAMProcessor class handles real-time SLAM processing with RF-vision fusion.

Methods

void start()
void stop()
void setParameters(const SLAMParameters& params)
SLAMResult getCurrentPose()
std::vector<MapPoint> getMapPoints()

SLAMParameters Structure

struct SLAMParameters {
    double map_scale;
    int update_rate;
    bool enable_rf_fusion;
    bool enable_vision_slam;
    bool enable_nerf_rendering;
};

SLAMResult Structure

struct SLAMResult {
    Eigen::Matrix4d pose;
    double confidence;
    std::chrono::steady_clock::time_point timestamp;
    std::vector<double> uncertainties;
};

UI/UX API

HolodeckUI Class

The HolodeckUI class provides a comprehensive PyQt6-based user interface.

Constructor

HolodeckUI()

Methods

Device Management
def start_device_discovery(self)
def stop_device_discovery(self)
def connect_to_device(self)
def disconnect_from_device(self)
SLAM Processing
def start_slam_processing(self)
def stop_slam_processing(self)
Azure Integration
def connect_to_azure(self)
def disconnect_from_azure(self)
def provision_gpu_resource(self)
def deprovision_gpu_resource(self)
def setup_ai_workspace(self)
def deploy_model(self)
View Management
def set_view(self, view_type: str)
def update_camera_position(self)
Configuration
def open_configuration(self)
def save_configuration(self)
def calibrate_camera(self)

View Types

  • "3D" - 3D OpenGL visualization
  • "2D" - 2D map view
  • "NeRF" - Neural Radiance Fields rendering

HolodeckGLWidget Class

OpenGL widget for 3D visualization.

Methods

def set_view_type(self, view_type: str)
def set_camera_position(self, x: float, y: float, z: float)
def update(self)

Configuration API

Configuration File Format

The system uses JSON configuration files for all settings.

Main Configuration

{
  "holodeck": {
    "enabled": true,
    "view_type": "3D",
    "update_rate": 30
  },
  "device_management": {
    "discovery_port": 8080,
    "max_connections": 100,
    "discovery_interval": 5,
    "connection_timeout": 30
  },
  "azure_integration": {
    "subscription_id": "your-subscription-id",
    "resource_group": "nowyouseeme-rg",
    "location": "eastus",
    "tenant_id": "your-tenant-id",
    "client_id": "your-client-id",
    "client_secret": "your-client-secret"
  },
  "slam_processing": {
    "map_scale": 1.0,
    "update_rate": 30,
    "enable_rf_fusion": true,
    "enable_vision_slam": true,
    "enable_nerf_rendering": true
  },
  "ui_settings": {
    "window_width": 1600,
    "window_height": 1200,
    "theme": "dark",
    "auto_save": true
  }
}

Network Protocols

Device Communication Protocol

Connection Establishment

  1. Device discovery via UDP broadcast on port 8080
  2. TCP connection establishment on device-specific port
  3. Handshake protocol for capability negotiation
  4. Data streaming with protocol headers

Message Format

[Header: 8 bytes][Payload: variable length]
Header: [MessageType: 2 bytes][Length: 4 bytes][Flags: 2 bytes]

Message Types

  • 0x0001 - Device Info
  • 0x0002 - Data Stream
  • 0x0003 - Control Command
  • 0x0004 - Status Update
  • 0x0005 - Error Report

Azure REST API Integration

Authentication

  • OAuth 2.0 client credentials flow
  • Bearer token authentication
  • Automatic token refresh

GPU Resource Management

  • Azure Compute REST API
  • VM provisioning with GPU extensions
  • Real-time status monitoring

AI Foundry Integration

  • Azure Machine Learning REST API
  • Model deployment and management
  • Workspace and compute target management

Error Handling

Error Codes

  • 0x0001 - Device not found
  • 0x0002 - Connection failed
  • 0x0003 - Authentication failed
  • 0x0004 - Resource unavailable
  • 0x0005 - Invalid parameters
  • 0x0006 - Timeout
  • 0x0007 - Network error
  • 0x0008 - Azure API error

Exception Handling

try {
    device_manager.connectToDevice(device_id);
} catch (const DeviceException& e) {
    spdlog::error("Device error: {}", e.what());
} catch (const NetworkException& e) {
    spdlog::error("Network error: {}", e.what());
} catch (const AzureException& e) {
    spdlog::error("Azure error: {}", e.what());
}

Performance Considerations

Multi-threading

  • Device discovery runs in separate thread
  • Data reception uses thread pool
  • UI updates on main thread
  • Azure operations use async/await pattern

Memory Management

  • Device connections use smart pointers
  • Data buffers are pre-allocated
  • GPU memory is managed by Azure
  • UI elements are properly disposed

Network Optimization

  • Connection pooling for multiple devices
  • Data compression for large payloads
  • Heartbeat mechanism for connection monitoring
  • Automatic reconnection on failure

Security

Authentication

  • Azure AD integration for cloud resources
  • Device-specific authentication tokens
  • Encrypted communication channels
  • Certificate-based device verification

Data Protection

  • End-to-end encryption for sensitive data
  • Secure storage of configuration files
  • Access control for device management
  • Audit logging for all operations

Examples

Basic Device Management

DeviceManager manager(8080, 100);
manager.setDeviceCallback([](const DeviceInfo& device) {
    std::cout << "Found device: " << device.name << std::endl;
});

manager.startDiscovery();
manager.connectToDevice("device-001");

Azure GPU Provisioning

AzureConfig config;
config.subscription_id = "your-subscription-id";
// ... set other config parameters

AzureIntegration azure(config);
azure.authenticate();
azure.provisionGPUResource("gpu-vm-001", "V100", 1, 32);

UI Integration

ui = HolodeckUI()
ui.start_device_discovery()
ui.connect_to_azure()
ui.start_slam_processing()

Troubleshooting

Common Issues

  1. Device Discovery Fails

    • Check network permissions
    • Verify firewall settings
    • Ensure discovery port is available

  2. Azure Authentication Fails

    • Verify credentials in configuration
    • Check Azure AD permissions
    • Ensure subscription is active

  3. SLAM Processing Errors

    • Check camera calibration
    • Verify sensor data quality
    • Monitor system resources

  4. UI Performance Issues

    • Reduce update rate
    • Disable unnecessary features
    • Monitor GPU usage

Debug Mode

Enable debug logging by setting environment variable:

export NOWYOUSEE_DEBUG=1

Log Files

  • Device logs: /var/log/nowyouseeme/device.log
  • Azure logs: /var/log/nowyouseeme/azure.log
  • SLAM logs: /var/log/nowyouseeme/slam.log
  • UI logs: /var/log/nowyouseeme/ui.log
Reference in New Issue View Git Blame Copy Permalink