5a8c26cf5d
- Backend: ShallowEtagHeaderFilter for /api/v1/*, API-VERSIONING.md, README (tenant, CORS, Flyway, ETag) - k8s: backend-deployment.yaml (Deployment, Service, Secret/ConfigMap) - Web: scaffold with directory pull, 304 handling, touch-friendly UI - Android 16: ANDROID-16-TARGET.md; BuildConfig STUN/signaling, SMOAApplication configures InfrastructureManager - Domain: CertificateManager revocation stub, ReportService signReports, ZeroTrust/ThreatDetection minimal docs - TODO.md and IMPLEMENTATION_STATUS.md updated; communications README for endpoint config Co-authored-by: Cursor <cursoragent@cursor.com>
50 lines
4.3 KiB
Markdown
50 lines
4.3 KiB
Markdown
# Connection-Speed-Aware Media and Point-to-Multipoint
|
||
|
||
## Overview
|
||
|
||
SMOA audio and video (Communications and Meetings modules) use **connection-speed-aware compression codecs** so that encoding adapts to available bandwidth, RTT, and packet loss. This is especially important for **point-to-multipoint** (one sender, many receivers), where different participants may have different link quality.
|
||
|
||
## Components
|
||
|
||
| Component | Location | Purpose |
|
||
|-----------|----------|---------|
|
||
| **ConnectionTier** | `communications/domain/AdaptiveCodecPolicy.kt` | Bandwidth tier (VERY_LOW … VERY_HIGH) for codec selection. |
|
||
| **AudioCodecConstraints** | Same | Opus codec limits: min/max bitrate, bandwidth mode (narrowband/wideband/fullband), DTX. |
|
||
| **VideoCodecConstraints** | Same | Video codec (VP8/VP9/H264), max resolution, max bitrate, simulcast/SVC options. |
|
||
| **MediaCodecPolicy** | Same | Maps each ConnectionTier to audio and video constraints; default policy is built-in. |
|
||
| **ConnectionQualityMonitor** | `communications/domain/ConnectionQualityMonitor.kt` | Interface for current quality (bandwidth, RTT, loss, tier). |
|
||
| **StubConnectionQualityMonitor** | `communications/domain/StubConnectionQualityMonitor.kt` | Stub implementation (fixed MEDIUM until WebRTC stats are wired). |
|
||
| **AdaptiveCodecSelector** | `communications/domain/AdaptiveCodecSelector.kt` | Selects current audio/video constraints from policy and quality monitor. |
|
||
| **WebRTCConfig / RTCConfiguration** | `communications/domain/WebRTCConfig.kt`, `WebRTCManager.kt` | Optional media policy; RTC config carries selected audio/video constraints into peer connection setup. |
|
||
|
||
## Connection Tiers and Default Policy
|
||
|
||
- **VERY_LOW** (e.g. < 100 kbps): Audio-only or minimal video; Opus narrowband, low bitrate.
|
||
- **LOW** (e.g. 100–256 kbps): Low-resolution video (e.g. 320×240), VP8, constrained audio.
|
||
- **MEDIUM** (e.g. 256–512 kbps): Moderate video (e.g. 640×360), VP8, wideband Opus.
|
||
- **HIGH** (e.g. 512 kbps–1 Mbps): Higher resolution (e.g. 720p), VP8, simulcast (2 layers), fullband Opus.
|
||
- **VERY_HIGH** (e.g. > 1 Mbps): 1080p, VP9, simulcast (3 layers), SVC preferred, fullband Opus.
|
||
|
||
Exact thresholds are in `connectionTierFromBandwidth()` in `ConnectionQualityMonitor.kt`.
|
||
|
||
## Point-to-Multipoint
|
||
|
||
- **Sender**: Uses `AdaptiveCodecSelector.getSendConstraints()` (or current tier) so the **single send** stream uses codec and bitrate appropriate for the current connection. For HIGH/VERY_HIGH, the policy enables **simulcast** (multiple resolution/bitrate layers) so an SFU or receivers can choose the best layer per participant.
|
||
- **Receivers**: When WebRTC stats are integrated, each receiver can use its own `ConnectionQualityMonitor` (or stats) to request the appropriate simulcast layer or SVC spatial/temporal layer from the server.
|
||
- **Stub**: Until WebRTC is fully integrated, `StubConnectionQualityMonitor` reports a fixed MEDIUM tier. Replace with an implementation that parses `RTCStatsReport` (e.g. outbound-rtp, remote-inbound-rtp, candidate-pair) and calls `update(estimatedBandwidthKbps, rttMs, packetLoss)` (or updates a tier) so the selector adapts in real time.
|
||
|
||
## Applying Constraints When WebRTC Is Integrated
|
||
|
||
When the WebRTC library is integrated:
|
||
|
||
1. When creating the peer connection, read `RTCConfiguration.audioConstraints` and `videoConstraints` (already set by `WebRTCManager` from `AdaptiveCodecSelector`).
|
||
2. For **audio**: create the audio track/sender with Opus and apply `minBitrateBps`/`maxBitrateBps` and bandwidth mode (narrowband/wideband/fullband) and DTX from `AudioCodecConstraints`.
|
||
3. For **video**: create the video track/sender with the requested codec (VP8/VP9/H264), cap resolution to `maxWidth`×`maxHeight`, set `maxBitrateBps`; if `useSimulcast` is true, configure the appropriate number of simulcast layers.
|
||
4. Periodically (e.g. from `getStats()` callback), compute estimated bandwidth (and optionally RTT/loss), call `StubConnectionQualityMonitor.update()` or the real monitor’s update, and optionally call `AdaptiveCodecSelector.selectForBandwidth()` so constraints are updated for the next negotiation or track reconfiguration.
|
||
|
||
## Related
|
||
|
||
- Communications module: `modules/communications/`
|
||
- Meetings (video transport): `modules/meetings/domain/VideoTransport.kt`
|
||
- WebRTC config: `WebRTCConfig.kt`, `WebRTCManager.kt`
|