50 lines
4.3 KiB
Markdown
50 lines
4.3 KiB
Markdown
|
# Connection-Speed-Aware Media and Point-to-Multipoint
|
|||
|
|
|
|||
|
|
## Overview
|
|||
|
|
|
|||
|
|
SMOA audio and video (Communications and Meetings modules) use **connection-speed-aware compression codecs** so that encoding adapts to available bandwidth, RTT, and packet loss. This is especially important for **point-to-multipoint** (one sender, many receivers), where different participants may have different link quality.
|
|||
|
|
|
|||
|
|
## Components
|
|||
|
|
|
|||
|
|
| Component | Location | Purpose |
|
|||
|
|
|-----------|----------|---------|
|
|||
|
|
| **ConnectionTier** | `communications/domain/AdaptiveCodecPolicy.kt` | Bandwidth tier (VERY_LOW … VERY_HIGH) for codec selection. |
|
|||
|
|
| **AudioCodecConstraints** | Same | Opus codec limits: min/max bitrate, bandwidth mode (narrowband/wideband/fullband), DTX. |
|
|||
|
|
| **VideoCodecConstraints** | Same | Video codec (VP8/VP9/H264), max resolution, max bitrate, simulcast/SVC options. |
|
|||
|
|
| **MediaCodecPolicy** | Same | Maps each ConnectionTier to audio and video constraints; default policy is built-in. |
|
|||
|
|
| **ConnectionQualityMonitor** | `communications/domain/ConnectionQualityMonitor.kt` | Interface for current quality (bandwidth, RTT, loss, tier). |
|
|||
|
|
| **StubConnectionQualityMonitor** | `communications/domain/StubConnectionQualityMonitor.kt` | Stub implementation (fixed MEDIUM until WebRTC stats are wired). |
|
|||
|
|
| **AdaptiveCodecSelector** | `communications/domain/AdaptiveCodecSelector.kt` | Selects current audio/video constraints from policy and quality monitor. |
|
|||
|
|
| **WebRTCConfig / RTCConfiguration** | `communications/domain/WebRTCConfig.kt`, `WebRTCManager.kt` | Optional media policy; RTC config carries selected audio/video constraints into peer connection setup. |
|
|||
|
|
|
|||
|
|
## Connection Tiers and Default Policy
|
|||
|
|
|
|||
|
|
- **VERY_LOW** (e.g. < 100 kbps): Audio-only or minimal video; Opus narrowband, low bitrate.
|
|||
|
|
- **LOW** (e.g. 100–256 kbps): Low-resolution video (e.g. 320×240), VP8, constrained audio.
|
|||
|
|
- **MEDIUM** (e.g. 256–512 kbps): Moderate video (e.g. 640×360), VP8, wideband Opus.
|
|||
|
|
- **HIGH** (e.g. 512 kbps–1 Mbps): Higher resolution (e.g. 720p), VP8, simulcast (2 layers), fullband Opus.
|
|||
|
|
- **VERY_HIGH** (e.g. > 1 Mbps): 1080p, VP9, simulcast (3 layers), SVC preferred, fullband Opus.
|
|||
|
|
|
|||
|
|
Exact thresholds are in `connectionTierFromBandwidth()` in `ConnectionQualityMonitor.kt`.
|
|||
|
|
|
|||
|
|
## Point-to-Multipoint
|
|||
|
|
|
|||
|
|
- **Sender**: Uses `AdaptiveCodecSelector.getSendConstraints()` (or current tier) so the **single send** stream uses codec and bitrate appropriate for the current connection. For HIGH/VERY_HIGH, the policy enables **simulcast** (multiple resolution/bitrate layers) so an SFU or receivers can choose the best layer per participant.
|
|||
|
|
- **Receivers**: When WebRTC stats are integrated, each receiver can use its own `ConnectionQualityMonitor` (or stats) to request the appropriate simulcast layer or SVC spatial/temporal layer from the server.
|
|||
|
|
- **Stub**: Until WebRTC is fully integrated, `StubConnectionQualityMonitor` reports a fixed MEDIUM tier. Replace with an implementation that parses `RTCStatsReport` (e.g. outbound-rtp, remote-inbound-rtp, candidate-pair) and calls `update(estimatedBandwidthKbps, rttMs, packetLoss)` (or updates a tier) so the selector adapts in real time.
|
|||
|
|
|
|||
|
|
## Applying Constraints When WebRTC Is Integrated
|
|||
|
|
|
|||
|
|
When the WebRTC library is integrated:
|
|||
|
|
|
|||
|
|
1. When creating the peer connection, read `RTCConfiguration.audioConstraints` and `videoConstraints` (already set by `WebRTCManager` from `AdaptiveCodecSelector`).
|
|||
|
|
2. For **audio**: create the audio track/sender with Opus and apply `minBitrateBps`/`maxBitrateBps` and bandwidth mode (narrowband/wideband/fullband) and DTX from `AudioCodecConstraints`.
|
|||
|
|
3. For **video**: create the video track/sender with the requested codec (VP8/VP9/H264), cap resolution to `maxWidth`×`maxHeight`, set `maxBitrateBps`; if `useSimulcast` is true, configure the appropriate number of simulcast layers.
|
|||
|
|
4. Periodically (e.g. from `getStats()` callback), compute estimated bandwidth (and optionally RTT/loss), call `StubConnectionQualityMonitor.update()` or the real monitor’s update, and optionally call `AdaptiveCodecSelector.selectForBandwidth()` so constraints are updated for the next negotiation or track reconfiguration.
|
|||
|
|
|
|||
|
|
## Related
|
|||
|
|
|
|||
|
|
- Communications module: `modules/communications/`
|
|||
|
|
- Meetings (video transport): `modules/meetings/domain/VideoTransport.kt`
|
|||
|
|
- WebRTC config: `WebRTCConfig.kt`, `WebRTCManager.kt`
|