mirror of
https://github.com/dz0ny/meshcore-sar.git
synced 2026-08-11 16:30:28 +00:00
fix: retain voice codec settings now
ref:
This commit is contained in:
Janez T
207
docs/voice-mode-technical.md
Normal file
207
docs/voice-mode-technical.md
Normal file
@@ -0,0 +1,207 @@
|
||||
# Voice Mode Technical Design
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Voice mode uses a **two-plane architecture**:
|
||||
|
||||
- **Control plane (text messages):**
|
||||
- `VE1:` voice envelope announces voice availability in chat.
|
||||
- `VR1:` direct fetch request asks sender to stream voice payload.
|
||||
- **Data plane (raw binary packets):**
|
||||
- `VoicePacket` payload streamed via `cmdSendRawData` and received through `pushRawData`.
|
||||
|
||||
This design avoids broadcasting full voice payloads to channels/rooms. Chat carries only metadata; audio is fetched on demand when user presses play.
|
||||
|
||||
## 2. Key Modules
|
||||
|
||||
- `lib/utils/voice_message_parser.dart`
|
||||
- `VoicePacket` (legacy text + binary packet format)
|
||||
- `VoiceEnvelope` (`VE1`)
|
||||
- `VoiceFetchRequest` (`VR1`)
|
||||
- `lib/screens/messages_tab.dart`
|
||||
- Capture/encode voice, cache encoded packets, send envelope only
|
||||
- `lib/providers/voice_provider.dart`
|
||||
- Reassembly/playback sessions
|
||||
- Outgoing session cache + deferred serving
|
||||
- `lib/providers/app_provider.dart`
|
||||
- Incoming routing for `VE1` and `VR1`
|
||||
- Handles raw packet ingestion
|
||||
- `lib/widgets/messages/voice_message_bubble.dart`
|
||||
- Play behavior (immediate play if complete, otherwise fetch + auto-play)
|
||||
- `lib/providers/messages_provider.dart`
|
||||
- Message-level voice detection (`VE1` + legacy `V:`)
|
||||
- `lib/services/message_storage_service.dart`
|
||||
- Persists `isVoice` and `voiceId`
|
||||
|
||||
## 3. Wire Formats
|
||||
|
||||
### 3.1 Voice Envelope (`VE1`)
|
||||
|
||||
Prefix: `VE1:` + colon-delimited compact payload
|
||||
|
||||
Fields:
|
||||
|
||||
- `sid` (string, 8 hex chars): session ID
|
||||
- `mode` (int): codec mode ID (`VoicePacketMode.id`)
|
||||
- `total` (int): packet count (1..255)
|
||||
- `durMs` (int): estimated duration in ms
|
||||
- `senderKey6` (string, 12 hex chars): sender public-key prefix (6 bytes)
|
||||
- `ts` (int): unix timestamp seconds
|
||||
- `ver` (int): protocol version (currently `1`)
|
||||
|
||||
Compact format:
|
||||
|
||||
```text
|
||||
VE1:{sid}:{mode}:{total}:{durMs}:{senderKey6}:{ts}:{ver}
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
VE1:deadbeef:1:4:3200:aabbccddeeff:1700000000:1
|
||||
```
|
||||
|
||||
### 3.2 Voice Fetch Request (`VR1`)
|
||||
|
||||
Prefix: `VR1:` + colon-delimited compact payload
|
||||
|
||||
Fields:
|
||||
|
||||
- `sid` (string, 8 hex chars): requested session
|
||||
- `want` (string): currently `a` (compact token for `all`)
|
||||
- `requesterKey6` (string, 12 hex chars): requester key prefix
|
||||
- `ts` (int): unix timestamp seconds
|
||||
- `ver` (int): protocol version (`1`)
|
||||
|
||||
Compact format:
|
||||
|
||||
```text
|
||||
VR1:{sid}:{want}:{requesterKey6}:{ts}:{ver}
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
VR1:deadbeef:a:112233445566:1700000010:1
|
||||
```
|
||||
|
||||
### 3.3 Raw Voice Packet (data plane)
|
||||
|
||||
Binary payload structure:
|
||||
|
||||
- Byte 0: magic `0x56` (`'V'`)
|
||||
- Bytes 1..4: session ID (4 bytes)
|
||||
- Byte 5: mode ID
|
||||
- Byte 6: packet index
|
||||
- Byte 7: total packets
|
||||
- Bytes 8..N: codec2 data
|
||||
|
||||
## 4. Outgoing Flow (Send)
|
||||
|
||||
1. Recorder captures PCM chunks.
|
||||
2. Each chunk is codec2-encoded into `VoicePacket` objects.
|
||||
3. Packets are cached in `VoiceProvider` outgoing cache (TTL 15 min).
|
||||
4. Sender inserts local voice placeholder message (`isVoice=true`, `voiceId=sessionId`).
|
||||
5. Sender sends one envelope (`VE1`) through normal message path:
|
||||
- channel/room: `sendChannelMessage`
|
||||
- direct: `sendTextMessage`
|
||||
6. **No raw audio packets are sent during initial send.**
|
||||
|
||||
## 5. Incoming Routing
|
||||
|
||||
### 5.1 `VE1` envelope received
|
||||
|
||||
`AppProvider` marks message as voice (`isVoice`, `voiceId`) and adds it to chat.
|
||||
|
||||
### 5.2 `VR1` request received
|
||||
|
||||
`AppProvider` treats it as control-plane only:
|
||||
|
||||
- request is not added to chat
|
||||
- validates requester prefix match against sender metadata
|
||||
- resolves requester contact via key prefix
|
||||
- calls `voiceProvider.serveSessionTo(...)`
|
||||
|
||||
### 5.3 Raw packet received (`pushRawData`)
|
||||
|
||||
`AppProvider.onRawDataReceived` parses `VoicePacket` binary and appends to session in `VoiceProvider`.
|
||||
|
||||
## 6. Play / Fetch Behavior
|
||||
|
||||
In `VoiceMessageBubble`:
|
||||
|
||||
- If session already complete: play immediately.
|
||||
- If incomplete/missing:
|
||||
1. Resolve sender contact (message sender prefix or `VE1.senderKey6` fallback)
|
||||
2. Send direct `VR1` fetch request
|
||||
3. Show requesting state in UI
|
||||
4. Auto-play when session becomes complete
|
||||
|
||||
If sender cannot be resolved or request cannot be sent, bubble remains and shows: **"Voice unavailable right now"**.
|
||||
|
||||
## 7. Outgoing Cache Details
|
||||
|
||||
`VoiceProvider` outgoing cache:
|
||||
|
||||
- key: `sessionId`
|
||||
- value: encoded packet list + cached timestamp
|
||||
- TTL: 15 minutes (`_outgoingSessionTtl`)
|
||||
- eviction: lazy (on cache access/add/serve)
|
||||
|
||||
Serving prerequisites:
|
||||
|
||||
- session exists in cache
|
||||
- `sendRawPacketCallback` configured
|
||||
- requester has direct path (`outPathLen >= 0`)
|
||||
|
||||
## 8. Persistence
|
||||
|
||||
`MessageStorageService` now stores and restores:
|
||||
|
||||
- `isVoice`
|
||||
- `voiceId`
|
||||
|
||||
This ensures envelope messages remain voice bubbles across app restart.
|
||||
|
||||
## 9. Validation and Safety
|
||||
|
||||
Parser validation enforces:
|
||||
|
||||
- strict hex lengths for IDs and key prefixes
|
||||
- valid mode range
|
||||
- valid packet counts and duration bounds
|
||||
- fixed protocol version (`ver == 1`)
|
||||
- `VR1.want` token `a` (internally normalized to `all`)
|
||||
|
||||
`VR1` handling verifies sender prefix matches `requesterKey6` to reduce spoofing risk.
|
||||
|
||||
## 10. Operational Constraints
|
||||
|
||||
- No firmware changes required.
|
||||
- On-demand fetch works only if sender app is online and has cached session.
|
||||
- Raw return path needs a currently valid direct route to requester.
|
||||
- Voice capture in UI is currently iOS-only (`MessagesTab._voiceSupported`).
|
||||
|
||||
## 11. Backward Compatibility
|
||||
|
||||
- Legacy `V:` text packet parsing is still supported.
|
||||
- Message voice detection accepts both new `VE1` and legacy `V:` formats.
|
||||
|
||||
## 12. High-Level Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant A as Sender App
|
||||
participant M as Mesh Chat
|
||||
participant B as Receiver App
|
||||
|
||||
A->>A: Record + encode voice packets
|
||||
A->>A: Cache session packets (TTL 15m)
|
||||
A->>M: Send VE1 envelope
|
||||
M->>B: Deliver VE1
|
||||
B->>B: Render voice bubble (metadata only)
|
||||
B->>A: Send VR1 request on Play
|
||||
A->>B: Stream raw VoicePacket packets
|
||||
B->>B: Reassemble session
|
||||
B->>B: Auto-play when complete
|
||||
```
|
||||
Reference in New Issue
Block a user