ua1zbe/meshcore-sar_android
1
0
Fork 0
mirror of https://github.com/dz0ny/meshcore-sar.git synced 2026-08-11 08:20:36 +00:00
Code Issues Packages Projects Releases 1 Wiki Activity

Add swarm mode transport doc

Browse Source
This commit is contained in:
Janez T
2026-03-07 14:13:54 +01:00
parent 4e76898c8d
commit 0e4f727e26
43 changed files with 3831 additions and 1078 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
docs/voice-mode-technical.md
View File

@@ -2,10 +2,11 @@
## 1. Overview
Voice mode uses a **two-plane architecture**:
Voice mode uses a **two-plane architecture** with optional swarm-assisted
recovery:
- **Control plane (text messages):**
- `VE2:` voice envelope announces voice availability in chat.
- `VE3:` voice envelope announces voice availability in chat.
- **Control plane (raw binary request):**
- Binary voice fetch request (same raw route as voice packets).
- **Data plane (raw binary packets):**
@@ -13,11 +14,13 @@ Voice mode uses a **two-plane architecture**:
This design avoids broadcasting full voice payloads to channels/rooms. Chat carries only metadata; audio is fetched on demand when user presses play.
Swarm fallback is documented in [Swarm Mode Technical Design](./swarm-mode-technical.md).
## 2. Key Modules
- `lib/utils/voice_message_parser.dart`
- `VoicePacket` (legacy text + binary packet format)
- `VoiceEnvelope` (`VE2`)
- `VoicePacket` (binary direct-packet format)
- `VoiceEnvelope` (`VE3`)
- `VoiceFetchRequest` (binary)
- `lib/screens/messages_tab.dart`
- Capture/encode voice, cache encoded packets, send envelope only
@@ -25,20 +28,20 @@ This design avoids broadcasting full voice payloads to channels/rooms. Chat carr
- Reassembly/playback sessions
- Outgoing session cache + deferred serving
- `lib/providers/app_provider.dart`
- Incoming routing for `VE2` and binary voice fetch requests
- Incoming routing for `VE3`, binary voice fetch requests, and raw swarm control payloads
- Handles raw packet ingestion
- `lib/widgets/messages/voice_message_bubble.dart`
- Play behavior (immediate play if complete, otherwise fetch + auto-play)
- `lib/providers/messages_provider.dart`
- Message-level voice detection (`VE2` + legacy `V:`)
- Message-level voice detection (`VE3`)
- `lib/services/message_storage_service.dart`
- Persists `isVoice` and `voiceId`
## 3. Wire Formats
### 3.1 Voice Envelope (`VE2`)
### 3.1 Voice Envelope (`VE3`)
Prefix: `VE2:` + colon-delimited compact payload (base36 numeric fields)
Prefix: `VE3:` + colon-delimited compact payload (base36 numeric fields)
Fields:
@@ -46,21 +49,19 @@ Fields:
- `mode` (base36): codec mode ID (`VoicePacketMode.id`)
- `total` (base36): packet count (1..255)
- `durS` (base36): estimated duration in seconds
- `senderKey6` (string, 12 hex chars): sender public-key prefix (6 bytes)
- `ts` (base36): unix timestamp seconds
`sid` is base36 on wire and expands to 8-hex internally.
Compact format:
```text
VE2:{sid}:{mode}:{total}:{durS}:{senderKey6}:{ts}
VE3:{sid}:{mode}:{total}:{durS}
```
Example:
```text
VE2:a:1:4:4:aabbccddeeff:s44we8
VE3:a:1:4:4
```
### 3.2 Voice Fetch Request (binary)
@@ -68,7 +69,7 @@ VE2:a:1:4:4:aabbccddeeff:s44we8
Binary payload format:
```text
[magic=0x72][sid:4B][flags:1B][requesterKey6:6B][ts:4B][missingCount:1B][missingIndices...]
[magic=0x72][sid:4B][flags:1B][requesterKey6:6B][missingCount:1B][missingIndices...]
```
### 3.3 Raw Voice Packet (data plane)
@@ -77,10 +78,10 @@ Binary payload structure:
- Byte 0: magic `0x56` (`'V'`)
- Bytes 1..4: session ID (4 bytes)
- Byte 5: mode ID
- Byte 6: packet index
- Byte 7: total packets
- Bytes 8..N: codec2 data
- Byte 5: packet index
- Bytes 6..N: codec2 data
Header is 6 bytes. Mode and total packet count come from the `VE3` envelope.
## 4. Outgoing Flow (Send)
@@ -88,27 +89,38 @@ Binary payload structure:
2. Each chunk is codec2-encoded into `VoicePacket` objects.
3. Packets are cached in `VoiceProvider` outgoing cache (TTL 15 min).
4. Sender inserts local voice placeholder message (`isVoice=true`, `voiceId=sessionId`).
5. Sender sends one envelope (`VE2`) through normal message path:
5. Sender sends one envelope (`VE3`) through normal message path:
- channel/room: `sendChannelMessage`
- direct: `sendTextMessage`
6. **No raw audio packets are sent during initial send.**
## 5. Incoming Routing
### 5.1 `VE2` envelope received
### 5.1 `VE3` envelope received
`AppProvider` marks message as voice (`isVoice`, `voiceId`) and adds it to chat.
`AppProvider` records sender identity from message metadata, registers the voice
session envelope, marks the message as voice (`isVoice`, `voiceId`), and adds it to chat.
### 5.2 Binary voice fetch request received
`AppProvider` treats it as control-plane only:
- request is not added to chat
- validates requester prefix match against sender metadata
- resolves requester contact via key prefix
- calls `voiceProvider.serveSessionTo(...)`
### 5.3 Raw packet received (`pushRawData`)
### 5.3 Swarm control messages received
`AppProvider` also handles raw swarm control payloads:
- binary swarm requests advertise which voice fragments are still missing
- binary swarm availability responses advertise which fragments a peer can relay
- swarm control payloads arrive via `pushRawData` and are not added to chat history
Swarm semantics are shared with image mode and documented in
[Swarm Mode Technical Design](./swarm-mode-technical.md).
### 5.4 Raw packet received (`pushRawData`)
`AppProvider.onRawDataReceived` parses `VoicePacket` binary and appends to session in `VoiceProvider`.
@@ -118,10 +130,15 @@ In `VoiceMessageBubble`:
- If session already complete: play immediately.
- If incomplete/missing:
1. Resolve sender contact (message sender prefix or `VE2.senderKey6` fallback)
2. Send direct binary fetch request
3. Show requesting state in UI
4. Auto-play when session becomes complete
1. Resolve sender contact from message sender metadata
2. Prefer a direct fetch from the original sender if its raw route is healthy
3. If the sender path does not respond, fan out a raw swarm request with the
exact missing packet indices to reachable peers
4. Wait up to 10 seconds for raw peer availability responses
5. Send a direct binary fetch request to the best alternate peer for the
missing subset it advertised
6. Show requesting state in UI
7. Auto-play when session becomes complete
If sender cannot be resolved or request cannot be sent, bubble remains and shows: **"Voice unavailable right now"**.
@@ -136,10 +153,13 @@ If sender cannot be resolved or request cannot be sent, bubble remains and shows
Serving prerequisites:
- session exists in cache
- session exists in outgoing cache or already-received session state
- `sendRawPacketCallback` configured
- requester has direct path (`outPathLen >= 0`)
Received partial sessions can therefore act as relay sources during swarm
recovery.
## 8. Persistence
`MessageStorageService` now stores and restores:
@@ -167,13 +187,13 @@ Voice bubbles and Message Technical Details show an **estimated transmit time**
The estimate is airtime-based (LoRa packet model), not file-duration-only:
- Source inputs:
- `packetCount` and `durationMs` from `VE2` envelope, or
- `packetCount` and `durationMs` from `VE3` envelope, or
- numeric envelope values decoded from base36
- actual received `VoicePacket.codec2Data.length` bytes when local session packets exist
- `pathLen` from message metadata
- current radio params from `deviceInfo`: `radioBw`, `radioSf`, `radioCr`
- Per-packet payload model:
- `meshHeader(2)` + `pathLen` + `voiceHeader(8)` + `codec2Bytes`
- `meshHeader(2)` + `pathLen` + `voiceHeader(6)` + `codec2Bytes`
- LoRa airtime:
- standard symbol-time formula (preamble + payload symbols)
- Mesh pacing/hops:
@@ -192,9 +212,12 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
## 11. Operational Constraints
- No firmware changes required.
- On-demand fetch works only if sender app is online and has cached session.
- On-demand fetch prefers the original sender, but a partial session can also be
completed from alternate peers that already hold packets.
- Raw return path needs a currently valid direct route to requester.
- Voice capture is available on iOS and Android (`Platform.isIOS || Platform.isAndroid`).
- Swarm discovery uses the same `cmdSendRawData` / `pushRawData` path as voice
fetch and packet delivery.
### 11.1 Raw Binary Routing Semantics
@@ -206,26 +229,27 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
- only nodes on that path relay it;
- it is **not** received by everyone in the mesh.
## 12. Backward Compatibility
- Legacy `V:` text packet parsing is still supported.
- Message voice detection accepts `VE2` and legacy `V:` formats.
## 13. High-Level Sequence
## 12. High-Level Sequence
```mermaid
sequenceDiagram
participant A as Sender App
participant M as Mesh Chat
participant P as Peer With Packets
participant N as Reachable Peers
participant B as Receiver App
A->>A: Record + encode voice packets
A->>A: Cache session packets (TTL 15m)
A->>M: Send VE2 envelope
M->>B: Deliver VE2
A->>M: Send VE3 envelope
M->>B: Deliver VE3
B->>B: Render voice bubble (metadata only)
B->>A: Send binary fetch request on Play
A->>B: Stream raw VoicePacket packets
A->>B: Stream raw VoicePacket packets (partial)
Note over A,B: Sender path stops responding
B->>N: Raw swarm requests with missing voice packet indices
P->>B: Raw swarm availability response
B->>P: Direct binary fetch request for missing subset
P->>B: Stream remaining raw VoicePacket packets
B->>B: Reassemble session
B->>B: Auto-play when complete
```