mirror of
https://github.com/dz0ny/meshcore-sar.git
synced 2026-08-11 16:30:28 +00:00
Add swarm mode transport doc
This commit is contained in:
Janez T
@@ -2,10 +2,11 @@
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Image mode mirrors the voice on-demand architecture exactly:
|
||||
Image mode mirrors the voice on-demand architecture exactly, including
|
||||
swarm-assisted recovery for stalled partial transfers:
|
||||
|
||||
- **Control plane (text messages):**
|
||||
- `IE2:` image envelope announces image availability in chat.
|
||||
- `IE4:` image envelope announces image availability in chat.
|
||||
- **Control plane (raw binary request):**
|
||||
- Binary image fetch request (same raw route as image fragments).
|
||||
- **Data plane (raw binary packets):**
|
||||
@@ -14,11 +15,14 @@ Image mode mirrors the voice on-demand architecture exactly:
|
||||
Images are never broadcast in full to channels. Chat carries only metadata;
|
||||
pixels are fetched on demand when the user taps the image bubble.
|
||||
|
||||
Shared swarm fallback is documented in
|
||||
[Swarm Mode Technical Design](./swarm-mode-technical.md).
|
||||
|
||||
## 2. Key Modules
|
||||
|
||||
- `lib/utils/image_message_parser.dart`
|
||||
- `ImagePacket` (binary fragment format)
|
||||
- `ImageEnvelope` (`IE2`)
|
||||
- `ImageEnvelope` (`IE4`)
|
||||
- `ImageFetchRequest` (binary)
|
||||
- `fragmentImage()` — split compressed bytes into packets
|
||||
- `reassembleImage()` — join received fragments into bytes
|
||||
@@ -27,8 +31,9 @@ pixels are fetched on demand when the user taps the image bubble.
|
||||
- `lib/providers/image_provider.dart`
|
||||
- Reassembly sessions, outgoing cache, deferred serving
|
||||
- Outgoing sessions also registered as complete incoming sessions for immediate local display
|
||||
- Received partial sessions can be re-served during swarm recovery
|
||||
- `lib/providers/app_provider.dart`
|
||||
- Incoming routing for `IE2`, binary image fetch requests, binary `0x49` packets
|
||||
- Incoming routing for `IE4`, binary image fetch requests, binary `0x49` packets, and raw swarm control payloads
|
||||
- `lib/widgets/messages/image_message_bubble.dart`
|
||||
- Square cover thumbnail (up to 256 px); tap-to-load for received images;
|
||||
progress ring during fetch; full-screen `InteractiveViewer` on tap
|
||||
@@ -40,9 +45,9 @@ pixels are fetched on demand when the user taps the image bubble.
|
||||
|
||||
## 3. Wire Formats
|
||||
|
||||
### 3.1 Image Envelope (`IE2`)
|
||||
### 3.1 Image Envelope (`IE4`)
|
||||
|
||||
Prefix: `IE2:` + colon-delimited compact payload (base36 numeric fields)
|
||||
Prefix: `IE4:` + colon-delimited compact payload (base36 numeric fields)
|
||||
|
||||
Fields:
|
||||
|
||||
@@ -54,51 +59,50 @@ Fields:
|
||||
| `w` | base36 | Actual image width after compression (pixels) |
|
||||
| `h` | base36 | Actual image height after compression (pixels) |
|
||||
| `bytes` | base36 | Total compressed size in bytes |
|
||||
| `senderKey6` | string | 12 hex chars (6 bytes sender prefix) |
|
||||
| `ts` | base36 | Unix timestamp (seconds) |
|
||||
|
||||
Compact format:
|
||||
|
||||
```text
|
||||
IE2:{sid}:{fmt}:{total}:{w}:{h}:{bytes}:{senderKey6}:{ts}
|
||||
IE4:{sid}:{fmt}:{total}:{w}:{h}:{bytes}
|
||||
```
|
||||
|
||||
Example (256×171 landscape image, 14 fragments):
|
||||
|
||||
```text
|
||||
IE2:a:0:e:74:4r:1mc:aabbccddeeff:s44we8
|
||||
IE4:a:0:e:74:4r:1mc
|
||||
```
|
||||
|
||||
Note: `sid` is base36 on wire and expands to 8-hex internally.
|
||||
`w` and `h` reflect the actual post-compression dimensions, which preserve
|
||||
the source aspect ratio (contain within the configured max size).
|
||||
|
||||
### 3.2 Image Fetch Request (binary)
|
||||
### 3.2 Image Fetch Request (`IR4` + binary)
|
||||
|
||||
Text format:
|
||||
|
||||
```text
|
||||
IR4:{sid}:{want}:{requesterKey6}
|
||||
```
|
||||
|
||||
Binary payload format:
|
||||
|
||||
```text
|
||||
[magic=0x69][sid:4B][flags:1B][requesterKey6:6B][ts:4B][missingCount:1B][missingIndices...]
|
||||
[magic=0x69][sid:4B][flags:1B][requesterKey6:6B][missingCount:1B][missingIndices...]
|
||||
```
|
||||
|
||||
| Field | Value |
|
||||
|------------------|--------------------------|
|
||||
| `flags` | bit0=1 => request missing indices, else all |
|
||||
| `requesterKey6` | 6-byte requester key prefix |
|
||||
| `ts` | unix timestamp seconds (u32) |
|
||||
|
||||
### 3.3 Raw Image Packet (data plane)
|
||||
|
||||
Binary payload structure:
|
||||
|
||||
- Byte 0: magic `0x49` (`'I'`)
|
||||
- Bytes 1..4: session ID (4 bytes)
|
||||
- Byte 5: format ID
|
||||
- Byte 6: fragment index (0-based)
|
||||
- Byte 7: total fragments
|
||||
- Bytes 8..N: image data (max 152 bytes per fragment)
|
||||
- Byte 5: fragment index (0-based)
|
||||
- Bytes 6..N: image data (max 152 bytes per fragment)
|
||||
|
||||
Header is 8 bytes — identical layout to `VoicePacket`.
|
||||
Header is 6 bytes. Image format and total fragment count come from the `IE4` envelope.
|
||||
|
||||
## 4. Compression Pipeline
|
||||
|
||||
@@ -152,11 +156,11 @@ only the shorter axis is padded — no cropping occurs.
|
||||
7. Envelope sent via normal message path:
|
||||
- Channel: `sendChannelMessage`
|
||||
- Direct: `sendTextMessage`
|
||||
8. Local placeholder message added (`IE2:` text, `deliveryStatus.sending`).
|
||||
8. Local placeholder message added (`IE4:` text, `deliveryStatus.sending`).
|
||||
|
||||
## 6. Incoming Flow (Receive)
|
||||
|
||||
### 6.1 `IE2` envelope received
|
||||
### 6.1 `IE4` envelope received
|
||||
|
||||
`AppProvider` calls `imageProvider.registerEnvelope()` and adds the message to
|
||||
chat. The bubble shows a grey square placeholder with a download icon.
|
||||
@@ -165,11 +169,21 @@ chat. The bubble shows a grey square placeholder with a download icon.
|
||||
|
||||
`AppProvider` treats it as control-plane only (not added to chat):
|
||||
|
||||
- Validates requester key prefix.
|
||||
- Resolves requester contact.
|
||||
- Calls `imageProvider.serveSessionTo()` which streams all cached fragments.
|
||||
|
||||
### 6.3 Raw packet received (`pushRawData`, magic `0x49`)
|
||||
### 6.3 Swarm control messages received
|
||||
|
||||
`AppProvider` also handles shared raw swarm discovery payloads:
|
||||
|
||||
- binary swarm requests advertise which image fragments are still missing
|
||||
- binary swarm availability responses advertise which fragments another peer can relay
|
||||
- swarm payloads arrive via `pushRawData` and are intercepted instead of being added to chat
|
||||
|
||||
Shared discovery and responder semantics are documented in
|
||||
[Swarm Mode Technical Design](./swarm-mode-technical.md).
|
||||
|
||||
### 6.4 Raw packet received (`pushRawData`, magic `0x49`)
|
||||
|
||||
`AppProvider.onRawDataReceived` parses `ImagePacket` binary and calls
|
||||
`imageProvider.addFragment()`. When the session becomes complete, the bubble
|
||||
@@ -188,6 +202,9 @@ When `cacheOutgoingSession()` is called it also writes all fragments into
|
||||
`_sessions[sessionId]`, so the sender sees the image immediately in the bubble
|
||||
(no tap-to-load required for own messages).
|
||||
|
||||
If a peer later receives only part of an image session, those received fragments
|
||||
can also be served onward to another requester during swarm recovery.
|
||||
|
||||
## 8. Display
|
||||
|
||||
`ImageMessageBubble` (max width 256 px):
|
||||
@@ -195,7 +212,8 @@ When `cacheOutgoingSession()` is called it also writes all fragments into
|
||||
- **Complete session**: `AspectRatio(1.0)` → `AvifImage.memory(fit: cover)`
|
||||
square thumbnail; tap → full-screen `InteractiveViewer` with fade transition.
|
||||
- **Incomplete/missing**: grey square placeholder with download icon;
|
||||
tap → sends binary fetch request.
|
||||
tap → sends binary fetch request, with raw swarm fallback if the original
|
||||
sender path stalls.
|
||||
- **Loading**: circular progress indicator showing `received/total` count.
|
||||
- **Error**: broken-image icon.
|
||||
|
||||
@@ -213,12 +231,12 @@ Image bubbles and Message Technical Details show an **estimated transmit time**
|
||||
The estimate is airtime-based (LoRa packet model), not just compressed image size:
|
||||
|
||||
- Source inputs:
|
||||
- `total` fragments and `bytes` from `IE2` envelope
|
||||
- `total` fragments and `bytes` from `IE4` envelope
|
||||
- all numeric envelope values are decoded from base36
|
||||
- `pathLen` from message metadata
|
||||
- current radio params from `deviceInfo`: `radioBw`, `radioSf`, `radioCr`
|
||||
- Per-fragment payload model:
|
||||
- `meshHeader(2)` + `pathLen` + `imageHeader(8)` + `fragmentBytes`
|
||||
- `meshHeader(2)` + `pathLen` + `imageHeader(6)` + `fragmentBytes`
|
||||
- LoRa airtime:
|
||||
- standard symbol-time formula (preamble + payload symbols)
|
||||
- Mesh pacing/hops:
|
||||
@@ -246,9 +264,12 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
|
||||
## 11. Operational Constraints
|
||||
|
||||
- No firmware changes required (reuses `cmdSendRawData` / `pushRawData`).
|
||||
- On-demand fetch works only if sender app is online and has cached session.
|
||||
- On-demand fetch prefers the original sender, but a partial image can also be
|
||||
completed from alternate peers that already hold matching fragments.
|
||||
- Raw return path requires a valid direct route to requester.
|
||||
- Available on iOS and Android (`image_picker` + `flutter_avif`).
|
||||
- Swarm discovery uses the same `cmdSendRawData` / `pushRawData` path as image
|
||||
fetch and fragment delivery.
|
||||
|
||||
### 11.1 Raw Binary Routing Semantics
|
||||
|
||||
@@ -260,7 +281,28 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
|
||||
- only nodes on that path relay it;
|
||||
- it is **not** received by everyone in the mesh.
|
||||
|
||||
## 12. High-Level Sequence
|
||||
## 12. Swarm Fallback Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant A as Original Sender
|
||||
participant P as Peer With Fragments
|
||||
participant N as Reachable Peers
|
||||
participant B as Receiver App
|
||||
|
||||
A->>M: Send IE4 envelope
|
||||
M->>B: Deliver IE4
|
||||
B->>A: Direct binary image fetch request
|
||||
A->>B: Stream raw ImagePacket fragments (partial)
|
||||
Note over A,B: Sender path stops responding
|
||||
B->>N: Raw swarm requests with missing image fragment indices
|
||||
P->>B: Raw swarm availability response
|
||||
B->>P: Direct binary fetch request for missing subset
|
||||
P->>B: Stream remaining raw ImagePacket fragments
|
||||
B->>B: Reassemble completed image
|
||||
```
|
||||
|
||||
## 13. High-Level Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@@ -272,8 +314,8 @@ sequenceDiagram
|
||||
A->>A: Compress: contain resize → grayscale → PNG → AVIF
|
||||
A->>A: Fragment into ≤152B packets
|
||||
A->>A: Cache outgoing + populate local session (immediate display)
|
||||
A->>M: Send IE2 envelope (actual w×h, fragment count)
|
||||
M->>B: Deliver IE2
|
||||
A->>M: Send IE4 envelope (actual w×h, fragment count)
|
||||
M->>B: Deliver IE4
|
||||
B->>B: Render grey placeholder bubble
|
||||
B->>A: Tap → send binary fetch request
|
||||
A->>B: Stream binary ImagePackets
|
||||
|
||||
148
docs/swarm-mode-technical.md
Normal file
148
docs/swarm-mode-technical.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# Swarm Mode Technical Design
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Swarm mode is a shared media-recovery transport used by both voice and image
|
||||
sessions when the original sender path stops responding after some fragments
|
||||
have already propagated through the mesh.
|
||||
|
||||
It adds a lightweight **swarm discovery plane** on top of the existing
|
||||
**direct raw-data transfer plane**:
|
||||
|
||||
- **Discovery plane (raw custom control payloads):**
|
||||
- `MediaSwarmRequest`
|
||||
- `MediaSwarmAvailability`
|
||||
- **Transfer plane (direct raw binary):**
|
||||
- existing `VoiceFetchRequest` / `ImageFetchRequest`
|
||||
- existing `VoicePacket` / `ImagePacket`
|
||||
|
||||
Swarm mode does not broadcast media payloads. It fans out raw control requests
|
||||
to reachable peers, collects raw availability responses, and then fetches media
|
||||
from the best responder.
|
||||
|
||||
## 2. Problem It Solves
|
||||
|
||||
Without swarm mode, media fetch is limited to the original sender's direct raw
|
||||
path. If that sender goes offline, moves, or stops responding, a receiver can
|
||||
be left with a partial session even when other peers already hold useful
|
||||
fragments.
|
||||
|
||||
Swarm mode lets the receiver discover alternate peers and fetch the missing
|
||||
subset directly from them.
|
||||
|
||||
## 3. Control Messages
|
||||
|
||||
### 3.1 Media Swarm Request (binary)
|
||||
|
||||
```text
|
||||
[magic=0x6d][kind=0x01][mediaType:1B][sessionId:4B][requesterKey6:6B][missingCount:1B][missingIndices...]
|
||||
```
|
||||
|
||||
Fields:
|
||||
|
||||
- `mediaType` — `voice` or `image`
|
||||
- `sessionId` — 8 hex chars
|
||||
- `requesterKey6` — 12 hex chars identifying the requesting device
|
||||
- `missingCount` — `0` means the requester needs all fragments
|
||||
- `missingIndices` — exact missing fragment indices when `missingCount > 0`
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
6d 01 02 de ad be ef aa bb cc dd ee ff 03 00 02 09
|
||||
```
|
||||
|
||||
### 3.2 Media Swarm Availability (binary)
|
||||
|
||||
```text
|
||||
[magic=0x6d][kind=0x02][mediaType:1B][sessionId:4B][requesterKey6:6B][responderKey6:6B][availableCount:1B][availableIndices...]
|
||||
```
|
||||
|
||||
Fields:
|
||||
|
||||
- `mediaType` — `voice` or `image`
|
||||
- `sessionId` — 8 hex chars
|
||||
- `requesterKey6` — copied from the swarm request
|
||||
- `responderKey6` — 12 hex chars identifying the responding peer
|
||||
- `availableCount` — `0` means the responder can satisfy the full request
|
||||
- `availableIndices` — exact fragment indices held when `availableCount > 0`
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
6d 02 02 de ad be ef aa bb cc dd ee ff 11 22 33 44 55 66 02 02 09
|
||||
```
|
||||
|
||||
## 4. Requester Behavior
|
||||
|
||||
When a voice or image session is incomplete:
|
||||
|
||||
1. Prefer the original sender if its direct raw route is healthy.
|
||||
2. If the original sender path is unavailable or not responding, send a raw
|
||||
swarm request to reachable peers with the exact missing fragment indices.
|
||||
3. Wait up to **10 seconds** for raw availability responses.
|
||||
4. Rank responders by overlap with the current missing set.
|
||||
5. Skip the original sender when choosing alternate peers.
|
||||
6. Send a direct binary fetch request to the best responder for only the
|
||||
missing subset that responder advertised.
|
||||
|
||||
Actual media transfer uses the same `cmdSendRawData` / `pushRawData` path as the
|
||||
swarm control messages.
|
||||
|
||||
## 5. Responder Behavior
|
||||
|
||||
A peer that receives a swarm request:
|
||||
|
||||
- checks whether it has fragments for the requested media session
|
||||
- replies only if it has at least one requested fragment
|
||||
- advertises only fragments it actually holds
|
||||
- may reply from:
|
||||
- its outgoing cache, or
|
||||
- a partially/fully received incoming session
|
||||
|
||||
This enables torrent-like relay behavior without requiring the peer to be the
|
||||
original sender.
|
||||
|
||||
## 6. Visibility and Routing
|
||||
|
||||
- Swarm control payloads are carried by `cmdSendRawData` and received through
|
||||
`pushRawData`.
|
||||
- They are intercepted by the app and **must not be surfaced in chat**.
|
||||
- Discovery is a direct raw fan-out to reachable peers, not a public-channel
|
||||
broadcast.
|
||||
- Media packets themselves remain direct-route raw packets.
|
||||
|
||||
## 7. Constraints
|
||||
|
||||
- No firmware changes are required.
|
||||
- Swarm mode only helps if at least one peer has already received some useful
|
||||
fragments.
|
||||
- Peers can only relay fragments they actually have.
|
||||
- Alternate peers still need a valid direct raw route back to the requester.
|
||||
- Swarm mode improves recovery from sender-path failure, but it does not change
|
||||
the underlying raw-packet size, airtime, or hop constraints.
|
||||
|
||||
## 8. High-Level Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as Original Sender
|
||||
participant P as Peer With Fragments
|
||||
participant N as Reachable Peers
|
||||
participant R as Requester
|
||||
|
||||
S->>R: Direct raw fragments (partial)
|
||||
Note over S,R: Original sender path stops responding
|
||||
R->>N: Raw swarm requests with exact missing fragments
|
||||
P->>R: Raw swarm availability response
|
||||
R->>P: Direct binary fetch request for missing subset
|
||||
P->>R: Direct raw fragments
|
||||
R->>R: Reassemble completed session
|
||||
```
|
||||
|
||||
## 9. Integration Points
|
||||
|
||||
- Voice-specific behavior is summarized in `docs/voice-mode-technical.md`.
|
||||
- Image-specific behavior is summarized in `docs/image-mode-technical.md`.
|
||||
- This document is the shared source of truth for swarm discovery and fallback
|
||||
semantics.
|
||||
@@ -2,10 +2,11 @@
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Voice mode uses a **two-plane architecture**:
|
||||
Voice mode uses a **two-plane architecture** with optional swarm-assisted
|
||||
recovery:
|
||||
|
||||
- **Control plane (text messages):**
|
||||
- `VE2:` voice envelope announces voice availability in chat.
|
||||
- `VE3:` voice envelope announces voice availability in chat.
|
||||
- **Control plane (raw binary request):**
|
||||
- Binary voice fetch request (same raw route as voice packets).
|
||||
- **Data plane (raw binary packets):**
|
||||
@@ -13,11 +14,13 @@ Voice mode uses a **two-plane architecture**:
|
||||
|
||||
This design avoids broadcasting full voice payloads to channels/rooms. Chat carries only metadata; audio is fetched on demand when user presses play.
|
||||
|
||||
Swarm fallback is documented in [Swarm Mode Technical Design](./swarm-mode-technical.md).
|
||||
|
||||
## 2. Key Modules
|
||||
|
||||
- `lib/utils/voice_message_parser.dart`
|
||||
- `VoicePacket` (legacy text + binary packet format)
|
||||
- `VoiceEnvelope` (`VE2`)
|
||||
- `VoicePacket` (binary direct-packet format)
|
||||
- `VoiceEnvelope` (`VE3`)
|
||||
- `VoiceFetchRequest` (binary)
|
||||
- `lib/screens/messages_tab.dart`
|
||||
- Capture/encode voice, cache encoded packets, send envelope only
|
||||
@@ -25,20 +28,20 @@ This design avoids broadcasting full voice payloads to channels/rooms. Chat carr
|
||||
- Reassembly/playback sessions
|
||||
- Outgoing session cache + deferred serving
|
||||
- `lib/providers/app_provider.dart`
|
||||
- Incoming routing for `VE2` and binary voice fetch requests
|
||||
- Incoming routing for `VE3`, binary voice fetch requests, and raw swarm control payloads
|
||||
- Handles raw packet ingestion
|
||||
- `lib/widgets/messages/voice_message_bubble.dart`
|
||||
- Play behavior (immediate play if complete, otherwise fetch + auto-play)
|
||||
- `lib/providers/messages_provider.dart`
|
||||
- Message-level voice detection (`VE2` + legacy `V:`)
|
||||
- Message-level voice detection (`VE3`)
|
||||
- `lib/services/message_storage_service.dart`
|
||||
- Persists `isVoice` and `voiceId`
|
||||
|
||||
## 3. Wire Formats
|
||||
|
||||
### 3.1 Voice Envelope (`VE2`)
|
||||
### 3.1 Voice Envelope (`VE3`)
|
||||
|
||||
Prefix: `VE2:` + colon-delimited compact payload (base36 numeric fields)
|
||||
Prefix: `VE3:` + colon-delimited compact payload (base36 numeric fields)
|
||||
|
||||
Fields:
|
||||
|
||||
@@ -46,21 +49,19 @@ Fields:
|
||||
- `mode` (base36): codec mode ID (`VoicePacketMode.id`)
|
||||
- `total` (base36): packet count (1..255)
|
||||
- `durS` (base36): estimated duration in seconds
|
||||
- `senderKey6` (string, 12 hex chars): sender public-key prefix (6 bytes)
|
||||
- `ts` (base36): unix timestamp seconds
|
||||
|
||||
`sid` is base36 on wire and expands to 8-hex internally.
|
||||
|
||||
Compact format:
|
||||
|
||||
```text
|
||||
VE2:{sid}:{mode}:{total}:{durS}:{senderKey6}:{ts}
|
||||
VE3:{sid}:{mode}:{total}:{durS}
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
VE2:a:1:4:4:aabbccddeeff:s44we8
|
||||
VE3:a:1:4:4
|
||||
```
|
||||
|
||||
### 3.2 Voice Fetch Request (binary)
|
||||
@@ -68,7 +69,7 @@ VE2:a:1:4:4:aabbccddeeff:s44we8
|
||||
Binary payload format:
|
||||
|
||||
```text
|
||||
[magic=0x72][sid:4B][flags:1B][requesterKey6:6B][ts:4B][missingCount:1B][missingIndices...]
|
||||
[magic=0x72][sid:4B][flags:1B][requesterKey6:6B][missingCount:1B][missingIndices...]
|
||||
```
|
||||
|
||||
### 3.3 Raw Voice Packet (data plane)
|
||||
@@ -77,10 +78,10 @@ Binary payload structure:
|
||||
|
||||
- Byte 0: magic `0x56` (`'V'`)
|
||||
- Bytes 1..4: session ID (4 bytes)
|
||||
- Byte 5: mode ID
|
||||
- Byte 6: packet index
|
||||
- Byte 7: total packets
|
||||
- Bytes 8..N: codec2 data
|
||||
- Byte 5: packet index
|
||||
- Bytes 6..N: codec2 data
|
||||
|
||||
Header is 6 bytes. Mode and total packet count come from the `VE3` envelope.
|
||||
|
||||
## 4. Outgoing Flow (Send)
|
||||
|
||||
@@ -88,27 +89,38 @@ Binary payload structure:
|
||||
2. Each chunk is codec2-encoded into `VoicePacket` objects.
|
||||
3. Packets are cached in `VoiceProvider` outgoing cache (TTL 15 min).
|
||||
4. Sender inserts local voice placeholder message (`isVoice=true`, `voiceId=sessionId`).
|
||||
5. Sender sends one envelope (`VE2`) through normal message path:
|
||||
5. Sender sends one envelope (`VE3`) through normal message path:
|
||||
- channel/room: `sendChannelMessage`
|
||||
- direct: `sendTextMessage`
|
||||
6. **No raw audio packets are sent during initial send.**
|
||||
|
||||
## 5. Incoming Routing
|
||||
|
||||
### 5.1 `VE2` envelope received
|
||||
### 5.1 `VE3` envelope received
|
||||
|
||||
`AppProvider` marks message as voice (`isVoice`, `voiceId`) and adds it to chat.
|
||||
`AppProvider` records sender identity from message metadata, registers the voice
|
||||
session envelope, marks the message as voice (`isVoice`, `voiceId`), and adds it to chat.
|
||||
|
||||
### 5.2 Binary voice fetch request received
|
||||
|
||||
`AppProvider` treats it as control-plane only:
|
||||
|
||||
- request is not added to chat
|
||||
- validates requester prefix match against sender metadata
|
||||
- resolves requester contact via key prefix
|
||||
- calls `voiceProvider.serveSessionTo(...)`
|
||||
|
||||
### 5.3 Raw packet received (`pushRawData`)
|
||||
### 5.3 Swarm control messages received
|
||||
|
||||
`AppProvider` also handles raw swarm control payloads:
|
||||
|
||||
- binary swarm requests advertise which voice fragments are still missing
|
||||
- binary swarm availability responses advertise which fragments a peer can relay
|
||||
- swarm control payloads arrive via `pushRawData` and are not added to chat history
|
||||
|
||||
Swarm semantics are shared with image mode and documented in
|
||||
[Swarm Mode Technical Design](./swarm-mode-technical.md).
|
||||
|
||||
### 5.4 Raw packet received (`pushRawData`)
|
||||
|
||||
`AppProvider.onRawDataReceived` parses `VoicePacket` binary and appends to session in `VoiceProvider`.
|
||||
|
||||
@@ -118,10 +130,15 @@ In `VoiceMessageBubble`:
|
||||
|
||||
- If session already complete: play immediately.
|
||||
- If incomplete/missing:
|
||||
1. Resolve sender contact (message sender prefix or `VE2.senderKey6` fallback)
|
||||
2. Send direct binary fetch request
|
||||
3. Show requesting state in UI
|
||||
4. Auto-play when session becomes complete
|
||||
1. Resolve sender contact from message sender metadata
|
||||
2. Prefer a direct fetch from the original sender if its raw route is healthy
|
||||
3. If the sender path does not respond, fan out a raw swarm request with the
|
||||
exact missing packet indices to reachable peers
|
||||
4. Wait up to 10 seconds for raw peer availability responses
|
||||
5. Send a direct binary fetch request to the best alternate peer for the
|
||||
missing subset it advertised
|
||||
6. Show requesting state in UI
|
||||
7. Auto-play when session becomes complete
|
||||
|
||||
If sender cannot be resolved or request cannot be sent, bubble remains and shows: **"Voice unavailable right now"**.
|
||||
|
||||
@@ -136,10 +153,13 @@ If sender cannot be resolved or request cannot be sent, bubble remains and shows
|
||||
|
||||
Serving prerequisites:
|
||||
|
||||
- session exists in cache
|
||||
- session exists in outgoing cache or already-received session state
|
||||
- `sendRawPacketCallback` configured
|
||||
- requester has direct path (`outPathLen >= 0`)
|
||||
|
||||
Received partial sessions can therefore act as relay sources during swarm
|
||||
recovery.
|
||||
|
||||
## 8. Persistence
|
||||
|
||||
`MessageStorageService` now stores and restores:
|
||||
@@ -167,13 +187,13 @@ Voice bubbles and Message Technical Details show an **estimated transmit time**
|
||||
The estimate is airtime-based (LoRa packet model), not file-duration-only:
|
||||
|
||||
- Source inputs:
|
||||
- `packetCount` and `durationMs` from `VE2` envelope, or
|
||||
- `packetCount` and `durationMs` from `VE3` envelope, or
|
||||
- numeric envelope values decoded from base36
|
||||
- actual received `VoicePacket.codec2Data.length` bytes when local session packets exist
|
||||
- `pathLen` from message metadata
|
||||
- current radio params from `deviceInfo`: `radioBw`, `radioSf`, `radioCr`
|
||||
- Per-packet payload model:
|
||||
- `meshHeader(2)` + `pathLen` + `voiceHeader(8)` + `codec2Bytes`
|
||||
- `meshHeader(2)` + `pathLen` + `voiceHeader(6)` + `codec2Bytes`
|
||||
- LoRa airtime:
|
||||
- standard symbol-time formula (preamble + payload symbols)
|
||||
- Mesh pacing/hops:
|
||||
@@ -192,9 +212,12 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
|
||||
## 11. Operational Constraints
|
||||
|
||||
- No firmware changes required.
|
||||
- On-demand fetch works only if sender app is online and has cached session.
|
||||
- On-demand fetch prefers the original sender, but a partial session can also be
|
||||
completed from alternate peers that already hold packets.
|
||||
- Raw return path needs a currently valid direct route to requester.
|
||||
- Voice capture is available on iOS and Android (`Platform.isIOS || Platform.isAndroid`).
|
||||
- Swarm discovery uses the same `cmdSendRawData` / `pushRawData` path as voice
|
||||
fetch and packet delivery.
|
||||
|
||||
### 11.1 Raw Binary Routing Semantics
|
||||
|
||||
@@ -206,26 +229,27 @@ Fallback defaults are used when radio params are unavailable: `SF10`, `BW250kHz`
|
||||
- only nodes on that path relay it;
|
||||
- it is **not** received by everyone in the mesh.
|
||||
|
||||
## 12. Backward Compatibility
|
||||
|
||||
- Legacy `V:` text packet parsing is still supported.
|
||||
- Message voice detection accepts `VE2` and legacy `V:` formats.
|
||||
|
||||
## 13. High-Level Sequence
|
||||
## 12. High-Level Sequence
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant A as Sender App
|
||||
participant M as Mesh Chat
|
||||
participant P as Peer With Packets
|
||||
participant N as Reachable Peers
|
||||
participant B as Receiver App
|
||||
|
||||
A->>A: Record + encode voice packets
|
||||
A->>A: Cache session packets (TTL 15m)
|
||||
A->>M: Send VE2 envelope
|
||||
M->>B: Deliver VE2
|
||||
A->>M: Send VE3 envelope
|
||||
M->>B: Deliver VE3
|
||||
B->>B: Render voice bubble (metadata only)
|
||||
B->>A: Send binary fetch request on Play
|
||||
A->>B: Stream raw VoicePacket packets
|
||||
A->>B: Stream raw VoicePacket packets (partial)
|
||||
Note over A,B: Sender path stops responding
|
||||
B->>N: Raw swarm requests with missing voice packet indices
|
||||
P->>B: Raw swarm availability response
|
||||
B->>P: Direct binary fetch request for missing subset
|
||||
P->>B: Stream remaining raw VoicePacket packets
|
||||
B->>B: Reassemble session
|
||||
B->>B: Auto-play when complete
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user