smyrgeorge/qrt: QRT: A screen-to-camera data transfer protocol

This project explores data transfer using a screen-to-camera approach. The idea is simple: encode information into a
sequence of QR codes, display them as a video on a screen, and then use a camera to capture and decode the video frames
to retrieve the original data.

This method enables:

• Air-gapped data transfer between devices
• One-way communication without network connectivity
• Visual data encoding that can be recorded and processed later

The project includes both encoding (generating QR code videos) and decoding (extracting data from captured
videos).

The QR Transfer (QRT) protocol defines how data is structured within each QR code frame. It provides reliable data
transfer with integrity verification and support for out-of-order frame reassembly.

All frames use | as field separator and start with a magic header QRT1 (version 1).

Header Frame (HDR) – First frame, contains transfer metadata:

Data Frame (DAT) – Contains actual payload chunks:

Footer Frame (END) – Last frame, marks end of transfer:

• Session tracking: Multiple concurrent transfers distinguished by session ID
• Ordering: Sequence numbers enable out-of-order reassembly
• Integrity: CRC32 checksums at chunk and message level
• Completeness: Header declares expected chunks, footer confirms end
• Versioning: Magic header includes version for future compatibility

Generates a QR code video from text input (single stream):

1. Text Input: Accepts text directly or generates Lorem Ipsum placeholder text
2. Protocol Framing: Wraps data in QRT protocol with header, data frames, and footer
3. QR Generation: Creates a QR code image for each protocol frame
4. Video Assembly: Uses ffmpeg to combine all QR code frames into a single video file

The output is an MP4 video where each frame displays a QR code containing a protocol-wrapped chunk.

Generates a video with multiple QR codes per frame for high-throughput transfer:

1. Parallel Streams: Displays 1-12 QR codes simultaneously
2. Grid Layout: Automatic arrangement in optimal grid
3. Increased Throughput: Up to 400x+ faster than single-stream baseline
4. Frame Repeat: Optional frame repetition for improved reliability

Supported grid configurations:

• 1 stream: 1×1 (baseline)
• 2 streams: 2×1
• 4 streams: 2×2 (recommended)
• 5-6 streams: 3×2
• 7-8 streams: 4×2
• 9 streams: 3×3
• 10 streams: 5×2
• 11-12 streams: 4×3 (maximum)

See BITRATE.md for detailed throughput analysis.

Decodes a captured video and extracts the original message:

1. Frame Extraction: Reads video frames using OpenCV
2. QR Detection: Detects and decodes QR codes from each frame using pyzbar
3. Deduplication: Filters duplicate frames (same QR captured multiple times)
4. Integrity Check: Verifies CRC32 checksums for each chunk
5. Reassembly: Orders chunks by sequence number and reconstructs the message
6. Verification: Validates final message checksum against header/footer

The decoder handles real-world conditions: camera-captured videos with motion blur, duplicates, and potentially missing
frames.

Decodes QR codes directly from a live camera feed:

1. Camera Selection: Lists available cameras including iPhone (via Continuity Camera on macOS)
2. Live Detection: Continuously scans for QR codes in the camera feed
3. Visual Feedback: Displays bounding boxes around detected QR codes with status
4. Progress Tracking: Shows real-time progress bar and statistics
5. Session Management: Automatically detects new transfer sessions

Features:

• Visual overlay showing session ID, progress, and chunk count
• Color-coded QR detection (green=valid, yellow=header, magenta=footer, red=error)
• Keyboard controls for reset and save operations

• Python 3.10+
• ffmpeg installed and available in PATH
• zbar library (for QR decoding)

On macOS:

python3 -m venv .venv
source .venv/bin/activate

pip install -r requirements.txt

Generate a QR video with Lorem Ipsum:

python src/generate_qr_video.py

Generate with custom text:

python src/generate_qr_video.py -t "Your text to encode here"

Encoder options:

Generate a 4-stream (2×2 grid) video:

python src/generate_qr_video_multistream.py

Generate with 8 streams for high throughput:

python src/generate_qr_video_multistream.py -s 8 -c 500 -f 10

Generate with 12 streams for maximum throughput:

python src/generate_qr_video_multistream.py -s 12 -c 400 -f 10 -r 1920x1080

Multi-stream encoder options:

Decode a captured video:

python src/decode_qr_video.py captured_video.mov

Save decoded message to file:

python src/decode_qr_video.py captured_video.mov -o decoded.txt

Decoder options:

List available cameras:

python src/decode_realtime.py -l

Start real-time decoder (interactive camera selection):

python src/decode_realtime.py

Use specific camera (e.g., iPhone via Continuity Camera):

python src/decode_realtime.py -c 1

Keyboard controls during real-time decoding:

Real-time decoder options: