V1.0: Class 1 fax — real-world 4-page send to external number confirmed
Core features: - Class 1 T.30 protocol: full send/receive implementation - HDLC: DLE-stuffing, FCS strip, USR5637 bit-reversal handling - T.4 MH encoder/decoder (1728px A4 standard) - Document pipeline: PDF (Ghostscript), PNG, TIFF input - Width clamping: US Letter 1734px → 1728px fax standard - Cover page: CJK rasterization (TW/CN/JP/EN), TIFF + HTML output - OCR verification: Tesseract 5 with eng+chi_tra, CJK space-tolerant - API server (axum): health, send, jobs, cover, retry, cancel - Background worker: auto-poll queue, speed fallback, retry policy - Modem detection, pool management Real-world test results (2026-07-23): - V90 → 25153038: 4 pages, V.17 12000 bps, 2:33 ✅ - USR5637 → 25153038: 4 pages, V.17 12000 bps, 2:26 ✅ - Both faxes confirmed received on remote machine Tested: loopback (100% pixel match), multi-page, all input formats, cover pages, OCR verify, API endpoints, worker processing. 13 unit tests pass, 0 new clippy warnings.
This commit is contained in:
Warren
@@ -0,0 +1,225 @@
|
||||
# Telfax Class 1 Fax Design Document
|
||||
|
||||
## Overview
|
||||
|
||||
Telfax is a standalone fax server built in Rust, implementing ITU-T T.30 fax protocol via Class 1 modem control. This document covers the architecture, protocol implementation, and key design decisions for reliable real-world fax transmission over POTS lines.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Module Structure
|
||||
|
||||
```
|
||||
telfax/src/
|
||||
├── fax/
|
||||
│ ├── class1/
|
||||
│ │ ├── send.rs — TX: T.30 state machine, HDLC, page data
|
||||
│ │ ├── recv.rs — RX: T.30 state machine, page data decode
|
||||
│ │ └── session.rs — Shared T.30 event types
|
||||
│ ├── class2/ — Class 2 (modem-managed T.30)
|
||||
│ ├── hdlc.rs — HDLC framing, DLE-stuffing, FCS, bit-reversal
|
||||
│ ├── t4.rs — T.4 G3 decode (MH → pixels)
|
||||
│ ├── encoder/
|
||||
│ │ └── mh.rs — T.4 Modified Huffman encoder (pixels → MH)
|
||||
│ └── negotiate.rs — DIS/DCS capability negotiation
|
||||
├── document/
|
||||
│ ├── pdf.rs — PDF → TIFF-F via Ghostscript
|
||||
│ ├── convert.rs — Image → fax format (width clamp, threshold)
|
||||
│ ├── cover.rs — Cover page generation (CJK rasterization)
|
||||
│ └── tiff.rs — TIFF-F writer (MH compressed output)
|
||||
├── modem/
|
||||
│ ├── driver.rs — Serial port I/O (ATSPEED, read_until)
|
||||
│ ├── at.rs — AT command channel
|
||||
│ └── pool.rs — Multi-modem pool management
|
||||
├── ocr/
|
||||
│ └── mod.rs — Tesseract OCR wrapper (cover verification)
|
||||
├── api/
|
||||
│ └── routes.rs — HTTP API (axum): send, jobs, cover, health
|
||||
└── worker/
|
||||
└── executor.rs — Background job processor (queue → modem)
|
||||
```
|
||||
|
||||
## Class 1 Protocol Implementation
|
||||
|
||||
### T.30 State Machine
|
||||
|
||||
Class 1 offloads HDLC framing and modulation to the host software. The modem acts as a raw data pump.
|
||||
|
||||
#### Sender (send.rs)
|
||||
|
||||
```
|
||||
Dial → Wait CONNECT → Escape → FRH (receive DIS)
|
||||
→ Parse DIS capabilities → Build DCS
|
||||
→ FTH=3 (send DCS) → FTM=n (send TCF)
|
||||
→ FRH (receive CFR/FTT)
|
||||
→ [For each page]:
|
||||
FTM=n (send page data) → FTH=3 (send MPS/EOP)
|
||||
→ FRH (receive MCF)
|
||||
→ FTH=3 (send DCN) → Hangup
|
||||
```
|
||||
|
||||
#### Receiver (recv.rs)
|
||||
|
||||
```
|
||||
ATA → Wait RING → Answer → Wait DIS/DCS
|
||||
→ FRM=n (receive page data) → FTH=3 (send MCF)
|
||||
→ [For each page]:
|
||||
FRH (receive MPS/EOP/DCN)
|
||||
→ FTH=3 (send MCF if more pages)
|
||||
→ Hangup
|
||||
```
|
||||
|
||||
### HDLC Layer (hdlc.rs)
|
||||
|
||||
The modem strips HDLC flags and bit-stuffing on FRH=3. We handle:
|
||||
|
||||
1. **DLE-stuffing**: Per T.31, 0x10 → 0x10 0x10. `dle_unstuff()` reverses this.
|
||||
2. **FCS stripping**: USR5637 outputs the 2-byte CRC-CCITT as part of frame data. `parse_modem_hdlc()` strips it.
|
||||
3. **Bit reversal**: USR5637 reverses ALL bytes during FTH=3 transmission. The receiver must reverse them back to get correct frame content. `parse_modem_hdlc()` applies `reverse_bits()` to all bytes.
|
||||
4. **FCF masking**: Bit 7 of the FCF byte is an address extension bit set by the calling station. `parse_modem_hdlc()` masks it with `0x7F`.
|
||||
|
||||
### Bit Reversal Details
|
||||
|
||||
USR5637 modem reverses bit order (MSB↔LSB) on **every byte** during FTH=3. This is not documented in the modem manual but was discovered during testing.
|
||||
|
||||
```
|
||||
Original: 0xFF → 0xFF (all ones, no change)
|
||||
0x01 → 0x80
|
||||
0x03 → 0xC0
|
||||
0x28 → 0x14
|
||||
```
|
||||
|
||||
The `reverse_bits()` function pre-reverses bytes before FTH=3 transmission. On FRH=3 receive, the modem does NOT reverse, so received data is already in correct orientation — but the *sender's* FTH=3 reversal must be undone by the parser.
|
||||
|
||||
### T.4 Page Data
|
||||
|
||||
#### Encoding (send)
|
||||
|
||||
1. Grayscale pixels → threshold to B&W (128/255 threshold)
|
||||
2. Each scan line → MH run-length encoding via `MhEncoder`
|
||||
3. MH-encoded bytes → DLE-stuff → transmit via FTM=n
|
||||
|
||||
#### Decoding (recv)
|
||||
|
||||
1. Raw bytes from FRM=n → DLE-unstuff
|
||||
2. MH bitstream → run-length decode → pixel array
|
||||
3. `find_first_eol_bit()` locates the EOL marker (11+0...01 pattern)
|
||||
4. `transitions_to_pixels()` converts alternating run lengths to pixel data
|
||||
|
||||
### Data Rates
|
||||
|
||||
| Modulation | FRM/FTM value | Speed | Use |
|
||||
|---|---|---|---|
|
||||
| V.21 (300 baud) | 3 | HDLC signaling | DIS/DCS/MCF/etc. |
|
||||
| V.27ter (4800) | 5 | Page data (low) | Fallback |
|
||||
| V.27ter (9600) | 6 | Page data (default) | Standard |
|
||||
| V.29 (9600) | 7 | Page data | Alternate |
|
||||
| V.29 (14400) | 8 | Page data (fast) | High speed |
|
||||
| V.17 (12000) | 145 | Training (TCF) | Negotiation |
|
||||
| V.17 (12000) | 146 | Page data | Fastest stable |
|
||||
|
||||
### FCS Computation
|
||||
|
||||
CRC-CCITT polynomial: x^16 + x^12 + x^5 + 1 (0x1021)
|
||||
- Initial value: 0xFFFF
|
||||
- Reflected input/output
|
||||
- Final XOR: 0xFFFF
|
||||
- FCS = complement of CRC
|
||||
|
||||
## Modem Compatibility
|
||||
|
||||
### Tested Modems
|
||||
|
||||
| Modem | Class 1 | Class 2 | Notes |
|
||||
|---|---|---|---|
|
||||
| USR5637 | ✅ | ✅ | Bit reversal on FTH=3. FCS not stripped. Reliable. |
|
||||
| V90 CX93001 | ✅ | ❌ | Class 2 FDT returns OK but internal T.30 fails (FHNG:025). Use Class 1. |
|
||||
|
||||
### Modem Detection
|
||||
|
||||
```bash
|
||||
AT+GMI → Manufacturer
|
||||
AT+GMM → Model
|
||||
ATI3 → Model (fallback)
|
||||
AT+FCLASS=? → Class support
|
||||
AT+FTM=? → TX rate capabilities
|
||||
AT+FRM=? → RX rate capabilities
|
||||
```
|
||||
|
||||
## Document Pipeline
|
||||
|
||||
### Input Formats
|
||||
|
||||
| Format | Path | Conversion |
|
||||
|---|---|---|
|
||||
| TIFF (existing) | Direct | Parse pages, use as-is |
|
||||
| PDF | pdf.rs | Ghostscript → TIFF-G3 → pixel extraction |
|
||||
| PNG/JPEG | convert.rs | image crate → grayscale → threshold → width clamp |
|
||||
|
||||
### Width Handling
|
||||
|
||||
Standard fax width: **1728 pixels** (A4 at 204 DPI).
|
||||
|
||||
- PDFs at US Letter (8.5") → 1734px at 204 DPI → clamped to 1728 via `width.min(1728)`
|
||||
- Images wider than 1728 → scaled down proportionally
|
||||
- Images narrower than 1728 → no scaling
|
||||
|
||||
### Cover Pages
|
||||
|
||||
Generated via `cover.rs`:
|
||||
- Rasterizes text to grayscale bitmap using PingFang CJK font
|
||||
- Supports Traditional Chinese, Simplified Chinese, Japanese, English
|
||||
- Output: 1728×2291 pixels (standard fax page)
|
||||
- Formats: TIFF (for fax) or HTML (for preview)
|
||||
|
||||
### OCR Verification
|
||||
|
||||
`verify-cover` command:
|
||||
1. Generate cover page TIFF
|
||||
2. Run Tesseract OCR (eng + chi_tra) with PSM 3
|
||||
3. Strip whitespace from OCR output (CJK characters get spaces inserted)
|
||||
4. Compare fields: FACSIMILE, TO, FROM, DATE, PAGES, SUBJECT, NOTES
|
||||
|
||||
## API Server
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | /api/health | Server health + modem status |
|
||||
| POST | /api/fax/send | Queue a fax job |
|
||||
| GET | /api/fax/jobs | List all jobs |
|
||||
| GET | /api/fax/jobs/{id} | Get job details |
|
||||
| PUT | /api/fax/jobs/{id}/cover | Update cover page fields |
|
||||
| POST | /api/fax/jobs/{id}/retry | Retry a failed job |
|
||||
| DELETE | /api/fax/jobs/{id} | Cancel a job |
|
||||
|
||||
### Worker
|
||||
|
||||
Background `FaxWorker` polls the queue every 5 seconds:
|
||||
1. Dequeue pending job
|
||||
2. Load document (PDF/TIFF/image)
|
||||
3. Prepend cover page if cover fields present
|
||||
4. Open modem, negotiate, transmit
|
||||
5. Handle retries with speed fallback (V.17 → V.29 → V.27ter)
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
1. **Class 1 over Class 2**: Class 1 gives full control over T.30 protocol. Class 2 depends on modem firmware which varies wildly (V90 Class 2 is broken).
|
||||
|
||||
2. **Pre-reverse bytes for FTH=3**: Instead of relying on modem firmware, we pre-reverse all bytes before FTH=3 to compensate for USR5637's undocumented bit reversal.
|
||||
|
||||
3. **Split parser**: `parse_hdlc_payload()` (clean) vs `parse_modem_hdlc()` (modem-specific with FCS strip + bit reversal). Tests use the clean parser; production code uses the modem parser.
|
||||
|
||||
4. **Width clamping at 1728**: Rather than failing on slightly wider documents, clamp to standard fax width. Minor clipping at edges is acceptable.
|
||||
|
||||
5. **MH encoding always**: Even though TIFF-G3 supports MR and MMR, we use MH (Group 3) for maximum compatibility with all receiving fax machines.
|
||||
|
||||
## Test Hardware
|
||||
|
||||
| Component | Detail |
|
||||
|---|---|
|
||||
| Modem 1 | Agere USR5637 USB, phone: 25289852 |
|
||||
| Modem 2 | CONEXANT V90 CX93001 USB, phone: 27486656 |
|
||||
| External fax | Phone: 25153038 |
|
||||
| Lines | Two separate POTS lines |
|
||||
| Host | macOS (Apple Silicon) |
|
||||
Reference in New Issue
Block a user