cuframes-composer/docs/en/user.md

# cfc-grid — user guide

> Audience: installation admin. Configures cameras, layouts, overlays,
> watches the RTSP stream on TV or browser. Does not edit C++ code.
>
> Developer extending Cell/Decoration/widget types → see
> [developer.md](developer.md).

## 1. What this is

**cfc-grid** is a CUDA compositor that combines N cameras into a single
RTSP stream `rtsp://192.168.88.23:554/cfc-grid` (1920×1080, H.264, NVENC).
Layout is selected automatically by motion (from Frigate) or manually via
ONVIF presets on the TV.

In addition to camera frames, the following are drawn on top:

- **Borders** — grey 2 px borders around each cell
- **Labels** — `name prio=N` caption in the corner of each camera
- **Detection boxes** — object rectangles from Frigate, tracking camera
  position when layout changes
- **MQTT overlays** — text fields bound to MQTT topics (temperature,
  statuses, chats)

## 2. Architecture in one sentence

```
cuframes-pub-* (per camera)
       ↓ shared VMM
   cfc-grid (composer) ── ZMQ control ──┐
       ↓ pipe (H.264)                   │
   cfc-grid-ffmpeg (relay) ─→ mediamtx ─┴─→ TV / VLC / Frigate / ...
                                ↑
                          ONVIF discovery from cctv-onvif
```

Frames go through cuframes zero-copy (a single VMM buffer shared between
publisher and composer). The composer takes the NV12 surface, resizes/blits
into its output, adds decorations, hands off to NVENC, NVENC writes H.264
into a pipe, `cfc-grid-ffmpeg` re-muxes the pipe → RTSP push to mediamtx.

## 3. Motion mode — the main operating mode

### 3.1 What happens

Each frame the composer:

1. Reads `last_motion_ms` for each camera (updated from Frigate MQTT
   `frigate/events`)
2. Treats as "active" any camera with `(now - last_motion_ms) < motion_ttl_ms`
   (default **45 seconds**)
3. Sorts active by `priority` (integer; higher = more important)
4. Selects template from `templates.json` by **best-fit** rule:
   minimal template with `nb_camera_cells >= active_count`
5. If template has more camera-cells than active — extra ones are filled
   with remaining drawable cameras from pool (by priority)
6. Applies **asymmetric hysteresis**: growth in active count switches
   layout immediately, shrinkage waits 3 seconds (to avoid flicker)

### 3.2 What "drawable" means

A camera is **excluded** from pool if its `cfc_source_state_t` =
`CONNECTING`, `DISCONNECTED` or `DEAD` (cuframes publisher silent for
longer than `dead_threshold_ms`, default 5 seconds).

`STALE` (frames arrive infrequently) — counts, last available frame is
drawn.

### 3.3 Manual override via PTZ

In the TV, ONVIF PTZ presets list template names (`tpl_1, tpl_3, tpl_4,
..., tpl_16`). Pressing `GotoPreset` or movement keys:

- Applies the chosen layout immediately
- **Freezes** motion-mode for 60 seconds
- After expiry — returns to auto mode

ContinuousMove (arrows): pan/tilt cycles through preset list, zoom-in =
`tpl_1` (full screen), zoom-out = `tpl_16` (4×4 grid).

## 4. templates.json — screen layout

### 4.1 Schema

```json
{
  "version": 1,
  "grid_cols": 8,
  "grid_rows": 8,
  "templates": [
    {
      "name": "tpl_N",
      "_desc": "Description",
      "priority": 0,
      "cells": [
        {
          "col": 0, "row": 0,
          "cs": 4, "rs": 4,
          "role": "camera",
          "order": 0
        },
        {
          "col": 6, "row": 0,
          "cs": 2, "rs": 6,
          "role": "widget",
          "widget": "temp_chart"
        }
      ]
    }
  ]
}
```

**Grid: 8×8 microcells** (240×135 px each = 16:9 on 1920×1080 output).
Any N×N square of microcells is also 16:9.

| Field | Value |
|---|---|
| `col`, `row` | Top-left corner of cell in microcells (0..7) |
| `cs`, `rs` | Size in microcells |
| `role` | `camera` or `widget` |
| `order` | For `camera`: placement order for active cameras (0 = main, usually the largest cell) |
| `widget` | For `widget`: placeholder name (caption text) |

### 4.2 Best-fit selection

The composer selects a template for current active count:

```
candidates = [t for t in templates if t.nb_camera_cells >= n_active]
pick = min(candidates, key=lambda t: t.nb_camera_cells - n_active)
# On tie: higher priority wins
```

If active count exceeds the largest template's cells — the largest is
taken, extra cameras are dropped (lowest priority).

### 4.3 Built-in templates

By default 9 templates in `/opt/templates.json`:

| Name | Cells | Description |
|---|---|---|
| `tpl_1` | 1 cam | One camera fullscreen |
| `tpl_3` | 3 cam + 2 widgets | Main 1440×810 + 2 previews + 2 widget areas |
| `tpl_4` | 4 cam | Quad 2×2, 960×540 each |
| `tpl_5` | 5 cam + 1 widget | Main + 4 previews stacked right + widget bottom |
| `tpl_6` | 6 cam + 1 widget | Main + 3 right + 2 bottom + widget |
| `tpl_7` | 7 cam + 1 widget | Main + 3 right + 3 bottom + widget |
| `tpl_8` | 8 cam (1+3+4) | Main + 3 right + 4 bottom row |
| `tpl_9` | 9 cam + 2 widgets | 3×3 mains + widget strips |
| `tpl_16` | 16 cam | 4×4 grid, 480×270 each |

Details — see `docker/templates.json` in the repo.

### 4.4 Adding your own template

1. Open `docker/templates.json` (or mounted override)
2. Add a block to `"templates": [...]` per schema above
3. Restart cfc-grid (or invoke ZMQ when hot-reload is available — Phase 12)

### 4.5 Coordinate math

Each microcell = `1920/8 = 240 px` wide, `1080/8 = 135 px` tall (16:9).
Cell `{col=2, row=4, cs=4, rs=2}`:

- pixel x = `2 * 240 = 480`
- pixel y = `4 * 135 = 540`
- pixel w = `4 * 240 = 960`
- pixel h = `2 * 135 = 270`

Aspect cell = `cs/rs * 16/9`. For 16:9 cells: **cs == rs**.

## 5. mqtt_overlays.json — text overlays from MQTT

### 5.1 Schema

```json
{
  "version": 1,
  "overlays": [
    {
      "id": "temp_outside",
      "topic": "zigbee2mqtt/Outdoor temp sensor",
      "json_field": "temperature",
      "format": "%+.1f°C",
      "anchor": "right-bottom",
      "margin_x": 32, "margin_y": 24,
      "pixel_size": 32,
      "color": [255, 255, 255],
      "alpha": 230,
      "bg_alpha": 160,
      "bg_y": 16, "bg_u": 128, "bg_v": 128,
      "bg_pad": 10,
      "placeholder": "—",
      "font_path": "/fonts/DejaVuSans-Bold.ttf"
    }
  ]
}
```

| Field | Description |
|---|---|
| `id` | Unique overlay identifier (used by ZMQ for lookup) |
| `topic` | MQTT topic to subscribe (via `cctv-mosquitto`) |
| `json_field` | If payload is JSON — field name to extract; **empty = raw payload as string** |
| `format` | `printf`-style for the formatted value (e.g. `"%+.1f°C"`, `"%s"`) |
| `anchor` | Positioning anchor: `right-bottom`, `right-top`, `left-bottom`, `left-top`, `center` |
| `margin_x`, `margin_y` | Offset from nearest screen edge (px) |
| `pixel_size` | Font size in pixels |
| `color` | RGB text color |
| `alpha` | Overall text opacity (0..255) |
| `bg_alpha` | Background opacity (0..255); **0 = no background** |
| `bg_y`, `bg_u`, `bg_v` | BT.709 limited-range background color; default black |
| `bg_pad` | Background padding around text (px) |
| `placeholder` | What to show before first MQTT message; empty = "—" |
| `font_path` | Path to font (.ttf/.otf) inside container |

### 5.2 Supported symbols

Font `DejaVuSans-Bold.ttf` — standard from `fonts-dejavu` package.
**Covers Basic Multilingual Plane** (latin, cyrillic, basic symbols),
including:

- `❯` (U+276F), `✎` (U+270E), `➤` (U+27A4), `→` (U+2192)
- `★` (U+2605), `▶` (U+25B6), `✉` (U+2709)

**Emoji from Supplementary Multilingual Plane (>U+10000)** — e.g.
`🗣` (U+1F5E3), `🤖` (U+1F916), `💬` (U+1F4AC) — **are not rendered**:
font lacks those glyphs. Placeholder square is drawn instead.

To add color emoji — bind Noto Color Emoji and extend renderer for
COLR/CPAL/SBIX (see developer.md).

### 5.3 Examples

**Plain number**: payload = `"23.5"` (raw), `json_field: ""`,
`format: "%s°"`

**JSON with field**: payload = `{"temperature": 23.5, "humidity": 45}`,
`json_field: "temperature"`, `format: "%+.1f°C"`

**Multiple overlays** stacked top-right: first with `margin_y: 24`,
second `margin_y: 72`, third `margin_y: 120`.

### 5.4 How to add

1. Open `docker/mqtt_overlays.json` (or mounted override)
2. Add block to `"overlays": [...]` array
3. Restart cfc-grid

Hot-reload via ZMQ — Phase 12 (`reload_overlays` verb).

## 6. Composer CLI flags

In compose override `docker/cctv/cuframes-composer/docker-compose.override.yml`:

```yaml
command:
  - "--out=/out/grid.h264"          # named pipe for ffmpeg-relay
  - "--fps=25"
  - "--bitrate=6000"                # kbps
  - "--width=1920"
  - "--height=1080"
  - "--intra-refresh"               # instead of IDR-bursts (low-latency)
  - "--control=tcp://0.0.0.0:5599"  # ZMQ control plane
  - "--mqtt=cctv-mosquitto:1883"    # MQTT for health publishing
  - "--mqtt-instance=cfc-grid"
  - "--mqtt-user=composer"
  - "--mqtt-pass=${COMPOSER_MQTT_PASSWORD}"

  # Sources (cameras) — repeatable
  - "--source=cam-parking,frigate=parking_overview,priority=100,zones=parking_zone:canopy:private_area"
  - "--source=cam-back_yard,frigate=back_yard,priority=70"
  # ...

  - "--motion-mode"                  # enable auto-layout
  - "--motion-ttl=45000"             # ms

  - "--templates=/opt/templates.json"
  - "--mqtt-overlays=/opt/mqtt_overlays.json"

  # Frigate motion-driver and detection boxes
  - "--frigate-mqtt=cctv-mosquitto:1883"
  - "--frigate-topic=frigate/events"
  - "--detection-cell=parking,parking_overview,0,0,960,540,640,480,parking_zone:canopy:private_area"
```

### 6.1 `--source` syntax

```
--source=<cuframes_key>,frigate=<camera>,priority=<N>[,zones=<z1>:<z2>:...]
```

- `cuframes_key` — name of cuframes publisher (e.g. `cam-parking`)
- `frigate=NAME` — camera name in Frigate (for motion event matching)
- `priority=N` — integer, higher = more important
- `zones=...` — optional whitelist of Frigate zones; motion counts
  only if `event.current_zones` intersects the list (filters street flood)

### 6.2 `--detection-cell` syntax

```
--detection-cell=<key>,<frigate_camera>,<x>,<y>,<w>,<h>,<detect_w>,<detect_h>[,zones]
```

- `key` — arbitrary overlay identifier for logs
- `frigate_camera` — Frigate name (for `event.camera` matching)
- `x,y,w,h` — initial geometry (composer recalculates dynamically)
- `detect_w,detect_h` — Frigate detector resolution (e.g. 640×480)
- `zones` — bbox whitelist

## 7. ZMQ control plane

Default endpoint: `tcp://192.168.88.23:5599`. All verbs — JSON request/reply.

### 7.1 Verb list

| Command | Parameters | What it does |
|---|---|---|
| `ping` | — | Health check |
| `health` | — | `{total, active, stale, dead}` over pool |
| `set_text` | `id, text, r, g, b, x, y, visible` | Update text overlay (for CLI `--text=...`) |
| `set_visible` | `id, visible` | Hide/show overlay |
| `list_overlays` | — | List of overlays |
| `set_layout` | `name` | Apply named template (manual override 60s in motion-mode) |
| `list_layouts` | — | List of templates with cells |
| `get_layout` | — | Name of current template |
| `set_motion_mode` | `on, ttl_ms` | Toggle motion mode |
| `get_motion_mode` | — | Motion mode state |
| `get_template` | `name` | Full template JSON |
| `reload_templates` | `path?` | Reload templates from file |

### 7.2 Example

```bash
python3 <<EOF
import zmq, json
s = zmq.Context().socket(zmq.REQ)
s.connect("tcp://192.168.88.23:5599")
s.send_json({"cmd": "list_layouts"})
print(json.dumps(s.recv_json(), indent=2, ensure_ascii=False))
EOF
```

Plain `nc` **does not work** — REP socket expects ZMQ wire-protocol.
Use `zmq` Python/Go/JS or `mosquitto_pub` via RPC-bridge (Phase 12).

## 8. ONVIF and PTZ

`cctv-onvif` service binds to host network, responds to WS-Discovery
multicast (239.255.255.250:3702) and SOAP requests over HTTP `:8085`.

### 8.1 Adding to TV

In the client (TV / IP-CamViewer / Synology / Frigate):

- ONVIF host: `192.168.88.23`
- Port: `8085`
- User / Password: empty (auth not configured)

WS-Discovery in LAN 192.168.88.0/24 finds device `cfc-grid (Goldix)`.
RTSP URL is automatic — `rtsp://192.168.88.23:554/cfc-grid`.

### 8.2 PTZ presets

List (`GetPresets`): `tpl_1, tpl_3, tpl_4, tpl_5, tpl_6, tpl_7, tpl_8,
tpl_9, tpl_16`.

GotoPreset(name) → composer applies template + freezes motion-mode for
60 seconds → auto-return.

### 8.3 PTZ movement (ContinuousMove)

| Command | Action |
|---|---|
| Pan right / Tilt down | Next template in list |
| Pan left / Tilt up | Previous |
| Zoom in (+) | `tpl_1` (full screen) |
| Zoom out (−) | `tpl_16` (4×4 grid) |

## 9. Where to view RTSP

| Method | URL |
|---|---|
| VLC / mpv / ffplay | `rtsp://192.168.88.23:554/cfc-grid` |
| Browser (HLS) | `http://192.168.88.23:8888/cfc-grid` |
| WebRTC | `http://192.168.88.23:8889/cfc-grid` |
| OBS / FFmpeg input | `rtsp://192.168.88.23:554/cfc-grid` |
| ONVIF clients | via WS-Discovery (see §8) |

## 10. Known limitations

- **Color emoji not rendered** (needs Noto Color Emoji + COLR/CPAL
  support in text renderer — Phase 12)
- **Hot-reload `mqtt_overlays.json`** — no ZMQ verb, requires cfc-grid
  restart
- **Per-overlay broker** — all MQTT overlays use the common broker
  (set as `--mqtt`); subscribing to a foreign broker separately —
  not yet
- **Widget rendering** — placeholder (dark rect + label), real widgets
  (graph, chat) — Phase 12+

## 11. FAQ

**Q: TV shows old layout names (`quad`, `dual_horizontal`). What to do?**

A: TV cached ONVIF presets. In the client, delete the camera and add
again — it will re-read `GetPresets` with current names.

**Q: Parking camera is DEAD but logs show active=3. Why?**

A: `cfc_composer_get_health` shows **pool-wide** state, but motion-active
counts by `last_motion_ms` independently of source state. DEAD is excluded
in `is_camera_drawable()` inside `compose_motion_relayout`.

**Q: Pressed PTZ on TV, layout switched, but reverted after a minute.**

A: This is by design — `set_layout` in motion-mode freezes auto for
60 seconds (`manual_override_duration_ms`). To pin layout permanently —
disable motion-mode entirely via ZMQ:

```json
{"cmd": "set_motion_mode", "on": 0}
```

**Q: I want text overlay backgrounds in different colors.**

A: `bg_y/bg_u/bg_v` fields accept BT.709 limited-range. For red — Y≈80,
U≈90, V≈240. For cyan — Y≈170, U≈170, V≈100. Calculator:
https://www.rapidtables.com/convert/color/rgb-to-yuv.html
(use BT.709 limited).

**Q: With motion on 5 cameras, the layout doesn't change, stays at quad.**

A: Check `docker logs cfc-grid | grep "loaded N templates"` — must be ≥5
(should include `tpl_5..tpl_8`, `tpl_9`, `tpl_16`). If not — templates.json
didn't load (check syntax via `jq` or `python3 -m json.tool`).

**Q: Frigate bbox is drawn on the wrong camera.**

A: Check `--detection-cell` — `frigate_camera` must match
`event.after.camera`. Composer binds detbox-overlay to pool-entry via
`frigate_camera` (see `cfc_composer::pool::by_frigate_camera`).

## 12. Next

- [developer.md](developer.md) — internals, extension
- [operations.md](operations.md) — build, deploy, troubleshooting
- repo README: brief overview