diff --git a/README.md b/README.md index 06e49e7..2e679f2 100644 --- a/README.md +++ b/README.md @@ -1,55 +1,105 @@ # cuframes-composer -Стандалонный композитор-демон для multi-source видео grid через CUDA + NVENC + RTSP. +CUDA-композитор multi-source видео в один H.264 RTSP-поток с +авто-раскладкой по motion и runtime-управлением через ONVIF/ZMQ. -Заменяет монолитный ffmpeg-конвейер (`ffmpeg + vf_cuda_grid` фильтр) для случаев, когда нужно: +Заменяет монолитный ffmpeg-конвейер (`ffmpeg + vf_cuda_grid` фильтр) для +случаев, когда нужно: -- Поток продолжает работать при потере любого числа источников (graceful degradation) -- Композитор сам управляет частотой кадров и обработкой ошибок без зависимости от семантики ffmpeg-демухера -- Минимум перемещений данных: zero-copy CUDA от источника `cuframes` напрямую в NVENC +- Поток продолжает работать при потере любого числа источников + (graceful degradation, blank cells вместо crash'а) +- Композитор сам управляет частотой кадров без зависимости от ffmpeg-демухера +- Минимум перемещений данных: zero-copy CUDA от cuframes-publisher до NVENC +- Auto-layout по движению (Frigate-driven), без оператора +- Управление с TV через ONVIF PTZ + +## Что внутри + +- **CUDA-композитор** (C++ ООП-ядро + extern "C" ABI, Phase 11b) +- **Auto-layout** с asymmetric hysteresis + best-fit selection +- **PTZ-override** через ONVIF (с auto-возвратом в motion-mode) +- **MQTT-driven text overlays** (температура, статусы, etc.) +- **Detection box overlay** от Frigate, следует за камерой при смене layout +- **ZMQ control plane** для runtime-управления (set_layout, set_text, ...) +- **NVENC** через `dlopen` (LGPL-совместимая интеграция) + +## Документация + +| | Русский | English | +|---|---|---| +| Для пользователя | [docs/ru/user.md](docs/ru/user.md) | [docs/en/user.md](docs/en/user.md) | +| Для разработчика | [docs/ru/developer.md](docs/ru/developer.md) | [docs/en/developer.md](docs/en/developer.md) | +| Operations / deploy | [docs/ru/operations.md](docs/ru/operations.md) | [docs/en/operations.md](docs/en/operations.md) | ## Статус -**Phase 1 — MVP.** В разработке. Не для боевой эксплуатации. +**Phase 11b — production.** Развёрнут на R9-88.23 в составе CCTV-стека. -См. [дизайн-документ](https://git.goldix.org/gx/cuframes/raw/branch/main/docs/DESIGN-composer-daemon.md) для архитектурных решений и поэтапного плана. +См. [STATE.md](../../localhost-infra/STATE.md) для текущего состояния +infra и git-history `main` для эволюции по фазам. ## Зависимости -- [cuframes](https://git.goldix.org/gx/cuframes) — библиотека zero-copy передачи кадров. Подключена как git submodule. -- [nv-codec-headers](https://github.com/FFmpeg/nv-codec-headers) — MIT-licensed заголовки NVENC API. Подключена как git submodule. Сама библиотека `libnvidia-encode.so` грузится через `dlopen` при старте (это даёт LGPL-совместимость — см. дизайн-документ часть 1.6). -- CUDA Toolkit 12.x+ (для cuda runtime и компиляции) -- NVIDIA драйвер 525+ (для NVENC и `cuMemCreate` POSIX FD) -- Linux 64-bit (POSIX shm, SCM_RIGHTS) +- [cuframes](https://git.goldix.org/gx/cuframes) — zero-copy frame + delivery от RTSP-publisher'а к composer'у. Подключена как git submodule. +- [nv-codec-headers](https://github.com/FFmpeg/nv-codec-headers) — + MIT-licensed заголовки NVENC API (submodule). `libnvidia-encode.so` + грузится через `dlopen` в runtime для LGPL-совместимости. +- CUDA Toolkit 12.x+ (cudart, nvcc, driver API) +- NVIDIA driver 525+ (NVENC, cuMemCreate POSIX FD) +- FreeType, libpng, libzmq, libjson-c, libmosquitto, libavformat/avcodec/avutil +- Linux 64-bit (POSIX shm, SCM_RIGHTS, named pipes) -Дополнительно по фазам: -- Phase 3: `libfreetype` (текст), `lodepng` через submodule (PNG-декодирование) -- Phase 4: `libzmq` (управление) - -## Сборка +## Quick start (host build) ```bash -git clone --recursive git@git.goldix.org:gx/cuframes-composer.git -cd cuframes-composer -cmake -B build -G Ninja -ninja -C build +git submodule update --init --recursive +cmake -B build -DCMAKE_BUILD_TYPE=Release +cmake --build build -j$(nproc) ``` -## Поэтапный план +Артефакты: +- `build/src/libcuframes_composer.so` — shared library +- `build/examples/grid_record` — main CLI entry (C) +- `build/examples/grid_record_cpp` — C++ smoke test -| фаза | срок | результат | -|---|---|---| -| 1 | 1 неделя | один источник → NVENC → файл .h264 (доказательство zero-copy) | -| 2 | 2 недели | четыре источника + композиция через `libcugrid` | -| 3 | 2 недели | оверлеи + RTSP push к mediamtx + AAC passthrough из `/live-audio` | -| 4 | 1 неделя | паритет ZMQ-управления с фильтром `vf_cuda_grid` | -| 5 | 1 неделя | боевое развёртывание + MQTT health + watchdog | -| 6 | 2 недели | тесты + бенчмарки + документация | +Запуск (1 камера, motion-mode, JSON-templates): +```bash +build/examples/grid_record \ + --out=/tmp/grid.h264 --fps=25 --bitrate=6000 \ + --width=1920 --height=1080 \ + --source=cam-parking,frigate=parking_overview,priority=100 \ + --motion-mode --motion-ttl=45000 \ + --templates=docker/templates.json \ + --mqtt-overlays=docker/mqtt_overlays.json +``` -Итого ~9 недель для одного разработчика. +Production deploy и jammy-build — см. +[docs/ru/operations.md](docs/ru/operations.md). + +## Архитектура одной картинкой + +``` +Frigate ──MQTT events──→ frigate_mqtt subscriber + │ + ↓ motion_pulse +cuframes-pub-* ──VMM──→ Composer (C++ Cell/Layout/Decoration) + │ + ↓ best-fit + hysteresis + Layout::apply() + │ + ↓ NV12 zero-copy + NVENC ──→ H.264 pipe + ↓ + cfc-grid-ffmpeg + ↓ RTSP push + mediamtx ──→ TV/VLC/HLS/WebRTC + ↑ + cctv-onvif (PTZ → ZMQ set_layout) +``` + +Подробности — [docs/ru/developer.md](docs/ru/developer.md) §1. ## Лицензия -LGPL-2.1-or-later. См. файл [LICENSE](LICENSE). - -NVENC SDK headers (`third_party/nv-codec-headers`) — MIT license, совместима с LGPL. +LGPL-2.1+ diff --git a/docs/en/developer.md b/docs/en/developer.md new file mode 100644 index 0000000..435c42f --- /dev/null +++ b/docs/en/developer.md @@ -0,0 +1,612 @@ +# cfc-grid — developer guide + +> Audience: developer who edits composer C++ code, adds new +> Cell/Decoration/overlay types, or changes auto-layout logic. +> +> If you're a user — see [user.md](user.md). For deploy/troubleshooting +> see [operations.md](operations.md). + +## 1. Architecture (40000-foot view) + +``` + ┌──────────────────────────────────────────────┐ + │ cfc::Composer (C++) │ + │ │ + │ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │ + │ │SourcePool│ │ Layout │ │OutputSurface │ │ + │ │ (pool of │ │ vector< │ │ CudaBuffer │ │ + │ │ cfc_ │ │ unique_ │ │ (VMM NV12) │ │ + │ │source_t*)│ │ ptr│ │ │ │ + │ └──────────┘ └──────────┘ └──────────────┘ │ + │ ↓ ↓ ↑ │ + │ motion_pulse apply() NVENC │ + └──────────────────────────────────────────────┘ + ↑ ↓ + extern "C" ABI shim H.264 pipe + (composer_c_api.cpp, ↓ + layouts_c_api.cpp, ffmpeg-relay + mqtt_overlay_c_api.cpp) ↓ + ↑ mediamtx + ┌────────┴────────┐ ↓ + │ │ ↓ + grid_record.c control.c clients + (C, CLI parsing) (ZMQ verbs) + ↑ ↑ + │ │ + frigate_mqtt.c health.c, audio.c, writer.c, source.c + (Frigate events) (C-only modules) +``` + +**Principles:** + +- **C++17 host-side**, CUDA kernels remain on C +- **Zero-copy:** output VMM buffer is passed as `NV12Ref` to all + cells/decorations (read-write reference, not copy) +- **Coexistence:** C modules (`source.c`, `nvenc.c`, `frigate_mqtt.c`, + `health.c`, `writer.c`, `audio.c`, `overlay.c`) are kept, accessed via + `extern "C"`. Only `composer.c` and `layouts.c` were removed (replaced + by C++ + ABI shim). +- **`grid_record.c`** (CLI entry point) remains in C, uses public + `cfc_composer_*` API unchanged + +## 2. Source tree + +``` +include/cuframes_composer/ +├── *.h # Public C API (extern "C") +│ ├── composer.h # cfc_composer_* ABI +│ ├── layouts.h # cfc_layout_* ABI +│ ├── overlay.h # cfc_overlay_* (text/png/border/detbox) +│ ├── source.h, nvenc.h, frigate_mqtt.h, ... +│ └── ... +└── cpp/ # C++ headers — public classes + ├── cuda_raii.hpp # CudaBuffer, CudaStream (RAII) + ├── types.hpp # Rect, NV12Ref + ├── cell.hpp # abstract Cell + draw() + ├── camera_cell.hpp # CameraCell : Cell + ├── widget_cell.hpp # WidgetCell : Cell + ├── blank_cell.hpp # BlankCell : Cell + ├── decoration.hpp # abstract Decoration + ├── border_decoration.hpp # BorderDecoration + ├── label_decoration.hpp # LabelDecoration (FreeType internal) + ├── template.hpp # LayoutTemplate, CellTemplate, kGridCols/Rows + ├── template_loader.hpp # JSON parser, current_templates() + ├── source_pool.hpp # SourcePool, PoolEntry + ├── layout.hpp # Layout + ├── composer.hpp # Composer (main class) + └── mqtt_overlay.hpp # MqttOverlayItem + Manager + +src/ +├── *.c # C modules (source/nvenc/health/...) +├── overlay.c # All cfc_overlay_t types +├── frigate_mqtt.c # Subscribe + motion_pulse + zone-filter +├── cugrid/cugrid.cu # CUDA kernels (resize, fill, blit) +└── cpp/ # C++ implementations + ├── camera_cell.cpp, widget_cell.cpp, blank_cell.cpp + ├── border_decoration.cpp, label_decoration.cpp + ├── source_pool.cpp + ├── template_loader.cpp + ├── layout.cpp + ├── composer.cpp + ├── composer_c_api.cpp # extern "C" shim for composer + ├── layouts_c_api.cpp # extern "C" shim for layouts + ├── mqtt_overlay.cpp + └── mqtt_overlay_c_api.cpp + +examples/ +├── simple_record.c # 1-source → H.264 file +├── grid_record.c # Main CLI entry point (C) +└── grid_record_cpp.cpp # C++ smoke (uses cfc::Composer directly) +``` + +## 3. Single frame lifecycle + +```cpp +// In compose loop (grid_record.c, via cfc_composer_compose): +NV12Ref ref = composer.compose_frame(); +// 1. maybe_relayout(): motion-mode? best-fit template; apply Layout +// 2. compose_clear(): fill output buffer BT.709 black +// 3. Layout::render(): for each cell: draw_content() + decorations[] +// 4. for each overlay: sync detbox geom + cfc_overlay_draw() +// cudaStreamSynchronize(0); +// nvenc.encode(ref.y_ptr, ref.pitch_y, ref.uv_ptr, ref.pitch_uv, ...); +``` + +All operations execute on CUDA default stream. Zero-copy: +`ref.y_ptr` / `ref.uv_ptr` are the same `CUdeviceptr`s inside +`composer.output_` (RAII `CudaBuffer`). + +## 4. Cell — hierarchy and extension + +### 4.1 Abstraction + +```cpp +class Cell { +public: + explicit Cell(const Rect& geom); + virtual ~Cell() = default; + Cell(const Cell&) = delete; + + const Rect& geometry() const noexcept; + void set_geometry(const Rect& r) noexcept; + void add_decoration(std::unique_ptr); + + void draw(CUstream stream, NV12Ref& dst); // public — calls draw_content() + decorations + +protected: + virtual void draw_content(CUstream stream, NV12Ref& dst) = 0; + Rect geom_; + std::vector> decorations_; +}; +``` + +`Cell::draw()` is final (non-virtual — NVI pattern): +```cpp +void Cell::draw(CUstream stream, NV12Ref& dst) { + if (geom_.empty()) return; + draw_content(stream, dst); // subclass renders content + for (auto& d : decorations_) { + d->draw(stream, dst, geom_); // overlay decorations on top + } +} +``` + +### 4.2 Existing subclasses + +| Class | draw_content implementation | +|---|---| +| `CameraCell` | `cfc_source_get_latest()` + `cfc_cugrid_resize_nv12` | +| `WidgetCell` | `cfc_cugrid_fill_nv12` dark grey Y=40 (placeholder) | +| `BlankCell` | `cfc_cugrid_fill_nv12` BT.709 black Y=16 | + +### 4.3 How to add a new Cell type + +Example: `GraphCell` — draws scrolling chart from MQTT subscription. + +1. Create `include/cuframes_composer/cpp/graph_cell.hpp`: + +```cpp +#ifndef CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP +#define CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP +#include "cell.hpp" +#include + +namespace cfc { + +class GraphCell : public Cell { +public: + GraphCell(const Rect& geom, std::size_t max_points = 60) + : Cell(geom), max_points_(max_points) {} + + void push_value(double v) { + if (values_.size() >= max_points_) values_.pop_front(); + values_.push_back(v); + } + +protected: + void draw_content(CUstream stream, NV12Ref& dst) override; + +private: + std::deque values_; + std::size_t max_points_; +}; + +} // namespace cfc +#endif +``` + +2. Implementation `src/cpp/graph_cell.cpp`: + +```cpp +#include "../../include/cuframes_composer/cpp/graph_cell.hpp" +#include "../../include/cuframes_composer/cugrid.h" + +namespace cfc { + +void GraphCell::draw_content(CUstream stream, NV12Ref& dst) { + if (geom_.empty() || values_.empty()) return; + + // 1. BG fill + cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + geom_.x, geom_.y, geom_.w, geom_.h, + /*Y*/ 30, 128, 128, 255); + + // 2. Compute min/max + double mn = *std::min_element(values_.begin(), values_.end()); + double mx = *std::max_element(values_.begin(), values_.end()); + if (mx == mn) mx = mn + 1.0; + + // 3. Render as thin filled rects (lazy, no bitmap kernel) + int n = static_cast(values_.size()); + int dx = geom_.w / n; + for (int i = 0; i < n; i++) { + double norm = (values_[i] - mn) / (mx - mn); + int y_px = geom_.y + geom_.h - 2 - (int)(norm * (geom_.h - 4)); + cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + geom_.x + i * dx, y_px, dx & ~1, 2, + /*Y*/ 235, 128, 64, 255); // limited-range bright + } +} + +} // namespace cfc +``` + +3. Add to `src/CMakeLists.txt`: +```cmake +set(COMPOSER_SOURCES_CPP + ... + cpp/graph_cell.cpp +) +``` + +4. Use in `Layout::apply()`: + +```cpp +// In layout.cpp, widget cells handling: +if (wt->widget == "graph_temp") { + cells_.push_back(std::make_unique(r, /*max_points=*/120)); +} else { + cells_.push_back(std::make_unique(r, wt->widget)); +} +``` + +5. Wire MQTT → `GraphCell::push_value()` — either via MqttOverlayManager +extension, or new API `cfc::Composer::register_graph_feed()`. + +## 5. Decoration — composition in Cell + +### 5.1 Abstraction + +```cpp +class Decoration { +public: + virtual ~Decoration() = default; + virtual void draw(CUstream stream, NV12Ref& dst, const Rect& parent_rect) = 0; +}; +``` + +Decoration knows only parent cell's pixel-rect — positions **relative +to it**. Has no access to Layout/Composer. + +### 5.2 Existing subclasses + +- **`LabelDecoration`** — FreeType atlas + `cfc_cugrid_blit_rgba_nv12`. + Persistent VRAM (rebuild on `set_text`). Supports background fill + (via `cfc_overlay_text_config_t.bg_*` parameters). +- **`BorderDecoration`** — 4 calls of `cfc_cugrid_fill_nv12` (top, bottom, + left, right rects by `thickness`). + +### 5.3 How to add a new Decoration type + +Example: `BadgeDecoration` — corner icon (e.g. red "recording" dot). + +1. Header `badge_decoration.hpp`: + +```cpp +class BadgeDecoration : public Decoration { +public: + struct Style { + int corner = 0; // 0=top-left, 1=top-right, 2=bottom-right, 3=bottom-left + int size = 12; + int margin = 8; + int color_y = 80, color_u = 90, color_v = 240; // red-ish + int alpha = 240; + bool visible = true; + }; + explicit BadgeDecoration(const Style& s) : style_(s) {} + void set_visible(bool v) noexcept { style_.visible = v; } + void draw(CUstream stream, NV12Ref& dst, const Rect& p) override; +private: + Style style_; +}; +``` + +2. Implementation: + +```cpp +void BadgeDecoration::draw(CUstream s, NV12Ref& dst, const Rect& p) { + if (!style_.visible || style_.alpha <= 0) return; + int x, y; + switch (style_.corner) { + case 0: x = p.x + style_.margin; y = p.y + style_.margin; break; + case 1: x = p.x + p.w - style_.size - style_.margin; y = p.y + style_.margin; break; + case 2: x = p.x + p.w - style_.size - style_.margin; y = p.y + p.h - style_.size - style_.margin; break; + case 3: x = p.x + style_.margin; y = p.y + p.h - style_.size - style_.margin; break; + } + x &= ~1; y &= ~1; + cfc_cugrid_fill_nv12(s, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + x, y, style_.size, style_.size, + style_.color_y, style_.color_u, style_.color_v, style_.alpha); +} +``` + +3. Wire in `Layout::apply()`: + +```cpp +if (cc->source()->state() == CFC_SOURCE_STALE) { + BadgeDecoration::Style bs; + bs.corner = 1; bs.color_y = 180; bs.color_v = 100; // yellow + cell->add_decoration(std::make_unique(bs)); +} +``` + +## 6. SourcePool and motion state + +```cpp +struct PoolEntry { + std::string cuframes_key; + std::string frigate_camera; + int priority; + cfc_source_t* source; + std::atomic last_motion_ms; + std::vector required_zones; + + cfc_source_state_t state() const; // calls cfc_source_get_latest + bool drawable() const; // ACTIVE || STALE +}; + +class SourcePool { +public: + int add(key, frigate_camera, priority, zones, SubscribeOpts); + PoolEntry* by_key(const std::string&); + PoolEntry* by_frigate_camera(const std::string&); + template void for_each(F&&); + void motion_pulse(const std::string& frigate_camera, + const std::vector& current_zones); +}; +``` + +`motion_pulse` is called from `frigate_mqtt.c` for each event. If +`required_zones` is non-empty — match by intersection with +`event.current_zones`, otherwise accept all. + +## 7. Auto-layout algorithms + +See `cfc::Composer::maybe_relayout()` (src/cpp/composer.cpp). + +### 7.1 Best-fit selection + +```cpp +const LayoutTemplate* Composer::pick_best_fit(int need) const { + const auto& reg = current_templates(); + const LayoutTemplate* best = nullptr; + int best_waste = -1, best_prio = -1; + for (auto& t : reg) { + int n = t.nb_camera_cells(); + if (n < need) continue; + int waste = n - need; + if (!best || waste < best_waste || + (waste == best_waste && t.priority > best_prio)) { + best = &t; best_waste = waste; best_prio = t.priority; + } + } + if (best) return best; + // overflow → largest + for (auto& t : reg) { + if (!best || t.nb_camera_cells() > best->nb_camera_cells()) best = &t; + } + return best; +} +``` + +### 7.2 Collect active + +```cpp +std::vector Composer::collect_active() const { + std::vector active; + int64_t now = now_ms_mono(); + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; // DEAD/CONNECTING — skip + int64_t last = e.last_motion_ms.load(); + if (last == 0) return; // never had motion + if (now - last > cfg_.motion_ttl_ms) return; // TTL expired + active.push_back(&e); + }); + // idle fallback: top-priority drawable + if (active.empty()) { + PoolEntry* best = nullptr; + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; + if (!best || e.priority > best->priority) best = &e; + }); + if (best) active.push_back(best); + } + std::sort(active.begin(), active.end(), + [](PoolEntry* a, PoolEntry* b) { return a->priority > b->priority; }); + return active; +} +``` + +### 7.3 Asymmetric hysteresis + +Signature `template_name + "|" + sorted_keys` is remembered. If new +set ⊇ committed (growth) → commit immediately. Otherwise wait +`shrink_hysteresis_ms` (default 3000) → commit. + +```cpp +bool is_grow = std::includes(nkeys.begin(), nkeys.end(), + ckeys.begin(), ckeys.end()); +if (is_grow && nkeys.size() == ckeys.size()) is_grow = false; +if (is_grow) { /* commit */ } else { + if (sig != pending_signature_) { + pending_signature_ = sig; + pending_first_seen_ms_ = now; + } + if (now - pending_first_seen_ms_ < cfg_.shrink_hysteresis_ms) return; + /* commit */ +} +``` + +### 7.4 Fill free cells + +After picking template and capping by `nb_camera_cells`: + +```cpp +if (active.size() < cap) { + std::vector extras; + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; + if (e ∈ already_active) return; + extras.push_back(&e); + }); + std::sort(extras.begin(), extras.end(), priority_desc); + while (active.size() < cap && !extras.empty()) { + active.push_back(extras.front()); + extras.erase(extras.begin()); + } +} +``` + +### 7.5 Manual PTZ override + +`Composer::set_layout(name)` in motion-mode: +```cpp +manual_override_until_ms_ = now + manual_override_duration_ms_; // default 60s +``` + +`maybe_relayout()` skips work while `now < manual_override_until_ms_`. +After expiry — `committed_signature_.clear()` → forced relayout. + +## 8. extern "C" ABI shim + +`composer_c_api.cpp` — thin wrapper: + +```cpp +extern "C" int cfc_composer_create(const cfc_composer_config_t* cfg, + cfc_composer_t** out) +{ + cfc::ComposerConfig cpp_cfg = {/* ... */}; + auto* comp = new (std::nothrow) cfc::Composer(cpp_cfg); + if (!comp->ok()) { delete comp; return -1; } + // manual_cells from cfg->cells → set_manual_cells() + *out = reinterpret_cast(comp); + return 0; +} +``` + +`cfc_composer_t` is opaque in C, `reinterpret_cast` to/from `cfc::Composer*` +in shim. + +`layouts_c_api.cpp` is analogous for `cfc_layout_*`. Holds static cache +`vector` resynced with `cfc::current_templates()` on reload. + +## 9. Build + +### 9.1 CMake + +```cmake +project(cuframes-composer LANGUAGES C CXX CUDA) +set(CMAKE_CXX_STANDARD 17) +``` + +`COMPOSER_SOURCES_CPP` in `src/CMakeLists.txt` lists all .cpp files. + +### 9.2 Host build (for CI / dev machine Ubuntu 24.04) + +```bash +cd /home/claude/projects/cuframes-composer +mkdir -p build && cd build +cmake -DCMAKE_BUILD_TYPE=Release .. +make -j$(nproc) +``` + +### 9.3 Production build (Ubuntu 22.04 jammy) + +See [operations.md](operations.md). Briefly: +```bash +docker run --rm --gpus all \ + -v "$PWD":/src -w /src/build-jammy \ + cuframes-composer-builder:cached \ + bash -c 'cmake .. && make -j$(nproc)' +``` + +## 10. Performance + +### 10.1 Zero-copy guarantees + +- One `CudaBuffer output_` per `Composer`, passed as `NV12Ref` to all + cells/decorations +- Cell does not create VRAM allocations per frame (only `Decoration::draw` + may do FreeType rebuild on `set_text` — that's offline) +- Layout::apply() recreates `vector>` only on template + change; typically every N seconds + +### 10.2 Virtual call overhead + +`Cell::draw()` → `draw_content()` virtual call: 1 indirect call per cell +per frame. At 25 fps × 16 cells = **400 calls/sec** — irrelevant. + +### 10.3 Hot path + +CUDA kernels (`cugrid.cu`): +- `fill_nv12` — 1 kernel launch per rect +- `resize_nv12` — bilinear via 2 kernels (Y plane + UV plane) +- `blit_rgba_nv12` — 1 kernel, RGBA → NV12 + alpha-blend + +**All Cell operations convert to N×fill_nv12 + 1×resize_nv12 + +M×blit_rgba_nv12** — batching not yet needed (GPU is not sticky on +current load). + +## 11. Where to edit common tasks + +| I want to… | File | +|---|---| +| Change hysteresis | `composer.hpp` → `ComposerConfig::shrink_hysteresis_ms` | +| Change cell border color | `layout.cpp` → `border_style.color_y/u/v` | +| Change label font | `mqtt_overlay.cpp` → `LabelDecoration` style + `LabelStyle::font_path` | +| Add ZMQ verb | `control.c::dispatch()` + new `cmd_*` function | +| Change manual override duration | `composer.hpp` → `manual_override_duration_ms_` | +| Add new MQTT-overlay anchor | `mqtt_overlay.cpp::reposition_overlay()` switch | +| Support color emoji | `overlay.c::text_rebuild_atlas()` — handle `FT_LOAD_COLOR` + bitmap BGRA → blit as RGBA | + +## 12. Testing + +### 12.1 Unit (not implemented) + +Planned: catch2 tests for `pick_best_fit`, `collect_active`, hysteresis. +CUDA dependency — mock via `cfc_source_get_latest` shim. + +### 12.2 Integration smoke (`examples/grid_record_cpp.cpp`) + +Minimal C++ smoke: init Composer, compose loop, dump NV12 → file. Does +not use NVENC (tests composition only). + +```bash +build/examples/grid_record_cpp \ + --out=/tmp/dump.nv12 --frames=10 --width=1920 --height=1080 \ + --templates=docker/templates.json \ + --source=cam-parking,frigate=parking_overview,priority=100 \ + --motion-mode +``` + +### 12.3 Production smoke + +`docker logs cfc-grid | grep -E "loaded|template|motion|grow|shrink"` — +live composer telemetry. + +## 13. Known pitfalls + +- **`cfc_overlay_t` is not RAII** — managed via `cfc_composer_add_overlay` + / `cfc_overlay_destroy` (Composer.cpp::~Composer destroys all). +- **`pthread_mutex_t` in SourcePool** vs `std::mutex` — chose `std::mutex` + for C++ layer, `pthread_mutex_t` for C layer (PoolEntry::last_motion_ms + uses `std::atomic`). +- **Compose loop is NOT blocking** on CUDA ops — `cudaStreamSynchronize` + called by caller (grid_record.c) before NVENC. +- **Frigate event may arrive BEFORE Layout::apply** — `motion_pulse` + updates `last_motion_ms`, but cell for that camera may not exist yet. + On next frame `maybe_relayout` recalculates. +- **`dynamic_cast` in `Layout::find_camera_cell_rect`** — + uses RTTI. Enabled `-frtti` by default in g++/nvcc. + +## 14. Phase task change workflow + +1. Branch `feature/-` off `main` +2. Implementation, host build PASS, jammy build PASS (see operations.md) +3. Bake image `gx/cuframes-composer:-stepN` +4. Deploy to dev target → smoke verify via VLC/logs +5. Commit + push branch +6. (If needed) — merge to `main` via `--no-ff` PR-style + +For multi-commit phase: one WIP merge commit on main describing key +changes. diff --git a/docs/en/operations.md b/docs/en/operations.md new file mode 100644 index 0000000..cfca6be --- /dev/null +++ b/docs/en/operations.md @@ -0,0 +1,402 @@ +# cfc-grid — operations / deploy / troubleshooting + +> Audience: who builds, deploys, monitors cfc-grid in production. +> +> If you're a user — see [user.md](user.md). For developer documentation +> — see [developer.md](developer.md). + +## 1. Production setup (R9-88.23) + +### 1.1 Stack + +``` +docker compose -f docker-compose.yml \ + -f cuda-grid/docker-compose.override.yml \ + -f cuframes-composer/docker-compose.override.yml \ + -f onvif/docker-compose.override.yml \ + up -d +``` + +Files — in `localhost-infra/hosts/R9-88.23/docker/cctv/`. + +| Service | Image | Purpose | +|---|---|---| +| `cuframes-ipc-anchor` | `gx/cuframes:0.4` | Shared VMM IPC anchor for cuframes | +| `cuframes-pub-*` (parking/back_yard/front_yard/gate_lpr) | `gx/cuframes:0.4` | RTSP → cuframes per-camera publishers | +| `cuda-grid-mediamtx` | `bluenviron/mediamtx` | RTSP/HLS/WebRTC gateway | +| `cctv-mosquitto` | `eclipse-mosquitto` | MQTT broker (+bridge to 192.168.88.4) | +| **`cfc-grid`** | `gx/cuframes-composer:0.11b-step1` | Composer (main service) | +| `cfc-grid-ffmpeg` | `ffmpeg-vf-cuda-grid:phase4b-final` | H.264 pipe → RTSP push | +| `cfc-grid-watchdog` | `gx/cuda-grid-watchdog:0.4` | Restart cfc-grid on stuck inboundBytes | +| `cctv-onvif` | `gx/cctv-onvif:0.6` | ONVIF discovery + PTZ → ZMQ | +| `cctv-frigate` | `ghcr.io/blakeblackshear/frigate` | Object detection → MQTT events | + +### 1.2 Frame flow + +``` +cuframes-pub-X ──VMM──┐ +cuframes-pub-Y ──VMM──┼──→ cfc-grid (composer) +cuframes-pub-Z ──VMM──┘ │ + │ H.264 NVENC + ↓ named pipe /tmp/cfc-pipe-dir/grid.h264 + cfc-grid-ffmpeg (re-mux) + │ RTSP push + ↓ + cuda-grid-mediamtx + rtsp://*/cfc-grid (TCP/UDP) + http://*:8888/cfc-grid (HLS) + http://*:8889/cfc-grid (WebRTC) +``` + +### 1.3 Networks + +- Internal docker network: `cctv` +- External ports on R9-88.23: + - `554/tcp` — RTSP (mediamtx) + - `8888/tcp` — HLS (mediamtx) + - `8889/tcp` — WebRTC (mediamtx) + - `5599/tcp` — ZMQ composer control plane + - `8085/tcp` — ONVIF SOAP (cctv-onvif) + - `3702/udp` — WS-Discovery multicast (cctv-onvif) + +## 2. Build + +### 2.1 Local host build (Ubuntu 24.04, dev machine) + +```bash +cd /home/claude/projects/cuframes-composer +cmake -B build -DCMAKE_BUILD_TYPE=Release +cmake --build build -j$(nproc) +``` + +Artifacts in `build/src/libcuframes_composer.so` and `build/examples/grid_record`. + +**IMPORTANT:** host binary (Ubuntu 24.04, glibc 2.39, libavformat60) is +**incompatible** with runtime container (Ubuntu 22.04 jammy, glibc 2.35, +libavformat58). See memory `incremental-ffmpeg-rebuild`. + +### 2.2 Jammy build (for production image) + +Uses cached builder container `cuframes-composer-builder:cached` +(Ubuntu 22.04 + nvidia/cuda:12.4.1-devel + apt-deps): + +```bash +cd /home/claude/projects/cuframes-composer + +# If builder not yet cached: +docker image inspect cuframes-composer-builder:cached >/dev/null 2>&1 || { + docker run -d --name cfc-builder-tmp \ + nvidia/cuda:12.4.1-devel-ubuntu22.04 sleep 3600 + docker exec cfc-builder-tmp bash -c ' + apt-get update -qq && apt-get install -y -qq --no-install-recommends \ + build-essential cmake git pkg-config \ + libpng-dev libfreetype-dev \ + libzmq3-dev libjson-c-dev libmosquitto-dev \ + libavformat-dev libavcodec-dev libavutil-dev' + docker commit cfc-builder-tmp cuframes-composer-builder:cached + docker rm -f cfc-builder-tmp +} + +# Actual build: +docker run --rm --gpus all -v "$PWD":/src -w /src/build-jammy \ + cuframes-composer-builder:cached \ + bash -c 'cmake -DCMAKE_BUILD_TYPE=Release .. && make -j$(nproc)' +``` + +Artifacts in `build-jammy/`. + +### 2.3 Bake image (incremental — without `docker build`) + +We don't use `docker build` (4GB CUDA pull on cache miss). Instead: + +```bash +docker rmi gx/cuframes-composer:0.11b-step1 -f 2>/dev/null +CID=$(docker create gx/cuframes-composer:0.10) +docker cp build-jammy/examples/grid_record "$CID":/usr/local/bin/grid_record +docker cp build-jammy/src/libcuframes_composer.so.0.1.0 \ + "$CID":/usr/lib/x86_64-linux-gnu/libcuframes_composer.so.0 +docker cp docker/templates.json "$CID":/opt/templates.json +docker cp docker/mqtt_overlays.json "$CID":/opt/mqtt_overlays.json +docker commit \ + --change 'ENTRYPOINT ["/usr/local/bin/grid_record"]' \ + --change 'CMD ["--help"]' \ + --change 'ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility,video' \ + "$CID" gx/cuframes-composer:0.11b-step1 +docker rm "$CID" +``` + +Uses `gx/cuframes-composer:0.10` as base (with runtime deps already +installed) + overlays fresh artifacts. Faster and no network traffic. + +### 2.4 Build ONVIF image + +```bash +cd hosts/R9-88.23/docker/cctv/onvif +docker build -t gx/cctv-onvif:0.6 -f Dockerfile . +``` + +Python image, lightweight. If you change `server.py` — rebuild image +(bump tag) + update `docker-compose.override.yml`. + +## 3. Deploy + +### 3.1 Production (R9-88.23) + +```bash +cd /home/claude/projects/localhost-infra/hosts/R9-88.23/docker/cctv +docker compose -f docker-compose.yml \ + -f cuda-grid/docker-compose.override.yml \ + -f cuframes-composer/docker-compose.override.yml \ + -f onvif/docker-compose.override.yml \ + up -d cfc-grid +``` + +Compose auto-recreates container if image tag changed in +`docker-compose.override.yml`. + +### 3.2 Verify post-deploy + +```bash +docker logs --tail 30 cfc-grid 2>&1 | grep -iE "loaded|template|pool|motion" + +# Expect something like: +# [cfc/loader] /opt/templates.json: loaded 9 templates +# [cfc/composer] templates loaded: 9 (path='/opt/templates.json') +# [cfc/composer] pool+ 'cam-parking' (frigate=parking_overview prio=100) +# [cfc/composer] motion_mode=1 ttl=45000ms pool=4 +# [cfc/composer] grow → template='tpl_3' active=3 +``` + +### 3.3 Rollback + +```bash +sed -i 's|gx/cuframes-composer:0.11b-step1|gx/cuframes-composer:0.10|' \ + hosts/R9-88.23/docker/cctv/cuframes-composer/docker-compose.override.yml +docker compose ... up -d cfc-grid +``` + +## 4. Logs + +### 4.1 Live tail + +```bash +docker logs -f --tail 50 cfc-grid +docker logs -f --tail 30 cfc-grid-ffmpeg +docker logs -f --tail 30 cuda-grid-mediamtx +docker logs -f --tail 30 cctv-onvif +docker logs -f --tail 30 cctv-frigate +``` + +### 4.2 Telemetry patterns + +| Marker | Meaning | +|---|---| +| `[grid_record] N kadrov, M IDR, X MB ...` | Composer successfully encoding every ~50 frames | +| `[cfc/composer] grow → template='X'` | New template applied (growth, immediate) | +| `[cfc/composer] shrink → template='X'` | New template applied after hysteresis (shrink) | +| `[cfc/composer] manual override 'X' до +60000ms` | PTZ via ONVIF | +| `[cfc/composer] manual override expired, возврат в motion-mode` | Auto-return after 60s | +| `[cfc/mqtt-overlay/] ''` | MQTT-overlay received/rendered new text | +| `[cfc/frigate] connected, subscribe 'frigate/events'` | Frigate subscriber connected | + +### 4.3 When something breaks + +| Symptom | Where to look | +|---|---| +| `src active=0 stale=0 dead=4` | cuframes-pub-* containers; check `docker ps` and network access to cameras | +| `overlay 0 draw failed` | `cfc_overlay_text_rebuild_atlas` — usually invalid font or text | +| RTSP stream not delivering | `cfc-grid-ffmpeg` logs; see §6.1 | +| TV/ONVIF can't find | `cctv-onvif` logs; check multicast WS-Discovery in LAN | + +## 5. Monitoring + +### 5.1 MQTT health + +`cfc-grid` publishes health to `cuda_grid/health/composer/cfc-grid` +every ~10 seconds: + +```json +{ + "uptime_s": 3600, + "frames_encoded": 90000, + "fps_actual": 25.0, + "bitrate_kbps": 6000, + "src_active": 4, + "src_stale": 0, + "src_dead": 0, + "idr_count": 1 +} +``` + +```bash +PW=$(grep '^COMPOSER_MQTT_PASSWORD=' \ + /home/claude/projects/localhost-infra/hosts/R9-88.23/docker/cctv/.env | cut -d= -f2) +mosquitto_sub -h 192.168.88.23 -u composer -P "$PW" \ + -t 'cuda_grid/health/composer/cfc-grid' -v +``` + +### 5.2 Watchdog + +`cfc-grid-watchdog` is a separate service, monitors mediamtx +`inboundBytes` for `cfc-grid` path. If **30 seconds of silence** — +`docker restart cfc-grid`. + +Watchdog logs: +```bash +docker logs --tail 30 cfc-grid-watchdog +``` + +On trigger — publishes to `cuda_grid/health/watchdog/cfc-grid`. + +## 6. Troubleshooting + +### 6.1 RTSP not delivering / `cfc-grid-ffmpeg` "Broken pipe" + +**Symptom:** `docker logs cfc-grid-ffmpeg` shows +`[out#0/rtsp] Task finished with error code: -32 (Broken pipe)`. + +**Cause:** `--intra-refresh` in composer (no IDR bursts), mediamtx +tears RTSP publisher when it can't deliver start-frame to a new client. + +**Treatment:** +- Full pipeline restart: + ```bash + docker compose ... restart cfc-grid-ffmpeg cfc-grid cuda-grid-mediamtx + ``` +- If recurring — disable `--intra-refresh` in compose override + (cost: IDR bursts in bitrate, but more stable for downstream clients + with frequent disconnect/reconnect) + +### 6.2 ffmpeg doesn't receive frames from RTSP + +**Symptom:** `ffmpeg -i rtsp://192.168.88.23:554/cfc-grid -frames:v 1 out.jpg` +hangs for 30+ seconds. + +**Cause:** Composer writes H.264 without regular IDR (intra-refresh). +A new RTSP client waits for a keyframe to start decoding. ffmpeg in +default config doesn't wait long enough. + +**Workaround:** +```bash +ffmpeg -rtsp_transport tcp \ + -analyzeduration 10000000 -probesize 10000000 \ + -i rtsp://192.168.88.23:554/cfc-grid \ + -frames:v 1 -y out.jpg +``` + +Or use HLS: +```bash +ffmpeg -i http://192.168.88.23:8888/cfc-grid/index.m3u8 \ + -frames:v 1 -y out.jpg +``` + +### 6.3 MQTT-overlay not updating + +**Checklist:** + +1. Bridge to HA broker (192.168.88.4) working? + ```bash + docker logs cctv-mosquitto 2>&1 | grep -i 'bridge' + ``` + Look for `Connecting bridge ha-bridge` and connect confirmation. + +2. Required topic in bridge config? + ```bash + docker exec cctv-mosquitto grep 'topic.*in 0' /mosquitto/config/mosquitto.conf + ``` + If new prefix — add `topic XXX/# in 0` and restart mosquitto. + +3. Subscriber connected? + ```bash + docker logs cfc-grid 2>&1 | grep 'mqtt-overlay/.*connected' + ``` + +4. Test publish: + ```bash + mosquitto_pub -h 192.168.88.4 -t '' -m 'test' -r + ``` + In composer logs, should appear `[cfc/mqtt-overlay/] 'test'`. + +### 6.4 Motion-mode not switching layouts + +**Checklist:** + +1. Frigate sending events? + ```bash + mosquitto_sub -h 192.168.88.23 -u composer -P "$PW" \ + -t 'frigate/events' -C 3 + ``` + +2. Composer receiving events? + ```bash + docker logs cfc-grid 2>&1 | grep 'frigate.*started\|grow\|shrink' + ``` + +3. Camera-name matches? + `frigate=` in `--source` must match `event.after.camera`. + +4. Zone-filter not blocking? + If `zones=A:B:C` in `--source` — check Frigate event `current_zones`. + If empty or doesn't intersect — pulse is discarded. + +5. TTL not expired? + Logs `motion_ttl=45000` (45 sec) — if events come less frequently — + camera drops from active. + +### 6.5 ONVIF PTZ presets empty in TV + +**Cause:** TV cached old `GetPresets` response (Phase 9 names). + +**Treatment:** delete and re-add camera in TV client. + +### 6.6 Templates loaded but motion-mode doesn't use new + +Composer reads global registry `cfc::current_templates()` on every frame +— changes via `cfc_layout_load_file` (ZMQ or CLI) should be picked up +immediately. If not — check: + +```bash +echo '{"cmd":"list_layouts"}' | python3 -c " +import zmq,json,sys +s = zmq.Context().socket(zmq.REQ) +s.connect('tcp://192.168.88.23:5599') +s.send_json({'cmd':'list_layouts'}) +print(json.dumps(s.recv_json(), indent=2, ensure_ascii=False))" +``` + +`source` field shows currently loaded path. If built-in (only `tpl_1` + +`tpl_4`) — JSON didn't load (syntax error, wrong path). + +## 7. Configs in repo + +| What | Where | +|---|---| +| templates.json | `cuframes-composer/docker/templates.json` | +| mqtt_overlays.json | `cuframes-composer/docker/mqtt_overlays.json` | +| compose override | `localhost-infra/hosts/R9-88.23/docker/cctv/cuframes-composer/docker-compose.override.yml` | +| ONVIF config | `localhost-infra/.../onvif/onvif.yaml` | +| ONVIF server | `localhost-infra/.../onvif/server.py` | +| Mosquitto config | `localhost-infra/.../cctv/mosquitto/config/mosquitto.conf` | +| .env (passwords) | `localhost-infra/.../cctv/.env` (gitignored) | + +After changing compose override — `docker compose ... up -d cfc-grid` +auto-recreates. + +## 8. Known limitations / TODO + +- **`--intra-refresh` ↔ RTSP clients**: trade-off bitrate vs latency + (see §6.1) +- **Watchdog only for cfc-grid**: cfc-grid-ffmpeg in zombie state not + detected directly; only full restart helps +- **Hot-reload of mqtt_overlays.json**: no ZMQ verb +- **Per-overlay MQTT broker config**: all via single broker; for + foreign broker — need to extend `MqttBrokerCfg` per-item + +## 9. See also + +- [user.md](user.md) — composer configuration +- [developer.md](developer.md) — internals, adding modules +- `memory/host-and-project.md` — general R9-88.23 infra +- `memory/project_cfc-grid-deployed.md` — first prod deploy +- `memory/project_cfc-grid-cpp-refactor.md` — Phase 11b refactor +- `memory/incremental-ffmpeg-rebuild.md` — incremental docker recipe diff --git a/docs/en/user.md b/docs/en/user.md new file mode 100644 index 0000000..60e715b --- /dev/null +++ b/docs/en/user.md @@ -0,0 +1,450 @@ +# cfc-grid — user guide + +> Audience: installation admin. Configures cameras, layouts, overlays, +> watches the RTSP stream on TV or browser. Does not edit C++ code. +> +> Developer extending Cell/Decoration/widget types → see +> [developer.md](developer.md). + +## 1. What this is + +**cfc-grid** is a CUDA compositor that combines N cameras into a single +RTSP stream `rtsp://192.168.88.23:554/cfc-grid` (1920×1080, H.264, NVENC). +Layout is selected automatically by motion (from Frigate) or manually via +ONVIF presets on the TV. + +In addition to camera frames, the following are drawn on top: + +- **Borders** — grey 2 px borders around each cell +- **Labels** — `name prio=N` caption in the corner of each camera +- **Detection boxes** — object rectangles from Frigate, tracking camera + position when layout changes +- **MQTT overlays** — text fields bound to MQTT topics (temperature, + statuses, chats) + +## 2. Architecture in one sentence + +``` +cuframes-pub-* (per camera) + ↓ shared VMM + cfc-grid (composer) ── ZMQ control ──┐ + ↓ pipe (H.264) │ + cfc-grid-ffmpeg (relay) ─→ mediamtx ─┴─→ TV / VLC / Frigate / ... + ↑ + ONVIF discovery from cctv-onvif +``` + +Frames go through cuframes zero-copy (a single VMM buffer shared between +publisher and composer). The composer takes the NV12 surface, resizes/blits +into its output, adds decorations, hands off to NVENC, NVENC writes H.264 +into a pipe, `cfc-grid-ffmpeg` re-muxes the pipe → RTSP push to mediamtx. + +## 3. Motion mode — the main operating mode + +### 3.1 What happens + +Each frame the composer: + +1. Reads `last_motion_ms` for each camera (updated from Frigate MQTT + `frigate/events`) +2. Treats as "active" any camera with `(now - last_motion_ms) < motion_ttl_ms` + (default **45 seconds**) +3. Sorts active by `priority` (integer; higher = more important) +4. Selects template from `templates.json` by **best-fit** rule: + minimal template with `nb_camera_cells >= active_count` +5. If template has more camera-cells than active — extra ones are filled + with remaining drawable cameras from pool (by priority) +6. Applies **asymmetric hysteresis**: growth in active count switches + layout immediately, shrinkage waits 3 seconds (to avoid flicker) + +### 3.2 What "drawable" means + +A camera is **excluded** from pool if its `cfc_source_state_t` = +`CONNECTING`, `DISCONNECTED` or `DEAD` (cuframes publisher silent for +longer than `dead_threshold_ms`, default 5 seconds). + +`STALE` (frames arrive infrequently) — counts, last available frame is +drawn. + +### 3.3 Manual override via PTZ + +In the TV, ONVIF PTZ presets list template names (`tpl_1, tpl_3, tpl_4, +..., tpl_16`). Pressing `GotoPreset` or movement keys: + +- Applies the chosen layout immediately +- **Freezes** motion-mode for 60 seconds +- After expiry — returns to auto mode + +ContinuousMove (arrows): pan/tilt cycles through preset list, zoom-in = +`tpl_1` (full screen), zoom-out = `tpl_16` (4×4 grid). + +## 4. templates.json — screen layout + +### 4.1 Schema + +```json +{ + "version": 1, + "grid_cols": 8, + "grid_rows": 8, + "templates": [ + { + "name": "tpl_N", + "_desc": "Description", + "priority": 0, + "cells": [ + { + "col": 0, "row": 0, + "cs": 4, "rs": 4, + "role": "camera", + "order": 0 + }, + { + "col": 6, "row": 0, + "cs": 2, "rs": 6, + "role": "widget", + "widget": "temp_chart" + } + ] + } + ] +} +``` + +**Grid: 8×8 microcells** (240×135 px each = 16:9 on 1920×1080 output). +Any N×N square of microcells is also 16:9. + +| Field | Value | +|---|---| +| `col`, `row` | Top-left corner of cell in microcells (0..7) | +| `cs`, `rs` | Size in microcells | +| `role` | `camera` or `widget` | +| `order` | For `camera`: placement order for active cameras (0 = main, usually the largest cell) | +| `widget` | For `widget`: placeholder name (caption text) | + +### 4.2 Best-fit selection + +The composer selects a template for current active count: + +``` +candidates = [t for t in templates if t.nb_camera_cells >= n_active] +pick = min(candidates, key=lambda t: t.nb_camera_cells - n_active) +# On tie: higher priority wins +``` + +If active count exceeds the largest template's cells — the largest is +taken, extra cameras are dropped (lowest priority). + +### 4.3 Built-in templates + +By default 9 templates in `/opt/templates.json`: + +| Name | Cells | Description | +|---|---|---| +| `tpl_1` | 1 cam | One camera fullscreen | +| `tpl_3` | 3 cam + 2 widgets | Main 1440×810 + 2 previews + 2 widget areas | +| `tpl_4` | 4 cam | Quad 2×2, 960×540 each | +| `tpl_5` | 5 cam + 1 widget | Main + 4 previews stacked right + widget bottom | +| `tpl_6` | 6 cam + 1 widget | Main + 3 right + 2 bottom + widget | +| `tpl_7` | 7 cam + 1 widget | Main + 3 right + 3 bottom + widget | +| `tpl_8` | 8 cam (1+3+4) | Main + 3 right + 4 bottom row | +| `tpl_9` | 9 cam + 2 widgets | 3×3 mains + widget strips | +| `tpl_16` | 16 cam | 4×4 grid, 480×270 each | + +Details — see `docker/templates.json` in the repo. + +### 4.4 Adding your own template + +1. Open `docker/templates.json` (or mounted override) +2. Add a block to `"templates": [...]` per schema above +3. Restart cfc-grid (or invoke ZMQ when hot-reload is available — Phase 12) + +### 4.5 Coordinate math + +Each microcell = `1920/8 = 240 px` wide, `1080/8 = 135 px` tall (16:9). +Cell `{col=2, row=4, cs=4, rs=2}`: + +- pixel x = `2 * 240 = 480` +- pixel y = `4 * 135 = 540` +- pixel w = `4 * 240 = 960` +- pixel h = `2 * 135 = 270` + +Aspect cell = `cs/rs * 16/9`. For 16:9 cells: **cs == rs**. + +## 5. mqtt_overlays.json — text overlays from MQTT + +### 5.1 Schema + +```json +{ + "version": 1, + "overlays": [ + { + "id": "temp_outside", + "topic": "zigbee2mqtt/Outdoor temp sensor", + "json_field": "temperature", + "format": "%+.1f°C", + "anchor": "right-bottom", + "margin_x": 32, "margin_y": 24, + "pixel_size": 32, + "color": [255, 255, 255], + "alpha": 230, + "bg_alpha": 160, + "bg_y": 16, "bg_u": 128, "bg_v": 128, + "bg_pad": 10, + "placeholder": "—", + "font_path": "/fonts/DejaVuSans-Bold.ttf" + } + ] +} +``` + +| Field | Description | +|---|---| +| `id` | Unique overlay identifier (used by ZMQ for lookup) | +| `topic` | MQTT topic to subscribe (via `cctv-mosquitto`) | +| `json_field` | If payload is JSON — field name to extract; **empty = raw payload as string** | +| `format` | `printf`-style for the formatted value (e.g. `"%+.1f°C"`, `"%s"`) | +| `anchor` | Positioning anchor: `right-bottom`, `right-top`, `left-bottom`, `left-top`, `center` | +| `margin_x`, `margin_y` | Offset from nearest screen edge (px) | +| `pixel_size` | Font size in pixels | +| `color` | RGB text color | +| `alpha` | Overall text opacity (0..255) | +| `bg_alpha` | Background opacity (0..255); **0 = no background** | +| `bg_y`, `bg_u`, `bg_v` | BT.709 limited-range background color; default black | +| `bg_pad` | Background padding around text (px) | +| `placeholder` | What to show before first MQTT message; empty = "—" | +| `font_path` | Path to font (.ttf/.otf) inside container | + +### 5.2 Supported symbols + +Font `DejaVuSans-Bold.ttf` — standard from `fonts-dejavu` package. +**Covers Basic Multilingual Plane** (latin, cyrillic, basic symbols), +including: + +- `❯` (U+276F), `✎` (U+270E), `➤` (U+27A4), `→` (U+2192) +- `★` (U+2605), `▶` (U+25B6), `✉` (U+2709) + +**Emoji from Supplementary Multilingual Plane (>U+10000)** — e.g. +`🗣` (U+1F5E3), `🤖` (U+1F916), `💬` (U+1F4AC) — **are not rendered**: +font lacks those glyphs. Placeholder square is drawn instead. + +To add color emoji — bind Noto Color Emoji and extend renderer for +COLR/CPAL/SBIX (see developer.md). + +### 5.3 Examples + +**Plain number**: payload = `"23.5"` (raw), `json_field: ""`, +`format: "%s°"` + +**JSON with field**: payload = `{"temperature": 23.5, "humidity": 45}`, +`json_field: "temperature"`, `format: "%+.1f°C"` + +**Multiple overlays** stacked top-right: first with `margin_y: 24`, +second `margin_y: 72`, third `margin_y: 120`. + +### 5.4 How to add + +1. Open `docker/mqtt_overlays.json` (or mounted override) +2. Add block to `"overlays": [...]` array +3. Restart cfc-grid + +Hot-reload via ZMQ — Phase 12 (`reload_overlays` verb). + +## 6. Composer CLI flags + +In compose override `docker/cctv/cuframes-composer/docker-compose.override.yml`: + +```yaml +command: + - "--out=/out/grid.h264" # named pipe for ffmpeg-relay + - "--fps=25" + - "--bitrate=6000" # kbps + - "--width=1920" + - "--height=1080" + - "--intra-refresh" # instead of IDR-bursts (low-latency) + - "--control=tcp://0.0.0.0:5599" # ZMQ control plane + - "--mqtt=cctv-mosquitto:1883" # MQTT for health publishing + - "--mqtt-instance=cfc-grid" + - "--mqtt-user=composer" + - "--mqtt-pass=${COMPOSER_MQTT_PASSWORD}" + + # Sources (cameras) — repeatable + - "--source=cam-parking,frigate=parking_overview,priority=100,zones=parking_zone:canopy:private_area" + - "--source=cam-back_yard,frigate=back_yard,priority=70" + # ... + + - "--motion-mode" # enable auto-layout + - "--motion-ttl=45000" # ms + + - "--templates=/opt/templates.json" + - "--mqtt-overlays=/opt/mqtt_overlays.json" + + # Frigate motion-driver and detection boxes + - "--frigate-mqtt=cctv-mosquitto:1883" + - "--frigate-topic=frigate/events" + - "--detection-cell=parking,parking_overview,0,0,960,540,640,480,parking_zone:canopy:private_area" +``` + +### 6.1 `--source` syntax + +``` +--source=,frigate=,priority=[,zones=::...] +``` + +- `cuframes_key` — name of cuframes publisher (e.g. `cam-parking`) +- `frigate=NAME` — camera name in Frigate (for motion event matching) +- `priority=N` — integer, higher = more important +- `zones=...` — optional whitelist of Frigate zones; motion counts + only if `event.current_zones` intersects the list (filters street flood) + +### 6.2 `--detection-cell` syntax + +``` +--detection-cell=,,,,,,,[,zones] +``` + +- `key` — arbitrary overlay identifier for logs +- `frigate_camera` — Frigate name (for `event.camera` matching) +- `x,y,w,h` — initial geometry (composer recalculates dynamically) +- `detect_w,detect_h` — Frigate detector resolution (e.g. 640×480) +- `zones` — bbox whitelist + +## 7. ZMQ control plane + +Default endpoint: `tcp://192.168.88.23:5599`. All verbs — JSON request/reply. + +### 7.1 Verb list + +| Command | Parameters | What it does | +|---|---|---| +| `ping` | — | Health check | +| `health` | — | `{total, active, stale, dead}` over pool | +| `set_text` | `id, text, r, g, b, x, y, visible` | Update text overlay (for CLI `--text=...`) | +| `set_visible` | `id, visible` | Hide/show overlay | +| `list_overlays` | — | List of overlays | +| `set_layout` | `name` | Apply named template (manual override 60s in motion-mode) | +| `list_layouts` | — | List of templates with cells | +| `get_layout` | — | Name of current template | +| `set_motion_mode` | `on, ttl_ms` | Toggle motion mode | +| `get_motion_mode` | — | Motion mode state | +| `get_template` | `name` | Full template JSON | +| `reload_templates` | `path?` | Reload templates from file | + +### 7.2 Example + +```bash +python3 < Аудитория: разработчик который правит C++ код композитора, добавляет +> новые типы Cell/Decoration/overlay-type, или меняет логику auto-layout. +> +> Если ты пользователь — см. [user.md](user.md). Если занимаешься +> deploy/troubleshooting — см. [operations.md](operations.md). + +## 1. Архитектура (40000-foot view) + +``` + ┌──────────────────────────────────────────────┐ + │ cfc::Composer (C++) │ + │ │ + │ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │ + │ │SourcePool│ │ Layout │ │OutputSurface │ │ + │ │ (pool of │ │ vector< │ │ CudaBuffer │ │ + │ │ cfc_ │ │ unique_ │ │ (VMM NV12) │ │ + │ │source_t*)│ │ ptr│ │ │ │ + │ └──────────┘ └──────────┘ └──────────────┘ │ + │ ↓ ↓ ↑ │ + │ motion_pulse apply() NVENC │ + └──────────────────────────────────────────────┘ + ↑ ↓ + extern "C" ABI shim H.264 pipe + (composer_c_api.cpp, ↓ + layouts_c_api.cpp, ffmpeg-relay + mqtt_overlay_c_api.cpp) ↓ + ↑ mediamtx + ┌────────┴────────┐ ↓ + │ │ ↓ + grid_record.c control.c clients + (C, CLI parsing) (ZMQ verbs) + ↑ ↑ + │ │ + frigate_mqtt.c health.c, audio.c, writer.c, source.c + (Frigate events) (C-only modules) +``` + +**Принципы:** + +- **C++17 host-side**, CUDA kernels остались на C +- **Zero-copy:** output VMM-буфер передаётся как `NV12Ref` всем + cells/decorations (read-write reference, не копия) +- **Coexistence:** C-модули (`source.c`, `nvenc.c`, `frigate_mqtt.c`, + `health.c`, `writer.c`, `audio.c`, `overlay.c`) сохранены, доступ через + `extern "C"`. Только `composer.c` и `layouts.c` удалены (заменены C++ + + ABI shim). +- **`grid_record.c`** (CLI entry-point) остался на C, использует + публичный `cfc_composer_*` API без правок + +## 2. Дерево исходников + +``` +include/cuframes_composer/ +├── *.h # Публичный C API (extern "C") +│ ├── composer.h # cfc_composer_* ABI +│ ├── layouts.h # cfc_layout_* ABI +│ ├── overlay.h # cfc_overlay_* (text/png/border/detbox) +│ ├── source.h, nvenc.h, frigate_mqtt.h, ... +│ └── ... +└── cpp/ # C++ headers — публичные классы + ├── cuda_raii.hpp # CudaBuffer, CudaStream (RAII) + ├── types.hpp # Rect, NV12Ref + ├── cell.hpp # абстрактный Cell + draw() + ├── camera_cell.hpp # CameraCell : Cell + ├── widget_cell.hpp # WidgetCell : Cell + ├── blank_cell.hpp # BlankCell : Cell + ├── decoration.hpp # абстрактный Decoration + ├── border_decoration.hpp # BorderDecoration + ├── label_decoration.hpp # LabelDecoration (FreeType internal) + ├── template.hpp # LayoutTemplate, CellTemplate, kGridCols/Rows + ├── template_loader.hpp # JSON parser, current_templates() + ├── source_pool.hpp # SourcePool, PoolEntry + ├── layout.hpp # Layout + ├── composer.hpp # Composer (главный класс) + └── mqtt_overlay.hpp # MqttOverlayItem + Manager + +src/ +├── *.c # C-модули (source/nvenc/health/...) +├── overlay.c # Все типы cfc_overlay_t +├── frigate_mqtt.c # Subscribe + motion_pulse + zone-filter +├── cugrid/cugrid.cu # CUDA kernels (resize, fill, blit) +└── cpp/ # C++ реализации + ├── camera_cell.cpp, widget_cell.cpp, blank_cell.cpp + ├── border_decoration.cpp, label_decoration.cpp + ├── source_pool.cpp + ├── template_loader.cpp + ├── layout.cpp + ├── composer.cpp + ├── composer_c_api.cpp # extern "C" shim для composer + ├── layouts_c_api.cpp # extern "C" shim для layouts + ├── mqtt_overlay.cpp + └── mqtt_overlay_c_api.cpp + +examples/ +├── simple_record.c # 1-источник → H.264 в файл +├── grid_record.c # Main CLI entry-point (C) +└── grid_record_cpp.cpp # C++ smoke (использует cfc::Composer напрямую) +``` + +## 3. Жизненный цикл одного кадра + +```cpp +// В compose loop (grid_record.c, через cfc_composer_compose): +NV12Ref ref = composer.compose_frame(); +// 1. maybe_relayout(): motion-mode? best-fit template; apply Layout +// 2. compose_clear(): fill output буфера BT.709 black +// 3. Layout::render(): for each cell: draw_content() + decorations[] +// 4. for each overlay: sync detbox geom + cfc_overlay_draw() +// cudaStreamSynchronize(0); +// nvenc.encode(ref.y_ptr, ref.pitch_y, ref.uv_ptr, ref.pitch_uv, ...); +``` + +Все операции выполняются на CUDA default stream. Zero-copy: +`ref.y_ptr` / `ref.uv_ptr` — те же `CUdeviceptr`, что внутри +`composer.output_` (RAII `CudaBuffer`). + +## 4. Cell — иерархия и расширение + +### 4.1 Абстракция + +```cpp +class Cell { +public: + explicit Cell(const Rect& geom); + virtual ~Cell() = default; + Cell(const Cell&) = delete; + + const Rect& geometry() const noexcept; + void set_geometry(const Rect& r) noexcept; + void add_decoration(std::unique_ptr); + + void draw(CUstream stream, NV12Ref& dst); // public — calls draw_content() + decorations + +protected: + virtual void draw_content(CUstream stream, NV12Ref& dst) = 0; + Rect geom_; + std::vector> decorations_; +}; +``` + +`Cell::draw()` финальный (не виртуальный — final pattern через NVI): +```cpp +void Cell::draw(CUstream stream, NV12Ref& dst) { + if (geom_.empty()) return; + draw_content(stream, dst); // subclass renders content + for (auto& d : decorations_) { + d->draw(stream, dst, geom_); // overlay decorations on top + } +} +``` + +### 4.2 Существующие подклассы + +| Класс | draw_content реализация | +|---|---| +| `CameraCell` | `cfc_source_get_latest()` + `cfc_cugrid_resize_nv12` | +| `WidgetCell` | `cfc_cugrid_fill_nv12` тёмно-серым Y=40 (placeholder) | +| `BlankCell` | `cfc_cugrid_fill_nv12` BT.709 черным Y=16 | + +### 4.3 Как добавить новый тип Cell + +Пример: `GraphCell` — рисует scrolling-chart из подписки на MQTT. + +1. Создать `include/cuframes_composer/cpp/graph_cell.hpp`: + +```cpp +#ifndef CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP +#define CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP +#include "cell.hpp" +#include + +namespace cfc { + +class GraphCell : public Cell { +public: + GraphCell(const Rect& geom, std::size_t max_points = 60) + : Cell(geom), max_points_(max_points) {} + + void push_value(double v) { + if (values_.size() >= max_points_) values_.pop_front(); + values_.push_back(v); + } + +protected: + void draw_content(CUstream stream, NV12Ref& dst) override; + +private: + std::deque values_; + std::size_t max_points_; +}; + +} // namespace cfc +#endif +``` + +2. Реализация `src/cpp/graph_cell.cpp`: + +```cpp +#include "../../include/cuframes_composer/cpp/graph_cell.hpp" +#include "../../include/cuframes_composer/cugrid.h" + +namespace cfc { + +void GraphCell::draw_content(CUstream stream, NV12Ref& dst) { + if (geom_.empty() || values_.empty()) return; + + // 1. BG fill + cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + geom_.x, geom_.y, geom_.w, geom_.h, + /*Y*/ 30, 128, 128, 255); + + // 2. Compute min/max + double mn = *std::min_element(values_.begin(), values_.end()); + double mx = *std::max_element(values_.begin(), values_.end()); + if (mx == mn) mx = mn + 1.0; + + // 3. Render line as thin filled rects (lazy, без kernel'я для bitmap'а) + int n = static_cast(values_.size()); + int dx = geom_.w / n; + for (int i = 0; i < n; i++) { + double norm = (values_[i] - mn) / (mx - mn); + int y_px = geom_.y + geom_.h - 2 - (int)(norm * (geom_.h - 4)); + cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + geom_.x + i * dx, y_px, dx & ~1, 2, + /*Y*/ 235, 128, 64, 255); // limited-range bright + } +} + +} // namespace cfc +``` + +3. Добавить в `src/CMakeLists.txt`: +```cmake +set(COMPOSER_SOURCES_CPP + ... + cpp/graph_cell.cpp +) +``` + +4. Использовать в `Layout::apply()`: + +```cpp +// В layout.cpp, в обработке widget cells: +if (wt->widget == "graph_temp") { + cells_.push_back(std::make_unique(r, /*max_points=*/120)); +} else { + cells_.push_back(std::make_unique(r, wt->widget)); +} +``` + +5. Подключить MQTT → `GraphCell::push_value()` — либо через MqttOverlayManager +расширение, либо через `cfc::Composer::register_graph_feed()` (новый API). + +## 5. Decoration — композиция в Cell + +### 5.1 Абстракция + +```cpp +class Decoration { +public: + virtual ~Decoration() = default; + virtual void draw(CUstream stream, NV12Ref& dst, const Rect& parent_rect) = 0; +}; +``` + +Decoration знает только pixel-rect parent-cell'а — позиционируется +**относительно него**. Не имеет access к Layout/Composer. + +### 5.2 Существующие подклассы + +- **`LabelDecoration`** — FreeType atlas + `cfc_cugrid_blit_rgba_nv12`. + Persistent VRAM (rebuild при `set_text`). Поддерживает background fill + (через `cfc_overlay_text_config_t.bg_*` параметры). +- **`BorderDecoration`** — 4 вызова `cfc_cugrid_fill_nv12` (top, bottom, + left, right rect'ы по `thickness`). + +### 5.3 Как добавить новый тип Decoration + +Пример: `BadgeDecoration` — иконка в углу (например красная точка +«recording»). + +1. Header `badge_decoration.hpp`: + +```cpp +class BadgeDecoration : public Decoration { +public: + struct Style { + int corner = 0; // 0=top-left, 1=top-right, 2=bottom-right, 3=bottom-left + int size = 12; + int margin = 8; + int color_y = 80, color_u = 90, color_v = 240; // red-ish + int alpha = 240; + bool visible = true; + }; + explicit BadgeDecoration(const Style& s) : style_(s) {} + void set_visible(bool v) noexcept { style_.visible = v; } + void draw(CUstream stream, NV12Ref& dst, const Rect& p) override; +private: + Style style_; +}; +``` + +2. Реализация: + +```cpp +void BadgeDecoration::draw(CUstream s, NV12Ref& dst, const Rect& p) { + if (!style_.visible || style_.alpha <= 0) return; + int x, y; + switch (style_.corner) { + case 0: x = p.x + style_.margin; y = p.y + style_.margin; break; + case 1: x = p.x + p.w - style_.size - style_.margin; y = p.y + style_.margin; break; + case 2: x = p.x + p.w - style_.size - style_.margin; y = p.y + p.h - style_.size - style_.margin; break; + case 3: x = p.x + style_.margin; y = p.y + p.h - style_.size - style_.margin; break; + } + x &= ~1; y &= ~1; + cfc_cugrid_fill_nv12(s, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv, + x, y, style_.size, style_.size, + style_.color_y, style_.color_u, style_.color_v, style_.alpha); +} +``` + +3. Подключать в `Layout::apply()`: + +```cpp +if (cc->source()->state() == CFC_SOURCE_STALE) { + BadgeDecoration::Style bs; + bs.corner = 1; bs.color_y = 180; bs.color_v = 100; // yellow + cell->add_decoration(std::make_unique(bs)); +} +``` + +## 6. SourcePool и motion-state + +```cpp +struct PoolEntry { + std::string cuframes_key; + std::string frigate_camera; + int priority; + cfc_source_t* source; + std::atomic last_motion_ms; + std::vector required_zones; + + cfc_source_state_t state() const; // вызывает cfc_source_get_latest + bool drawable() const; // ACTIVE || STALE +}; + +class SourcePool { +public: + int add(key, frigate_camera, priority, zones, SubscribeOpts); + PoolEntry* by_key(const std::string&); + PoolEntry* by_frigate_camera(const std::string&); + template void for_each(F&&); + void motion_pulse(const std::string& frigate_camera, + const std::vector& current_zones); +}; +``` + +`motion_pulse` вызывается из `frigate_mqtt.c` при каждом event. Если +`required_zones` непустой — match по intersection с `event.current_zones`, +иначе принимаем всё. + +## 7. Auto-layout алгоритмы + +См. `cfc::Composer::maybe_relayout()` (src/cpp/composer.cpp). + +### 7.1 Best-fit selection + +```cpp +const LayoutTemplate* Composer::pick_best_fit(int need) const { + const auto& reg = current_templates(); + const LayoutTemplate* best = nullptr; + int best_waste = -1, best_prio = -1; + for (auto& t : reg) { + int n = t.nb_camera_cells(); + if (n < need) continue; + int waste = n - need; + if (!best || waste < best_waste || + (waste == best_waste && t.priority > best_prio)) { + best = &t; best_waste = waste; best_prio = t.priority; + } + } + if (best) return best; + // overflow → largest + for (auto& t : reg) { + if (!best || t.nb_camera_cells() > best->nb_camera_cells()) best = &t; + } + return best; +} +``` + +### 7.2 Collect active + +```cpp +std::vector Composer::collect_active() const { + std::vector active; + int64_t now = now_ms_mono(); + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; // DEAD/CONNECTING — skip + int64_t last = e.last_motion_ms.load(); + if (last == 0) return; // never had motion + if (now - last > cfg_.motion_ttl_ms) return; // TTL expired + active.push_back(&e); + }); + // idle fallback: top-priority drawable + if (active.empty()) { + PoolEntry* best = nullptr; + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; + if (!best || e.priority > best->priority) best = &e; + }); + if (best) active.push_back(best); + } + std::sort(active.begin(), active.end(), + [](PoolEntry* a, PoolEntry* b) { return a->priority > b->priority; }); + return active; +} +``` + +### 7.3 Asymmetric hysteresis + +Сигнатура `template_name + "|" + sorted_keys` запоминается. Если новый +набор ⊇ committed (рост) → commit мгновенно. Иначе ждём +`shrink_hysteresis_ms` (default 3000) → commit. + +```cpp +bool is_grow = std::includes(nkeys.begin(), nkeys.end(), + ckeys.begin(), ckeys.end()); +if (is_grow && nkeys.size() == ckeys.size()) is_grow = false; +if (is_grow) { /* commit */ } else { + if (sig != pending_signature_) { + pending_signature_ = sig; + pending_first_seen_ms_ = now; + } + if (now - pending_first_seen_ms_ < cfg_.shrink_hysteresis_ms) return; + /* commit */ +} +``` + +### 7.4 Fill свободных cells + +После выбора template и cap'а по `nb_camera_cells`: + +```cpp +if (active.size() < cap) { + std::vector extras; + pool_.for_each([&](PoolEntry& e) { + if (!e.drawable()) return; + if (e ∈ already_active) return; + extras.push_back(&e); + }); + std::sort(extras.begin(), extras.end(), priority_desc); + while (active.size() < cap && !extras.empty()) { + active.push_back(extras.front()); + extras.erase(extras.begin()); + } +} +``` + +### 7.5 Manual PTZ override + +`Composer::set_layout(name)` в motion-mode: +```cpp +manual_override_until_ms_ = now + manual_override_duration_ms_; // default 60s +``` + +`maybe_relayout()` пропускает работу пока `now < manual_override_until_ms_`. +По истечении — `committed_signature_.clear()` → форс relayout. + +## 8. extern "C" ABI shim + +`composer_c_api.cpp` — тонкая обёртка: + +```cpp +extern "C" int cfc_composer_create(const cfc_composer_config_t* cfg, + cfc_composer_t** out) +{ + cfc::ComposerConfig cpp_cfg = {/* ... */}; + auto* comp = new (std::nothrow) cfc::Composer(cpp_cfg); + if (!comp->ok()) { delete comp; return -1; } + // manual_cells из cfg->cells → set_manual_cells() + *out = reinterpret_cast(comp); + return 0; +} +``` + +`cfc_composer_t` — opaque в C, `reinterpret_cast` к/от `cfc::Composer*` +в shim'е. + +`layouts_c_api.cpp` — аналогично для `cfc_layout_*`. Держит static +кеш `vector` который пересинхронизируется с +`cfc::current_templates()` при reload. + +## 9. Build + +### 9.1 CMake + +```cmake +project(cuframes-composer LANGUAGES C CXX CUDA) +set(CMAKE_CXX_STANDARD 17) +``` + +`COMPOSER_SOURCES_CPP` в `src/CMakeLists.txt` перечисляет все .cpp. + +### 9.2 Host build (для CI / dev-машины Ubuntu 24.04) + +```bash +cd /home/claude/projects/cuframes-composer +mkdir -p build && cd build +cmake -DCMAKE_BUILD_TYPE=Release .. +make -j$(nproc) +``` + +### 9.3 Production build (Ubuntu 22.04 jammy) + +См. [operations.md](operations.md). Кратко: +```bash +docker run --rm --gpus all \ + -v "$PWD":/src -w /src/build-jammy \ + cuframes-composer-builder:cached \ + bash -c 'cmake .. && make -j$(nproc)' +``` + +## 10. Производительность + +### 10.1 Гарантии zero-copy + +- Один `CudaBuffer output_` на `Composer`, передаётся как `NV12Ref` всем + cells/decorations +- Cell не создаёт VRAM-allocation'ов на кадр (только `Decoration::draw` + может делать FreeType-rebuild при `set_text` — это offline) +- Layout::apply() пересоздаёт `vector>` только при + смене template'а; обычно раз в N секунд + +### 10.2 Virtual call overhead + +`Cell::draw()` → `draw_content()` virtual call: 1 indirect call per cell +per frame. При 25 fps × 16 cells = **400 calls/sec** — нерелевантно. + +### 10.3 Hot path + +CUDA kernels (`cugrid.cu`): +- `fill_nv12` — 1 kernel launch на rect +- `resize_nv12` — bilinear через 2 kernels (Y plane + UV plane) +- `blit_rgba_nv12` — 1 kernel, RGBA → NV12 + alpha-blend + +**Все Cell-операции конвертируются в N×fill_nv12 + 1×resize_nv12 + +M×blit_rgba_nv12** — оптимизировать через batching пока не нужно +(GPU не sticky на текущей нагрузке). + +## 11. Где править распространённые задачи + +| Хочу… | Файл | +|---|---| +| Изменить hysteresis | `composer.hpp` → `ComposerConfig::shrink_hysteresis_ms` | +| Поменять цвет border'а cells | `layout.cpp` → `border_style.color_y/u/v` | +| Поменять label-font | `mqtt_overlay.cpp` → `LabelDecoration` style + `LabelStyle::font_path` | +| Добавить ZMQ-verb | `control.c::dispatch()` + новая `cmd_*` функция | +| Поменять manual override длительность | `composer.hpp` → `manual_override_duration_ms_` | +| Добавить новый MQTT-overlay anchor | `mqtt_overlay.cpp::reposition_overlay()` switch | +| Поддержать color emoji | `overlay.c::text_rebuild_atlas()` — handle `FT_LOAD_COLOR` + bitmap BGRA → blit как RGBA | + +## 12. Тестирование + +### 12.1 Юнит (не реализован) + +Запланировано: catch2-тесты для `pick_best_fit`, `collect_active`, +hysteresis. Зависимость на CUDA — mock через `cfc_source_get_latest` +шим. + +### 12.2 Integration smoke (`examples/grid_record_cpp.cpp`) + +Минимальный C++ smoke: init Composer, compose loop, dump NV12 → файл. +Не использует NVENC (тестирует только композицию). + +```bash +build/examples/grid_record_cpp \ + --out=/tmp/dump.nv12 --frames=10 --width=1920 --height=1080 \ + --templates=docker/templates.json \ + --source=cam-parking,frigate=parking_overview,priority=100 \ + --motion-mode +``` + +### 12.3 Production smoke + +`docker logs cfc-grid | grep -E "loaded|template|motion|grow|shrink"` — +живая телеметрия композитора. + +## 13. Известные подводные камни + +- **`cfc_overlay_t` не RAII** — managed через `cfc_composer_add_overlay` / + `cfc_overlay_destroy` (Composer.cpp::~Composer уничтожает все). +- **`pthread_mutex_t` в SourcePool** vs `std::mutex` — выбор `std::mutex` + для C++ слоя, `pthread_mutex_t` для C-слоя (PoolEntry::last_motion_ms + использует `std::atomic`). +- **Compose loop НЕ blocking** на CUDA-операциях — `cudaStreamSynchronize` + вызывается caller'ом (grid_record.c) перед NVENC. +- **Frigate event может прийти ДО Layout::apply** — `motion_pulse` + обновляет `last_motion_ms`, но cell для этой камеры ещё может не + существовать. На следующем кадре `maybe_relayout` пересчитает. +- **`dynamic_cast` в `Layout::find_camera_cell_rect`** — + использует RTTI. Включён `-frtti` по умолчанию в g++/nvcc. + +## 14. Workflow по изменению Phase-задач + +1. Branch `feature/-` от `main` +2. Реализация, host build PASS, jammy build PASS (см. operations.md) +3. Bake image `gx/cuframes-composer:-stepN` +4. Deploy на dev-target → smoke verify через VLC/logs +5. Commit + push branch +6. (Если нужно) — merge в `main` через `--no-ff` PR-style + +Для multi-commit фазы: один WIP-merge commit на main с описанием +ключевых изменений. diff --git a/docs/ru/operations.md b/docs/ru/operations.md new file mode 100644 index 0000000..6d3691e --- /dev/null +++ b/docs/ru/operations.md @@ -0,0 +1,404 @@ +# cfc-grid — operations / deploy / troubleshooting + +> Аудитория: тот кто билдит, деплоит, мониторит cfc-grid в проде. +> +> Если ты пользователь — см. [user.md](user.md). Если разработчик — +> см. [developer.md](developer.md). + +## 1. Production setup (R9-88.23) + +### 1.1 Стек + +``` +docker compose -f docker-compose.yml \ + -f cuda-grid/docker-compose.override.yml \ + -f cuframes-composer/docker-compose.override.yml \ + -f onvif/docker-compose.override.yml \ + up -d +``` + +Файлы — в `localhost-infra/hosts/R9-88.23/docker/cctv/`. + +| Сервис | Image | Назначение | +|---|---|---| +| `cuframes-ipc-anchor` | `gx/cuframes:0.4` | Shared VMM IPC anchor для cuframes | +| `cuframes-pub-*` (parking/back_yard/front_yard/gate_lpr) | `gx/cuframes:0.4` | RTSP → cuframes per-camera publishers | +| `cuda-grid-mediamtx` | `bluenviron/mediamtx` | RTSP/HLS/WebRTC gateway | +| `cctv-mosquitto` | `eclipse-mosquitto` | MQTT broker (+bridge к 192.168.88.4) | +| **`cfc-grid`** | `gx/cuframes-composer:0.11b-step1` | Композитор (главный сервис) | +| `cfc-grid-ffmpeg` | `ffmpeg-vf-cuda-grid:phase4b-final` | H.264 pipe → RTSP push | +| `cfc-grid-watchdog` | `gx/cuda-grid-watchdog:0.4` | Restart cfc-grid при stuck inboundBytes | +| `cctv-onvif` | `gx/cctv-onvif:0.6` | ONVIF discovery + PTZ → ZMQ | +| `cctv-frigate` | `ghcr.io/blakeblackshear/frigate` | Object detection → MQTT events | + +### 1.2 Поток кадров + +``` +cuframes-pub-X ──VMM──┐ +cuframes-pub-Y ──VMM──┼──→ cfc-grid (composer) +cuframes-pub-Z ──VMM──┘ │ + │ H.264 NVENC + ↓ named pipe /tmp/cfc-pipe-dir/grid.h264 + cfc-grid-ffmpeg (re-mux) + │ RTSP push + ↓ + cuda-grid-mediamtx + rtsp://*/cfc-grid (TCP/UDP) + http://*:8888/cfc-grid (HLS) + http://*:8889/cfc-grid (WebRTC) +``` + +### 1.3 Сети + +- Внутренний docker network: `cctv` +- Внешние порты на R9-88.23: + - `554/tcp` — RTSP (mediamtx) + - `8888/tcp` — HLS (mediamtx) + - `8889/tcp` — WebRTC (mediamtx) + - `5599/tcp` — ZMQ control plane composer'а + - `8085/tcp` — ONVIF SOAP (cctv-onvif) + - `3702/udp` — WS-Discovery multicast (cctv-onvif) + +## 2. Build + +### 2.1 Local host build (Ubuntu 24.04, dev машина) + +```bash +cd /home/claude/projects/cuframes-composer +cmake -B build -DCMAKE_BUILD_TYPE=Release +cmake --build build -j$(nproc) +``` + +Артефакты в `build/src/libcuframes_composer.so` и `build/examples/grid_record`. + +**ВАЖНО:** host'овый бинарь (Ubuntu 24.04, glibc 2.39, libavformat60) +**несовместим** с runtime контейнером (Ubuntu 22.04 jammy, glibc 2.35, +libavformat58). См. memory `incremental-ffmpeg-rebuild`. + +### 2.2 Jammy build (для production image) + +Использует кешированный builder-контейнер `cuframes-composer-builder:cached` +(Ubuntu 22.04 + nvidia/cuda:12.4.1-devel + apt-deps): + +```bash +cd /home/claude/projects/cuframes-composer + +# Если builder ещё не закеширован: +docker image inspect cuframes-composer-builder:cached >/dev/null 2>&1 || { + docker run -d --name cfc-builder-tmp \ + nvidia/cuda:12.4.1-devel-ubuntu22.04 sleep 3600 + docker exec cfc-builder-tmp bash -c ' + apt-get update -qq && apt-get install -y -qq --no-install-recommends \ + build-essential cmake git pkg-config \ + libpng-dev libfreetype-dev \ + libzmq3-dev libjson-c-dev libmosquitto-dev \ + libavformat-dev libavcodec-dev libavutil-dev' + docker commit cfc-builder-tmp cuframes-composer-builder:cached + docker rm -f cfc-builder-tmp +} + +# Сам build: +docker run --rm --gpus all -v "$PWD":/src -w /src/build-jammy \ + cuframes-composer-builder:cached \ + bash -c 'cmake -DCMAKE_BUILD_TYPE=Release .. && make -j$(nproc)' +``` + +Артефакты в `build-jammy/`. + +### 2.3 Bake image (incremental — без `docker build`) + +Не используем `docker build` (4GB CUDA pull при cache miss). Вместо: + +```bash +docker rmi gx/cuframes-composer:0.11b-step1 -f 2>/dev/null +CID=$(docker create gx/cuframes-composer:0.10) +docker cp build-jammy/examples/grid_record "$CID":/usr/local/bin/grid_record +docker cp build-jammy/src/libcuframes_composer.so.0.1.0 \ + "$CID":/usr/lib/x86_64-linux-gnu/libcuframes_composer.so.0 +docker cp docker/templates.json "$CID":/opt/templates.json +docker cp docker/mqtt_overlays.json "$CID":/opt/mqtt_overlays.json +docker commit \ + --change 'ENTRYPOINT ["/usr/local/bin/grid_record"]' \ + --change 'CMD ["--help"]' \ + --change 'ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility,video' \ + "$CID" gx/cuframes-composer:0.11b-step1 +docker rm "$CID" +``` + +Использует базу `gx/cuframes-composer:0.10` (с уже установленными runtime +deps) + накладывает свежие артефакты. Быстрее и без сетевого трафика. + +### 2.4 Build ONVIF image + +```bash +cd hosts/R9-88.23/docker/cctv/onvif +docker build -t gx/cctv-onvif:0.6 -f Dockerfile . +``` + +Python image, лёгкий. Если меняешь `server.py` — rebuild image (тег +поднимать) + правишь image в `docker-compose.override.yml`. + +## 3. Deploy + +### 3.1 Прод (R9-88.23) + +```bash +cd /home/claude/projects/localhost-infra/hosts/R9-88.23/docker/cctv +docker compose -f docker-compose.yml \ + -f cuda-grid/docker-compose.override.yml \ + -f cuframes-composer/docker-compose.override.yml \ + -f onvif/docker-compose.override.yml \ + up -d cfc-grid +``` + +Compose автоматически recreate'нет контейнер если image tag поменялся в +`docker-compose.override.yml`. + +### 3.2 Verify post-deploy + +```bash +# Logs композитора +docker logs --tail 30 cfc-grid 2>&1 | grep -iE "loaded|template|pool|motion" + +# Ожидаем что-то типа: +# [cfc/loader] /opt/templates.json: loaded 9 templates +# [cfc/composer] templates loaded: 9 (path='/opt/templates.json') +# [cfc/composer] pool+ 'cam-parking' (frigate=parking_overview prio=100) +# [cfc/composer] motion_mode=1 ttl=45000ms pool=4 +# [cfc/composer] grow → template='tpl_3' active=3 +``` + +### 3.3 Rollback + +```bash +sed -i 's|gx/cuframes-composer:0.11b-step1|gx/cuframes-composer:0.10|' \ + hosts/R9-88.23/docker/cctv/cuframes-composer/docker-compose.override.yml +docker compose ... up -d cfc-grid +``` + +## 4. Logs + +### 4.1 Live tail + +```bash +docker logs -f --tail 50 cfc-grid +docker logs -f --tail 30 cfc-grid-ffmpeg +docker logs -f --tail 30 cuda-grid-mediamtx +docker logs -f --tail 30 cctv-onvif +docker logs -f --tail 30 cctv-frigate +``` + +### 4.2 Telemetry pattern + +| Маркер | Что значит | +|---|---| +| `[grid_record] N кадров, M IDR, X МБ за Y.0с (25.0 fps)` | Composer успешно encode'ит каждые ~50 кадров | +| `[cfc/composer] grow → template='X'` | Применился новый template (расширение, мгновенно) | +| `[cfc/composer] shrink → template='X'` | Применился новый template после hysteresis (сжатие) | +| `[cfc/composer] manual override 'X' до +60000ms` | PTZ через ONVIF | +| `[cfc/composer] manual override expired, возврат в motion-mode` | Auto-возврат после 60s | +| `[cfc/mqtt-overlay/] ''` | MQTT-overlay получил/отрендерил новый text | +| `[cfc/frigate] connected, subscribe 'frigate/events'` | Frigate-subscriber подключился | +| `[cfc/temp] update: 'XX.X°C'` | (старый код, deprecated — теперь mqtt-overlay) | + +### 4.3 Когда что-то сломалось + +| Симптом | Где искать | +|---|---| +| `src active=0 stale=0 dead=4` | cuframes-pub-* контейнеры; проверь `docker ps` и сетевой доступ к камерам | +| `overlay 0 draw failed` | `cfc_overlay_text_rebuild_atlas` — обычно невалидный шрифт или текст | +| RTSP стрим не отдаёт | `cfc-grid-ffmpeg` логи; смотри §6.1 | +| TV/ONVIF не находит | `cctv-onvif` логи; проверь multicast WS-Discovery в LAN | + +## 5. Monitoring + +### 5.1 MQTT health + +`cfc-grid` публикует health в `cuda_grid/health/composer/cfc-grid` +каждые ~10 секунд: + +```json +{ + "uptime_s": 3600, + "frames_encoded": 90000, + "fps_actual": 25.0, + "bitrate_kbps": 6000, + "src_active": 4, + "src_stale": 0, + "src_dead": 0, + "idr_count": 1 +} +``` + +```bash +PW=$(grep '^COMPOSER_MQTT_PASSWORD=' \ + /home/claude/projects/localhost-infra/hosts/R9-88.23/docker/cctv/.env | cut -d= -f2) +mosquitto_sub -h 192.168.88.23 -u composer -P "$PW" \ + -t 'cuda_grid/health/composer/cfc-grid' -v +``` + +### 5.2 Watchdog + +`cfc-grid-watchdog` — отдельный сервис, мониторит mediamtx +`inboundBytes` для пути `cfc-grid`. Если **30 секунд молчания** — +`docker restart cfc-grid`. + +Логи watchdog'а: +```bash +docker logs --tail 30 cfc-grid-watchdog +``` + +При срабатывании — публикует в `cuda_grid/health/watchdog/cfc-grid`. + +## 6. Troubleshooting + +### 6.1 RTSP не отдаёт / `cfc-grid-ffmpeg` в "Broken pipe" + +**Симптом:** `docker logs cfc-grid-ffmpeg` показывает +`[out#0/rtsp] Task finished with error code: -32 (Broken pipe)`. + +**Причина:** `--intra-refresh` в composer'е (без IDR-burst'ов), mediamtx +рвёт RTSP-publisher если не может отдать новому клиенту start-frame. + +**Лечение:** +- Полный restart pipeline: + ```bash + docker compose ... restart cfc-grid-ffmpeg cfc-grid cuda-grid-mediamtx + ``` +- Если повторяется — отключить `--intra-refresh` в compose override + (стоимость: IDR-bursts в bitrate, но стабильнее для downstream + клиентов с frequent disconnect/reconnect) + +### 6.2 ffmpeg не получает кадры от RTSP + +**Симптом:** при `ffmpeg -i rtsp://192.168.88.23:554/cfc-grid -frames:v 1 out.jpg` +зависает на 30+ секунд. + +**Причина:** Composer пишет H.264 без regular IDR (intra-refresh). Новый +RTSP-клиент ждёт keyframe для старта декодинга. ffmpeg в default +конфигурации не ждёт достаточно долго. + +**Workaround:** +```bash +ffmpeg -rtsp_transport tcp \ + -analyzeduration 10000000 -probesize 10000000 \ + -i rtsp://192.168.88.23:554/cfc-grid \ + -frames:v 1 -y out.jpg +``` + +Или используй HLS: +```bash +ffmpeg -i http://192.168.88.23:8888/cfc-grid/index.m3u8 \ + -frames:v 1 -y out.jpg +``` + +### 6.3 MQTT-overlay не обновляется + +**Чек-лист:** + +1. Бридж к HA broker (192.168.88.4) работает? + ```bash + docker logs cctv-mosquitto 2>&1 | grep -i 'bridge' + ``` + Ищи `Connecting bridge ha-bridge` и подтверждение connect. + +2. Нужный topic в bridge config? + ```bash + docker exec cctv-mosquitto grep 'topic.*in 0' /mosquitto/config/mosquitto.conf + ``` + Если новый префикс — добавь `topic XXX/# in 0` и restart mosquitto. + +3. Subscriber подключился? + ```bash + docker logs cfc-grid 2>&1 | grep 'mqtt-overlay/.*connected' + ``` + +4. Тестовый publish: + ```bash + mosquitto_pub -h 192.168.88.4 -t '<твой topic>' -m 'test' -r + ``` + В логах composer'а должно появиться `[cfc/mqtt-overlay/] 'test'`. + +### 6.4 Motion-mode не переключает layout + +**Чек-лист:** + +1. Frigate шлёт events? + ```bash + mosquitto_sub -h 192.168.88.23 -u composer -P "$PW" \ + -t 'frigate/events' -C 3 + ``` + +2. Composer получает events? + ```bash + docker logs cfc-grid 2>&1 | grep 'frigate.*started\|grow\|shrink' + ``` + +3. Camera-name матчится? + `frigate=<имя>` в `--source` должно совпадать с `event.after.camera`. + +4. Zone-filter не отсекает? + Если `zones=A:B:C` в `--source` — посмотри в Frigate event + `current_zones`. Если пусто или не пересекается — pulse отбрасывается. + +5. TTL не истёк? + Logs `motion_ttl=45000` (45 сек) — если события приходят реже — + камера выпадает из active. + +### 6.5 ONVIF PTZ presets пусты в TV + +**Причина:** TV закешировал старый ответ `GetPresets` (Phase 9 имена). + +**Лечение:** удалить и заново добавить камеру в TV-клиенте. + +### 6.6 Templates загрузились но motion-mode не использует новый + +Composer читает global registry `cfc::current_templates()` на каждом +кадре — изменение через `cfc_layout_load_file` (ZMQ или CLI) должно +быть подхвачено сразу. Если нет — проверь: + +```bash +echo '{"cmd":"list_layouts"}' | python3 -c " +import zmq,json,sys +s = zmq.Context().socket(zmq.REQ) +s.connect('tcp://192.168.88.23:5599') +s.send_json({'cmd':'list_layouts'}) +print(json.dumps(s.recv_json(), indent=2, ensure_ascii=False))" +``` + +Поле `source` показывает текущий загруженный path. Если built-in (только +`tpl_1` + `tpl_4`) — JSON не подгрузился (syntax error, кривой path). + +## 7. Конфиги в репо + +| Что | Где | +|---|---| +| templates.json | `cuframes-composer/docker/templates.json` | +| mqtt_overlays.json | `cuframes-composer/docker/mqtt_overlays.json` | +| compose override | `localhost-infra/hosts/R9-88.23/docker/cctv/cuframes-composer/docker-compose.override.yml` | +| ONVIF config | `localhost-infra/.../onvif/onvif.yaml` | +| ONVIF server | `localhost-infra/.../onvif/server.py` | +| Mosquitto config | `localhost-infra/.../cctv/mosquitto/config/mosquitto.conf` | +| .env (passwords) | `localhost-infra/.../cctv/.env` (gitignored) | + +После изменения compose override — `docker compose ... up -d cfc-grid` +автоматически recreate'нет. + +## 8. Известные ограничения / TODO + +- **`--intra-refresh` ↔ RTSP-clients**: trade-off bitrate vs latency + (см. §6.1) +- **Watchdog только cfc-grid**: cfc-grid-ffmpeg в зомби-state не + детектится напрямую; помогает только полный restart +- **Hot-reload mqtt_overlays.json**: нет ZMQ verb'а +- **MQTT-overlay per-broker config**: всё через один broker; для + внешнего broker'а нужно расширить `MqttBrokerCfg` per-item + +## 9. См. также + +- [user.md](user.md) — настройка композитора +- [developer.md](developer.md) — внутренности, добавление модулей +- `memory/host-and-project.md` — общая инфра R9-88.23 +- `memory/project_cfc-grid-deployed.md` — deploy 1-го прода +- `memory/project_cfc-grid-cpp-refactor.md` — Phase 11b refactor +- `memory/incremental-ffmpeg-rebuild.md` — incremental docker recipe diff --git a/docs/ru/user.md b/docs/ru/user.md new file mode 100644 index 0000000..cb10627 --- /dev/null +++ b/docs/ru/user.md @@ -0,0 +1,470 @@ +# cfc-grid — руководство пользователя + +> Аудитория: администратор инсталляции. Кто настраивает камеры, layout'ы, +> overlay'и, смотрит RTSP-поток на TV или в браузере. Не правит C++ код. +> +> Если ты разработчик и хочешь добавить новый тип ячейки/декорации/виджета — +> см. [developer.md](developer.md). + +## 1. Что это + +**cfc-grid** — CUDA-композитор, собирающий N камер в один RTSP-поток +`rtsp://192.168.88.23:554/cfc-grid` (1920×1080, H.264, NVENC). Раскладка +выбирается автоматически по движению (от Frigate) или вручную через +ONVIF-presets с TV. + +Параллельно с видеокадром поверх рисуются: + +- **Borders** — серые рамки 2 px вокруг каждой ячейки +- **Labels** — подпись `имя prio=N` в углу каждой камеры +- **Detection boxes** — рамки объектов от Frigate, повторяющие позицию + камеры при смене layout +- **MQTT-overlays** — текстовые поля привязанные к топикам MQTT + (температура, статусы, чаты) + +## 2. Архитектура одной фразой + +``` +cuframes-pub-* (на камеру) + ↓ shared VMM + cfc-grid (composer) ── ZMQ control ──┐ + ↓ pipe (H.264) │ + cfc-grid-ffmpeg (relay) ─→ mediamtx ─┴─→ TV / VLC / Frigate / ... + ↑ + ONVIF discovery от cctv-onvif +``` + +Кадры через cuframes идут zero-copy (один VMM-буфер, разделяемый между +publisher'ом и composer'ом). Композитор берёт NV12-поверхность, +ресайзит/блитит в свой output, добавляет декорации, отдаёт NVENC, +NVENC пишет H.264 в pipe, `cfc-grid-ffmpeg` транскодирует pipe → RTSP +к mediamtx. + +## 3. Motion-mode — основной режим работы + +### 3.1 Что происходит + +На каждом кадре композитор: + +1. Берёт `last_motion_ms` для каждой камеры (обновляется из Frigate MQTT + `frigate/events`) +2. Считает «активными» те у кого `(now - last_motion_ms) < motion_ttl_ms` + (по умолчанию **45 секунд**) +3. Сортирует активных по `priority` (число; больше = главнее) +4. Выбирает template из `templates.json` по правилу **best-fit**: + минимальный template с `nb_camera_cells >= количество_активных` +5. Если в template'е больше camera-cells, чем активных — лишние + заполняются остальными drawable камерами из pool (по приоритету) +6. Применяет **asymmetric hysteresis**: рост числа активных переключает + layout мгновенно, уменьшение — ждёт 3 секунды (чтобы не мелькало) + +### 3.2 Что значит «drawable» + +Камера **исключается** из pool если её `cfc_source_state_t` = +`CONNECTING`, `DISCONNECTED` или `DEAD` (cuframes publisher молчит +дольше `dead_threshold_ms`, по умолчанию 5 секунд). + +`STALE` (кадры приходят редко) — считается, рисуется последний доступный +кадр. + +### 3.3 Manual override через PTZ + +В TV в ONVIF PTZ-presets отображаются имена templates (`tpl_1, tpl_3, +tpl_4, ..., tpl_16`). Нажатие `GotoPreset` или movement-кнопок: + +- Применяет выбранный layout мгновенно +- **Замораживает** motion-mode на 60 секунд +- По истечении — возвращается в auto-режим + +ContinuousMove (стрелки): pan/tilt циклируют по списку presets, zoom-in +=`tpl_1` (full screen), zoom-out=`tpl_16` (4×4 grid). + +## 4. templates.json — раскладка экрана + +### 4.1 Схема + +```json +{ + "version": 1, + "grid_cols": 8, + "grid_rows": 8, + "templates": [ + { + "name": "tpl_N", + "_desc": "Описание", + "priority": 0, + "cells": [ + { + "col": 0, "row": 0, + "cs": 4, "rs": 4, + "role": "camera", + "order": 0 + }, + { + "col": 6, "row": 0, + "cs": 2, "rs": 6, + "role": "widget", + "widget": "temp_chart" + } + ] + } + ] +} +``` + +**Грид: 8×8 микроячеек** (240×135 px каждая = 16:9 на output 1920×1080). +Любой квадрат N×N микроячеек тоже 16:9. + +| Поле | Значение | +|---|---| +| `col`, `row` | Top-left угол cell в микроячейках (0..7) | +| `cs`, `rs` | Размер в микроячейках | +| `role` | `camera` либо `widget` | +| `order` | Для `camera`: порядок placement'а активных (0 = главная, обычно крупнейшая cell) | +| `widget` | Для `widget`: имя placeholder'а (текст подписи) | + +### 4.2 Best-fit selection + +Композитор выбирает template для текущего числа активных: + +``` +candidates = [t for t in templates if t.nb_camera_cells >= n_active] +pick = min(candidates, key=lambda t: t.nb_camera_cells - n_active) +# При равенстве: больший priority побеждает +``` + +Если активных больше чем cell'ов в самом большом template — берётся +самый большой, лишние камеры обрезаются (низший priority вылетает). + +### 4.3 Встроенные templates + +По умолчанию 9 templates в `/opt/templates.json`: + +| Имя | Cells | Описание | +|---|---|---| +| `tpl_1` | 1 cam | Одна камера во весь экран | +| `tpl_3` | 3 cam + 2 widgets | Главная 1440×810 слева + 2 превью справа + 2 widget | +| `tpl_4` | 4 cam | Quad 2×2, 960×540 каждая | +| `tpl_5` | 5 cam + 1 widget | Главная + 4 превью справа стопкой + widget снизу | +| `tpl_6` | 6 cam + 1 widget | Главная + 3 правые + 2 нижние + widget | +| `tpl_7` | 7 cam + 1 widget | Главная + 3 правые + 3 нижние + widget | +| `tpl_8` | 8 cam (1+3+4) | Главная + 3 правые + 4 в нижней строке | +| `tpl_9` | 9 cam + 2 widgets | 3×3 главных + widget справа + widget снизу | +| `tpl_16` | 16 cam | 4×4 grid, 480×270 каждая | + +Подробности — см. `docker/templates.json` в репо. + +### 4.4 Как добавить свой template + +1. Открыть `docker/templates.json` (или подмонтированный override) +2. Добавить блок в `"templates": [...]` по схеме выше +3. Перезапустить cfc-grid (либо `docker exec cfc-grid sh -c 'kill -HUP 1'` + когда добавим hot-reload в Phase 12), либо вызвать ZMQ: + +```bash +mosquitto_pub -h 192.168.88.23 -p 5599 ... # пока нет CLI, см. operations.md +``` + +### 4.5 Координатная математика + +Каждая микроячейка = `1920/8 = 240 px` ширина, `1080/8 = 135 px` высота +(aspect 16:9). Cell `{col=2, row=4, cs=4, rs=2}`: + +- pixel x = `2 * 240 = 480` +- pixel y = `4 * 135 = 540` +- pixel w = `4 * 240 = 960` +- pixel h = `2 * 135 = 270` + +Aspect cell = `cs/rs * 16/9`. Для 16:9 cell'ов **cs == rs**. + +## 5. mqtt_overlays.json — text overlay'и из MQTT + +### 5.1 Схема + +```json +{ + "version": 1, + "overlays": [ + { + "id": "temp_outside", + "topic": "zigbee2mqtt/Температура на улице", + "json_field": "temperature", + "format": "%+.1f°C", + "anchor": "right-bottom", + "margin_x": 32, "margin_y": 24, + "pixel_size": 32, + "color": [255, 255, 255], + "alpha": 230, + "bg_alpha": 160, + "bg_y": 16, "bg_u": 128, "bg_v": 128, + "bg_pad": 10, + "placeholder": "—", + "font_path": "/fonts/DejaVuSans-Bold.ttf" + } + ] +} +``` + +| Поле | Описание | +|---|---| +| `id` | Уникальный идентификатор overlay'я (используется ZMQ для lookup'а) | +| `topic` | MQTT topic для subscribe (через `cctv-mosquitto`) | +| `json_field` | Если payload JSON — имя поля для извлечения значения; **пусто = raw payload как строка** | +| `format` | `printf`-стиль для отформатированного значения (например `"%+.1f°C"`, `"%s"`) | +| `anchor` | Якорь позиционирования: `right-bottom`, `right-top`, `left-bottom`, `left-top`, `center` | +| `margin_x`, `margin_y` | Отступ от ближайшего края экрана (px) | +| `pixel_size` | Размер шрифта в пикселях | +| `color` | RGB цвет текста | +| `alpha` | Общая непрозрачность текста (0..255) | +| `bg_alpha` | Непрозрачность подложки (0..255); **0 = без фона** | +| `bg_y`, `bg_u`, `bg_v` | BT.709 limited-range цвет подложки (Y=16..235, UV=16..240); default чёрный | +| `bg_pad` | Отступ подложки вокруг текста (px) | +| `placeholder` | Что показывать пока не пришло MQTT-сообщение; пусто = "—" | +| `font_path` | Путь к шрифту (.ttf/.otf) в контейнере | + +### 5.2 Поддерживаемые символы + +Шрифт `DejaVuSans-Bold.ttf` — стандартный из пакета `fonts-dejavu`. +**Покрывает Basic Multilingual Plane** (latin, cyrillic, базовые +символы), включая: + +- `❯` (U+276F), `✎` (U+270E), `➤` (U+27A4), `→` (U+2192) +- `★` (U+2605), `▶` (U+25B6), `✉` (U+2709) + +**Emoji из Supplementary Multilingual Plane (>U+10000)** — например +`🗣` (U+1F5E3), `🤖` (U+1F916), `💬` (U+1F4AC) — **не рендерятся**: +шрифт не содержит таких глифов. Рисуется placeholder-квадрат. + +Чтобы добавить color emoji — нужно подключить Noto Color Emoji и +расширить рендерер для COLR/CPAL/SBIX (см. developer.md). + +### 5.3 Примеры + +**Только число**: payload = `"23.5"` (raw), `json_field: ""`, `format: "%s°"` + +**JSON с полем**: payload = `{"temperature": 23.5, "humidity": 45}`, +`json_field: "temperature"`, `format: "%+.1f°C"` + +**Несколько overlay'ев** в правом-верхнем углу столбиком: первый с +`margin_y: 24`, второй с `margin_y: 72`, третий с `margin_y: 120`. + +### 5.4 Как добавить + +1. Открыть `docker/mqtt_overlays.json` (либо подмонтированный override) +2. Добавить блок в массив `"overlays": [...]` +3. Перезапустить cfc-grid + +Hot-reload через ZMQ — Phase 12 (`reload_overlays` verb). + +## 6. CLI флаги composer'а + +В compose override `docker/cctv/cuframes-composer/docker-compose.override.yml`: + +```yaml +command: + - "--out=/out/grid.h264" # named pipe для ffmpeg-relay + - "--fps=25" + - "--bitrate=6000" # kbps + - "--width=1920" + - "--height=1080" + - "--intra-refresh" # вместо IDR-burst'ов (low-latency) + - "--control=tcp://0.0.0.0:5599" # ZMQ control plane + - "--mqtt=cctv-mosquitto:1883" # MQTT для health-публикации + - "--mqtt-instance=cfc-grid" + - "--mqtt-user=composer" + - "--mqtt-pass=${COMPOSER_MQTT_PASSWORD}" + + # Источники (камеры) — повторяемое + - "--source=cam-parking,frigate=parking_overview,priority=100,zones=parking_zone:canopy:private_area" + - "--source=cam-back_yard,frigate=back_yard,priority=70" + # ... + + - "--motion-mode" # включить auto-layout + - "--motion-ttl=45000" # ms + + - "--templates=/opt/templates.json" + - "--mqtt-overlays=/opt/mqtt_overlays.json" + + # Frigate motion-driver и detection boxes + - "--frigate-mqtt=cctv-mosquitto:1883" + - "--frigate-topic=frigate/events" + - "--detection-cell=parking,parking_overview,0,0,960,540,640,480,parking_zone:canopy:private_area" +``` + +### 6.1 `--source` синтаксис + +``` +--source=,frigate=,priority=[,zones=::...] +``` + +- `cuframes_key` — имя cuframes-publisher'а (например `cam-parking`) +- `frigate=NAME` — имя камеры в Frigate (для матчинга motion-событий) +- `priority=N` — целое, больше = главнее +- `zones=...` — опциональный whitelist Frigate zones; motion засчитывается + только если `event.current_zones` пересекается со списком (отсев + street-флуда) + +### 6.2 `--detection-cell` синтаксис + +``` +--detection-cell=,,,,,,,[,zones] +``` + +- `key` — произвольный идентификатор overlay'я для логов +- `frigate_camera` — имя в Frigate (для матчинга event.camera) +- `x,y,w,h` — initial geometry (composer пересчитает динамически) +- `detect_w,detect_h` — разрешение детектора Frigate (например 640×480) +- `zones` — whitelist для bbox + +## 7. ZMQ control plane + +Default endpoint: `tcp://192.168.88.23:5599`. Все verb'ы — JSON request/reply. + +### 7.1 Список verbs + +| Команда | Параметры | Что делает | +|---|---|---| +| `ping` | — | Health-check | +| `health` | — | `{total, active, stale, dead}` по pool'у | +| `set_text` | `id, text, r, g, b, x, y, visible` | Обновить текстовый overlay (для CLI `--text=...`) | +| `set_visible` | `id, visible` | Скрыть/показать overlay | +| `list_overlays` | — | Список overlay'ев | +| `set_layout` | `name` | Применить named template (manual override на 60s в motion-mode) | +| `list_layouts` | — | Список доступных templates с cells | +| `get_layout` | — | Имя текущего template'а | +| `set_motion_mode` | `on, ttl_ms` | Включить/выключить motion-режим | +| `get_motion_mode` | — | Состояние motion-mode | +| `get_template` | `name` | Полный JSON template'а | +| `reload_templates` | `path?` | Перезагрузить templates из файла (default — последний путь) | + +### 7.2 Пример + +```bash +# Python +python3 <