cuframes-composer/docs/en/developer.md

# cfc-grid — developer guide

> Audience: developer who edits composer C++ code, adds new
> Cell/Decoration/overlay types, or changes auto-layout logic.
>
> If you're a user — see [user.md](user.md). For deploy/troubleshooting
> see [operations.md](operations.md).

## 1. Architecture (40000-foot view)

```
                ┌──────────────────────────────────────────────┐
                │            cfc::Composer (C++)              │
                │                                              │
                │  ┌──────────┐ ┌──────────┐ ┌──────────────┐  │
                │  │SourcePool│ │  Layout  │ │OutputSurface │  │
                │  │ (pool of │ │ vector<  │ │  CudaBuffer  │  │
                │  │  cfc_    │ │ unique_  │ │   (VMM NV12) │  │
                │  │source_t*)│ │ ptr<Cell>│ │              │  │
                │  └──────────┘ └──────────┘ └──────────────┘  │
                │       ↓             ↓              ↑          │
                │   motion_pulse   apply()         NVENC        │
                └──────────────────────────────────────────────┘
                       ↑                              ↓
              extern "C" ABI shim                  H.264 pipe
              (composer_c_api.cpp,                    ↓
               layouts_c_api.cpp,                ffmpeg-relay
               mqtt_overlay_c_api.cpp)                ↓
                       ↑                          mediamtx
              ┌────────┴────────┐                     ↓
              │                 │                     ↓
        grid_record.c       control.c              clients
        (C, CLI parsing)    (ZMQ verbs)
              ↑                 ↑
              │                 │
         frigate_mqtt.c   health.c, audio.c, writer.c, source.c
         (Frigate events)  (C-only modules)
```

**Principles:**

- **C++17 host-side**, CUDA kernels remain on C
- **Zero-copy:** output VMM buffer is passed as `NV12Ref` to all
  cells/decorations (read-write reference, not copy)
- **Coexistence:** C modules (`source.c`, `nvenc.c`, `frigate_mqtt.c`,
  `health.c`, `writer.c`, `audio.c`, `overlay.c`) are kept, accessed via
  `extern "C"`. Only `composer.c` and `layouts.c` were removed (replaced
  by C++ + ABI shim).
- **`grid_record.c`** (CLI entry point) remains in C, uses public
  `cfc_composer_*` API unchanged

## 2. Source tree

```
include/cuframes_composer/
├── *.h                            # Public C API (extern "C")
│   ├── composer.h                 # cfc_composer_* ABI
│   ├── layouts.h                  # cfc_layout_* ABI
│   ├── overlay.h                  # cfc_overlay_* (text/png/border/detbox)
│   ├── source.h, nvenc.h, frigate_mqtt.h, ...
│   └── ...
└── cpp/                           # C++ headers — public classes
    ├── cuda_raii.hpp              # CudaBuffer, CudaStream (RAII)
    ├── types.hpp                  # Rect, NV12Ref
    ├── cell.hpp                   # abstract Cell + draw()
    ├── camera_cell.hpp            # CameraCell : Cell
    ├── widget_cell.hpp            # WidgetCell : Cell
    ├── blank_cell.hpp             # BlankCell : Cell
    ├── decoration.hpp             # abstract Decoration
    ├── border_decoration.hpp      # BorderDecoration
    ├── label_decoration.hpp       # LabelDecoration (FreeType internal)
    ├── template.hpp               # LayoutTemplate, CellTemplate, kGridCols/Rows
    ├── template_loader.hpp        # JSON parser, current_templates()
    ├── source_pool.hpp            # SourcePool, PoolEntry
    ├── layout.hpp                 # Layout
    ├── composer.hpp               # Composer (main class)
    └── mqtt_overlay.hpp           # MqttOverlayItem + Manager

src/
├── *.c                            # C modules (source/nvenc/health/...)
├── overlay.c                      # All cfc_overlay_t types
├── frigate_mqtt.c                 # Subscribe + motion_pulse + zone-filter
├── cugrid/cugrid.cu               # CUDA kernels (resize, fill, blit)
└── cpp/                           # C++ implementations
    ├── camera_cell.cpp, widget_cell.cpp, blank_cell.cpp
    ├── border_decoration.cpp, label_decoration.cpp
    ├── source_pool.cpp
    ├── template_loader.cpp
    ├── layout.cpp
    ├── composer.cpp
    ├── composer_c_api.cpp         # extern "C" shim for composer
    ├── layouts_c_api.cpp          # extern "C" shim for layouts
    ├── mqtt_overlay.cpp
    └── mqtt_overlay_c_api.cpp

examples/
├── simple_record.c                # 1-source → H.264 file
├── grid_record.c                  # Main CLI entry point (C)
└── grid_record_cpp.cpp            # C++ smoke (uses cfc::Composer directly)
```

## 3. Single frame lifecycle

```cpp
// In compose loop (grid_record.c, via cfc_composer_compose):
NV12Ref ref = composer.compose_frame();
//   1. maybe_relayout():  motion-mode? best-fit template; apply Layout
//   2. compose_clear():    fill output buffer BT.709 black
//   3. Layout::render():   for each cell: draw_content() + decorations[]
//   4. for each overlay:   sync detbox geom + cfc_overlay_draw()
// cudaStreamSynchronize(0);
// nvenc.encode(ref.y_ptr, ref.pitch_y, ref.uv_ptr, ref.pitch_uv, ...);
```

All operations execute on CUDA default stream. Zero-copy:
`ref.y_ptr` / `ref.uv_ptr` are the same `CUdeviceptr`s inside
`composer.output_` (RAII `CudaBuffer`).

## 4. Cell — hierarchy and extension

### 4.1 Abstraction

```cpp
class Cell {
public:
    explicit Cell(const Rect& geom);
    virtual ~Cell() = default;
    Cell(const Cell&) = delete;

    const Rect& geometry() const noexcept;
    void set_geometry(const Rect& r) noexcept;
    void add_decoration(std::unique_ptr<Decoration>);

    void draw(CUstream stream, NV12Ref& dst);   // public — calls draw_content() + decorations

protected:
    virtual void draw_content(CUstream stream, NV12Ref& dst) = 0;
    Rect geom_;
    std::vector<std::unique_ptr<Decoration>> decorations_;
};
```

`Cell::draw()` is final (non-virtual — NVI pattern):
```cpp
void Cell::draw(CUstream stream, NV12Ref& dst) {
    if (geom_.empty()) return;
    draw_content(stream, dst);                 // subclass renders content
    for (auto& d : decorations_) {
        d->draw(stream, dst, geom_);            // overlay decorations on top
    }
}
```

### 4.2 Existing subclasses

| Class | draw_content implementation |
|---|---|
| `CameraCell` | `cfc_source_get_latest()` + `cfc_cugrid_resize_nv12` |
| `WidgetCell` | `cfc_cugrid_fill_nv12` dark grey Y=40 (placeholder) |
| `BlankCell` | `cfc_cugrid_fill_nv12` BT.709 black Y=16 |

### 4.3 How to add a new Cell type

Example: `GraphCell` — draws scrolling chart from MQTT subscription.

1. Create `include/cuframes_composer/cpp/graph_cell.hpp`:

```cpp
#ifndef CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP
#define CUFRAMES_COMPOSER_CPP_GRAPH_CELL_HPP
#include "cell.hpp"
#include <deque>

namespace cfc {

class GraphCell : public Cell {
public:
    GraphCell(const Rect& geom, std::size_t max_points = 60)
        : Cell(geom), max_points_(max_points) {}

    void push_value(double v) {
        if (values_.size() >= max_points_) values_.pop_front();
        values_.push_back(v);
    }

protected:
    void draw_content(CUstream stream, NV12Ref& dst) override;

private:
    std::deque<double> values_;
    std::size_t max_points_;
};

} // namespace cfc
#endif
```

2. Implementation `src/cpp/graph_cell.cpp`:

```cpp
#include "../../include/cuframes_composer/cpp/graph_cell.hpp"
#include "../../include/cuframes_composer/cugrid.h"

namespace cfc {

void GraphCell::draw_content(CUstream stream, NV12Ref& dst) {
    if (geom_.empty() || values_.empty()) return;

    // 1. BG fill
    cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv,
                          geom_.x, geom_.y, geom_.w, geom_.h,
                          /*Y*/ 30, 128, 128, 255);

    // 2. Compute min/max
    double mn = *std::min_element(values_.begin(), values_.end());
    double mx = *std::max_element(values_.begin(), values_.end());
    if (mx == mn) mx = mn + 1.0;

    // 3. Render as thin filled rects (lazy, no bitmap kernel)
    int n = static_cast<int>(values_.size());
    int dx = geom_.w / n;
    for (int i = 0; i < n; i++) {
        double norm = (values_[i] - mn) / (mx - mn);
        int y_px = geom_.y + geom_.h - 2 - (int)(norm * (geom_.h - 4));
        cfc_cugrid_fill_nv12(stream, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv,
                              geom_.x + i * dx, y_px, dx & ~1, 2,
                              /*Y*/ 235, 128, 64, 255);  // limited-range bright
    }
}

} // namespace cfc
```

3. Add to `src/CMakeLists.txt`:
```cmake
set(COMPOSER_SOURCES_CPP
    ...
    cpp/graph_cell.cpp
)
```

4. Use in `Layout::apply()`:

```cpp
// In layout.cpp, widget cells handling:
if (wt->widget == "graph_temp") {
    cells_.push_back(std::make_unique<GraphCell>(r, /*max_points=*/120));
} else {
    cells_.push_back(std::make_unique<WidgetCell>(r, wt->widget));
}
```

5. Wire MQTT → `GraphCell::push_value()` — either via MqttOverlayManager
extension, or new API `cfc::Composer::register_graph_feed()`.

## 5. Decoration — composition in Cell

### 5.1 Abstraction

```cpp
class Decoration {
public:
    virtual ~Decoration() = default;
    virtual void draw(CUstream stream, NV12Ref& dst, const Rect& parent_rect) = 0;
};
```

Decoration knows only parent cell's pixel-rect — positions **relative
to it**. Has no access to Layout/Composer.

### 5.2 Existing subclasses

- **`LabelDecoration`** — FreeType atlas + `cfc_cugrid_blit_rgba_nv12`.
  Persistent VRAM (rebuild on `set_text`). Supports background fill
  (via `cfc_overlay_text_config_t.bg_*` parameters).
- **`BorderDecoration`** — 4 calls of `cfc_cugrid_fill_nv12` (top, bottom,
  left, right rects by `thickness`).

### 5.3 How to add a new Decoration type

Example: `BadgeDecoration` — corner icon (e.g. red "recording" dot).

1. Header `badge_decoration.hpp`:

```cpp
class BadgeDecoration : public Decoration {
public:
    struct Style {
        int corner = 0;          // 0=top-left, 1=top-right, 2=bottom-right, 3=bottom-left
        int size = 12;
        int margin = 8;
        int color_y = 80, color_u = 90, color_v = 240;  // red-ish
        int alpha = 240;
        bool visible = true;
    };
    explicit BadgeDecoration(const Style& s) : style_(s) {}
    void set_visible(bool v) noexcept { style_.visible = v; }
    void draw(CUstream stream, NV12Ref& dst, const Rect& p) override;
private:
    Style style_;
};
```

2. Implementation:

```cpp
void BadgeDecoration::draw(CUstream s, NV12Ref& dst, const Rect& p) {
    if (!style_.visible || style_.alpha <= 0) return;
    int x, y;
    switch (style_.corner) {
        case 0: x = p.x + style_.margin; y = p.y + style_.margin; break;
        case 1: x = p.x + p.w - style_.size - style_.margin; y = p.y + style_.margin; break;
        case 2: x = p.x + p.w - style_.size - style_.margin; y = p.y + p.h - style_.size - style_.margin; break;
        case 3: x = p.x + style_.margin; y = p.y + p.h - style_.size - style_.margin; break;
    }
    x &= ~1; y &= ~1;
    cfc_cugrid_fill_nv12(s, dst.y_ptr, dst.pitch_y, dst.uv_ptr, dst.pitch_uv,
                          x, y, style_.size, style_.size,
                          style_.color_y, style_.color_u, style_.color_v, style_.alpha);
}
```

3. Wire in `Layout::apply()`:

```cpp
if (cc->source()->state() == CFC_SOURCE_STALE) {
    BadgeDecoration::Style bs;
    bs.corner = 1; bs.color_y = 180; bs.color_v = 100;  // yellow
    cell->add_decoration(std::make_unique<BadgeDecoration>(bs));
}
```

## 6. SourcePool and motion state

```cpp
struct PoolEntry {
    std::string cuframes_key;
    std::string frigate_camera;
    int priority;
    cfc_source_t* source;
    std::atomic<int64_t> last_motion_ms;
    std::vector<std::string> required_zones;

    cfc_source_state_t state() const;     // calls cfc_source_get_latest
    bool drawable() const;                 // ACTIVE || STALE
};

class SourcePool {
public:
    int add(key, frigate_camera, priority, zones, SubscribeOpts);
    PoolEntry* by_key(const std::string&);
    PoolEntry* by_frigate_camera(const std::string&);
    template<typename F> void for_each(F&&);
    void motion_pulse(const std::string& frigate_camera,
                       const std::vector<std::string>& current_zones);
};
```

`motion_pulse` is called from `frigate_mqtt.c` for each event. If
`required_zones` is non-empty — match by intersection with
`event.current_zones`, otherwise accept all.

## 7. Auto-layout algorithms

See `cfc::Composer::maybe_relayout()` (src/cpp/composer.cpp).

### 7.1 Best-fit selection

```cpp
const LayoutTemplate* Composer::pick_best_fit(int need) const {
    const auto& reg = current_templates();
    const LayoutTemplate* best = nullptr;
    int best_waste = -1, best_prio = -1;
    for (auto& t : reg) {
        int n = t.nb_camera_cells();
        if (n < need) continue;
        int waste = n - need;
        if (!best || waste < best_waste ||
            (waste == best_waste && t.priority > best_prio)) {
            best = &t; best_waste = waste; best_prio = t.priority;
        }
    }
    if (best) return best;
    // overflow → largest
    for (auto& t : reg) {
        if (!best || t.nb_camera_cells() > best->nb_camera_cells()) best = &t;
    }
    return best;
}
```

### 7.2 Collect active

```cpp
std::vector<PoolEntry*> Composer::collect_active() const {
    std::vector<PoolEntry*> active;
    int64_t now = now_ms_mono();
    pool_.for_each([&](PoolEntry& e) {
        if (!e.drawable()) return;            // DEAD/CONNECTING — skip
        int64_t last = e.last_motion_ms.load();
        if (last == 0) return;                 // never had motion
        if (now - last > cfg_.motion_ttl_ms) return;  // TTL expired
        active.push_back(&e);
    });
    // idle fallback: top-priority drawable
    if (active.empty()) {
        PoolEntry* best = nullptr;
        pool_.for_each([&](PoolEntry& e) {
            if (!e.drawable()) return;
            if (!best || e.priority > best->priority) best = &e;
        });
        if (best) active.push_back(best);
    }
    std::sort(active.begin(), active.end(),
              [](PoolEntry* a, PoolEntry* b) { return a->priority > b->priority; });
    return active;
}
```

### 7.3 Asymmetric hysteresis

Signature `template_name + "|" + sorted_keys` is remembered. If new
set ⊇ committed (growth) → commit immediately. Otherwise wait
`shrink_hysteresis_ms` (default 3000) → commit.

```cpp
bool is_grow = std::includes(nkeys.begin(), nkeys.end(),
                              ckeys.begin(), ckeys.end());
if (is_grow && nkeys.size() == ckeys.size()) is_grow = false;
if (is_grow) { /* commit */ } else {
    if (sig != pending_signature_) {
        pending_signature_ = sig;
        pending_first_seen_ms_ = now;
    }
    if (now - pending_first_seen_ms_ < cfg_.shrink_hysteresis_ms) return;
    /* commit */
}
```

### 7.4 Fill free cells

After picking template and capping by `nb_camera_cells`:

```cpp
if (active.size() < cap) {
    std::vector<PoolEntry*> extras;
    pool_.for_each([&](PoolEntry& e) {
        if (!e.drawable()) return;
        if (e ∈ already_active) return;
        extras.push_back(&e);
    });
    std::sort(extras.begin(), extras.end(), priority_desc);
    while (active.size() < cap && !extras.empty()) {
        active.push_back(extras.front());
        extras.erase(extras.begin());
    }
}
```

### 7.5 Manual PTZ override

`Composer::set_layout(name)` in motion-mode:
```cpp
manual_override_until_ms_ = now + manual_override_duration_ms_;  // default 60s
```

`maybe_relayout()` skips work while `now < manual_override_until_ms_`.
After expiry — `committed_signature_.clear()` → forced relayout.

## 8. extern "C" ABI shim

`composer_c_api.cpp` — thin wrapper:

```cpp
extern "C" int cfc_composer_create(const cfc_composer_config_t* cfg,
                                     cfc_composer_t** out)
{
    cfc::ComposerConfig cpp_cfg = {/* ... */};
    auto* comp = new (std::nothrow) cfc::Composer(cpp_cfg);
    if (!comp->ok()) { delete comp; return -1; }
    // manual_cells from cfg->cells → set_manual_cells()
    *out = reinterpret_cast<cfc_composer_t*>(comp);
    return 0;
}
```

`cfc_composer_t` is opaque in C, `reinterpret_cast` to/from `cfc::Composer*`
in shim.

`layouts_c_api.cpp` is analogous for `cfc_layout_*`. Holds static cache
`vector<cfc_layout_t>` resynced with `cfc::current_templates()` on reload.

## 9. Build

### 9.1 CMake

```cmake
project(cuframes-composer LANGUAGES C CXX CUDA)
set(CMAKE_CXX_STANDARD 17)
```

`COMPOSER_SOURCES_CPP` in `src/CMakeLists.txt` lists all .cpp files.

### 9.2 Host build (for CI / dev machine Ubuntu 24.04)

```bash
cd /home/claude/projects/cuframes-composer
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$(nproc)
```

### 9.3 Production build (Ubuntu 22.04 jammy)

See [operations.md](operations.md). Briefly:
```bash
docker run --rm --gpus all \
  -v "$PWD":/src -w /src/build-jammy \
  cuframes-composer-builder:cached \
  bash -c 'cmake .. && make -j$(nproc)'
```

## 10. Performance

### 10.1 Zero-copy guarantees

- One `CudaBuffer output_` per `Composer`, passed as `NV12Ref` to all
  cells/decorations
- Cell does not create VRAM allocations per frame (only `Decoration::draw`
  may do FreeType rebuild on `set_text` — that's offline)
- Layout::apply() recreates `vector<unique_ptr<Cell>>` only on template
  change; typically every N seconds

### 10.2 Virtual call overhead

`Cell::draw()` → `draw_content()` virtual call: 1 indirect call per cell
per frame. At 25 fps × 16 cells = **400 calls/sec** — irrelevant.

### 10.3 Hot path

CUDA kernels (`cugrid.cu`):
- `fill_nv12` — 1 kernel launch per rect
- `resize_nv12` — bilinear via 2 kernels (Y plane + UV plane)
- `blit_rgba_nv12` — 1 kernel, RGBA → NV12 + alpha-blend

**All Cell operations convert to N×fill_nv12 + 1×resize_nv12 +
M×blit_rgba_nv12** — batching not yet needed (GPU is not sticky on
current load).

## 11. Where to edit common tasks

| I want to… | File |
|---|---|
| Change hysteresis | `composer.hpp` → `ComposerConfig::shrink_hysteresis_ms` |
| Change cell border color | `layout.cpp` → `border_style.color_y/u/v` |
| Change label font | `mqtt_overlay.cpp` → `LabelDecoration` style + `LabelStyle::font_path` |
| Add ZMQ verb | `control.c::dispatch()` + new `cmd_*` function |
| Change manual override duration | `composer.hpp` → `manual_override_duration_ms_` |
| Add new MQTT-overlay anchor | `mqtt_overlay.cpp::reposition_overlay()` switch |
| Support color emoji | `overlay.c::text_rebuild_atlas()` — handle `FT_LOAD_COLOR` + bitmap BGRA → blit as RGBA |

## 12. Testing

### 12.1 Unit (not implemented)

Planned: catch2 tests for `pick_best_fit`, `collect_active`, hysteresis.
CUDA dependency — mock via `cfc_source_get_latest` shim.

### 12.2 Integration smoke (`examples/grid_record_cpp.cpp`)

Minimal C++ smoke: init Composer, compose loop, dump NV12 → file. Does
not use NVENC (tests composition only).

```bash
build/examples/grid_record_cpp \
  --out=/tmp/dump.nv12 --frames=10 --width=1920 --height=1080 \
  --templates=docker/templates.json \
  --source=cam-parking,frigate=parking_overview,priority=100 \
  --motion-mode
```

### 12.3 Production smoke

`docker logs cfc-grid | grep -E "loaded|template|motion|grow|shrink"` —
live composer telemetry.

## 13. Known pitfalls

- **`cfc_overlay_t` is not RAII** — managed via `cfc_composer_add_overlay`
  / `cfc_overlay_destroy` (Composer.cpp::~Composer destroys all).
- **`pthread_mutex_t` in SourcePool** vs `std::mutex` — chose `std::mutex`
  for C++ layer, `pthread_mutex_t` for C layer (PoolEntry::last_motion_ms
  uses `std::atomic`).
- **Compose loop is NOT blocking** on CUDA ops — `cudaStreamSynchronize`
  called by caller (grid_record.c) before NVENC.
- **Frigate event may arrive BEFORE Layout::apply** — `motion_pulse`
  updates `last_motion_ms`, but cell for that camera may not exist yet.
  On next frame `maybe_relayout` recalculates.
- **`dynamic_cast<CameraCell*>` in `Layout::find_camera_cell_rect`** —
  uses RTTI. Enabled `-frtti` by default in g++/nvcc.

## 14. Phase task change workflow

1. Branch `feature/<phase>-<feature>` off `main`
2. Implementation, host build PASS, jammy build PASS (see operations.md)
3. Bake image `gx/cuframes-composer:<phase>-stepN`
4. Deploy to dev target → smoke verify via VLC/logs
5. Commit + push branch
6. (If needed) — merge to `main` via `--no-ff` PR-style

For multi-commit phase: one WIP merge commit on main describing key
changes.