cuframes/docs/launch/frigate-integration-issue.md

Frigate integration issue

Status: DRAFT — review перед публикацией.

Куда: https://github.com/blakeblackshear/frigate

Тип: GitHub Discussion (category: Ideas), не Issue. Причина: это feature proposal, не баг. Frigate активно использует discussions (см. #17033, #20191, #21559 — все три уже жалуются на эту проблему).

Альтернатива: ответить в одной из существующих discussion'ов о NVDEC saturation. Может быть лучше — там уже собралась audience.

---

Title

[Ideas] Reduce NVDEC duplication on multi-consumer cameras via shared CUDA frame buffer (cuframes)

Body

## Problem

When Frigate co-exists with other GPU-using video consumers on the same
camera stream (separate AI processor, custom analytics, recording to a
second NVR, etc.), each process opens its own NVDEC session and decodes
the same H.264/HEVC stream independently. On 16+ cameras at 25 fps this
becomes the bottleneck on consumer GPUs:

- NVDEC sessions are limited (4 concurrent on RTX 30xx/40xx, more on
  workstation cards). Decoder context creation / destruction is not free.
- Each duplicate decode burns PCIe bandwidth pushing the same NV12 frame
  to host memory (in setups that go through `hwdownload`).
- Power draw and thermals scale with redundant decoding.

Related discussions: #17033, #20191, #21559.

## Existing workarounds

- Single Frigate restream and have everything else pull from go2rtc — works
  for re-encoding to TCP/UDP, but every downstream still re-decodes.
- DeepStream `nvstreammux` — solves it but is closed-source NVIDIA stack,
  GStreamer-only, not co-installable with current Frigate ffmpeg pipeline.

## Proposal: cuframes ingest source

[cuframes](https://git.goldix.org/gx/cuframes) (LGPL-2.1+) is a small
library that lets one process decode once into a CUDA ring buffer and any
number of other processes import that buffer via CUDA IPC and consume
**zero-copy** in VRAM.

Concretely for Frigate this would mean a new ffmpeg input source like:

```yaml
cameras:
  driveway:
    ffmpeg:
      inputs:
        - path: cuframes://driveway
          input_args: preset-cuframes
          roles: [detect]

where a sentinel container (one per camera, ~5MB RAM, runs cuframes-rtsp-source) does the actual RTSP pull + NVDEC and Frigate attaches to that pre-decoded stream.

Working integration (early proof)

I've been running this in production for ~24h: a single cuframes-rtsp-source container per camera serves both Frigate (detection role) and a separate C++ analytics pipeline from the same NVDEC session. Frigate gets pre-decoded NV12 frames; no detection or recording behaviour was changed.

Integration guide with full docker-compose and a patched Frigate Dockerfile: https://git.goldix.org/gx/cuframes/src/branch/main/docs/integrations/frigate.md

What I'm asking for

Not a PR yet — first I'd like maintainer / community input on:

1. Would Frigate be open to upstream a cuframes:// input source, or should this stay a third-party patched Frigate image?
2. If upstream — what's the preferred shape: new ffmpeg preset only (zero core code changes), or a first-class decoder: cuframes option in the Frigate config schema?
3. The cuframes library currently requires --ipc and --pid namespace sharing between producer and consumer containers. Frigate uses s6-overlay which is incompatible with --pid share (s6 needs PID 1). The current integration uses a small race-window workaround (troubleshooting #2); a cleaner solution requires either making s6 optional in the Frigate image or moving the IPC handshake to a sidecar pattern.

Limitations of cuframes (full disclosure)

• NVIDIA GPUs only.
• Linux only.
• Producer + consumer must share the same CUDA device.
• NV12 frame format only in v0.1.
• Requires patching FFmpeg with a small (~400 LOC) demuxer; an upstream FFmpeg RFC is in flight separately.

If this looks worth pursuing I'm happy to open a draft PR against a feature branch and iterate.

## Notes на review

- **Tone:** Frigate maintainer (Blake) ценит конкретику и production proof — без них любой feature request кладётся в backlog. У нас есть production proof (24h+) — это сильный аргумент, использован прямо.
- **Не обещаем upstream без request'а** — спрашиваем discussion'ом, не PR'ом. Если Blake скажет "не наш scope, оставайтесь third-party" — это OK; integration guide уже валиден как standalone.
- **Прозрачно про s6-overlay constraint** — это блокирующий issue для clean upstream'а. Лучше упомянуть сразу чем спрятать и получить отказ через 2 недели review.
- **Линки на 3 existing discussions** — показывает что problem подтверждена сообществом, не наша одинокая боль.
- **Не упоминать другие AI-системы** (ANPR, face recognition итд) — Blake уже несколько раз говорил что Frigate scope = детектор и NVR, не platform. Подача "cuframes решает вашу проблему" работает лучше чем "cuframes построит экосистему".