gx
a1090a5f4c
Phase 4a deliverable (no filter rendering yet — это Phase 4b).
End-to-end pipeline: HA/HTTP/MQTT → controller → ZMQ → FFmpeg (logged).
Modules:
- overlays.py — 7 discriminated union types через pydantic:
rect, text, icon, image, dim, graph, chat. Normalized coords (0.0-1.0),
optional cell binding, z_order, opacity, visible.
- state.py — overlay storage per instance (CRUD: add/remove/update/get/clear)
- dispatch.py — overlay.add/remove/clear actions:
- parses JSON payload в Overlay через TypeAdapter
- serializes to ZMQ string: "<id> <type> <full-json>"
- sends via FFmpeg process_command (filter will парсить в Phase 4b)
- updates state + publishes events (overlay_added, overlay_removed, overlays_cleared)
- http_api.py — REST endpoints:
- POST /overlay/{inst}/add (body = Overlay JSON, returns id)
- GET /overlay/{inst} — list all
- DELETE /overlay/{inst}/{id} — single
- DELETE /overlay/{inst} — clear all
- PATCH /overlay/{inst}/{id} — update
- mqtt_loop.py — already subscribes cuda_grid/cmd/<inst>/+/+; teper handles
overlay/add (JSON payload), overlay/remove (id), overlay/clear
- frigate_bridge.py — FrigateBridge skeleton:
- subscribe frigate/+/motion + frigate/events
- mapping camera_name → target_instance + cell index
- Phase 4a: log received events (rendering в Phase 4b)
- config.py — frigate: optional section
- examples/controller.yaml — frigate mappings для 4 наших камер
State management:
- ControllerState.add/remove/update/get/clear_overlay (asyncio.Lock guarded)
- InstanceState.overlays: dict[str, Overlay]
- IDs generated via uuid4()[:8]
Phase 4a limitations:
- Filter side ничего не рендерит (just logs ZMQ commands)
- Frigate bridge принимает events но не auto-generates overlays
- HA Discovery не имеет overlay-specific entities (overlays через REST API)
Phase 4b: filter-side AVFrame side data + CUDA kernels (rect first, NPP-based,
потом text via freetype atlas, потом icon sprite blit).
cuda-grid-controller
Control-plane sidecar для vf_cuda_grid FFmpeg filter. Transforms
MQTT / HTTP REST commands от Home Assistant / Node-RED / custom apps
в FFmpeg process_command через ZeroMQ. Publishes state + events наружу
для bidirectional integration.
Статус: Phase 3 work-in-progress — basic skeleton + HA Discovery + ZMQ command bridge. Phase 4+ добавит overlays, audio orchestration.
Что умеет (Phase 3)
- ✅ MQTT subscribe
cuda_grid/cmd/<instance>/layout/set→ ZMQ send → FFmpeg layout switch - ✅ MQTT publish
cuda_grid/state/<instance>/layout(retained) +cuda_grid/event/.../layout_switched - ✅ HA MQTT Discovery —
select.layout+sensor.current_layout+binary_sensor.onlineper instance - ✅ HTTP REST API:
/health,/layouts,/state,POST /layout/{instance}/set - ✅ Multi-instance (несколько FFmpeg pipelines с разными ZMQ endpoints)
- ✅ Auto-reconnect MQTT, LWT для availability tracking
Использование
pip install -e .
export MQTT_USERNAME=mqtt
export MQTT_PASSWORD=secret
cuda-grid-controller --config examples/controller.yaml
FFmpeg side — как подключить cuda_grid + zmq filter
ffmpeg -i cam1.mp4 -i cam2.mp4 -i cam3.mp4 -i cam4.mp4 \
-filter_complex "
[0:v][1:v][2:v][3:v]cuda_grid=layout=quad,
zmq=bind_address=tcp\\\\://127.0.0.1\\\\:5555
" \
-c:v h264_nvenc out.mp4
(После этого controller с zmq_endpoint: tcp://127.0.0.1:5555 сможет
переключать layout командой cuda_grid@... layout six_grid.)
HA dashboard
После startup controller'а — в Home Assistant появятся entities:
select.cuda_grid_<instance>_layout— dropdown с layoutssensor.cuda_grid_<instance>_current_layout— текущий выбранныйbinary_sensor.cuda_grid_controller_online— online/offline
Roadmap
| Phase | Что |
|---|---|
| 3 (this) | Basic skeleton + HA Discovery + ZMQ bridge для layout switching |
| 4 | Overlay API (rect/text/icon через side data) |
| 5 | Rich overlays (image/dim/graph/chat) + privacy filtering |
| 6 | Audio orchestration (state machine, domofon use case) |
Полный design: docs/design.md.