Enable visualization by default for inference and replay

This commit is contained in:
zmv
2026-09-24 15:15:51 +08:00
parent daab1db58f
commit 475fe2440b
7 changed files with 43 additions and 10 deletions
+6 -4
View File
@@ -1,7 +1,7 @@
# Vision Anti-Pinch # Vision Anti-Pinch
C++ / TensorRT 实时摄像头风险推理,算法来自最新 run_risk.py。 C++ / TensorRT 实时摄像头风险推理,算法来自最新 run_risk.py。
持续取图、推理并更新内存中的 FrameResult;可视化可选。 持续取图、推理并更新内存中的 FrameResult;实时推理、录制和回灌默认开启可视化。
相机每次取帧默认等待 **2000 ms**,连续超时 **6 次**才停止流水线(纯超时约 **12 秒**);成功取得有效帧后清零超时计数。单次等待时间由 `config/camera.json` 的 `camera.timeout_ms` 配置。运行中的坏帧、取流错误共用该重试期限;设备失效时尝试重新打开相机,恢复有效帧后继续推理。GPU 错误仍按原有异常处理退出。 相机每次取帧默认等待 **2000 ms**,连续超时 **6 次**才停止流水线(纯超时约 **12 秒**);成功取得有效帧后清零超时计数。单次等待时间由 `config/camera.json` 的 `camera.timeout_ms` 配置。运行中的坏帧、取流错误共用该重试期限;设备失效时尝试重新打开相机,恢复有效帧后继续推理。GPU 错误仍按原有异常处理退出。
@@ -14,15 +14,17 @@ C++ / TensorRT 实时摄像头风险推理,算法来自最新 run_risk.py。
git clone https://git.kendeji.fun/zmv/vision-anti-pinch.git git clone https://git.kendeji.fun/zmv/vision-anti-pinch.git
cd vision-anti-pinch/orin cd vision-anti-pinch/orin
./scripts/initialize_camera.sh # 开机后初始化 /dev/video5 摄像头链路 ./scripts/initialize_camera.sh # 开机后初始化 /dev/video5 摄像头链路
./run.sh # 持续推理,结果只保存在内存 ./run.sh # 持续推理并显示预览,结果只保存在内存
``` ```
可选板端本地可视化: 默认显示本地预览,叠加门区、检测框、手臂/手部关键点及风险信息。需要关闭预览时:
```bash ```bash
./run.sh --visualize ./run.sh --no-visualize
``` ```
`--visualize` 仍可显式开启;没有 DISPLAY / WAYLAND_DISPLAY 时自动跳过窗口并继续推理。
Ctrl+C / SIGTERM 停止程序。 Ctrl+C / SIGTERM 停止程序。
关闭预览窗口或按 q/Esc 只关闭预览,推理继续。 关闭预览窗口或按 q/Esc 只关闭预览,推理继续。
+1 -1
View File
@@ -14,7 +14,7 @@ sha256sum -c SHA256SUMS
./run.sh ./run.sh
``` ```
可视化可选:`./run.sh --visualize`,需要板端本地桌面 DISPLAY。 实时推理、录制和回灌默认开启可视化,显示门区、检测框、手臂/手部关键点及风险信息。使用 `./run.sh --no-visualize` 关闭;`--visualize` 仍可显式开启。需要板端桌面 DISPLAY / WAYLAND_DISPLAY,没有显示环境时自动跳过窗口并继续推理。
默认持续推理直到 Ctrl+C / SIGTERM;正常退出没有统计输出。 默认持续推理直到 Ctrl+C / SIGTERM;正常退出没有统计输出。
预览窗口关闭后推理继续。结果通过 C++ CameraPipeline::latest() 获取,不通过端口或结果文件传递。 预览窗口关闭后推理继续。结果通过 C++ CameraPipeline::latest() 获取,不通过端口或结果文件传递。
+2 -2
View File
@@ -1,11 +1,11 @@
6d5e874fd27fcfd75d303d7e35d9672028af71041f52dcc82a1e83676faefa65 ./bin/drivers/max9295.ko 6d5e874fd27fcfd75d303d7e35d9672028af71041f52dcc82a1e83676faefa65 ./bin/drivers/max9295.ko
8a78d04ef8be44e9b62a5e4ee315d3d6d679e34aedff02a234bbf4d40e222933 ./bin/drivers/max9296.ko 8a78d04ef8be44e9b62a5e4ee315d3d6d679e34aedff02a234bbf4d40e222933 ./bin/drivers/max9296.ko
3da628587477cc35d5e3f8aee86e718e2ddf16808e893e00ba6e164b2307e843 ./bin/drivers/sgx-yuv-gmsl2.ko 3da628587477cc35d5e3f8aee86e718e2ddf16808e893e00ba6e164b2307e843 ./bin/drivers/sgx-yuv-gmsl2.ko
0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11 ./build/camera_risk cd9711d71c6d04f40adf9434f893abe1744e9e2013f1459ddf736a80dcbbfd64 ./build/camera_risk
8a572f661e056e3916758a442e73212842bfe89361dd1b07b87de8f6ae523cf8 ./config/camera.json 8a572f661e056e3916758a442e73212842bfe89361dd1b07b87de8f6ae523cf8 ./config/camera.json
9ba768d559fd3b64e833b50c62c70f2adfd6c169620aea5fba32cfb1b4c5b0e9 ./config/door_mask.json 9ba768d559fd3b64e833b50c62c70f2adfd6c169620aea5fba32cfb1b4c5b0e9 ./config/door_mask.json
ebcd09e8c2e35008adbb0c1d222e580b17e0df1c8093df3481ba3374cd9a8174 ./engines/rtmw_l_384.engine ebcd09e8c2e35008adbb0c1d222e580b17e0df1c8093df3481ba3374cd9a8174 ./engines/rtmw_l_384.engine
f6a65b926ac1144484f4aa93cd26d7997d32d62310ab93fe867f2bd4412222e9 ./engines/yolo11s.engine f6a65b926ac1144484f4aa93cd26d7997d32d62310ab93fe867f2bd4412222e9 ./engines/yolo11s.engine
eb90be468ff835bc2a1a79b83dbb3c9cfe78449e79d005596f887e8ac9981b86 ./README.md 011e8d1e9b735536630a26cb0ee9930f4f6ae52052e26427156fb4d9a6f96e83 ./README.md
7f42b0e7f9396e0d82313a892fc62c153261dab1206a206a466bce6376f58a36 ./run.sh 7f42b0e7f9396e0d82313a892fc62c153261dab1206a206a466bce6376f58a36 ./run.sh
54e1d63e3aa72ad9b056a224d55968d4947ca6703a8d1c405570de5707a7e7ef ./scripts/initialize_camera.sh 54e1d63e3aa72ad9b056a224d55968d4947ca6703a8d1c405570de5707a7e7ef ./scripts/initialize_camera.sh
Binary file not shown.
+7 -3
View File
@@ -2,6 +2,7 @@
#include "config.hpp" #include "config.hpp"
#include "stop_signals.hpp" #include "stop_signals.hpp"
#include "visualizer.hpp" #include "visualizer.hpp"
#include <cstdlib>
#include <functional> #include <functional>
#include <iostream> #include <iostream>
#include <map> #include <map>
@@ -10,7 +11,7 @@ int main(int argc,char** argv)try {
auto root=std::filesystem::canonical("/proc/self/exe").parent_path().parent_path(); auto root=std::filesystem::canonical("/proc/self/exe").parent_path().parent_path();
auto configPath=root/"config/camera.json"; auto configPath=root/"config/camera.json";
cms::OutputOptions output;output.directory=root/"output"; cms::OutputOptions output;output.directory=root/"output";
bool visualize=false; bool visualize=true;
std::filesystem::path recordSession,replaySession; std::filesystem::path recordSession,replaySession;
std::map<std::string,std::function<void(int&)>> arguments{ std::map<std::string,std::function<void(int&)>> arguments{
{"--record-session",[&](int&i){if(++i>=argc)throw std::invalid_argument("--record-session needs a directory");recordSession=argv[i];}}, {"--record-session",[&](int&i){if(++i>=argc)throw std::invalid_argument("--record-session needs a directory");recordSession=argv[i];}},
@@ -19,8 +20,9 @@ int main(int argc,char** argv)try {
{"--save-annotated",[&](int&){output.annotated=true;}}, {"--save-annotated",[&](int&){output.annotated=true;}},
{"--save-results",[&](int&){output.results=true;}}, {"--save-results",[&](int&){output.results=true;}},
{"--visualize",[&](int&){visualize=true;}}, {"--visualize",[&](int&){visualize=true;}},
{"--no-visualize",[&](int&){visualize=false;}},
{"--config",[&](int&i){if(++i>=argc)throw std::invalid_argument("--config needs a file");configPath=argv[i];}}, {"--config",[&](int&i){if(++i>=argc)throw std::invalid_argument("--config needs a file");configPath=argv[i];}},
{"--help",[](int&){std::cout<<"camera_risk [--record-session NEW_DIR] [--replay-session DIR] [--save-original] [--save-annotated] [--save-results] [--config FILE] [--visualize]\nContinuous camera inference; replay runs to EOF without a camera.\nH.264 sessions require a new recording directory. Stop with Ctrl+C or SIGTERM to finalize video.\n";std::exit(0);}} {"--help",[](int&){std::cout<<"camera_risk [--record-session NEW_DIR] [--replay-session DIR] [--save-original] [--save-annotated] [--save-results] [--config FILE] [--visualize|--no-visualize]\nVisualization is enabled by default for live inference, recording and replay.\nContinuous camera inference; replay runs to EOF without a camera.\nH.264 sessions require a new recording directory. Stop with Ctrl+C or SIGTERM to finalize video.\n";std::exit(0);}}
}; };
for(int i=1;i<argc;++i){auto handler=arguments.find(argv[i]);if(handler==arguments.end())throw std::invalid_argument("Unknown option: "+std::string(argv[i]));handler->second(i);} for(int i=1;i<argc;++i){auto handler=arguments.find(argv[i]);if(handler==arguments.end())throw std::invalid_argument("Unknown option: "+std::string(argv[i]));handler->second(i);}
@@ -43,7 +45,9 @@ int main(int argc,char** argv)try {
*/ */
// Optional observer. Closing/failing the window never requests inference stop. // Optional observer. Closing/failing the window never requests inference stop.
if(visualize) { if(visualize) {
try {cms::runLocalVisualization(pipeline);} auto display=std::getenv("DISPLAY");auto wayland=std::getenv("WAYLAND_DISPLAY");
if((!display||!*display)&&(!wayland||!*wayland))std::cerr<<"No display session; inference continues without visualization.\n";
else try {cms::runLocalVisualization(pipeline);}
catch(std::exception const& e){std::cerr<<"Visualization unavailable; inference continues: "<<e.what()<<std::endl;} catch(std::exception const& e){std::cerr<<"Visualization unavailable; inference continues: "<<e.what()<<std::endl;}
} }
pipeline.join(); pipeline.join();
+7
View File
@@ -36,3 +36,10 @@
`0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11` `0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11`
独立包 `orin/build/camera_risk` 已更新,`orin/SHA256SUMS` 全部校验通过。使用独立包实际运行并完整回灌 13 帧,再次核对 `/proc/<pid>/exe` 与上述哈希一致;部署检查日志位于 `/tmp/vap-video-session-deployment-20260924/`。测试结束恢复原来的无常驻进程状态。 独立包 `orin/build/camera_risk` 已更新,`orin/SHA256SUMS` 全部校验通过。使用独立包实际运行并完整回灌 13 帧,再次核对 `/proc/<pid>/exe` 与上述哈希一致;部署检查日志位于 `/tmp/vap-video-session-deployment-20260924/`。测试结束恢复原来的无常驻进程状态。
## 默认开启可视化补充验证(2026-09-24)
实时推理、录制和回灌默认显示全部现有叠加;新增 `--no-visualize`,无显示环境自动跳过预览。已在 Orin 的真实桌面上通过 X11 窗口列表确认默认创建 `CMS Camera Risk` 窗口,并使用更新后的独立包完成真实摄像头 13 帧录制、13 帧完整回灌、录制 SIGTERM(1.267 秒)及回灌 SIGINT(0.916 秒)检查。另验证 `--no-visualize` 和清除 DISPLAY / WAYLAND_DISPLAY 后均能正常完成 TensorRT 回灌。
新二进制 SHA256:`cd9711d71c6d04f40adf9434f893abe1744e9e2013f1459ddf736a80dcbbfd64`。源码构建文件、独立包文件及 `/proc/<pid>/exe` 一致;`orin/SHA256SUMS` 已刷新。完整数据在 `orin_session_results.json` 的 `default_visualization` 中,日志位于 `/tmp/vap-default-preview-20260924/`。此次未重复模拟采集和板端故障注入;仍未覆盖有人员时的关键点显示、窗口按钮/键盘关闭操作。测试前后均无常驻推理进程。
+20
View File
@@ -19,5 +19,25 @@
"binary_sha256": "0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11", "binary_sha256": "0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11",
"running_binary_sha256": "0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11", "running_binary_sha256": "0f178057529eb8be44bb9a639504dfe57a1e353a84d2351fd907304f6d951c11",
"replayed_frames": 13 "replayed_frames": 13
},
"default_visualization": {
"platform": "orin-desktop",
"binary": "/home/orin/vision-anti-pinch-0923/orin/build/camera_risk",
"binary_sha256": "cd9711d71c6d04f40adf9434f893abe1744e9e2013f1459ddf736a80dcbbfd64",
"running_binary_sha256": "cd9711d71c6d04f40adf9434f893abe1744e9e2013f1459ddf736a80dcbbfd64",
"recorded_frames": 13,
"replayed_frames": 13,
"frames_with_people": 0,
"max_raw_pose_error_px": 0.0,
"matching_track_frames": 13,
"matching_decision_frames": 13,
"video_bytes": 317961,
"uncompressed_bgr_bytes": 115015680,
"sigterm_seconds": 1.266686278000634,
"replay_sigint_seconds": 0.9155338489999849,
"interrupted_frames": 2,
"default_window_observed": true,
"no_visualize_replay_passed": true,
"headless_replay_passed": true
} }
} }