base on # M-detector ## 1.Introduction **M-detector** is a moving event detection package, which determines if a point from LiDAR is moving immediately after its arrival, resulting in a point-by-point detection with a latency of just several microseconds. M-detector is designed based on occlusion principles and can be used in different environments with various types of LiDAR sensors. ### **1.1 Related paper** Our related papers has been accepted by Nature Communications. [Moving Event Detection from LiDAR Stream Points](https://www.nature.com/articles/s41467-023-44554-8). If our code is used in your project, please cite our paper. ### **1.2 Related video** Our accompanying videos are now available on **YouTube** (click below images to open) and [**Bilibili**](https://www.bilibili.com/video/BV1ke411i7t7/?share_source=copy_web). <div align="center"> <a href="https://www.youtube.com/watch?v=SYaig2eHV5I" target="_blank"><img src="img/cover.bmp" alt="video" width="60%" /></a> </div> ### 1.3 Developers The codes of this repo are contributed by: [Huajie Wu (吴花洁)](https://github.com/HuajieWu99), [Yihang Li (李一航)](https://github.com/yihangHKU) and [Wei Xu (徐威)](https://github.com/XW-HKU) ## 2. Prerequisites ### 2.1 **Ubuntu** and **ROS** Ubuntu ≥ 18.04. ROS ≥ Melodic. Follow [[ROS Installation](http://wiki.ros.org/ROS/Installation)] ### 2.2 **PCL** and **Eigen** PCL ≥ 1.8 `sudo apt install libpcl-dev` Eigen ≥ 3.3.4 `sudo apt install libeigen3-dev` ### 2.3 **livox_ros_driver** Follow [livox_ros_driver Installation](https://github.com/Livox-SDK/livox_ros_driver). *Remarks:* * Since the M-detector support Livox serials LiDAR firstly, so the **livox_ros_driver** must be installed and **sourced** before run any M-detector launch file. * How to source? The easiest way is add the line `source $Livox_ros_driver_dir$/devel/setup.bash` to the end of file `~/.bashrc`, where `$Livox_ros_driver_dir$` is the directory of the livox ros driver workspace (should be the `ws_livox` directory if you completely followed the livox official document). ### 2.4 TBB Install gcc-9 g++-9 `sudo add-apt-repository ppa:ubuntu-toolchain-r/test` `sudo apt update` `sudo apt install gcc-9 g++-9` `cd /usr/bin` `sudo rm gcc g++` `sudo ln -s gcc-9 gcc` `sudo ln -s g++-9 g++` Follow [[TBB Installation](https://solarianprogrammer.com/2019/05/09/cpp-17-stl-parallel-algorithms-gcc-intel-tbb-linux-macos/)] (**Note:** change the gcc-9.1/g++-9.1 to gcc-9/g++-9) ## 3. Build Clone the repository and catkin_make: `cd ~/catkin_ws/src` `git clone [email protected]:hku-mars/M-detector.git` `catkin_make` `source devel/setup.bash` (**Note:** change the path for TBB in CMakeList.txt) ## 4. Key Information ### 4.1 Key parameters ``` dataset: 3 #0 for kitti, 1 for nuscenes, 2 for waymo buffer_delay: 0.1 buffer_size: 100000 points_num_perframe: 30000 depth_map_dur: 0.2 max_depth_map_num: 5 max_pixel_points: 5 frame_dur: 0.1 hor_resolution_max: 0.005 ver_resolution_max: 0.01 ``` The parameters are provided in folder "config" for different LiDARs. For methods of parameters tuning, please follow the section 8 introduced in [[Supplementary Information](https://www.nature.com/articles/s41467-023-44554-8)]. To save the label files, please pass the parameter via the corresponding launch files. ### 4.2 Folder structure for dataset ``` ├── XXX (dataset name) │ ├── bags │ │ ├── XXX_0000.bag │ │ ├── ... │ ├── sequences │ │ ├── 0000 │ │ │ ├── labels │ │ │ ├── predictionsx_origin (results in point-out mode with xth parameter file) │ │ │ ├── predictionsx (in frame-out mode with xth parameter file) │ │ │ ├── ... │ │ ├── ... ├── ... ``` The dataset can be downloaded at [[this link](https://drive.google.com/drive/folders/1ASNfrjZB7n9Q-nB4Pm2IwvArFWnTcFAj?usp=drive_link)]. ## 5. Directly Run ### 5.1 Run with odometry and point clouds (in local frame) At first, please run a odometery node, such as [[Fast Lio](https://github.com/hku-mars/FAST_LIO)] (Download Fast Lio provided in Releases into the same location as M-detector's and complie them). Then: `roslaunch fast_lio mapping_XXX(for dataset).launch` `roslaunch m_detector detector_(dataset).launch` `rosbag play YOURBAG.bag` ### 5.2 Generate the label files for every point `roslaunch m_detector detector_XXX.launch out_path:="your path for frame-out results" out_origin_path:="your path for point-out results"` Note: Follow the folder structure introduced before, the `out_path` should be in the format of "(the path to dataset folder)/(dataset name)/sequences/(sequence number)/predictionsx(x is the parameter file's number)/", and the `out_origin_path` should be in the format of "(the path to dataset folder)/(dataset name)/sequences/(sequence number)/predictionsx_origin(x is the parameter file's number)/". ### 5.3 Calculate the IoU of results `roslaunch m_detector cal_recall.launch dataset:=(0 for kitti, 1 for nuscenes, 2 for waymo, 3 for avia) dataset_folder:="the path to the dataset_folder" start_se:=(the first sequence number for calculation) end_se:=(the last sequence number for calculation) start_param:=(the first parameter file's number for calculation) end_param:=(the last parameter file's number for calculation) is_origin:=(true for point-out results, false for frame-out results)` Note: Follow the folder structure introduced before, the `dataset_folder` should be the path to dataset folder. This step will calculate all the IoU for all designated results listed in the dataset folder and generate a new folder named "recall" or "recall_origin" containing the results. ## 6. Run with Embedded in FAST LIO Download the embedded version provided in Releases into a new workspace and complie them. `roslaunch fast_lio mapping_(dataset).launch` `rosbag play YOURBAG.bag` ## 7. Rosbag Download The bags used in paper can be download at [[this link](https://drive.google.com/drive/folders/1ASNfrjZB7n9Q-nB4Pm2IwvArFWnTcFAj?usp=sharing)]. ## 8. License The source code of this package is released under [**GPLv2**](http://www.gnu.org/licenses/) license. We only allow it free for **academic usage**. For commercial use, please contact Dr. Fu Zhang [[email protected]](mailto:[email protected]). For any technical issues, please contact me via email [[email protected]](mailto:[email protected]). ", Assign "at most 3 tags" to the expected json: {"id":"6827","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"