Deep Sort Pytorch

# Deep Sort with PyTorch  ## Update(1-1-2020) Changes - fix bugs - refactor code - accerate detection by adding nms on gpu ## Latest Update(07-22) Changes - bug fix (Thanks @JieChen91 and @yingsen1 for bug reporting). - using batch for feature extracting for each frame, which lead to a small speed up. - code improvement. Futher improvement direction - Train detector on specific dataset rather than the official one. - Retrain REID model on pedestrain dataset for better performance. - Replace YOLOv3 detector with advanced ones. **Any contributions to this repository is welcome!** ## Introduction This is an implement of MOT tracking algorithm deep sort. Deep sort is basicly the same with sort but added a CNN model to extract features in image of human part bounded by a detector. This CNN model is indeed a RE-ID model and the detector used in [PAPER](https://arxiv.org/abs/1703.07402) is FasterRCNN , and the original source code is [HERE](https://github.com/nwojke/deep_sort). However in original code, the CNN model is implemented with tensorflow, which I'm not familier with. SO I re-implemented the CNN feature extraction model with PyTorch, and changed the CNN model a little bit. Also, I use **YOLOv3** to generate bboxes instead of FasterRCNN. ## Dependencies - python 3 (python2 not sure) - numpy - scipy - opencv-python - sklearn - torch >= 0.4 - torchvision >= 0.1 - pillow - vizer - edict ## Quick Start 0. Check all dependencies installed ```bash pip install -r requirements.txt ``` for user in china, you can specify pypi source to accelerate install like: ```bash pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple ``` 1. Clone this repository ``` git clone git@github.com:ZQPei/deep_sort_pytorch.git ``` 2. Download YOLOv3 parameters ``` cd detector/YOLOv3/weight/ wget https://pjreddie.com/media/files/yolov3.weights wget https://pjreddie.com/media/files/yolov3-tiny.weights cd ../../../ ``` 3. Download deepsort parameters ckpt.t7 ``` cd deep_sort/deep/checkpoint # download ckpt.t7 from https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6 to this folder cd ../../../ ``` 4. Compile nms module ```bash cd detector/YOLOv3/nms sh build.sh cd ../../.. ``` Notice: If compiling failed, the simplist way is to **Upgrade your pytorch >= 1.1 and torchvision >= 0.3" and you can avoid the troublesome compiling problems which are most likely caused by either `gcc version too low` or `libraries missing`. 5. Run demo ``` usage: python yolov3_deepsort.py VIDEO_PATH [--help] [--frame_interval FRAME_INTERVAL] [--config_detection CONFIG_DETECTION] [--config_deepsort CONFIG_DEEPSORT] [--display] [--display_width DISPLAY_WIDTH] [--display_height DISPLAY_HEIGHT] [--save_path SAVE_PATH] [--cpu] # yolov3 + deepsort python yolov3_deepsort.py [VIDEO_PATH] # yolov3_tiny + deepsort python yolov3_deepsort.py [VIDEO_PATH] --config_detection ./configs/yolov3_tiny.yaml # yolov3 + deepsort on webcam python3 yolov3_deepsort.py /dev/video0 --camera 0 # yolov3_tiny + deepsort on webcam python3 yolov3_deepsort.py /dev/video0 --config_detection ./configs/yolov3_tiny.yaml --camera 0 ``` Use `--display` to enable display. Results will be saved to `./output/results.avi` and `./output/results.txt`. All files above can also be accessed from BaiduDisk! linker：[BaiduDisk](https://pan.baidu.com/s/1YJ1iPpdFTlUyLFoonYvozg) passwd：fbuw ## Training the RE-ID model The original model used in paper is in original_model.py, and its parameter here [original_ckpt.t7](https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6). To train the model, first you need download [Market1501](http://www.liangzheng.com.cn/Project/project_reid.html) dataset or [Mars](http://www.liangzheng.com.cn/Project/project_mars.html) dataset. Then you can try [train.py](deep_sort/deep/train.py) to train your own parameter and evaluate it using [test.py](deep_sort/deep/test.py) and [evaluate.py](deep_sort/deep/evalute.py).  ## Demo videos and images [demo.avi](https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6) [demo2.avi](https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6)   ## References - paper: [Simple Online and Realtime Tracking with a Deep Association Metric](https://arxiv.org/abs/1703.07402) - code: [nwojke/deep_sort](https://github.com/nwojke/deep_sort) - paper: [YOLOv3](https://pjreddie.com/media/files/papers/YOLOv3.pdf) - code: [Joseph Redmon/yolov3](https://pjreddie.com/darknet/yolo/)