Motion estimation

lastmod 2025-02-11created 2015-01-23146 edits

Motion Estimation (ME) means estimating (Estimation) where each object (Object) or processing-unit block (Macroblock) in a video has moved in the preceding and following frames over time. It is widely used in video compression such as MPEG and in video processing such as pixel demosaicking (Pixel Demosaicking) and frame interpolation (Frame Interpolation).

Human Pose Estimation

Estimating human motion.

ETC

Convolutional Pose Machines - Tensorflow: https://github.com/timctho/convolutional-pose-machines-tensorflow

PoseTrack Dataset and Benchmark: https://posetrack.net/

Detect And Track - Efficient Pose Estimation in Videos: https://github.com/facebookresearch/DetectAndTrack

Pose Animator: https://github.com/yemount/pose-animator/; Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh.

Critter Camera: Turns your phone/laptop into a motion-detection camera; https://critter.camera/; Uses an unused phone/tablet/laptop as a motion-detection camera for a specific location; A local app that runs only in the browser: all images are stored only locally, so it's safe; High-sensitivity motion detection that recognizes everything from people to very small objects; Leave it on, and later you can download captured images of only the moments with movement

unknown

Documentation

Discrete-Continuous Energy Minimization for Multi-Target Tracking: http://www.milanton.de/dctracking/; https://bitbucket.org/amilan/dctracking; MatLab Source code: Dctracking-v1.0.zip

Automatic Tracker Selection w.r.t Object Detection Performance: https://arxiv.org/abs/1404.2005v1; 1404.2005v1.pdf

Globally Optimal Solution to Multi-object Tracking with Merged Measurements: http://www.robots.ox.ac.uk/~joao/merged/; Henriques_iccv2011.pdf

Stable Multi-Target Tracking in Real-Time Surveillance Video (CVPR 2011): http://www.robots.ox.ac.uk/ActiveVision/Publications/benfold_reid_cvpr2011/benfold_reid_cvpr2011.html; http://www.robots.ox.ac.uk/ActiveVision/Publications/benfold_reid_cvpr2011/benfold_reid_cvpr2011.pdf; Benfold_reid_cvpr2011.pdf

Video Tracking with an Adaptive Particle Filter (Person): Youtube video

Discrete-Continuous Optimization for Multi-Target Tracking (CVPR 2012): Youtube video

Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies: https://arxiv.org/abs/1701.01909v2

Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking: https://arxiv.org/abs/1704.02781v1

[Recommended] Comparison of tracking algorithms implemented in OpenCV: Comparison of OpenCV tracking algorithms; Matecconf_cscc2016_04031.pdf; https://www.matec-conferences.org/articles/matecconf/pdf/2016/39/matecconf_cscc2016_04031.pdf

Video Segmentation via Object Flow: http://files.is.tue.mpg.de/black/papers/TsaiCVPR2016.pdf; TsaiCVPR2016.pdf

Real-Time Multiple Object Tracking - A Study on the Importance of Speed: https://arxiv.org/abs/1709.03572; Github - Code for my master thesis, titled "Real-Time Multiple Object Tracking: A Study on the Importance of Speed".

Deep Learning based Tracker

Learning A Deep Compact Image Representation for Visual Tracking (NIPS2013'): http://winsty.net/dlt.html

Robust Visual Tracking via Convolutional Networks: https://arxiv.org/abs/1501.04505; 1501.04505.pdf

Deep Tracking - Seeing Beyond Seeing Using Recurrent Neural Networks: https://arxiv.org/abs/1602.00991; 1602.00991v2.pdf

Fully-Convolutional Siamese Networks for Object Tracking (SiamFC): http://www.robots.ox.ac.uk/~luca/siamese-fc.html; https://github.com/bertinetto/siamese-fc

Tensorflow Object Tracking Video: https://github.com/DrewNF/Tensorflow_Object_Tracking_Video; Object Tracking in Tensorflow ( Localization Detection Classification ) developed to partecipate to ImageNET VID competition

Deep Reinforcement Learning for Visual Object Tracking in Videos: https://arxiv.org/abs/1701.08936v1; 1701.08936v1.pdf

Online Multi-Target Tracking Using Recurrent Neural Networks: https://arxiv.org/abs/1604.03635v2; 1604.03635v2.pdf

GOTURN (Learning to Track at 100 FPS with Deep Regression Networks): https://github.com/davheld/GOTURN; http://davheld.github.io/GOTURN/GOTURN.html; http://davheld.github.io/GOTURN/GOTURN.pdf; GOTURN.pdf

Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking: https://arxiv.org/abs/1707.02309

Tracking in Aerial Hyperspectral Videos using Deep Kernelized Correlation Filters: https://arxiv.org/abs/1711.07235v1; It seems to use KCF. Needs exact verification.

Efficient Video Object Segmentation via Network Modulation: See One-Shot Modulation Network for Semi-supervised Video Segmentation.

Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning (ADNet): https://sites.google.com/view/cvpr2017-adnet

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications: https://arxiv.org/abs/1808.01356

Learning Hierarchical Features for Visual Object Tracking with Recursive Neural Networks: https://arxiv.org/abs/1801.02021v1

Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking: https://arxiv.org/abs/1607.05781

Multi-Class Multi-Object Tracking using Changing Point Detection: https://arxiv.org/abs/1608.08434; https://www.youtube.com/watch?v=xOGQ1lP7pt8

Detect to Track and Track to Detect: https://arxiv.org/abs/1710.03958

Deep Network Flow for Multi-Object Tracking: http://www.nec-labs.com/uploads/images/Department-Images/MediaAnalytics/papers/cvpr17_deepnetworkflow.pdf

Joint detection and online multi-object tracking: http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w29/Kieritz_Joint_Detection_and_CVPR_2018_paper.pdf

Learning Dynamic Memory Networks for Object Tracking (MemTrack): https://arxiv.org/abs/1803.07268; http://openresearch.ai/t/memtrack-learning-dynamic-memory-networks-for-object-tracking/235

Learning Policies for Adaptive Tracking with Deep Feature Cascades (EAST): https://arxiv.org/abs/1708.02973

Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection: https://arxiv.org/abs/1704.05775

Parallel Tracking and Verifying - A Framework for Real-Time and High Accuracy Visual Tracking (Parallel Tracking and Verifying; PTAV): https://arxiv.org/abs/1708.00153

Robust and Real-time Deep Tracking Via Multi-Scale Domain Adaptation: https://arxiv.org/abs/1701.00561

Long-term face tracking in the wild using deep learning: https://arxiv.org/abs/1805.07646

Deep SORT: Simple Online Realtime Tracking with a Deep Association Metric; https://github.com/nwojke/deep_sort

TrackNet: Simultaneous Object Detection and Tracking and Its Application in Traffic Video Analysis; https://arxiv.org/abs/1902.01466

Multi Object Tracking

adipandas/multi-object-tracker - Multi-object trackers in Python

Deep Learning Based

Girish Varma - Multi Object Tracking using Deep Learning

Multi Object Tracking is the task tracking objects in video frames. We survey the various types of MOT methods, with special focus on the latest methods that use Deep Learning.

Visual tracking library based on PyTorch: https://github.com/visionml/pytracking

Learning to Track: Online Multi-object Tracking by Decision Making: http://cvgl.stanford.edu/papers/xiang_iccv15.pdf

Online Multi-Target Tracking Using Recurrent Neural Networks: https://arxiv.org/abs/1604.03635

Deep Network Flow for Multi-Object Tracking: https://arxiv.org/abs/1706.08482

Learning to Track at 100 FPS with Deep Regression Networks: https://link.springer.com/chapter/10.1007/978-3-319-46448-0_45

Deep tracking in the wild: End-to-end tracking using recurrent neural networks: http://www.robots.ox.ac.uk/~mobile/Papers/2017_IJRR_Dequaire.pdf

Survey Slides by Merc Benz: https://32e624bb-a-62cb3a1a-s-sites.googlegroups.com/site/dlitsc17/A%20Survey%20on%20Leveraging%20Deep%20Neural%20Networks%20for.pdf

Multiple Object Tracking - A Literature Review: https://arxiv.org/abs/1409.7618; https://pdfs.semanticscholar.org/8b42/8625c8a496c7e56b419197df55c751c22bc3.pdf

A survey on multiple object tracking algorithm: http://ieeexplore.ieee.org/document/7832121/

Target Re-identification and Multi-Target Multi-Camera Tracking (DeepCC): https://reid-mct.github.io/2019/

Deep Affinity Network for Multiple Object Tracking (Deep Affinity Network; DAN): https://github.com/shijieS/SST

bendidi/Tracking-with-darkflow: Real-time people Multitracker using YOLO v2 and deep_sort with tensorflow; https://github.com/bendidi/Tracking-with-darkflow

pan1028/deep-multi-tracking: Multiple Object Tracking Using Deep Learning and Kalman Filter; https://github.com/pan1028/deep-multi-tracking

ReIdentificationNet: Enhance Multi-Camera Tracking Accuracy by Fine-Tuning AI Models with Synthetic Data

Benchmark

Generic Object Tracking Benchmark (GOT-10k; got10k): http://got-10k.aitestunion.com/index

Visual Tracker Benchmark: http://www.visual-tracking.net; [Recommended] Visual Tracker Benchmark - result

foolwood/benchmark_results - visual tracker benchmark results: https://github.com/foolwood/benchmark_results; Foolwood-benchmark_results-master-712bc86-README.zip

Image Matching Benchmark: https://arxiv.org/abs/1709.03917

Long-Term Visual Object Tracking Benchmark: https://arxiv.org/abs/1712.01358v1

A Comparison of Feature Detectors and Descriptors forObject Class Matching: http://vision.cs.tut.fi/data/publications/neurocomputing2015_accepted.pdf

PoseTrack Dataset and Benchmark: https://posetrack.net/

Challenge

Welcome to MOTChallenge - The Multiple Object Tracking Benchmark!: https://motchallenge.net/; MOT17 Results

VOT challenges: http://votchallenge.net/; [Recommended] The sixth Visual Object Tracking VOT2018 challenge results ¹

VideoNet - A benchmark initiative for all things video: http://videonet.team/

VOT2018 Result

Evolution of Siamese Visual Tracking with Very Deep Networks (SiamRPN++): https://arxiv.org/abs/1812.11703

High Performance Visual Tracking with Siamese Region Proposal Network (SiamRPN): http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_High_Performance_Visual_CVPR_2018_paper.pdf; Li_High_Performance_Visual_CVPR_2018_paper.pdf

Distractor-aware Siamese Networks for Visual Object Tracking (DaSiamRPN): https://arxiv.org/abs/1808.06048

LADCF - No 1 Algorithm on the public dataset of VOT2018: Demo for Learning Adaptive Discriminative Correlation Filters (LADCF) via Temporal Consistency preserving Spatial Feature Selection for Robust Visual Tracking; https://github.com/XU-TIANYANG/LADCF

MBMD (MobileNet-based tracking by detection algorithm) for VOT2018 Long-Term Challenge.: https://github.com/xiaobai1217/MBMD; Learning regression and verification networks for long-term visual tracking

Multi-solution Fusion for Visual Tracking (MFT): https://github.com/ShuaiBai623/MFT; Multi-hierarchical Independent Correlation Filters for Visual Tracking

SiamFC series

Favorite site

Wikipedia (en) description of motion estimation
Naver Open Encyclopedia: Motion Estimation in video compression
OpenCV 3.0 Global Motion Estimation
Github - Deep-Learning-for-Tracking-and-Detection (Collection of papers and other resources for object tracking and detection using deep learning)
Deep Learning for Video: Object Tracking (UPC 2018)

Site collection

Object Tracking

Dark Programmer - Visual tracking program (visual tracker) (Mean shift, Camshift, Histogram, Optical Flow)
Dark Programmer - Visual tracking #2: TLD - Learning while tracking
Tracker Benchmark v1.0
OpenCV motion detection, object tracking
Condensation algorithm
Tracking with camshift (using functions provided by opencv)
[Recommended] Object Tracking using OpenCV (C++/Python) ⁶ (compares the pros and cons of OpenCV tracking algorithms)
Object Tracking - Object detection and tracking methods (1) ⁷
Object Tracking - Object detection and tracking methods (2) ⁸
Object Tracking - HOG-based object tracking using SVM ⁹
Object Tracking - Optical Flow comparison ¹⁰
Fast Object Tracking – Robot Computer Vision (Color based tracking, Shape based tracking)
Multi-object tracking with dlib (dlib)
[Recommended] Vehicle Detection and Tracking using Machine Learning and HOG ¹¹
- Github - Term 1, Project 5 - Udacity Self Driving Car Nanodegree
Tracking multiple objects with OpenCV - PyImageSearch
Visual Tracking Decomposition
Adaptive Color Attributes for Real-Time Visual Tracking (KCF)
- Accurate Scale Estimation for Robust Visual Tracking
tracking. OpenCV 3.0.0-dev documentation » OpenCV API Reference » Tracking API
- Tracker Algorithms
Tracking API Classes
Visual tracking #2: TLD - Learning while tracking
Visual tracking #1: Mean Shift tracking
Kernelized Correlation Filters
KCF(Kernelized Correlation Filters) (OpenCV sample: opencv_contrib/modules/tracking/samples/kcf.cpp)
OpenCV - Motion detection, object tracking (Mean-Shift, CamShift, Kalman Filter, background subtraction, optical flow)
- Flow accuracy and interpolation evaluation (a site that measures the performance of each Optical Flow algorithm)
Approaches to an OpenCV object tracking program (MeanShift and CAMShift)
Part 45: Tracking objects in video 1 - MeanShift
Optical flow - Pyramidal Lucas-Kanade (object tracking)
On object recognition - (3) Extracting SURF features
OpenCV - Motion detection references
OpenCV - Detecting screen movement. An example of Lucas-Kanade (cvCalcOpticalFlowPyrLK), an optical flow method
(Object Tracking) Object detection and tracking methods (1)

Multiple Object Tracking

Youtube: Multiple Object Tracker (OpenCV) (Demo)
Github - Smorodov/Multitarget-tracker - Multitarget tracker implementation, Hungarian algorithm + Kalman filter
OpenCV: Using MultiTracker

References

Kristan2018The.pdf ↩
February 17, 2019: 2015-10-09-tracking-master-eff293f.md.zip ↩
February 21, 2019: Abhineet123-Deep-Learning-for-Tracking-and-Detection-master-2419b81-ReadMe.md.zip ↩
April 28, 2018: Huanglianghua-mot-papers-master-0a24ffc-README.md.zip ↩
SpyderXu_-multi-object-tracking-paper-list-d91ac35-master-_README.zip ↩
Object_Tracking_using_OpenCV_-_Learn_OpenCV.pdf ↩
Enough_is_not_enough_-Object_Tracking-_how_to_01.pdf ↩
Enough_is_not_enough_-Object_Tracking-_how_to_02.pdf ↩
Enough_is_not_enough_-Object_Tracking-_SVM_and_HOG_tracker.pdf ↩
Enough_is_not_enough_-Object_Tracking-_Optical_Flow.pdf ↩
Vehicle_Detection_and_Tracking_using_Machine_Learning_and_HOG.pdf ↩

author Zer0 sitemap