Skip to content
Search

Faster R-CNN

lastmod 2025-11-12created 2016-03-1025 edits

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

It solved the bottleneck of Region Proposal by cleverly reusing the CNN weights used in Classification. The Region Proposal Network presented in the paper then infers Regions from that result.

Category

INTRODUCTION

State-of-the-art object detection methods are driven by region proposal methods and R-CNN.

  • Selective Search: Best quality. A CPU implementation takes 2 seconds per image.
  • EdgeBoxes: Takes 0.2 seconds per image. Good balance between quality and speed.

The methods above are implemented on the CPU, so they cannot use high-performance GPUs. They can also be implemented on the GPU, but this is found to ignore the Down-stream detection network. This misses the opportunity to share computation. (Needs further research. Seems to mean that Back-propagation is not possible. Need to check what the Down-stream detection network is.)

This paper introduces the Region Proposal Network (RPN), which addresses the drawback above (improves performance).

An RPN is a kind of Fully convolutional network (FCN). The RPN is built by adding a few functions to the Convolution Layer.

  • Region bounds.
  • OBjectness score.

It can be trained End-to-End for Detection Proposals.

The RPN effectively predicts Region Proposals across a wide range of Scales and Aspect Ratios.

It introduces "Anchor" boxes, which serve as references at multiple Scales and Aspect Ratios. This approach can be regarded as a Pyramid of regression references. It avoids multiple scaled images or filters. (It avoids the situations in a and b of Figure 1 and makes it work like c.)

To integrate Fast R-CNN and RPN, a training scheme is used that alternates between the tasks below while the Proposals are kept fixed.

  • Fine-tuning for Region Proposals.
  • Fine-tuning for Object Detection.

This scheme produces a unified network with Convolution Features shared between the two tasks (RPN, Fast R-CNN). (It was found that the network unifying Fast R-CNN and RPN can be trained efficiently with less training time.)

Finally, it compares on the PASCAL VOC benchmark.

  • Selective Search with Fast R-CNN
  • RPNs with Fast R-CNN

The tests include the following.

  • Eliminates all the time cost of Selective Search at test time.
  • The detection method using the GPU, including all steps, has a frame rate of 5fps.
Faster-RCNN_-_figure1.jpg

Figure 1: Different schemes for addressing multiple scales and sizes. (a) Pyramids of images and feature maps are built, and the classifier is run at all scales. (b) Pyramids of filters with multiple scales/sizes are run on the feature map. (c) We use pyramids of reference boxes in the regression functions.

FASTER R-CNN

Figure 2: Faster R-CNN is a single, unified network for object detection. The RPN module serves as the 'attention' of this unified network

Faster R-CNN consists of two modules.

  1. A Deep Fully Convolutional Network that proposes regions
  2. A Fast R-CNN that uses the proposed regions

The RPN module tells Fast R-CNN where to look (Look; Attention).

Region Proposal Networks

  • INPUT: Takes an image of Any size as input.
  • OUTPUT: Outputs rectangular Object Proposals regions and Objectness scores.
  • Fully Convolutional Network (FCN)
  • Shares the computation of the Common Convolution Network with Fest R-CNN.
  • The RPN module acts as the 'Attention' of the Unified network.
  • Inserts a network on top of the Convolutional feature map of the last shared Convolutional layer.
    • In other words, it takes the Convolutional feature map as input.
  • Sliding window
    • The RPN has an NxN spatial window. (This paper uses N=3.)
    • Mapped to a Lower-dimensional feature.
      • ZF: 256d
      • VGG: 512d
      • with ReLU following
    • The sliding window has the two FC (Fully Connected) Layers below.
      • box-regression layer (REG)
      • box-classification layer (CLS)

Terminology notes:

  • Region: Generally refers to a rectangular region.
  • Objectness: Computes Membership in the Object class versus the Background.
Faster-RCNN_-_figure3.jpg

Figure 3: Left: Region Proposal Network (RPN). Right: Example detections using RPN proposals on PASCAL VOC 2007 test. Our method detects objects in a wide range of scales and aspect ratios.

Anchors

  • At each sliding window, predicts multiple Region Proposals, up to K.
    • 4K REG outputs: Coordinates of the box position.
    • 2K CLS outputs: Estimate probability of Object or Not object.
      • For simplicity, CLS has a 2-class Softmax layer.
      • (As an alternative, the K Scores can be used with Logistic Regression.)
  • K Proposals ==> K reference box.
  • An Anchor is the center of the Sliding window and is associated with a Scale and an Aspect ratio.
  • At each Sliding position, the default values below are used.
    • 3 Scale
    • 3 Aspect ratio
    • K = 9 Anchor
Translation-Invariant Anchors
Multi-Scale Anchors as Regression References

Loss Function

Training RPNs

Sharing Features for RPN and Fast R-CNN

Implementation Details

EXPERIMENTS

Documentation

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (v3)
https://github.com/rbgirshick/py-faster-rcnn (Python Caffe version)
https://github.com/ShaoqingRen/faster_rcnn (Matlab version)
1506.01497v3.pdf

Troubleshooting

ImportError: No module named gpu_nms

When using rbgirshick/py-faster-rcnn with pycaffe, the following error may occur. 

2016-10-06T14:25:36,721 @140718718843328 [INFORMATIONAL] Request deep learning.
Traceback (most recent call last):
  File "/home/your/Project/americano/build/resource/python/fe.py", line 2, in <module>
    import pymain
  File "/home/your/Project/americano/build/resource/python/pymain/__init__.py", line 3, in <module>
    import faster as FASTER_RCNN
  File "/home/your/Project/americano/build/resource/python/pymain/faster.py", line 18, in <module>
    from fast_rcnn.test import im_detect
  File "/home/your/Project/americano/build/resource/python/pymain/../faster-rcnn/fast_rcnn/test.py", line 17, in <module>
    from fast_rcnn.nms_wrapper import nms
  File "/home/your/Project/americano/build/resource/python/pymain/../faster-rcnn/fast_rcnn/nms_wrapper.py", line 9, in <module>
    from nms.gpu_nms import gpu_nms
ImportError: No module named gpu_nms

In this case, check whether the Cython-related modules have been installed. 

$ cd $FRCN_ROOT/lib
$ make

Segmentation Fault

For how to fix segmentation faults, see Caffe#Segmentation Fault.

See also

Favorite site

References

  1. Faster_rcnn_-Towards_Real_Time_Object_detection-_Presentor_AndyTsai.zip ↩

  2. DeepLab_160923.pptx.zip ↩

  3. Faster_R-CNN_-_Pull_Requests_to_Tomorrow.pdf ↩

author Zer0 sitemap