RF-DETR:Review

lastmod 2026-09-02created 2026-09-021 edits

RF-DETR: Neural Architecture Search for Real-Time Detection Transformers

About

Original title: RF-DETR: Neural Architecture Search for Real-Time Detection Transformers

arXiv: 2511.09554

Authors: Isaac Robinson¹, Peter Robicheaux¹, Matvei Popov¹, Deva Ramanan², Neehar Peri²

• ¹Roboflow
• ²Carnegie Mellon University

Abstract

The paper points out that open-vocabulary detectors perform impressively on COCO but fail to generalize to real-world datasets with out-of-distribution classes not found in pre-training.

Key contributions:

• Introduces RF-DETR: a lightweight specialist detection transformer that discovers accuracy-latency Pareto curves for a target dataset through weight-sharing neural architecture search (weight-sharing NAS)
• On COCO, RF-DETR(nano): reaches 48.0 AP, beating D-FINE(nano) by 5.3 AP
• On Roboflow100-VL, RF-DETR(2x-large): outperforms GroundingDINO(tiny) by 1.2 AP while running 20x faster
• RF-DETR(2x-large) is the first real-time detector to exceed 60 AP on COCO

1. Introduction

1.1 Main problem

Limitations of open-vocabulary detectors:

• GroundingDINO, YOLO-World, etc. have strong zero-shot performance on common categories
• State-of-the-art vision-language models (VLMs) still struggle to generalize to out-of-distribution classes, tasks, and imaging modalities not found in pre-training

Specialist detectors vs open-vocabulary detectors:

• Fine-tuning VLMs: improves in-domain performance but reduces runtime efficiency and open-vocabulary generalization
• Specialist detectors such as D-FINE, RT-DETR: achieve real-time inference but fall short of fine-tuned VLM performance

1.2 Core argument

Are specialist detectors over-optimized for COCO?

The paper argues that standardized benchmarks such as PASCAL VOC and COCO have driven progress, but state-of-the-art specialist detectors implicitly overfit to COCO through bespoke model architectures, learning rate schedulers, and augmentation schedulers.

In practice, detectors such as YOLOv8 generalize poorly to real-world datasets whose distribution differs from COCO.

1.3 Revisiting neural architecture search (NAS)

Prior uses of NAS:

• Mainly applied to image classification (EfficientNet, OFA) and to components such as detection backbones and FPNs
• Simply swaps the standard backbone for a NAS backbone in an existing detection framework

RF-DETR's approach:

• First exploration of end-to-end weight-sharing NAS for object detection and segmentation
• Varies model inputs such as image resolution and patch size during training
• Inference configurations such as the number of decoder layers and query tokens can be changed at inference
• The search space is evaluated after the base model is fully trained on the target dataset
• Every possible subnet (a model configuration in the search space) performs well without additional fine-tuning

Interestingly, subnets never explicitly seen during training also perform well, suggesting generalization to unseen architectures.

1.4 Extension to segmentation

RF-DETR-Seg: adding a lightweight instance segmentation head makes it possible to discover Pareto-optimal architectures for real-time instance segmentation.

1.5 Standardizing latency evaluation

Current inconsistency: When each paper re-benchmarks the latency of prior work, D-FINE's latency evaluation of LW-DETR comes out 25% faster than originally reported.

• Main cause: GPU power throttling
• Fix: buffering to limit excess power use between forward passes

1.6 Key contributions

Three key contributions:

1. Presents RF-DETR: beats the state of the art on RF100-VL and achieves the state of the art among real-time methods with latency ≤40ms on COCO. RF-DETR is the first real-time detector to exceed 60 mAP on COCO
2. Explores the tunable knobs of weight-sharing NAS: improves the accuracy-latency trade-off for end-to-end object detection. Uses large-scale pre-training for effective transfer learning to small datasets
3. Revisits the benchmarking protocol for latency measurement: presents a standardized procedure to improve reproducibility

2. Related work

2.1 Neural architecture search (NAS)

Early NAS:

• NASNet, AmoebaNet: focused on maximizing accuracy, with little attention to efficiency, so computational cost is high

Hardware-aware NAS:

• ProxylessNAS, MNASNet, FBNet: integrate hardware feedback directly into the search process
• The search and training process must be repeated for each new hardware platform

OFA(One-For-All):

• Decouples training and search with weight-sharing NAS
• Simultaneously optimizes thousands of subnets with different accuracy-latency trade-offs
• Inspired by OFA, RF-DETR varies model inputs such as image resolution and patch size during training

2.2 Real-time object detectors

Evolution:

• Past: Mask-RCNN, Hybrid Task Cascade (accuracy first, high latency) vs YOLO, SSD (speed first)
• Present: modern detectors such as RT-DETR improve both axes at once

YOLO family:

• Achieves fast inference while keeping accuracy through innovations in architecture, data augmentation, and training techniques
• Reliance on NMS (non-maximum suppression) adds latency

DETR family:

• Original DETR: removes hand-crafted components such as NMS and anchor boxes, but runtime is high
• RT-DETR, LW-DETR: successfully adapt high-performance DETRs to real-time applications

2.3 Vision-language models (VLMs)

Features:

• Trained on large-scale weakly supervised image-text pairs
• A key enabling technology for open-vocabulary object detection

Examples:

• GLIP: phrase-based detection with a single text query
• Detic: improves long-tail detection with ImageNet supervision
• MQ-Det: extends with learnable modules that enable multimodal prompting

Limitations:

• These models were found to perform poorly on categories not found in pre-training
• Additional fine-tuning required
• Many VLMs are too slow for real-time tasks

Where RF-DETR stands: RF-DETR combines the fast inference of real-time detectors with the internet-scale pre-training of VLMs

3. RF-DETR: Weight-sharing NAS with foundation models

3.1 Foundation model architecture

Key improvements:

RF-DETR modernizes LW-DETR to improve generalization to diverse target domains:

1. Backbone change: CAEv2 → DINOv2
   • Initializing from DINOv2 pre-trained weights greatly improves detection accuracy on small datasets
   • CAEv2: 10 layers, patch size 16 vs DINOv2: 12 layers
   • DINOv2 is slower, but NAS makes up the latency

1. Friendly to consumer-grade GPUs: batch normalization → layer normalization
   • Easy to train with gradient accumulation

3.2 Real-time instance segmentation

RF-DETR-Seg design (inspired by MaskDINO):

• Adds a lightweight instance segmentation head to jointly predict high-quality segmentation masks
• Upsamples the encoder output with linear interpolation
• Produces a pixel embedding map with a lightweight projector
• Produces segmentation masks from the dot product of all projected query token embeddings (an FFN transform of each decoder layer's output) with the pixel embedding map
• Pixel embeddings can be read as segmentation prototypes (YOLACT style)

Pre-training: Objects-365 is pseudo-labeled with SAM2 (Segment Anything Model 2) to pre-train the detection and segmentation heads jointly

Unlike MaskDINO, it does not incorporate multi-scale backbone features, keeping latency to a minimum.

3.3 End-to-end neural architecture search

NAS search space:

Weight-sharing NAS trains thousands of model configurations that evaluate different input image resolutions, patch sizes, window attention blocks, numbers of decoder layers, and numbers of query tokens.

At every training iteration, a random model configuration is sampled uniformly and a gradient update is applied, so many subnets can be trained efficiently in parallel (similar to ensemble learning with dropout).

Interestingly, this weight-sharing NAS acts as "architecture augmentation", improving regularization and generalization.

Tunable knobs:

1. Patch size: smaller patches → higher accuracy, higher computational cost
   • Interpolates between patch sizes during training with a FlexiViT-style transform

1. Number of decoder layers: regression loss applied to every decoder layer's output (as in recent DETRs)
   • All decoder blocks can be removed at inference
   • With them removed, it behaves like a single-stage detector (YOLO style)
   • Segmentation latency is also controlled by shrinking the segmentation branch

1. Number of query tokens: learn spatial priors for bounding box regression and segmentation
   • At test time, dropping query tokens by confidence (the maximum sigmoid of the class logits from the encoder output) changes the maximum number of detections and reduces inference latency
   • The Pareto-optimal number of queries implicitly encodes the target dataset's statistic of average objects per image

1. Image resolution: higher resolution → better small object detection, lower resolution → faster runtime
   • Pre-allocates NN positional encodings corresponding to the largest resolution ÷ the smallest patch size
   • Interpolates these embeddings for smaller resolutions or larger patch sizes

1. Number of windows per window attention block: window attention → processes only a fixed set of neighboring tokens
   • Adding/removing windows per block balances accuracy, global information mixing, and computational efficiency

At inference: The model configuration is chosen by picking a specific operating point on the accuracy-latency Pareto curve. Even with the same parameter count, latency can differ significantly.

Benefit of fine-tuning: As with OFA, the benefit of fine-tuning on COCO is limited, but RF100-VL shows a modest improvement (architecture augmentation regularization needs more than 100 epochs to converge on small datasets).

3.4 Bias in training schedulers and augmentation

Problem: State-of-the-art detectors need careful hyperparameter tuning to maximize performance on standard benchmarks, but these bespoke training procedures carry an implicit bias toward specific dataset characteristics (e.g., number of images).

Limitation of cosine schedules: A cosine schedule assumes a known, fixed optimization horizon, which is unrealistic across diverse target datasets. (Observed concurrently with DINOv3)

Bias in data augmentation:

• Augmentation assumes prior knowledge of dataset characteristics
• Aggressive augmentation (VerticalFlip, RandomFlip, RandomResize, RandomCrop, YOLOXHSVRandomAug, CachedMixUp) increases the effective dataset size
• But augmentations such as VerticalFlip can be a harmful bias in safety-critical domains
  • e.g., a pedestrian detector for self-driving cars should not use VerticalFlip, to avoid false positives from reflections in puddles
• RF-DETR's approach: limited to horizontal flips and random crops

Batch-level resize:

• LW-DETR: random resize per image, padded to the largest image in the batch
  • Result: heavy padding on most images → window artifacts, wasted computation on padded regions
• RF-DETR: batch-level resize minimizes padded pixels and exposes all positional encoding resolutions equally

4. Experiments

4.1 Datasets and evaluation metrics

Evaluation datasets:

• COCO: fair comparison with prior work
• RF100-VL(Roboflow100-VL): evaluates generalization to real-world datasets through the diversity of 100 datasets
  • The paper's assumption: overall RF100-VL performance is a proxy for transferability to an arbitrary target domain

Evaluation metrics:

• Standard metric via pycocotools: mean average precision (mAP)
• Breakdown: AP50, AP75, AP_Small, AP_Medium, AP_Large
• Efficiency: GFLOPs, parameter count, and inference latency on an NVIDIA T4 GPU (TensorRT 10.4, CUDA 12.4)

4.2 Standardizing latency benchmarking

The main source of variance is power throttling and GPU overheating. A 200ms buffer between forward passes mitigates it.

This approach guarantees highly reproducible latency measurements, not sustained throughput measurements.

Problems observed:

• Could not reproduce the mAP results of YOLOv8 and YOLOv11 in TensorRT (assumed to be evaluated with multi-class NMS while only single-class NMS is used for inference)
• Used the standard NMS-tuned confidence threshold of 0.01
• Performance drops when quantizing from FP32 to FP16 (D-FINE: 55.1 AP → 0.5 AP)
  • Fix: switch to ONNX opset 17
• Accuracy and latency must be reported from the same model artifact

4.3 COCO detection evaluation

Key findings:

• RF-DETR(nano) beats D-FINE(nano) and LW-DETR(nano) by more than 5 AP
• The same trend holds at S and M sizes
• RF-DETR(nano) matches the medium YOLOv8 and YOLOv11
• GroundingDINO uses the mmdetection implementation (marked with * for no TensorRT support)

4.4 COCO instance segmentation evaluation

Key findings:

• RF-DETR-Seg(nano) beats every reported YOLOv8 and YOLOv11 model size
• RF-DETR-Seg(nano) beats FastInst by 4.4% while being nearly 10x faster
• RF-DETR(medium) comes close to MaskDINO(R50) at a tiny fraction of the runtime

4.5 RF100-VL evaluation

RF100-VL is a challenging detection benchmark made up of 100 diverse datasets.

Key findings:

• RF-DETR(2x-large) beats GroundingDINO and LLMDet at a tiny fraction of the runtime
• Interestingly, RT-DETR beats D-FINE (built on the same foundation) on mAP50 → suggests D-FINE's hyperparameters may be overfit to COCO
• RF-DETR benefits from scaling to larger backbone sizes
• YOLOv8 and YOLOv11 consistently fall short of DETR-based detectors, and scaling them up does not improve RF100-VL performance

4.6 Impact of neural architecture search

Key findings:

• Introducing gentle hyperparameters (smaller batch size, lower learning rate, batch normalization → layer normalization) costs 1% performance relative to LW-DETR
• Switching to the DINOv2 backbone recovers the lost performance and adds another 2% improvement
• The lower learning rate especially helps preserve DINOv2's pre-trained knowledge
• Additional Objects-365 pre-training makes up for the slower optimization
• Introducing weight-sharing NAS improves the base configuration's performance (even though patch size 14 is not in the search space)

4.7 Impact of backbone architecture and pre-training

Key findings:

• DINOv2 performs best, 2.4% above CAEv2
• SigLIPv2 and SAM2 perform poorly under FP16 quantization (reported with FP16 TensorRT latency and FP32 ONNX accuracy as an upper bound)
• SAM2 Hiera-S is markedly slower despite having fewer parameters
  • State-of-the-art vision transformer families usually do not release lightweight ViT-S, ViT-T variants, which makes them hard to repurpose for real-time applications

4.8 Revisiting standard accuracy benchmarking practice

Problem: Selecting and evaluating models only on the validation set can lead to overfitting.

e.g., D-FINE (based on RT-DETR) runs an extensive hyperparameter sweep on the COCO validation set and reports the best model. But evaluated on the RF100-VL test set, D-FINE falls short of RT-DETR.

In contrast: RF-DETR achieves the state of the art among all real-time detectors on both RF100-VL and COCO, demonstrating robustness.

Recommendation for the future: Future detectors should also be evaluated on datasets with public validation-test splits such as RF100-VL

4.9 Limitations

Despite controlling for power throttling from GPU overheating, latency measurements can still vary by up to 0.1ms because of nondeterministic behavior during TensorRT compilation.

Specifically, TensorRT can introduce power throttling during compilation, which affects the resulting engine and causes random variation in latency.

Measurements of a given TensorRT engine are generally consistent, but recompiling the same ONNX artifact can produce different latency results.

Latency is therefore reported only to one decimal place.

5. Conclusion

The paper introduces RF-DETR, a state-of-the-art NAS-based method for fine-tuning specialist end-to-end object detectors for a target dataset and hardware platform.

Key results:

• Beats the prior real-time state of the art on COCO and RF100-VL
• 5% AP improvement over D-FINE(nano) on COCO
• Highlights that current architectures, learning rate schedulers, and augmentation schedulers are tuned to maximize COCO performance
• The community needs to benchmark on diverse, large-scale datasets to avoid implicitly overfitting to COCO
• Highlights the high variance of latency benchmarking caused by power throttling
• Presents a standardized procedure to improve reproducibility

Appendix

A. Implementation details

Training hyperparameters: Extends LW-DETR; the main differences:

• Objects365 pseudo-labeled with SAM2 to pre-train the segmentation/detection heads jointly
• Learning rate: 1e-4 (LW-DETR: 4e-4)
• Batch size: 128
• Uses an EMA scheduler (as in DINOv3)
• No learning rate warm-up
• Gradient clipping: clip above 0.1
• Layer-wise multiplicative decay: 0.8 (preserves information, especially in the early DINOv2 backbone layers)
• Window attention block placement: {0, 1, 3, 4, 6, 7, 9, 10} (LW-DETR: {0, 1, 3, 6, 7, 9})
• Multi-scale resolution training: 0.5–1.5x (LW-DETR: 0.7–1.4)

Latency evaluation:

• Uses CUDA graphs to pre-queue all kernels in TensorRT (no serial CPU launches needed)
• Standard inference optimization
• Improves LW-DETR latency, does not improve D-FINE

B. Query token and decoder layer ablation

Evaluates the effect of dropping queries at inference for RF-DETR(nano).

Training uses 300 object queries, following standard real-time DETR practice.

Many datasets contain fewer than 300 objects per image. Processing all 300 queries can be wasted computation.

LW-DETR(tiny) showed that training with fewer queries can improve the latency-accuracy trade-off.

RF-DETR discards the lowest-confidence queries by confidence from the encoder output, so queries can be dropped at test time without retraining:

Dropping the 100 lowest-confidence queries gives a meaningful latency-accuracy trade-off without a significant drop in performance.

Also, since each decoder layer receives its own supervision (during training), decoder layers can be pruned at test time.

Interestingly, all decoder layers can be removed → relying only on the initial query proposals:

• No cross-attention to encoder states
• No self-attention between queries
• Substantial runtime reduction
• The resulting model is a YOLO-style single-stage architecture (no NMS)

Removing the final decoder layer cuts latency by 10% at a cost of only 2 mAP.

C. FLOPs benchmarking

FLOPs for RF-DETR, GroundingDINO, and YOLO-E are benchmarked with PyTorch's FlopCounterMode.

FlopCounterMode closely reproduces the FLOPs counts from the custom benchmarking tools of YOLOv11, D-FINE, and LW-DETR.

It also gives more reliable results than CalFLOPs.

Interestingly, LW-DETR's FLOPs come out about 2x the originally reported result → LW-DETR may have reported FLOPs instead of MACs.

D. Effect of class names on open-vocabulary detectors

Evaluates the effect of fine-tuning VLMs such as GroundingDINO on RF100-VL with class names vs class indices.

Intuitively, VLM vision-language pre-training should be more useful when prompted with class names (e.g., car, truck, bus).

Conversely, prompting with class indices (0, 1, 2) gives the model less information about the underlying data.

However, GroundingDINO fine-tuning results on RF100-VL:

• Class names: 62.3 AP
• Class indices: 62.5 AP (nearly equal)

→ Naive end-to-end fine-tuning of the model diminishes the benefit of internet-scale pre-training.

Future research direction: ways to preserve foundational pre-training when fine-tuning VLMs need to be investigated.

E. Benchmarking large model variants

Detectors such as LW-DETR and D-FINE design their larger variants by hand.

In contrast, RF-DETR, a NAS-based architecture, discovers an automatic scaling strategy through grid-based search.

Analysis of two RF-DETR model families:

1. Based on the DINOv2-S backbone
2. Based on the DINOv2-B backbone

Key findings:

• The DINOv2-S backbone family beats D-FINE at first but loses its edge at larger model sizes
• The DINOv2-B backbone family shows the opposite trend: the gap to D-FINE narrows as latency increases
• In fact, RF-DETR(2x-large) beats D-FINE
• Scaling the D-FINE family requires substantial extra engineering
• Scaling the RF-DETR family is simple: higher-latency variants can be sampled from the same NAS search without retraining

Note: D-FINE(x-large) beats RF-DETR(x-large) on mAP 50:95. But RF-DETR(2x-large) beats D-FINE by 0.8 AP, becoming the first real-time detector to exceed 60 AP on COCO.

F. Effect of NAS on COCO fine-tuning

Fine-tuning after NAS on COCO gives only a limited benefit.

Hypothesis: NAS "architecture augmentation" acts as a strong regularizer, and removing that regularization during additional training induces overfitting.

Trend: the fine-tuning benefit is limited for larger models. Interestingly, RF100-VL models can get a larger fine-tuning benefit (they need more than 100 epochs to converge).

G. Effect of a fixed architecture on RF100-VL

Evaluates transferring COCO-optimized architectures to RF100-VL to check the benefit of fixed-architecture models.

Even though the fixed architecture is not tuned for RF100-VL, it beats LW-DETR, and the RF-DETR(large) fixed architecture performs best among prior real-time models.

Dataset-specific NAS, however, gives a substantial additional benefit.

Key findings:

• The fixed architecture keeps strong performance
• Dataset-specific NAS adds further benefit
• Fine-tuning gives especially strong gains for smaller models

H. Discussion of notable discovered architectures

Patch size convergence: Pareto-optimal models tend to use the same patch size.

• RF-DETR DINOv2-S: converges to an optimal patch size of 16
• RF-DETR DINOv2-B: patch size 20
• RF-DETR-Seg DINOv2-S: patch size 12

Joint encoder-decoder scaling: Pareto-optimal models scale computation in both the encoder and the decoder. Resolution scales up as latency increases.

Number of windows: Pareto-optimal models generally prefer 2 windows (LW-DETR: 4)

• RF-DETR's DINOv2 backbone relies on a class token, while LW-DETR's CAEv2 omits the class token
• For window attention compatibility, the class token is duplicated into each window
• When scaling to more windows, class token duplication degrades runtime efficiency

Interaction of spatial positions with resolution/patch size: RF-DETR performance depends on the number of spatial positions (resolution ÷ patch size) rather than on resolution or patch size individually.

Scaling resolution with patch size fixed and scaling patch size with resolution fixed give similar results.

Verified with alternative family configurations:

• RF-DETR(nano): 640×640, patch size 27 (never seen in training)
• RF-DETR(small): 640×640, patch size 21 (unseen)
• RF-DETR(medium): 640×640, patch size 18 (unseen)

Result: nearly equal to the Pareto-optimal family → demonstrates RF-DETR's strong generalization to unseen patch sizes

A subtle interaction in RF-DETR-Seg: Segmentation head features are always upsampled from 1/4 of the input image resolution, so resolution affects both the number of spatial positions and the head resolution.

Examples:

• RF-DETR-Seg(nano): 312×312, patch size 12 → segmentation head 78×78, 26×26 spatial positions
• RF-DETR-Seg(small): 384×384, patch size 12 → segmentation head 96×96, 32×32 spatial positions
• RF-DETR-Seg(medium): 432×432, patch size 12 → segmentation head 108×108, 36×36 spatial positions

Dataset characteristics affect the optimal architecture: NAS run on RF100-VL datasets uses fewer queries than COCO models of equivalent latency.

• RF100-VL: tends to have fewer objects per image → fewer queries needed

I. Visualizing model predictions

RF-DETR(nano) detection vs LW-DETR(tiny): RF-DETR(nano) has fewer false positives (e.g., mistaking a sign pole for a person)

RF-DETR-Seg(nano) segmentation vs YOLOv11(nano): RF-DETR-Seg(nano) predicts more precise object boundaries

Figure descriptions

Figure 1: Accuracy-latency Pareto curves

Plots of the Pareto accuracy-latency frontier of real-time detectors on the COCO detection validation set (top left, bottom left), COCO segmentation validation set (top right), and RF100-VL test set (bottom right).

Because RF100-VL contains 100 different datasets, a target latency is chosen for each of the N, S, M, L, XL, 2XL configurations, RF-DETR variants with latency within 10% of the target are searched for and fine-tuned until convergence, and the average performance is reported.

Important: every point on RF-DETR's continuous COCO Pareto curve comes from the results of a single training run.

Figure 2: RF-DETR architecture

RF-DETR extracts multi-scale features of the input image with a pre-trained ViT backbone. Window and non-window attention blocks are interleaved to balance accuracy and latency. In particular, both the deformable cross-attention layers and the segmentation head use bilinear interpolation of the projector output to keep a consistent spatial organization of features. Finally, detection and segmentation losses are applied to every decoder layer to make decoder dropout at inference easy.

Figure 3: NAS search space

Varies (a) patch size, (b) number of decoder layers, (c) number of queries, (d) image resolution, and (e) number of windows per attention block.

Besides training thousands of network configurations in parallel, this "architecture augmentation" acts as a regularizer and improves generalization.

Figure 4: Effect of decoder layers vs query tokens

Evaluates the accuracy-latency trade-off of dropping queries at inference for RF-DETR(nano).

Interestingly, dropping the 100 lowest-confidence queries gives a meaningful latency improvement without a significant drop in performance.

Removing all decoder layers (like a single-stage detector) cuts latency by 10% at a cost of only 2 mAP.

Figure 5: RF100-VL fixed architecture ablation

Evaluates the benefit of transferring COCO-optimized NAS architectures to RF100-VL. The fixed architecture beats LW-DETR, but dataset-specific NAS gives significant additional gains. Especially for smaller models, fine-tuning consistently gives gains across all model sizes.

See also

• RF-DETR