Skip to content
Search

RF-DETR:Tutorial:15-Appendix

lastmod 2026-11-18created 2026-11-186 edits

Appendix: Debugging · Glossary · Next Steps is chapter 15 of the RF-DETR tutorial. When you build your own detector or fine-tune RF-DETR on your own data, sooner or later you will see boxes drawn in the wrong places, a loss that keeps falling while AP does not rise, and the paper's 2.3 ms turning into tens of milliseconds on your machine. This chapter is a reference to open at those moments; it is meant to be consulted section by section rather than read from start to finish. It contains a first-30-minutes checklist and a symptom → cause → check → fix debugging table, the default settings of the official implementation and the mini-model settings used in this tutorial, the key formulas of each chapter, the key numbers from each chapter's mini experiments along with the results that did not match expectations, a glossary of 260 terms, a paper reading order with public implementations and tools, frequently asked questions, and projects to try next. Every item links to the chapter (1 – 14) that explains it in detail.

1. Debugging: what to check in the first 30 minutes

A detector has many parts (backbone · projector · query selection · decoder · heads · matching · loss · evaluator), and if any one of them is off, the symptom is the same: “AP is low”. When something looks wrong, go through the checks below in order before you start tweaking hyperparameters. With detectors, many problems are caught in the first four steps (data · box representation · shapes · evaluator).

  1. Draw the ground truth on the images. Overlay the ground-truth boxes and class names on a few training images (draw_boxes in rf_shapes.py). If the boxes do not enclose the objects, the problem is in the data, not the model. The most common cause is confusion between box representations (chapter 2, chapter 14).
  2. Check the box representation and units. For every function, write down whether the four numbers you are handling are the center representation (cx, cy, w, h) or the corner representation (x0, y0, x1, y1), and whether they are pixels or normalized 0 – 1 coordinates, then verify with assert.
  3. Check tensor shapes. Write the expected shapes in comments, such as classification output (B, N, K), box output (B, N, 4), and all-pairs IoU (N, M), and verify them. Broadcasting sometimes produces wrong results without raising an error.
  4. Make the evaluator trustworthy first. Feeding the ground truth in as predictions must give an AP of 1 (in experiment 1 of chapter 3, AP is 1.000 at zero noise). If the evaluator is wrong, every number after it is meaningless.
  5. Compare the first loss with a hand calculation. The loss before training starts can be computed in advance. Depending on whether the classification head's bias is initialized to match the prior probability, the first loss differs by about a factor of twenty (chapter 4).
  6. Trace the matching for one image by hand. Print the cost matrix, compute one cell yourself, and check that the Hungarian matching assignment agrees with the pairs you would pick by eye (chapter 7, chapter 11).
  7. Overfit a few images. If you train repeatedly on only about eight images, the loss should drop sharply and the predictions on those images should overlap the ground truth. If not, there is a bug in the model or loss code.
  8. Log the loss separately by term and by output. Split classification · L1 · GIoU across the last decoder layer · auxiliary losses · encoder proposals. Looking only at the total, you cannot tell which term is the problem.
  9. Measure the resolution of your ruler first. Simply rerunning the same configuration changes AP slightly. In the mini experiments of chapter 11, that spread was about 0.005. Do not read anything into smaller differences; check whether the direction is the same across three seeds.
  10. Only then change hyperparameters. One at a time, starting with the learning rate.

The snippet below turns the first part of this checklist into code. Placing it between the data loader and the loss computation catches problems early.

import torch

def check_target_boxes(name, boxes):
    """정답 박스가 정규화한 중심 표현 (cx, cy, w, h) 인지 확인한다. boxes: (T, 4)"""
    assert boxes.ndim == 2 and boxes.shape[-1] == 4, f"{name}: 모양 {tuple(boxes.shape)}"
    assert torch.isfinite(boxes).all(), f"{name} 에 NaN 또는 inf 가 있습니다"
    assert (boxes[:, 2:] > 0).all(), f"{name}: 너비 · 높이가 0 이하입니다 (모서리 표현이 아닌가요?)"
    assert boxes.max() <= 1.0, f"{name}: 1 을 넘습니다 (픽셀 좌표가 아닌가요?)"
    x0y0 = boxes[:, :2] - boxes[:, 2:] / 2          # 왼쪽 위 모서리
    x1y1 = boxes[:, :2] + boxes[:, 2:] / 2          # 오른쪽 아래 모서리
    assert (x0y0 >= -1e-6).all() and (x1y1 <= 1 + 1e-6).all(), f"{name}: 그림 밖으로 나갑니다"

def check(name, x, shape=None):
    """모양과 유한성(NaN · inf 없음)을 확인한다."""
    if shape is not None:
        assert tuple(x.shape) == tuple(shape), f"{name}: {tuple(x.shape)} != {shape}"
    assert torch.isfinite(x).all(), f"{name} 에 NaN 또는 inf 가 있습니다"

def grad_norm(model):
    """모든 매개변수 기울기를 한 벡터로 이었을 때의 길이."""
    total = 0.0
    for p in model.parameters():
        if p.grad is not None:
            total += p.grad.detach().pow(2).sum().item()
    return total ** 0.5

# 학습 고리 안에서
# for t in targets: check_target_boxes("target", t["boxes"])
# check("pred_logits", out["pred_logits"], (B, N, K))
# check("pred_boxes", out["pred_boxes"], (B, N, 4))
# loss.backward()
# print("grad norm", grad_norm(model))    # RF-DETR 설정에서는 이 값이 늘 0.1 보다 커서 매 스텝 잘립니다 (11장)
Caution · the most common silent bug — which coordinates are these?
Four box numbers alone do not tell you whether they are in corner or center representation, or in pixels or normalized units. (0.2, 0.3, 0.4, 0.5) can be read either way. This tutorial uses the following conventions.
  • Model outputs and ground truth: normalized center representation (cx, cy, w, h)
  • Input to the IoU · GIoU functions (ch02_boxes.py): corner representation (x0, y0, x1, y1)
  • bbox in COCO annotation files: [x, y, w, h] in pixels, where x, y is the top-left corner, not the center
  • Detections.xyxy returned by predict() in rfdetr: corner representation in pixels of the original image
Keep every representation conversion in a single function, and put cxcywh · xyxy in the function names (chapter 2).

2. Symptom → cause → check → fix

Common symptoms, grouped by component. They come from the caution boxes in each chapter and from pitfalls actually hit while building the exercises for this tutorial. All numbers were measured in that chapter's mini experiments. The chapter in the rightmost column has the detailed explanation and derivation.

Boxes · coordinates

Symptom

Common cause

How to check

Where to fix

Ch.

Prediction and ground truth visibly overlap, but IoU is nearly 0

Center representation (cxcywh) and corner representation (xyxy) are mixed up. The model outputs center representation, but the IoU function expects corner representation

Convert one box by hand. In corner representation, both x1 > x0 and y1 > y0 must hold

box_cxcywh_to_xyxy before IoU · GIoU. State the expected representation in the function's docstring

02

The box loss starts in the tens to hundreds. Drawn boxes fall outside the image

Pixel coordinates and normalized coordinates are mixed up

Print the minimum and maximum of the ground-truth · predicted boxes. They must lie within 0 – 1

Divide pixel values by the image width · height

02

Ground-truth boxes loaded from COCO format are shifted by about half a box toward the bottom right

COCO bbox is [x, y, w, h] in pixels relative to the top-left corner, but it was read as the center

Draw the ground truth on the image

Convert with \(c_x = (x^{\text{px}} + w^{\text{px}}/2)/W\)

14

Two diagonally separated boxes have a positive intersection area

The overlap lengths were multiplied before being clamped to 0. Multiplying −2 horizontally by −3 vertically gives +6

Use a pair of diagonally separated boxes as a test input

Put clamp(min=0) before the multiplication

02

Computing all-pairs IoU gives shape (N,) instead of (N, N)

The None for broadcasting is missing. If the box counts happen to match, it silently computes element-wise pairs instead

Print the shape of the result. Test with two sets of different sizes

Keep separate all-pairs (for matching) and element-wise (for the loss) functions and distinguish them by name

02

Predicted boxes that do not overlap the ground truth do not move during training

Only the IoU loss is used. Without overlap, IoU is flat at 0 and the gradient is 0 (IoU still 0.0000 after 2,000 steps)

Look at the fraction of prediction–ground-truth pairs with IoU 0

Use GIoU loss together with L1 loss

02

A negative GIoU was taken to mean “no overlap”, but the boxes actually overlap

GIoU is negative even with overlap if the empty fraction is large (two boxes with IoU 0.1667 have GIoU −0.0333)

Decide overlap by whether IoU is 0

IoU for decisions, GIoU for the loss

02

Only small objects score noticeably worse

The same pixel error lowers IoU more for small boxes. L1 loss does not know the box size

Look at AP by size (APS · APM · APL) separately

Add GIoU loss (scale-invariant) to L1. Raise the input resolution

02 · 03

Box predictions go far outside 0 – 1 or pile up at one end

Box reparameterization (shifting and scaling the reference box without a sigmoid) is mixed with the chapter 8 approach (adding in logit space, then a sigmoid)

Set the box head output to 0 and check that the prediction equals the reference box

Use only one approach. RF-DETR uses reparameterization and applies no sigmoid to boxes

09 · 10

Evaluation · thresholds

Symptom

Common cause

How to check

Where to fix

Ch.

Feeding the ground truth in as predictions does not give AP 1

A bug in the evaluator (matching order, class IDs, box representation)

Test with perfect predictions and with slightly jittered boxes

Step through the evaluation code against the hand-calculated example

03

AP50 is high but AP is low

Objects are found but the boxes are loose. Boxes with mean IoU 0.715 had AP50 0.971 but AP75 0.170 and AP 0.377

Look at the AP at each of the ten IoU thresholds

Look at the box side (box loss, box refinement, number of decoder layers, training length)

03

Boxes look good but AP is low

The score ranking does not match box quality. Assigning different scores to the same boxes moved AP between 0.369 and 0.480

Collect (score, IoU) for each detected object and look at the rank correlation

IA-BCE, which mixes IoU into the classification target

03 · 09 · 11

There are far more predictions than ground-truth objects, and AP50 is low

Duplicate predictions with similar scores. With three duplicates per object, AP50 dropped from 1.000 to 0.763

Count the predictions with score 0.5 or higher, and the pairs among them that overlap with IoU above 0.5

Check that training used one-to-one matching and that the decoder has self-attention

03 · 07

The model outputs hundreds of low-score predictions, and it is unclear whether that is fine

It is fine. Adding 96 low-score false positives per image left AP unchanged. Only high-score false positives are fatal (0.700 → 0.192)

Look at the score distribution of the false positives

Nothing to fix. AP only looks at the score ranking

03

Changing the threshold leaves AP unchanged and only changes precision · recall

There are two thresholds. The IoU threshold is set by the evaluator; the score threshold is set by whoever uses the detector

Check which threshold was changed

To change AP, change the model; to change the precision · recall balance, change the score threshold

03

A model trained with focal loss misses many objects at threshold 0.5

Probabilities from focal-loss training come out low (median probability on positive cells: BCE 0.996, focal 0.709)

Lower the threshold to 0.3 and check recall again. Compare with AP, which does not depend on the threshold

Choose the threshold separately for each model

04 · 08

Applying NMS to DETR-family outputs lowered AP slightly

There are almost no duplicates to remove. NMS keeps the higher-scoring box, and there is no guarantee that it is the more accurate one

Look at AP and AP50 before and after NMS together (chapter 7: AP −0.0016, AP50 +0.0021)

Do not apply NMS to the last layer's output

07 · 11

AP 0.93 in the shape world was taken to mean the model beats one with 48.0 AP on COCO

AP can only be compared within the same dataset · the same evaluation protocol

—

Measure again on the same data

03

The mAP reported by rfdetr differs greatly from your own evaluator's value

The definition is the same (0.9828 vs. 0.9830 on the same predictions). A large difference means class IDs or the coordinate conversion are off

Draw a few predictions on the image and check classes and boxes

Class ID remapping, pixel ↔ normalized conversion

14

Matching · loss

Symptom

Common cause

How to check

Where to fix

Ch.

The classification loss at the first step exceeds 100

The classification head's bias is 0, so every cell starts at probability 0.5. With 432 cells and 2.5 positives, \(432 \times 0.6931 / 2.5 = 119.8\)

Compare the first loss with a hand calculation (chapter 4 experiment: 121.03 vs. 6.36)

Prior-probability bias initialization \(b = -\log\frac{1 - \pi}{\pi}\) (\(\pi = 0.01\))

04

Matching looks wrong. An unrelated prediction gets paired while a prediction with the correct class is left out

The three cost terms are on mismatched scales. The class term uses log probability instead of probability, or a coefficient is wrong

Compute one cell of the cost matrix by hand and compare it with the code's value

DETR: \(-\hat{p}_i(c_j) + 5 \cdot \text{L1} - 2 \cdot \operatorname{GIoU}\). RF-DETR: 2 · focal form + 5 · L1 + 2 · (−GIoU)

07 · 11

Pairing greedily from the closest ground truth gives a higher total cost

Greedy assignment is not optimal (example from chapter 7: −1.0011 vs. the optimal −1.9587)

On a small example, compare with the result of trying every assignment

scipy.optimize.linear_sum_assignment

07

The matching code raises gradient-related errors or uses a lot of memory

Matching only decides indices. It needs no gradients

—

Run matching inside torch.no_grad()

07

The “loss” in the log is several times larger than the weighted sum of classification · L1 · GIoU

The total loss is the sum over all outputs (each decoder layer + encoder proposals), while the per-term values in the log come from the last layer only

Sum \(1 \cdot \sum\) classification \(+\ 5 \cdot \sum\) L1 \(+\ 2 \cdot \sum\) GIoU over all outputs and compare with the logged value

Nothing to fix. Log the values for each output separately

07 · 11 · 14

DETR's classification loss differs slightly from the value computed with the paper's formula

F.cross_entropy(..., weight=...) divides the weighted sum by the sum of the weights (0.4518 vs. 0.4669 in the hand-calculated example)

Compute the denominator yourself

Nothing to fix. The intent, “scale the ∅ term down to one tenth”, is the same

07

The loss magnitude swings widely from batch to batch

It was divided by the number of images. Box · classification losses are divided by the number of ground-truth objects in the batch (times the number of groups, if groups are used)

Compare the loss of a batch with many objects and one with few

Divide by the number of ground-truth objects, with a minimum of 1 for batches with none

07 · 11

The IA-BCE classification loss stalls around 0.1

With soft targets below 1, the floor of BCE is above 0 (0.117 for a target of 0.975)

Compute the floor from the mean IoU of the matched predictions

Nothing to fix

04 · 11

Adding iterative box refinement made training unstable

The gradient of the reference box passed to the next layer was not cut

Check that the reference box has .detach()

Stop gradients between layers

08

The cells chosen by two-stage query selection have nothing to do with the objects

No loss is applied to the encoder proposals. That loss is the only signal that teaches query selection

Evaluate the proposals alone (chapter 11 mini model: AP 0.8928)

Apply the same matching · loss to the encoder proposals as well

10 · 11

cardinality_error in the rfdetr training log does not go down and stays around 29

This is normal. During training, each of the 13 query groups produces one prediction per object. With an average of 2.44 objects per image, \(12 \times 2.44 = 29.3\)

Compare with the average number of objects per image × 12

Nothing to fix

14

Several queries point to the same object with high scores

You are using the first decoder layer's output (chapter 7: 153 overlapping pairs at the first layer, 45 at the last), or using multiple query groups at inference

Count the overlapping pairs per layer

At inference, use only the first group and the output of as late a layer as possible

07 · 09

Training dynamics

Symptom

Common cause

How to check

Where to fix

Ch.

Validation AP goes up and down from epoch to epoch

The weights being trained were evaluated directly

Evaluate the EMA weights alongside the regular weights (chapter 11: EMA was higher in 19 of 20 runs, by +0.0091 on average)

Use the EMA weights for evaluation · saving

11

Early in training, the EMA model is much worse than the regular model

The decay was fixed at 0.993 from the start, so the EMA stays anchored to useless initial weights for a long time

Print the effective decay at each step

Decay warmup \(d_n = d\,(1 - e^{-n/\tau})\)

11

Increasing the effective batch with gradient accumulation gives different results from training with a large batch

The model has batch normalization. The loss of one image depends on the other images in the same batch

Check which kind of normalization layers the model has

Use layer normalization (what RF-DETR changed in the projector)

10 · 11

Validation during training looks fine, but a separate forward pass gives garbage

model.eval() was not called. Batch normalization behaves differently in training mode and inference mode

Feed the same input in both modes and compare the outputs

Call .eval() before inference. The same applies when you pull the nn.Module out of rfdetr and use it directly

04 · 14

The loss suddenly becomes NaN

The attention mask blocked every key for some query (softmax becomes 0/0). Or a log was taken after a sigmoid

Check that every row of the mask has at least one open position. Use torch.isfinite to find the first tensor where NaN appears

Fix the mask. Compute BCE directly from the logits

05 · 04

Training with SGD gives widely different results across seeds

A large initial loss turns directly into large updates (AP after 12 epochs: 0.661 ± 0.270 vs. 0.891 ± 0.030)

Look at the first loss and the loss curve over the first few dozen steps

Prior-probability bias initialization. An Adam-family optimizer

04

Using the official learning-rate settings on a backbone without pretraining makes it learn slowly

The low backbone learning rate and layer-wise learning-rate decay exist to protect a pretrained backbone. They do not suit a random backbone

Retrain with a constant learning rate and compare (chapter 11, 200 training images: 0.7170 vs. 0.6752)

If starting from random initialization, turn off layer-wise decay or raise the backbone learning rate

11

Changing only the number of epochs changed the results at the same epoch

A cosine schedule sets the learning rate as a fraction of the total length. Changing the length changes the learning rate at every step

Log the learning rate at each step

When the training length is unknown, use a constant learning rate (schedule-free training)

11

The gradient norm exceeds the clipping limit (0.1) at every step

This is normal. With these settings, gradients are clipped at every step, so the gradient fed into Adam has the same magnitude at every step

Log the norm before clipping

Nothing to fix

11

Two configurations differed by 0.003 AP, and one was concluded to be better

Rerunning the same configuration moves the result by that much (nondeterminism of GPU operations, seeds)

Measure the spread by running the same configuration twice and with three different seeds

Do not read anything into differences smaller than the ruler's resolution

08 · 11

The training loss went back up while validation AP keeps rising

The training loss is computed with a resolution that changes per batch and with augmentation, so the epoch average fluctuates a little

Look at the validation AP curve

Choose models by validation AP, not training loss

14

Calling train() again on the same model starts learning from scratch

It does not continue training; it restarts from the original pretrained weights

Read the warning message

Pass the last.ckpt path to resume=

14

The self-supervised pretraining loss is low, but the features are useless

Collapse. The model gives the same answer for every image

Check whether the entropy of the teacher output is 0 or stuck at the value of the uniform distribution. Put a linear classifier on the frozen features and compare with a randomly initialized backbone

Use centering and sharpening together

06

Attention · backbone · resolution

Symptom

Common cause

How to check

Where to fix

Ch.

Feeding an image at a different resolution raises “The size of tensor a (65) must match the size of tensor b (145)”

Position embeddings are a table with a fixed number of positions, so they are tied to the resolution

Compare the number of tokens with the number of positions in the position embedding

Reshape the position embeddings back into a 2D grid and interpolate

06

rfdetr rejects the resolution (ValueError)

A side is not a multiple of patch_size × num_windows (a multiple of 32 for Nano)

Divide the resolution by that number

Change it to the nearest multiple. Training · inference · export all have the same constraint

10 · 14

A model trained at a single resolution collapses in AP when given a different resolution

Weights trained with a fixed configuration only fit that configuration (chapter 12: a model with 0.9345 at resolution 96 scored 0.1329 at 64 and 0.3063 at 128)

Compare with the AP in the trained configuration

Train while cycling through several configurations (weight-sharing NAS, multi-resolution training). Or fine-tune in the configuration you will use

12

Bilinearly interpolating the patch embedding kernel to change the patch size lowers AP

Interpolation alone changes the magnitude of the tokens (chapter 12: 0.8585 → 0.7579 → 0.6315 as the scale factor grows)

Compare the original and new token values on the same image

Change the kernel with PI-resize, which preserves inner products (stays at 0.9348 – 0.9352)

12

The model gets the object type right but cannot locate it at all

Positional encoding · position embedding was not added. Attention has no notion of order (chapter 5 experiment: the error matched the 17.06-pixel baseline that always answers the mean)

Check whether the result is the same as when the positional encoding is zeroed out

Add positional encoding to the tokens

05 · 06

After switching every block to window attention, information from one window never reaches another

If every block is a window block, no information flows between windows at all

Change the input of one cell and check whether outputs in other windows change

Interleave global blocks

09

The paper and the code disagree on which blocks are window blocks

The notation in Appendix A of the paper differs from the actual behavior of the public code (the complement of out_feature_indexes)

Print the per-block settings from the model

If you use the public weights, follow the code's behavior (for Nano, global blocks 3 · 6 · 9)

10

A hand-written bilinear interpolation is off from F.grid_sample by about half a cell

The coordinate conventions differ. With align_corners=False, the cell index is the normalized coordinate times the width, minus 0.5

Compute one point on a 3×3 map by hand and compare

Fix the cell index formula and read 0 outside the map

08

Switching to deformable attention made it slower

That happens with small maps (on a 12×12 map, one cross-attention call took 2.6 times as long). The value projection is still applied to every cell

Time both attentions while increasing the map size

The gain comes from large maps. Measure after exporting the whole model

08

Window attention reduced the compute but did not make things faster

Only the term proportional to the square of the token count shrinks. With few tokens there is no gain, and the reordering operations eat up time

Measure while varying the number of tokens

Window-major layout. Check compute and latency separately

09

Reducing the number of queries at inference changed the outputs of the remaining queries as well

This is normal. Queries see each other through the decoder's self-attention

—

Nothing to fix. Keep queries from the top of the score ranking

10 · 12

The attention map is bright only where there are no objects

It may not be broken. With the same architecture · same data, some seeds produced a solution that “looks at everything except the shapes”

Look at that model's prediction error

Do not assume the attention map shows “where the model thinks matters”

05

Segmentation masks

Symptom

Common cause

How to check

Where to fix

Ch.

Mask boundaries are blocky like stair steps

Mask logits are computed at 1/4 of the image resolution. They were thresholded at 0 first and upsampled afterward

Check the order of post-processing

Threshold at 0 after upsampling the logits to the image size

13

Masks for small objects come out empty

BCE scores each cell and averages, so the missed cells of a small object get buried in the background (when missed entirely, BCE is 0.08 for a 9-cell object and 2.31 for a 288-cell object)

Look at mask IoU by object size

Add Dice loss (independent of size: 0.93, 0.98)

13

The rough region is right, but the boundaries are blunt

Only Dice loss was used (4-epoch ablation: mask AP 0.561, AP75 0.590)

Look at mask AP50 and AP75 separately

Use BCE and Dice together (0.743 in the same ablation)

13

Box AP dropped after training with the segmentation head attached

Gradients from the mask loss move the backbone · projector · decoder shared with detection (chapter 13: from 0.9347 to 0.8906 after one epoch)

Compare with a control trained for the same extra amount without the segmentation head

Train longer (it returned to 0.9349 by epoch 30)

13

With the same weights · same batch, the mask loss differs from run to run

Point sampling draws different points at each step

Fix the seed, or compute over all pixels and compare

Nothing to fix. The smaller the mask grid, the larger the noise

13

Reducing decoder layers hurt masks more than boxes

Segmentation head blocks are removed along with the decoder layers. The queries of layer \(l\) are dotted with a map that has passed through \(l\) blocks

Look at box AP and mask AP together for each layer count

Prune the decoder less in segmentation models

13

Inference · export · latency

Symptom

Common cause

How to check

Where to fix

Ch.

The 2.3 ms in the paper · README becomes tens of milliseconds on your machine

The measurement conditions differ. That number is for an NVIDIA T4, a TensorRT engine, FP16, forward pass only, batch 1

Write down the four things: hardware · runtime · precision · what was timed

Compare only on the same machine · same runtime · same precision · same scope

12 · 14

Times measured on the GPU are absurdly short

The timer was stopped without waiting for the GPU work to finish

Check that torch.cuda.synchronize() is called before stopping the timer

Read the time after synchronizing

12 · 14

Only the first few measurements are unusually slow

Warmup has not finished

Plot the measurements in order

Discard the first few runs, measure many times, and use the median

12 · 14

Latency differs every time it is measured

GPU power throttling (overheating), other jobs on the same machine

Measure the configurations being compared alternately over several rounds and look at the per-round medians

The paper's method: rest 200 ms between forward passes. Report to one decimal place only

12

The FP16 engine's latency and the FP32 model's AP were put in the same table

Accuracy (AP) and latency must be measured on the same artifact. Example from the paper: converting D-FINE (M) to FP16 with its public export code as-is collapsed AP to 0.5

Compare the exported model's outputs numerically with the original model, and re-evaluate with the exported model

Report AP and latency together from the same artifact

12

The model picked from the Pareto curve is not the best on your hardware

The Pareto frontier is a property of model + hardware + execution method. Just switching the horizontal axis from compute to latency changed the frontier

Remeasure the latency of each configuration on your hardware

Keep AP as is, remeasure only latency, and extract the frontier again

12

Boxes and logits in the ONNX output seem swapped

Outputs were told apart by shape. With 3 classes, the logits (3 + 1 slots) and the boxes have the same shape, [1, 300, 4]

Print the output names

Fetch them by name, as in session.run(["dets", "labels"], …)

14

Comparing ONNX and PyTorch outputs over all 300 queries shows large differences

The proposal scores of background queries are nearly equal, so their ranking flips. Even the highest score among the differing queries was 0.013

Compare only the queries whose score exceeds the threshold (max box difference 7.8e-05)

Nothing to fix

14

class_id of the fine-tuned model seems shifted by one

A fine-tuned model's class IDs are renumbered from 0 in the order of the category ids in the annotation file

Print the category ids and the model's IDs side by side

Build the name table based on the renumbered IDs

14

The same object appears twice with two different classes

The top K in post-processing are not K queries but the K highest-scoring (query, class) pairs

Check whether the two results' boxes came from the same query

Raise the score threshold. In a well-trained model, the second class scores far lower

14

The pretrained model calls objects in your images by the wrong names

A model trained on COCO only knows the COCO classes

—

Fine-tune on your own data

14

Prediction colors · classes look wrong

A numpy array was passed in BGR order (an image read with OpenCV)

Save the input image as is and open it

Flip it to RGB order before passing it in

14

AttributeError: ... has no attribute 'optimize_for_inference'

This is an example from older posts. The name was replaced by inference() and then removed

Check the installed version (this tutorial uses 1.12.1)

Use the inference engine returned by model.inference()

14

The validation data cannot be found

The folder is named val/

Look at the folder structure

Name it valid/, with the annotation file _annotations.coco.json

14

3. Settings reference

The values in the paper and the official implementation are “values that worked well on COCO and data of that scale”. Treat them as a starting point rather than using them as is. Values may differ between versions, so if you need exact reproduction, check rfdetr/config.py in your installed version directly.

Defaults of the official implementation

These are the values in the installed version checked for this tutorial (rfdetr 1.12.1).

Item

Value

Setting name

Ch.

Matching cost

2 · focal-form classification (α = 0.25, γ = 2) + 5 · L1 + 2 · (−GIoU). Matched separately for each query group

—

11

Loss coefficients

Classification 1, L1 5, GIoU 2. The same loss on every decoder layer and on the encoder proposals

cls_loss_coef, bbox_loss_coef, giou_loss_coef

11

Classification loss

IA-BCE (exponent 0.25)

ia_bce_loss=True

09

Number of queries · groups

300 · 13 (only the first group at inference)

num_queries, group_detr

09 · 10

Decoder

Dimension 256, 8 self-attention heads, 16 cross-attention heads, 2 sampling points, FFN 2048, number of layers varies by size (2 for Nano)

hidden_dim, sa_nheads, ca_nheads, dec_n_points, dim_feedforward, dec_layers

10

Box prediction

Box reparameterization; all layers share the initial reference boxes

bbox_reparam=True, lite_refpoint_refine=True

10

Classification head bias

Initialized to match a prior probability of 0.01

—

04 · 10

Learning rate

1e-4 (projector · heads, etc.), 1.5e-4 base for the backbone, no warmup, constant to the end

lr, lr_encoder, warmup_epochs=0

11

Learning-rate decay

0.8 per layer, 0.7 per component

lr_vit_layer_decay, lr_component_decay

11 · 14

Optimizer

AdamW, weight decay 1e-4

weight_decay

04 · 11

Gradient clipping

Max norm 0.1

clip_max_norm

11

EMA

Decay 0.993, warmup constant 100

ema_decay, ema_tau

11

Batch · epochs

Batch 4, gradient accumulation 1, 100 epochs. Effective batch size = batch × accumulation steps

batch_size, grad_accum_steps, epochs

14

Augmentation · resolution

Horizontal flip · random crop, per-batch multi-resolution

multi_scale, expanded_scales

11

Mask loss (segmentation)

BCE 5, Dice 5, masks at 1/4 of the image resolution, points at 1/16 of the number of cells

mask_ce_loss_coef, mask_dice_loss_coef, mask_downsample_ratio, mask_point_sample_ratio

13

Note · compared with the paper's training settings (Appendix A)
The RF-DETR paper lists a learning rate of 1e-4 (4e-4 for LW-DETR), batch 128, EMA, no learning-rate warmup, gradient clipping at 0.1, layer-wise learning-rate decay of 0.8 for the DINOv2 backbone, and multi-resolution at 0.5 – 1.5× (0.7 – 1.4× for LW-DETR). The package's default batch is 4, and its documentation recommends an effective batch of 16. Where the paper and the package differ, cite the source to keep them apart (chapter 11, chapter 14).

The mini models in this tutorial

These values are scaled down to run in a few minutes to a few tens of minutes on a personal computer. All data comes from the shape world (96×96, 3 classes, 1 – 4 objects per image), with 4,000 training images and 500 validation images (1,265 objects).

Ch. · file

Main settings

07 · ch07_detr.py

MiniDETR: 12 queries, 3 decoder layers, 12×12 feature map (144 tokens), 60 epochs (125 steps per epoch), learning rate lowered after epoch 48

08 · ch08_deformable.py

Mini Deformable DETR: 12 queries, 2 feature levels (12×12, 6×6), iterative box refinement, focal loss, 60 epochs. Ablations at 15 epochs

10 · ch10_rfdetr_model.py

MiniRFDETR: resolution 96 · patch 8 (12×12 grid), backbone dimension 96 · 6 blocks · 4 heads, window blocks [0, 1, 3, 5] · global blocks [2, 4], 2 windows per side, projector 3 × 96 → 128, 3 decoder layers, 12 queries × 4 groups, 1,832,707 parameters

11 · ch11_train.py

The chapter 10 model at batch 32 · 60 epochs, learning rate 2e-4 (the other learning-rate rules · EMA · gradient clipping · loss coefficients match the official ones), no augmentation, randomly initialized backbone. Ablations at 15 epochs · 3 seeds

12 · ch12_nas.py

During training: one of the 23 valid combinations of resolution (64, 80, 96, 112, 128) × patch (8, 16) × windows (1, 2, 4) per step. Search: 408 configurations, also varying decoder layers 0 – 3 and queries 4 · 8 · 12

13 · ch13_seg.py

A segmentation head (219,137 parameters) attached to the chapter 11 EMA weights, 30 epochs. 24×24 mask grid, 36 points used in the loss. Ablations at 4 epochs · 3 seeds

14 · ch14_finetune.py

RF-DETR Nano (384×384) on 1,000 shape-world images for 5 epochs. Batch 2 × gradient accumulation 4, learning rate 1e-4. Randomly initialized control for 3 epochs

Tip · what to change first on your own data
When things are not working, check roughly in this order.
  1. Data: are the annotations correct, and does each class have enough images?
  2. Did you start from pretrained weights? (This made the biggest difference in the experiments of chapter 14.)
  3. Input resolution and model size (resolution first if objects are small)
  4. Training length and effective batch size
  5. Then the learning rate. Loss coefficients and matching costs are usually left alone

4. Key formulas by chapter

The key formulas derived in the tutorial, in chapter order. Derivations and hand calculations are in the worked-example boxes of each chapter. The same letter sometimes means different things in different chapters (for example, N is the number of tokens in chapters 5 · 6 · 9 and the number of queries in chapters 7 · 8 · 10), so read the symbols in each formula with the meaning from the chapter given in that row's chapter column. Notation follows chapter 0 and the “Notation in this chapter” box at the start of each chapter.

Math · fundamentals (chapters 02 – 06)

Formula

Meaning

Ch.

\((x_0,\ y_0,\ x_1,\ y_1) = \left(c_x - \tfrac{w}{2},\ c_y - \tfrac{h}{2},\ c_x + \tfrac{w}{2},\ c_y + \tfrac{h}{2}\right)\)

Center representation → corner representation

02

\((c_x,\ c_y,\ w,\ h) = \left(\frac{x_0 + x_1}{2},\ \frac{y_0 + y_1}{2},\ x_1 - x_0,\ y_1 - y_0\right)\)

Corner representation → center representation

02

\(|A \cup B| = |A| + |B| - |A \cap B|\)

Area of the union (inclusion–exclusion principle)

02

\(\operatorname{IoU}(A, B) = \dfrac{|A \cap B|}{|A \cup B|}\)

IoU. 0 – 1, symmetric, scale-invariant

02

\(\operatorname{IoU} = \dfrac{w - d}{w + d}\)

IoU of two equal-sized boxes shifted horizontally by \(d\). An offset of just 1/3 of the width gives 0.5

02

\(\operatorname{GIoU}(A, B) = \operatorname{IoU}(A, B) - \dfrac{|C| - |A \cup B|}{|C|}\)

GIoU. \(C\) is the enclosing box. Even without overlap, it decreases with distance

02

\(\mathcal{L}_{\text{box}}(b, \hat{b}) = \lambda_{\text{L1}}\,\|b - \hat{b}\|_1 + \lambda_{\operatorname{GIoU}}\,\big(1 - \operatorname{GIoU}(b, \hat{b})\big), \quad \lambda_{\text{L1}} = 5,\ \lambda_{\operatorname{GIoU}} = 2\)

Box loss of the DETR family

02

\(P = \dfrac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FP}}, \quad R = \dfrac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}}\)

Precision and recall

03

\(\mathrm{AP} = \dfrac{1}{101} \sum_{i=0}^{100} p_{\mathrm{interp}}\!\left(\dfrac{i}{100}\right)\)

COCO's 101-point interpolated AP (one class · one IoU threshold)

03

\(\mathrm{AP} = \dfrac{1}{10} \sum_{\tau \in \{0.50,\, 0.55,\, \dots,\, 0.95\}} \;\dfrac{1}{C} \sum_{c=1}^{C} \mathrm{AP}_c^{\tau}\)

COCO AP: the average over classes and over ten IoU thresholds

03

\(\theta \leftarrow \theta - \eta\nabla_\theta L\)

Gradient descent

04

\(\sigma(z) = \dfrac{1}{1 + e^{-z}}, \quad \sigma' = \sigma(1 - \sigma), \quad z = \log\dfrac{p}{1 - p}\)

Sigmoid and its derivative, logit (log-odds)

04

\(\operatorname{BCE} = -y\log p - (1 - y)\log(1 - p) = \max(z, 0) - zy + \log(1 + e^{-|z|})\)

Binary cross-entropy, and the form computed directly from logits. The gradient with respect to the logit is \(p - y\)

04

\(\mathrm{FL}(p_t) = -\,\alpha_t\,(1 - p_t)^{\gamma}\,\log p_t\)

Focal loss. With \(\gamma = 0\) it is BCE

04

\(b = -\log\dfrac{1 - \pi}{\pi}\)

Prior-probability bias initialization (\(-4.5951\) for \(\pi = 0.01\))

04

\(\theta \leftarrow (1 - \eta\lambda)\theta\)

AdamW weight decay (separate from the gradient update)

04

\(\lfloor (n_{\text{in}} - F + 2P)/S \rfloor + 1\)

Output size of a convolution

04

\(\operatorname{Attention}(Q, K, V) = \operatorname{softmax}\!\left( \dfrac{Q K^{\top}}{\sqrt{d_k}} \right) V\)

Scaled dot-product attention

05

\(\operatorname{Var}\!\left( \dfrac{q \cdot k}{\sqrt{d_k}} \right) = \dfrac{d_k}{d_k} = 1\)

Why we divide by \(\sqrt{d_k}\) (when components are independent with variance 1)

05

\(4 \times (D^2 + D)\)

Number of parameters in multi-head attention. Independent of the number of heads

05

\(f(PX) = Pf(X)\)

Permutation equivariance: attention has no notion of order

05

\(N = (H/p)(W/p)\)

Number of patch tokens in a ViT

06

\(\text{블록 하나} \approx 12\,N D^2 + 2\,N^2 D\)

Number of multiplications in one ViT block ("블록 하나" = one block). A term linear in the token count and a term quadratic in it

06

\(\theta_t \leftarrow m\,\theta_t + (1 - m)\,\theta_s\)

DINO's teacher is an exponential moving average of the student

06

\(\mathrm{CE}(P_t, P_s) = \mathrm{Ent}(P_t) + D_{\mathrm{KL}}(P_t\,\|\,P_s)\)

Splitting DINO's loss (cross-entropy) into the teacher's entropy and the KL divergence

06

Formulas · the DETR lineage (chapters 07 – 09)

Formula

Meaning

Ch.

\(\hat{\sigma} = \operatorname*{arg\,min}_{\sigma} \sum_{j=0}^{T-1} C_{\sigma(j),\, j}\)

Bipartite matching: the one-to-one assignment with minimum total cost

07

\(C_{ij} = -\hat{p}_i(c_j) + 5\,\lVert \hat{b}_i - b_j \rVert_1 - 2\,\operatorname{GIoU}(\hat{b}_i, b_j)\)

DETR's matching cost

07

\(\mathcal{L}_{\text{Hungarian}} = \sum_{i=0}^{N-1} -\log \hat{p}_i(t_i) + \sum_{j=0}^{T-1} \mathcal{L}_{\text{box}}\big(b_j,\, \hat{b}_{\hat{\sigma}(j)}\big)\)

Set loss (Hungarian loss): classification for every query + boxes for matched queries

07

\(v = (1 - t_x)(1 - t_y)\,v_{00} + t_x(1 - t_y)\,v_{01} + (1 - t_x)\,t_y\,v_{10} + t_x\,t_y\,v_{11}\)

Bilinear interpolation: a weighted average of four neighbors

08

\(\dfrac{\partial v}{\partial x} = W\Big[(1 - t_y)(v_{01} - v_{00}) + t_y(v_{11} - v_{10})\Big]\)

Derivative of the sampled value with respect to the sampling location. This is why offsets can be learned with gradient descent

08

\(\operatorname{DeformAttn}(z_q,\ p_q,\ X) = \sum_{m=1}^{M} W^O_m \Big[ \sum_{k=1}^{K} A_{mqk} \cdot W^V_m\, X(p_q + \Delta p_{mqk}) \Big]\)

Deformable attention (single-scale): reads only the sampling points around the reference point

08

\(\hat{b}^{d} = \sigma(\Delta b^{d} + \sigma^{-1}(\operatorname{sg}(\hat{b}^{d-1}))), \quad \sigma^{-1}(x) = \log\frac{x}{1-x}\)

Iterative box refinement: add in logit space, and cut the gradient when passing to the next layer

08

\(\dfrac{N^2}{W_n^2}\)

Number of (query, key) pairs in one window attention. \(N^2\) for global attention

09

\(\ell_{\text{cls}} = \sum_{i=1}^{N_{\text{pos}}} \operatorname{BCE}(p_i,\ t_i) + \sum_{j=1}^{N_{\text{neg}}} p_j^{2}\, \operatorname{BCE}(p_j,\ 0), \quad t = p^{\alpha}\, u^{1-\alpha}\)

IoU-aware BCE loss (IA-BCE). The gradient \(p - t\) on a positive cell is zero at \(p = u\)

09

\(\hat{x} = \delta_x w_r + x_r, \quad \hat{w} = e^{\delta_w} w_r\)

Box reparameterization: shift and scale in units of the reference box size

09

\((\delta_x,\ \delta_y,\ \delta_w,\ \delta_h) = \left(\frac{x_g - x_r}{w_r},\ \frac{y_g - y_r}{h_r},\ \log\frac{w_g}{w_r},\ \log\frac{h_g}{h_r}\right)\)

Inverse transform: the deltas that move the reference box onto the ground-truth box

09

Formulas · RF-DETR and practice (chapters 10 – 14)

Formula

Meaning

Ch.

\(\operatorname{reparam}(\delta, r) = \big( \delta_x\, r_w + r_x,\ \delta_y\, r_h + r_y,\ e^{\delta_w}\, r_w,\ e^{\delta_h}\, r_h \big)\)

Box reparameterization in chapter 10 notation

10

\(r_q = \operatorname{reparam}\big( e_q,\ \operatorname{sg}(\hat{b}^{\text{enc}}_q) \big), \quad \hat{b}^{d}_q = \operatorname{reparam}\big( \delta^{d}_q,\ r_q \big)\)

Reference boxes come from the encoder proposals, and each layer's predicted boxes come from the same reference boxes

10

\(\mathcal{L} = \sum_{\ell \,\in\, \mathcal{S}} \Big( 1 \cdot \mathcal{L}^{(\ell)}_{\text{cls}} + 5 \cdot \mathcal{L}^{(\ell)}_{\text{L1}} + 2 \cdot \mathcal{L}^{(\ell)}_{\operatorname{GIoU}} \Big), \quad \mathcal{S} = \{\text{enc},\ 1,\ 2,\ \dots,\ L_{\text{dec}}\}\)

RF-DETR's total loss: the sum over the encoder proposals and each decoder layer

11

\(\eta_n = \eta_{\min} + \tfrac{1}{2}\,(\eta_0 - \eta_{\min})\left(1 + \cos\dfrac{\pi\, n}{n_{\text{end}}}\right)\)

Cosine schedule. The end point \(n_{\text{end}}\) must be known in advance (RF-DETR does not use it)

11

\(\mathbf{g} \leftarrow \mathbf{g} \cdot \min\!\left(1,\ \dfrac{c_{\max}}{\lVert \mathbf{g} \rVert}\right)\)

Gradient clipping (\(c_{\max} = 0.1\))

11

\(\theta_{\text{ema}} \leftarrow d\,\theta_{\text{ema}} + (1 - d)\,\theta, \quad d_n = d \cdot \left(1 - e^{-n/\tau}\right)\)

EMA and decay warmup (\(d = 0.993\), \(\tau = 100\))

11

\(\eta_{\text{enc}} \cdot \lambda^{\,L + 1 - i} \cdot \rho^{2}\)

Layer-wise learning-rate decay: the learning rate of backbone parameters

11

\(r_s = 1 - \dfrac{6 \sum_{k=1}^{P} \delta_k^{\,2}}{P\,(P^2 - 1)}\)

Spearman correlation: how well the score ranking agrees with the IoU ranking

11

\(J(\theta) = \mathbb{E}_{c \sim \mathcal{C}}\big[\, \mathcal{L}(\theta; c) \,\big] = \dfrac{1}{|\mathcal{C}|} \sum_{c \in \mathcal{C}} \mathcal{L}(\theta; c)\)

Training objective of weight-sharing NAS: the average loss over configurations

12

\(\hat{w} = R\,(R^\top R)^{-1}\, w = (R^\top)^{+}\, w\)

PI-resize: resizes the patch embedding kernel while preserving inner products

12

\(z_n(x, y) = \big\langle f(q_n),\ e(x, y) \big\rangle + \beta\)

Mask logit: the inner product of the query-side vector and the pixel embedding

13

\(L_{\text{dice}} = 1 - \dfrac{2 \sum_{i} p_i\, t_i + 1}{\sum_{i} p_i + \sum_{i} t_i + 1}\)

Dice loss (the form used in the official implementation)

13

\(\operatorname{Dice} = \dfrac{2\,\operatorname{IoU}}{1 + \operatorname{IoU}}\)

Relation between the Dice coefficient and IoU

13

\([x^{\text{px}},\ y^{\text{px}},\ w^{\text{px}},\ h^{\text{px}}] = \Big[ \Big(c_x - \frac{w}{2}\Big) W,\ \Big(c_y - \frac{h}{2}\Big) H,\ wW,\ hH \Big]\)

Normalized center representation → COCO pixel [x, y, w, h]

14

\(c_x = \dfrac{x^{\text{px}} + w^{\text{px}}/2}{W}, \quad c_y = \dfrac{y^{\text{px}} + h^{\text{px}}/2}{H}, \quad w = \dfrac{w^{\text{px}}}{W}, \quad h = \dfrac{h^{\text{px}}}{H}\)

COCO pixel [x, y, w, h] → normalized center representation

14

5. Experiment results at a glance

All experiments in this tutorial are small experiments in the shape world. The numbers below are copied directly from the text of each chapter and, unless noted otherwise, were measured on the 500-image validation set (1,265 objects) with a single seed. Do not read them alongside the numbers the papers report on COCO. The 0.93 AP in the shape world comes from the problem being easy (chapter 3).

Key numbers from each chapter's mini experiments

Ch.

Experiment

Key numbers

Takeaway

02

Fitting one box to the ground truth with gradient descent (2,000 steps)

Final IoU: IoU loss 0.0000, GIoU loss 0.9926, L1 1.0000, combined loss 0.7986. Step at which IoU reached 0.9: GIoU 244, L1 234, combined loss 45

IoU loss cannot move without overlap. The combined loss was faster because its coefficients effectively increased the learning rate

03

Using fake detectors to see what AP responds to

Box noise 0.10: AP50 0.971, AP75 0.170, AP 0.377. Same boxes, different scores only: AP 0.369 – 0.480. Three duplicates with similar scores each: AP 0.700 → 0.608. 96 low-score false positives each: unchanged at 0.700

AP responds to box precision and score ranking, but not to low-score false positives

04

Training an imbalanced classification under three settings

First loss: 121.03 with zero bias, 6.36 with the prior-probability bias. AP after 1 epoch: 0.016 vs. 0.341. AP after 30 epochs: BCE 0.928 ± 0.010, focal 0.914 ± 0.005

The prior-probability bias clearly helps early training. Focal loss did not beat BCE in this experiment

05

Finding shape centers with one cross-attention layer; token count vs. time

Queries put 93 – 97% of their attention weight on their own shape (three seeds). Without positional encoding, error 17.06 – 17.07 pixels (baseline that always answers the mean: 17.06). Time scales as token count to the power 2.14 (hand-written) · 1.88 (built-in module)

Attention searches by content and retrieves position. Without positional encoding, position cannot be known

06

Changing MiniViT's resolution; linear-probe evaluation of mini DINO

Fraction with all three labels correct: 0.920 at the trained resolution 96, 0.953 at 128 with interpolated position embeddings, 0.790 at 64. Type of a single shape: random initialization 0.59, mini DINO trained on single-shape images 0.934, supervised 0.999

Interpolating position embeddings makes other resolutions usable. The data decides what self-supervised learning learns

07

MiniDETR, 60 epochs

AP 0.8375, AP50 0.9863, AP75 0.9500. AP at decoder layers 1 · 2 · 3: 0.8067 · 0.8365 · 0.8375. AP difference before/after NMS −0.0016. Overlapping pairs 153 → 55 → 45

Duplicates are rare even without NMS. Predictions are available whichever decoder layer you stop at

08

Mini Deformable DETR, 60 epochs; 15-epoch ablations

AP 0.8982, AP50 0.9795. First reached AP 0.8 at epoch 7 (epoch 50 in chapter 7). Ablations: baseline 0.8763, without iterative box refinement 0.7500, two-stage 0.9100, single level 0.8563, 3 extra encoder layers 0.8720

Gets higher, faster than chapter 7. The clear effects in the ablations were iterative box refinement and two-stage query selection

09

Window-major vs. row-major layout

Max output difference between the two: 7.15e-07. No time difference showed up in measurements on this PC

Window-major layout does the same computation without reordering. The speed gain could not be confirmed in this tutorial

10

Measured shapes · parameters of RF-DETR Nano

30,467,298 parameters (26,863,662 used at inference). The mini model has 1,832,707 (1,675,870)

Some parameters are used only in training (query groups · per-group heads)

11

Mini RF-DETR, 60 epochs; 15-epoch ablations (3 seeds)

AP 0.9347 (EMA model) · 0.9292 (regular model). First reached AP 0.8 at epoch 3. AP at decoder layers 0 · 1 · 2 · 3: 0.8928 · 0.9289 · 0.9332 · 0.9347. Spearman correlation between score and IoU +0.760 (chapter 8 model +0.358). Mean ablation differences: without EMA −0.0135, focal loss −0.0057, 1 group −0.0025, 1 × 1 windows +0.0006

The clear effects were the gain from EMA and IA-BCE aligning scores with IoU

12

Weight-sharing NAS on mini RF-DETR, 60 epochs

AP of the default configuration 0.9498 (chapter 11 fixed-configuration model 0.9345). 321 of 408 configurations reached AP 0.8 or higher. 0.9208 at resolution 64 (fixed-configuration model 0.1329), 0.9567 at 128 (0.3063). With PI-resize, 0.9348 – 0.9352 at unseen patch sizes; with bilinear interpolation alone, 0.8585 → 0.6315

One set of weights yields a family of models. A fixed-configuration model collapses outside the configuration it was trained in

13

Segmentation head on the chapter 11 model, 30 epochs; 4-epoch ablations (3 seeds)

Box AP 0.9347 → 0.9349, mask AP 0.8766 (AP50 0.9884, AP75 0.9176). Ablation mask AP: BCE + Dice 0.743, BCE only 0.707, Dice only 0.561, masks at 1/2 resolution 0.775

Masks are gained without losing box AP. Using both losses together worked best

14

Fine-tuning the real RF-DETR Nano on 1,000 shape-world images

From pretrained weights: validation AP 0.9476 after 1 epoch, 0.9839 after 5 epochs (EMA), test AP 0.983. From random initialization: validation 0.7749 after 3 epochs, test 0.772. Latency on this laptop (median): PyTorch GPU FP32 forward pass 16.9 ms, onnxruntime CPU 112.3 ms

The power of fine-tuning comes from pretraining. Latency depends on how it is measured

Note · 0.9345 and 0.9347
The AP of the chapter 11 mini RF-DETR is given as 0.9347 in chapters 11 · 13 and as 0.9345 in the chapter 12 scorecard. They are the same weights. As chapter 12 explains, the difference is a floating-point error caused by a different batch size for grouping images during evaluation.

Three models run back to back on the same data

The models in chapters 7 · 8 · 11 were trained on the same data · with the same number of epochs · with the same batch size. Here is the table from chapter 11 (validation AP).

Epoch

Ch. 7 MiniDETR

Ch. 8 Deformable DETR

Ch. 11 mini RF-DETR (EMA)

Ch. 11 mini RF-DETR (regular)

1

0.0048

0.2173

0.3085

0.3011

3

0.0405

0.6698

0.8190

0.8133

7

0.4193

0.8109

0.8907

0.8766

10

0.5788

0.8380

0.9074

0.9023

20

0.7039

0.8654

0.9255

0.9135

40

0.7358

0.8959

0.9325

0.9265

50

0.8386

0.9020

0.9347

0.9277

60

0.8375

0.8982

0.9347

0.9292

First epoch reaching AP 0.5 / 0.8 / 0.9

9 / 50 / —

2 / 7 / 50

2 / 3 / 9

2 / 3 / 10

Caution · the differences in this table cannot be credited to any one component
Seven things changed at once between the chapter 8 model and the chapter 11 model.
  1. Backbone: a ViT (window + global attention) and a C2f projector instead of a small CNN. Also from two feature levels to one
  2. Two-stage query selection (the main chapter 8 model did not use it)
  3. Boxes: box reparameterization and lite reference-point refinement instead of iterative box refinement
  4. Classification loss: focal loss → IA-BCE
  5. 4 query groups
  6. EMA
  7. Learning rate: lowered after epoch 48 → constant to the end, with layer-wise decay
What chapter 11 concluded from this comparison goes no further than “a large part of the improvement over the main chapter 8 model can be explained by two-stage query selection and EMA”. The effect of a single component should be read from an ablation that changes only that component.

Results that did not match expectations

So that you do not read only the good numbers, here in one place are the mini-experiment results that did not match expectations or failed to reproduce what the papers report. “Not observed” does not mean “no effect”; it means the effect was smaller than the resolution of the ruler with this data · at this scale.

  • Chapter 2: in the single-box fitting experiment, there was no evidence that combining L1 and GIoU beats using either alone. The combined loss was faster because of its coefficients.
  • Chapter 4: focal loss learned more slowly than BCE, and its AP after 30 epochs was slightly lower (0.914 vs. 0.928). Recall at threshold 0.5 was clearly lower (0.710 vs. 0.834).
  • Chapter 5: with the same architecture · same data, some seeds produced attention maps that “look at everything except the assigned shape”. Those models still located the shapes correctly.
  • Chapter 6: the mini DINO trained on scenes learned object counts (0.754), but for which shapes are in a scene it was nearly the same as a randomly initialized backbone (0.737 vs. 0.712 · 0.721).
  • Chapter 7: DETR's slow convergence reported on COCO in the paper (500 epochs) was not reproduced. The mini DETR passed AP 0.5 at epoch 9. Size-based specialization of queries was not observed either.
  • Chapter 8:
    • AP50 was higher for the chapter 7 DETR (0.9863 vs. 0.9795).
    • On a 12×12 map, deformable attention was 2.6 times slower than ordinary attention.
    • The effects of multi-scale features and the encoder were within the range of chance (±0.02).
  • Chapter 9: the speed gain of window-major layout (3.9 → 2.9 ms on T4 · TensorRT in the paper) did not show up in PyTorch measurements on this PC. Window attention itself gave no speed gain with few tokens either.
  • Chapter 11:
    • The effects of Group DETR (4 groups vs. 1) and the number of windows were within the ruler's resolution (−0.0025, +0.0006).
    • IA-BCE clearly aligned scores with IoU (Spearman correlation +0.65 vs. +0.26), but the AP gain was small (switching to focal loss: −0.0057).
    • The gain from weak backbone pretraining was not reproduced (200 training images: +0.0109 with the official learning-rate rules, −0.0169 with a constant learning rate; 4,000 images: −0.0005).
    • For a randomly initialized backbone, a constant learning rate beat the official learning-rate rules (0.7170 vs. 0.6752).
  • Chapter 12:
    • With 0 decoder layers (encoder proposals only), the NAS-trained model was 0.09 lower than the fixed-configuration model. This is the subnetwork where the cost of weight sharing shows.
    • The Pareto frontier of the same configurations ranked by compute differed from the frontier ranked by latency on this PC.
    • Fine-tuning fixed to a single configuration made the other configurations collapse (fixed at resolution 96: 0.9208 → 0.5912 at 64 / patch 8).
  • Chapter 13:
    • BCE's weakness, “small objects get buried in the background”, did not show up strongly (mask AP 0.707 with BCE alone).
    • The mask term in the matching cost made no difference (0.739 without, 0.743 with).
    • For rectangles, masks did not describe the region better than boxes.
    • Attaching the segmentation head dropped box AP for a while before it recovered (0.9347 → 0.8906 → 0.9349).
  • Chapter 14: from epoch 4 to 5 of fine-tuning, the training loss rose (1.425 → 1.513) while validation AP kept rising. The EMA weights were slightly higher than the regular weights in all five epochs (difference 0.002 – 0.006).

6. Glossary

The 260 terms used in the tutorial, split into tables by topic. Within each table, the rows follow the order of the original Korean terms (numbers · Latin letters first, then Korean alphabetical order), so they are not in English alphabetical order. The “Meaning” column is the same sentence as the summary on the term card that pops up when you click the term in the original tutorial. The “Standard term” column gives the usual English name and abbreviations. “Explained in” is the chapter that first defines and explains the term in detail (chapters where the name merely appears earlier are not listed).

Terms · the big picture of detection (chapter 01)

Term

Standard term

Meaning

Explained in

One-stage detector

one-stage detector, single-stage detector

A detector that outputs boxes and classes directly in a single pass over the image, without a region proposal stage.

Ch. 01

Two-stage detector

two-stage detector

A detector that first extracts region proposals, then classifies each proposal and refines its box.

Ch. 01

Object detection

object detection

The problem of answering, for each object in an image, both where it is (a box) and what it is (a class).

Ch. 01

Benchmark

benchmark

A test set up so that everyone is scored on the same data in the same way.

Ch. 01

Vision-language model (VLM)

vision-language model, VLM

A model trained on large numbers of image – text pairs so that it can handle images and text together.

Ch. 01

Hand-designed component

hand-designed component

A detector component whose rules and numbers are set by people rather than learned by the network.

Ch. 01

Real-time

real-time

Producing an answer within the time the application requires. There is no single fixed number; each paper states its own criterion.

Ch. 01

Anchor

anchor, anchor box

A reference box laid out over the image in advance. The detector predicts how much to adjust each anchor.

Ch. 01

Open-vocabulary detector

open-vocabulary detector

A detector that takes what to find as text (words · sentences). Names not seen during training can also be given.

Ch. 01

Image classification

image classification

The problem of putting a class label on the whole image.

Ch. 01

Instance segmentation

instance segmentation

The problem of separately painting the pixels (mask) each object occupies.

Ch. 01

Ablation study

ablation study

An experiment that adds or removes changes one at a time and measures performance to find out what helps and by how much.

Ch. 01

Latency

latency

The time from feeding in one image until the answer comes out. Measured in milliseconds; smaller is better.

Ch. 01

Class

class

A kind of object the detector distinguishes. The list is fixed before training.

Ch. 01

Class imbalance

class imbalance

A situation where one answer (background, in detection) vastly outnumbers the others.

Ch. 01

Detector

detector

A model that solves the object detection problem.

Ch. 01

Specialist detector

specialist detector, closed-vocabulary detector

A detector that answers only within the class list fixed at training time.

Ch. 01

Region proposal

region proposal

A rectangular region picked out as likely to contain an object.

Ch. 01

COCO

Common Objects in Context

A photo dataset of common objects in everyday scenes. The standard benchmark for object detection (80 classes).

Ch. 01

Objects365

Objects365

A large-scale object detection dataset. RF-DETR and LW-DETR use it to pretrain the detector.

Ch. 01

RF100-VL

Roboflow100-VL

A benchmark bundling 100 object detection datasets from different domains.

Ch. 01

Terms · boxes and evaluation (chapters 02 – 03)

Term

Standard term

Meaning

Explained in

False positive (FP)

false positive

A prediction not paired with any ground truth. Spurious predictions · predictions with the wrong class · predictions whose box is too far off · duplicate predictions all count.

Ch. 03

False negative (FN)

false negative

A ground-truth object not paired with any prediction. An object the detector missed.

Ch. 03

Intersection (boxes)

intersection

The part contained in both boxes. A rectangle whose top left is found with max and whose bottom right is found with min.

Ch. 02

Enclosing box

smallest enclosing box

The smallest box C that covers both boxes. Used to compute GIoU.

Ch. 02

Corner representation (xyxy)

corner format, xyxy

Writing a box as the coordinates of its top-left and bottom-right corners.

Ch. 02

Box (bounding box)

bounding box

A rectangle enclosing an object, with sides parallel to the image's horizontal · vertical axes. Written as four numbers.

Ch. 02

Box regression

box regression

Fitting the four numbers of a predicted box to the ground-truth box.

Ch. 02

Interpolated precision

interpolated precision

The highest precision among the points with recall r or higher. The PR curve with its sawtooth filled in.

Ch. 03

Broadcasting

broadcasting

The rule that, when operating on tensors of different shapes, stretches axes of length 1 to match the other tensor's length.

Ch. 02

Recall

recall

The fraction of ground-truth objects that were found. The ability not to miss things.

Ch. 03

Score threshold

score threshold, confidence threshold

The cutoff above which predictions are accepted. Set by whoever uses the detector.

Ch. 03

Normalized coordinates

normalized coordinates

Coordinates written as values between 0 and 1, treating the image width and height as 1.

Ch. 02

Precision

precision

The fraction of predictions that are correct. The ability not to guess wrong.

Ch. 03

Precision – recall curve (PR curve)

precision-recall curve

The curve joining the (recall, precision) points obtained by accepting predictions one by one in score order.

Ch. 03

Accuracy

accuracy

The fraction of all problems answered correctly. Used in classification; not usable in detection.

Ch. 03

Duplicate prediction

duplicate detection

The second and later predictions pointing to the same object. They count as false positives in evaluation.

Ch. 03

Center representation (cxcywh)

center format, cxcywh

Writing a box as the coordinates of its center plus its width · height.

Ch. 02

True positive (TP)

true positive

A prediction paired with a ground-truth object. Its IoU with a ground truth of the same class must be at or above the threshold, and no other prediction may have claimed that ground truth first.

Ch. 03

Scale invariance

scale invariance

The property that a value does not change when both boxes are scaled up or down by the same factor.

Ch. 02

AP by size (APS · APM · APL)

AP across scales

AP computed separately after splitting ground-truth objects by area: small (under 32² pixels), medium (32² – 96²), large (over 96²).

Ch. 03

Inclusion–exclusion principle

inclusion–exclusion principle

Counting by adding the sizes of two sets and then subtracting once the overlap that was counted twice.

Ch. 02

Pixel coordinates

pixel coordinates

Coordinates measuring a position on the image in pixels. The origin is at the top left, and y increases downward.

Ch. 02

Union (boxes)

union

The part contained in at least one of the two boxes. Its area is the sum of the two areas minus the intersection.

Ch. 02

AP (average precision)

average precision

An approximation of the area under the PR curve. The COCO method averages interpolated precision at 101 recall points 0, 0.01, …, 1.

Ch. 03

AP50

AP at IoU = 0.50

AP scored with the single IoU threshold 0.50 (averaged over classes). Checks “was the object found at all, even roughly?”

Ch. 03

AP75

AP at IoU = 0.75

AP scored with the single IoU threshold 0.75 (averaged over classes). Checks whether the boxes are fairly accurate.

Ch. 03

F1 score

F1 score

The harmonic mean of precision and recall. Summarizes performance at a single score threshold in one number.

Ch. 03

GIoU (generalized IoU)

Generalized Intersection over Union

IoU minus the fraction of empty space inside the enclosing box. Even without overlap, it decreases with distance.

Ch. 02

GIoU loss

GIoU loss

The loss 1 minus GIoU (0 – 2). Provides gradients even for non-overlapping boxes.

Ch. 02

IoU (intersection over union)

Intersection over Union

The fraction of the total area covered by two boxes that both boxes cover. 0 (no overlap) – 1 (perfect match).

Ch. 02

IoU threshold

IoU threshold

The criterion for how much a predicted box must overlap a ground-truth box to count as correct. COCO uses ten values from 0.50 to 0.95.

Ch. 03

IoU loss

IoU loss

The loss 1 minus IoU. Scale-invariant, but its gradient is 0 when the boxes do not overlap.

Ch. 02

L1 distance (L1 loss)

L1 distance, L1 loss

The sum of the absolute differences between the four numbers of two boxes, compared one by one.

Ch. 02

mAP

mean average precision

The mean of the per-class AP values. In COCO it is used interchangeably with AP.

Ch. 03

NMS (non-maximum suppression)

non-maximum suppression

Post-processing that keeps only the highest-scoring box among overlapping predicted boxes and removes the rest.

Ch. 03

Terms · deep learning basics and classification losses (chapter 04)

Term

Standard term

Meaning

Explained in

1×1 convolution

1×1 convolution, pointwise convolution

A convolution with kernel size 1. Equivalent to applying the same linear layer at every position.

Ch. 04

Weight decay

weight decay

Pulling the weights slightly toward 0 at every step so they do not grow needlessly large.

Ch. 04

Gradient descent

gradient descent

Reducing the loss by moving the parameters a little at a time in the direction opposite to the gradient.

Ch. 04

Cross-entropy

cross-entropy, CE

The loss for classification that picks one of several classes. The −log of the probability given to the correct class.

Ch. 04

Log-odds

log-odds

The log of the probability of occurring ÷ the probability of not occurring (the odds). The inverse of the sigmoid.

Ch. 04

Logit

logit

The real number before it enters a sigmoid · softmax. The score before it becomes a probability.

Ch. 04

Parameter

parameter

The adjustable numbers inside a neural network (weights and biases). Written together as θ.

Ch. 04

Mini-batch

mini-batch

A small set drawn at random from the data and used for one update.

Ch. 04

Background class

background class, no-object class ∅

A fake (K + 1)-th class added to the K classes to represent “no object”.

Ch. 04

Batch normalization (BatchNorm)

batch normalization

A layer that computes the mean · variance of each feature (channel) along the batch axis and rescales to mean 0 · variance 1.

Ch. 04

Soft target

soft target

A classification target t given as a value between 0 and 1 rather than exactly 0 or 1.

Ch. 04

Prior-probability bias initialization

prior probability bias initialization

Initialization that sets the classification head's bias to b = −log((1 − π)/π) so that all initial predictions are near π.

Ch. 04

Linear layer

linear layer, fully connected layer || h = Wx + b. A layer that multiplies a vector by a weight matrix and adds a bias.

Ch. 04

Softmax

softmax

A function that turns K logits into K probabilities summing to 1.

Ch. 04

Loss function

loss function

A single number measuring how far the network's output is from the ground truth. Smaller is better.

Ch. 04

Forward pass

forward pass

Computing values in order from the input to the loss.

Ch. 04

Stride

stride

How many cells the kernel moves at a time in a convolution.

Ch. 04

Sigmoid

sigmoid, logistic function

The function σ(z) = 1/(1 + e^(−z)) that turns a real number (logit) into a probability between 0 and 1.

Ch. 04

Neural network

neural network

A function f_θ with parameters θ. It takes an input, produces an output, and learns by adjusting its parameters.

Ch. 04

Epoch

epoch

One full pass over the training data.

Ch. 04

Backpropagation

backpropagation

Computing the gradients of all parameters by going backward from the loss and passing on (incoming gradient) × (local derivative).

Ch. 04

Chain rule

chain rule

The derivative of a composite function is the product of the derivatives of each step.

Ch. 04

Binary cross-entropy (BCE)

binary cross-entropy

The loss for yes / no classification. The −log of the probability given to the correct side.

Ch. 04

Residual connection

residual connection, skip connection

Adding a layer's input directly to its output: x + f(x).

Ch. 04

Transposed convolution

transposed convolution

A convolution that enlarges a feature map. Each input number stamps a kernel-sized imprint onto the output, and overlapping imprints are summed.

Ch. 04

Maximum likelihood estimation

maximum likelihood estimation, MLE

The principle of choosing parameters that maximize the probability (likelihood) of the observed answers.

Ch. 04

Layer normalization (LayerNorm)

layer normalization

A layer that normalizes using the mean · variance of the features of a single sample (token). Independent of the batch.

Ch. 04

Kernel

kernel, filter

The small plate of weights that slides over the input in a convolution.

Ch. 04

Padding

padding

The number of rows of zeros added around the edges of the input before a convolution.

Ch. 04

Focal loss

focal loss

A classification loss that scales down the loss of well-predicted cells by (1 − p_t)^γ to focus on the hard cells.

Ch. 04

Learning rate

learning rate

The number η that sets how far the parameters move in one step.

Ch. 04

Convolution

convolution

A layer that slides a small plate of weights (the kernel) over the input and outputs the sum of products at each position.

Ch. 04

Activation function

activation function

A nonlinear function inserted between layers. Applied to each number separately.

Ch. 04

Adam

adaptive moment estimation

An optimizer that divides the moving average of the gradient by the square root of the moving average of the squared gradient, equalizing the update size for each parameter.

Ch. 04

AdamW

Adam with decoupled weight decay

Adam with weight decay applied separately instead of mixed into the gradient.

Ch. 04

GELU

Gaussian error linear unit

x·Φ(x). An activation function that passes x through smoothly using the cumulative distribution function of the standard normal distribution.

Ch. 04

MLP (multi-layer perceptron)

multi-layer perceptron

A neural network built by stacking linear layers and activation functions alternately.

Ch. 04

ReLU

rectified linear unit

max(0, x). An activation function that sets negatives to 0 and passes positives through unchanged.

Ch. 04

SGD (stochastic gradient descent)

stochastic gradient descent

Gradient descent using the gradient of a mini-batch.

Ch. 04

SiLU (Swish)

sigmoid linear unit, Swish

x·σ(x). An activation function that passes x through smoothly using the sigmoid.

Ch. 04

Terms · attention, ViT, DINOv2 (chapters 05 – 06)

Term

Standard term

Meaning

Explained in

Value (attention)

value

The content vector actually passed along in attention. Averaged with the weights to form the output.

Ch. 05

Cross-attention

cross-attention

Attention whose queries come from one set of tokens and whose keys · values come from another.

Ch. 05

Inductive bias

inductive bias

Assumptions built into a model's structure before it sees any data.

Ch. 06

Dot product

dot product

The sum of the products of corresponding components of two vectors. Larger when the vectors point in similar directions and are longer.

Ch. 05

Multi-head attention

multi-head attention

Attention that splits dimension D into h heads, attends separately in each head, then concatenates the results.

Ch. 05

Register tokens

register tokens

Extra learned tokens inserted into a ViT's token sequence. They give global information a separate place to be written, removing spikes in the patch features.

Ch. 06

Multi-crop

multi-crop

Augmentation that makes a few large global crops and many small local crops from one image.

Ch. 06

Fine-tuning

fine-tuning

Starting from pretrained weights and training a little more on the (small) data of the target task.

Ch. 06

Dense features

dense features

Features with one per position (patch) rather than one for the whole image.

Ch. 06

Collapse

collapse

A state where the model gives the same output whatever image comes in. The loss is low, but nothing has been learned.

Ch. 06

View

view

One image made by randomly transforming an image (cropping · flipping · color changes).

Ch. 06

Pretraining

pre-training

Learning weights in advance by solving a more general task on very large data, before the final task.

Ch. 06

Linear-probe evaluation (linear probe)

linear probe

Measuring feature quality by freezing the backbone and training only a single linear layer on its features.

Ch. 06

Permutation equivariance

permutation equivariance

The property that shuffling the order of input tokens only shuffles the outputs in the same way, without changing their values.

Ch. 05

Scaled dot-product attention

scaled dot-product attention

The basic Transformer attention formula: divide the dot-product scores by √d_k, then apply softmax · a weighted average.

Ch. 05

Attention

attention

An operation that sets weights from the similarity of queries and keys and takes a weighted average of the values.

Ch. 05

Attention weight

attention weight

The share a query gives to each key · value. All positive, summing to 1.

Ch. 05

Attention mask

attention mask

A mask that decides, for each (query, key) pair, whether it can be seen. Shape (N_q, N_k).

Ch. 05

Key

key

The vector in attention that announces “what I have”. Its dot product with a query becomes a score.

Ch. 05

Key padding mask

key padding mask

A mask on the key side that hides fake padding tokens from every query. Shape (B, N_k).

Ch. 05

Temperature (of softmax)

temperature

A positive number τ that divides the scores before the softmax. The lower it is, the sharper the distribution.

Ch. 06

Positional encoding

positional encoding

A fixed vector built from sines · cosines and added to tokens to tell them their position.

Ch. 05

Position embedding

position embedding

A learned vector, one per position. Added to tokens to tell them “where in the image they came from”.

Ch. 06

Position embedding interpolation

position embedding interpolation

Reshaping learned position embeddings back into a 2D grid, resizing them like an image, and flattening them again.

Ch. 06

Causal mask

causal mask

A triangular attention mask that keeps each position from seeing tokens after it.

Ch. 05

Self-attention

self-attention

Attention whose queries · keys · values all come from the same set of tokens. The tokens look at each other.

Ch. 05

Self-distillation

self-distillation

Distillation without a pretrained teacher, using the exponential moving average of the student itself as the teacher.

Ch. 06

Self-supervised learning

self-supervised learning, SSL

Learning in which the targets are created from the data itself instead of human-assigned labels.

Ch. 06

Distillation (knowledge distillation)

knowledge distillation

A technique that trains a student network to imitate the teacher network's output distribution instead of ground-truth labels.

Ch. 06

Query

query

The vector in attention that expresses “what I am looking for”. Produced by the side receiving information.

Ch. 05

Token

token

One vector, the unit attention works on. In this tutorial, one image patch or one object query.

Ch. 05

Projection head

projection head

A small MLP placed on top of the backbone output in self-supervised learning to compute the loss. Discarded after pretraining.

Ch. 06

Foundation model

foundation model

A model pretrained once at large scale and used as the starting point for many tasks.

Ch. 06

Patch

patch

A square piece of p × p pixels cut from the image without overlap. Becomes one ViT token.

Ch. 06

Patch embedding

patch embedding

Flattening one patch and turning it into a token of length D with a linear layer. The same computation as a convolution with kernel · stride p.

Ch. 06

Centering

centering

Subtracting the moving average (center c) from the teacher's scores. Prevents the collapse where every image piles onto one dimension.

Ch. 06

CLS token

class token, [CLS]

A learned vector inserted at the front of the token sequence. Its output at the last layer is used as a summary of the whole image.

Ch. 06

DINO

self-distillation with no labels

A self-supervised learning method in which, for different views of the same image, the student's output distribution follows that of an EMA teacher.

Ch. 06

DINOv2

DINOv2

A ViT given self-supervised pretraining on 142 million curated images using the DINO and iBOT losses together. The backbone of RF-DETR.

Ch. 06

FFN (feed-forward network)

feed-forward network

A two-layer network (D → rD → D) applied to each token separately after attention.

Ch. 05

Flash Attention

FlashAttention

A fast attention implementation that computes in blocks without building the full N×N weight matrix, with the same result.

Ch. 05

iBOT

image BERT pre-training with online tokenizer

Patch-level self-distillation that masks some patches of the image given to the student and makes the outputs at those positions follow the teacher's patch outputs.

Ch. 06

LVD-142M

LVD-142M

The curated dataset of 142 million unlabeled images used to pretrain DINOv2.

Ch. 06

post-norm

post-normalization

The Transformer layer form that applies layer normalization after adding the residual: LN(x + f(x)).

Ch. 05

pre-norm

pre-normalization

The Transformer layer form that applies layer normalization before the sublayer: x + f(LN(x)).

Ch. 05

Sharpening

sharpening

Keeping the teacher's softmax temperature low so its distribution leans to one side. Prevents collapse to the uniform distribution.

Ch. 06

Transformer

Transformer

A network architecture that stacks layers of multi-head attention and an FFN wrapped with residual connections · layer normalization.

Ch. 05

Vision Transformer (ViT)

Vision Transformer

A model that cuts an image into patches to form a token sequence and processes it with a standard Transformer encoder.

Ch. 06

Terms · the DETR lineage (chapters 07 – 09)

Term

Standard term

Meaning

Explained in

Two-stage scheme

two-stage

A scheme in which every cell of the feature map produces a proposal box and a score, and the top-scoring few become the decoder's initial reference boxes.

Ch. 08

Object query

object query

The \(N\) learned vectors fed into the DETR decoder. Each one is responsible for one prediction slot.

Ch. 07

Stop-gradient

stop-gradient

An operation that uses a value as is in the forward pass but treats it as a constant in the backward pass, letting no gradient through.

Ch. 08

Reference box

reference box

Using a box \((c_x, c_y, w, h)\) instead of a point as the reference for deformable attention and box prediction.

Ch. 08

Reference point

reference point

One normalized coordinate at which a query decides to “look around here”. The sampling points of deformable attention are this point plus offsets.

Ch. 08

Multi-scale features

multi-scale features

Using several feature maps of different resolutions together. Small objects are found in dense maps, large objects in coarse maps.

Ch. 08

Multi-level feature aggregation

multi-level feature aggregation

Taking feature maps not only from the last layer of the ViT encoder but also from intermediate layers, and feeding them together into the projector.

Ch. 09

Matching cost

matching cost

A value measuring how poorly prediction \(i\) fits ground truth \(j\). DETR uses the sum of three terms: class probability · L1 distance · GIoU.

Ch. 07

Head (prediction head)

head

A small network that turns the query vectors from the decoder into final predictions (class logits, boxes, masks).

Ch. 09

Iterative box refinement

iterative bounding box refinement

A scheme in which each decoder layer takes the boxes from the previous layer as reference and corrects them bit by bit.

Ch. 08

Box reparameterization

box regression target reparameterization

A box prediction scheme that reads the box head's output as “how many reference-box sizes to shift, and by what factor to scale”.

Ch. 09

Assignment

assignment

A correspondence \(\sigma\) that picks a distinct prediction for each ground truth.

Ch. 07

Backbone

backbone

The trunk of the network that extracts features from the image. The projector · decoder · heads sit on top of its output (feature maps).

Ch. 09

Deformable attention

deformable attention

Attention in which each query reads values only at a few sampling points around its reference point and takes their weighted average. The offsets and weights are predicted from the query vector with linear layers.

Ch. 08

Parallel decoding

parallel decoding

Feeding all \(N\) object queries at once and producing \(N\) predictions in a single forward pass.

Ch. 07

Auxiliary loss

auxiliary loss

The same loss applied to predictions made with the same heads on every intermediate decoder layer's output, not just the last layer.

Ch. 07

Cost matrix

cost matrix

A table listing, for every pair, the cost \(C_{ij}\) of assigning prediction \(i\) to ground truth \(j\). Shape (number of predictions, number of ground truths).

Ch. 07

Mixed query selection

mixed-query selection

A scheme in which a query's position (reference box) is set by picking the top-scoring cells of the image's feature map, while its content is a learned embedding.

Ch. 09

Linear interpolation

linear interpolation

Taking the value between two points from the straight line joining them. A weighted average that gives more weight to the closer point.

Ch. 08

Permutation invariance

permutation invariance

The property that changing the order of the inputs does not change the value. A loss for set prediction must have it.

Ch. 07

Bilinear interpolation

bilinear interpolation

Reading the value at a point between grid cells as a weighted average of the four surrounding cells.

Ch. 08

Inverse sigmoid

inverse sigmoid

The inverse of the sigmoid. Maps a value in 0 – 1 back to a logit (any real number).

Ch. 08

Prediction slot

prediction slot

One of DETR's fixed \(N\) prediction places. Each slot outputs one set of (class probabilities, box).

Ch. 07

Offset

sampling offset

A 2D vector giving how far from the reference point to read. Predicted from the query vector with a linear layer.

Ch. 08

Window attention

window attention

Self-attention that splits the token grid into non-overlapping windows and attends only among tokens in the same window. The number of pairs is divided by the number of windows.

Ch. 09

Window-major layout

window-major feature map organization

Laying tokens out window by window rather than row by row. Each window becomes a contiguous span of the sequence, so switching between window and global attention takes only a view.

Ch. 09

Bipartite matching

bipartite matching

The problem of choosing one-to-one pairs between two groups (predictions and ground truths) so that the total cost is minimal.

Ch. 07

One-to-one matching

one-to-one matching

Pairing exactly one prediction with each ground truth, and at most one ground truth with each prediction.

Ch. 07

Global attention

global attention

Ordinary self-attention in which every token sees every token. The name used to distinguish it from window attention.

Ch. 09

Proposal box

region proposal

In the first stage of the two-stage scheme, the candidate a single feature-map cell outputs: “if there is an object here, this is its box”.

Ch. 08

End-to-end

end-to-end

An approach in which one model produces the final answer from the input without hand-designed intermediate steps, and the whole thing is trained at once from the loss on that answer.

Ch. 07

Set loss

set loss, Hungarian loss

DETR's loss, computed on the pairs chosen by Hungarian matching: classification loss for every query + box loss for matched queries.

Ch. 07

Set prediction

set prediction

The problem of outputting an unordered set of elements all at once. DETR solves detection as set prediction of (class, box).

Ch. 07

Greedy assignment

greedy assignment

An assignment that goes through the ground truths in turn and gives each the cheapest remaining prediction at that moment. Fast, but may not be optimal.

Ch. 07

Feature level

feature level

Each individual feature map of a different resolution within multi-scale features.

Ch. 08

Feature map

feature map

A tensor with one vector in each cell of a grid laid over the image. Shape (B, C, H, W).

Ch. 09

Sampling point

sampling point

The reference point plus an offset. Where deformable attention actually reads values from the feature map.

Ch. 08

Projector

projector

The convolutional block connecting the backbone and the decoder. Turns the backbone's feature map(s) into a feature map with the channel count · resolution the decoder reads.

Ch. 09

Hungarian matching

Hungarian matching

How the DETR family pairs predictions with ground truths during training: find the one-to-one assignment with minimum total cost in the matching cost matrix.

Ch. 07

Hungarian algorithm

Hungarian algorithm

A classic algorithm (Kuhn, 1955) that finds the exact optimal solution of the assignment problem in time roughly cubic in the matrix size.

Ch. 07

C2f block

C2f block

A block that does a 1×1 convolution → splits the channels in two → runs n bottlenecks (two 3×3 convolutions each) in sequence → concatenates all intermediate results and merges them with a 1×1 convolution.

Ch. 09

Deformable DETR

Deformable DETR

A detector that replaces DETR's attention with deformable attention to speed up convergence and make multi-scale features usable (Zhu et al., 2020; ICLR 2021).

Ch. 08

DETR

DEtection TRansformer

A model that treats detection as set prediction and detects without anchors or NMS, using a one-to-one matching loss and a Transformer encoder · decoder (Carion et al., 2020).

Ch. 07

Group DETR

Group DETR

A technique that replicates the queries into several groups during training and runs one-to-one matching separately for each group. Positives multiply by the number of groups; inference uses only the first group.

Ch. 09

IoU-aware BCE loss (IA-BCE)

IoU-aware BCE loss

A classification loss that sets the target of a positive cell not to 1 but to t, a weighted geometric mean of the score and the IoU. The score is pulled toward the predicted box's IoU.

Ch. 09

LW-DETR

Light-Weight DETR

A real-time detector stacking a plain ViT encoder, a C2f projector, and a shallow (3-layer) deformable-attention decoder (Chen et al., 2024). The direct foundation of RF-DETR.

Ch. 09

RT-DETR

Real-Time DEtection TRansformer

A detector that made the DETR family real-time by redesigning the encoder after the CNN backbone to be lightweight (Zhao et al., 2023).

Ch. 09

Terms · RF-DETR architecture and training (chapters 10 – 11)

Term

Standard term

Meaning

Explained in

Two-stage query selection

two-stage query selection

The procedure that scores every cell of the projector output, picks the top N cells, and uses the boxes they produce as the reference boxes of the decoder queries.

Ch. 10 · Ch. 08

Gradient accumulation

gradient accumulation

Running forward · backward passes several times on small batches, summing the gradients, and updating the weights only once, to get a large batch's gradient while using only a small batch's memory.

Ch. 11

Gradient clipping

gradient clipping

When the norm of the whole model's gradient exceeds a limit, shrinking only its length to the limit while keeping its direction. The DETR family and RF-DETR use a limit of 0.1.

Ch. 11

Content query

content query

The initial value of the query vectors fed into the first decoder layer. In RF-DETR, it is the learned embedding query_feat, independent of the image.

Ch. 10

Multi-resolution training

multi-scale training

Augmentation that randomly changes the input image resolution during training. RF-DETR picks one resolution per batch and resizes the whole batch to it (0.5 – 1.5× the base resolution).

Ch. 11

Data augmentation

data augmentation

Showing training images flipped, cropped, or otherwise transformed so that one image has the effect of several. RF-DETR uses only horizontal flips and random crops.

Ch. 11

Drop path

drop path, stochastic depth

A regularization technique that skips the branch \(F(x)\) of a residual block entirely with probability \(q\) during training.

Ch. 11

Masked image modeling (MIM)

masked image modeling

Self-supervised pretraining that hides some patches of an image and has the model predict the hidden parts from the visible patches alone.

Ch. 11

Schedule-free training

scheduler-free training

RF-DETR's training approach, with no schedule that changes the learning rate or augmentation during training. The learning rate stays constant to the end, with no warmup.

Ch. 11

Spearman correlation

Spearman's rank correlation

A correlation coefficient between −1 and +1 that measures how well the rankings, rather than the values, of two quantities agree.

Ch. 11

Warmup

warm-up

Gradually raising the learning rate from near 0 to its target value during the very beginning of training. RF-DETR does not use it.

Ch. 11

Per-window CLS token replication

per-window class token

Replicating the CLS token once per window and prepending it to each window, so that window attention and DINOv2's CLS token work together.

Ch. 10

Encoder proposals

encoder proposals

The class logits and boxes of the cells chosen by two-stage query selection. They are enc_outputs in the output dictionary and receive the same loss as the decoder outputs.

Ch. 10

Exponential moving average (EMA)

exponential moving average

An average maintained by mixing in the old average with weight \(d\) and the new value with weight \(1 - d\) each time the value changes. RF-DETR keeps a separate EMA of the weights being trained and uses it for evaluation · saving.

Ch. 11

Query positional embedding

query positional embedding

A vector telling a query its assigned position and size. In RF-DETR, it is built by feeding the sinusoidal embedding of the reference box's four numbers into a 2-layer MLP (ref_point_head).

Ch. 10

Layer-wise learning-rate decay

layer-wise learning rate decay

In a pretrained backbone, repeatedly multiplying the learning rate by a number below 1 (0.8 in RF-DETR) for each block closer to the input, so shallow layers change little and deep layers change a lot.

Ch. 11

Cosine schedule

cosine schedule

A learning-rate schedule that lowers the learning rate from its starting value to its final value along half a period of a cosine curve.

Ch. 11

Learning-rate scheduler

learning rate scheduler

A mechanism that changes the learning rate during training according to a fixed rule. Usually it lowers the rate as training goes on.

Ch. 11

Lite reference-point refinement

lite reference point refinement

A scheme in which every decoder layer shares the initial reference boxes unchanged (public setting lite_refpoint_refine=True). Contrasts with iterative box refinement, which updates the reference boxes at every layer.

Ch. 10

P4

P4 feature level

The name of the feature-map stage where one cell corresponds to about 16 pixels of the original image. In RF-DETR, it means the projector output that keeps the backbone feature resolution (scale 1.0).

Ch. 10

RF-DETR

RF-DETR

A real-time detector that fits a DINOv2 backbone into the LW-DETR skeleton and adds weight-sharing NAS on top (Robinson et al., ICLR 2026).

Ch. 10

Terms · NAS, segmentation, practice (chapters 12 – 14)

Term

Standard term

Meaning

Explained in

Weight sharing

weight sharing

A NAS approach that trains many configurations to share one set of weights, so that after training a configuration can be picked out without retraining.

Ch. 12

Grid search

grid search

A search that fixes a list of values for each knob and evaluates every combination one by one.

Ch. 12

Number of spatial locations

number of spatial locations

The number of cells along one side of the token grid, i.e., resolution ÷ patch size.

Ch. 12

Architecture augmentation

architecture augmentation

The RF-DETR paper's interpretation that randomly changing the configuration during training acts as regularization, like data augmentation.

Ch. 12

Depthwise separable convolution

depthwise separable convolution

A convolution that separates “looking at neighbors” (a depthwise convolution done per channel) from “mixing channels” (a 1×1 pointwise convolution).

Ch. 13

Export

export

Converting a trained model into a file another runtime can read (ONNX, a TensorRT engine, etc.).

Ch. 14

Dice coefficient

Dice coefficient

How much two sets overlap. The size of the intersection divided by the mean size of the two sets.

Ch. 13

Dice loss

Dice loss

A mask loss equal to 1 minus the Dice coefficient computed on probabilities. Penalizes the overlap ratio almost independently of object size.

Ch. 13

Mask

mask

A grid the same size as the image with 0 or 1 in each cell, marking the pixels one object occupies.

Ch. 13

Mask AP

mask AP

AP computed by measuring the overlap between predictions and ground truth with mask IoU instead of box IoU. The standard metric for instance segmentation.

Ch. 13

Mask IoU

mask IoU

The IoU of two masks. The number of pixels that are 1 in both divided by the number that are 1 in at least one.

Ch. 13

Subnetwork

sub-network, sub-net

A model extracted by fixing one configuration of a large weight-sharing network.

Ch. 12

Segmentation head

segmentation head

A lightweight component that takes the detector's projector output and the per-decoder-layer query embeddings and outputs mask logits for each query.

Ch. 13

Knob

tunable knob

A setting whose value can be changed to trade accuracy against speed while leaving the trained weights as they are.

Ch. 12

Effective batch size

effective batch size

The number of images seen per weight update = batch_size × grad_accum_steps.

Ch. 14

Quantization

quantization

Making a model smaller and faster by converting weights and intermediate values to numbers with fewer bits (FP16, INT8).

Ch. 14

Pseudo-label

pseudo-label

A ground truth produced by another model rather than by a person.

Ch. 13

Pseudo-inverse

pseudo-inverse

For a matrix with no inverse, the matrix that gives the shortest solution when there are many, and the least-wrong solution when there are none.

Ch. 12

Power throttling

power throttling

The GPU slowing itself down when it hits its power · thermal limits. It makes latency measurements unstable.

Ch. 12

Preprocessing

preprocessing

Preparing an image before feeding it to the model: convert to 0 – 1, resize to the model resolution, and normalize per channel.

Ch. 14

Point sampling

point sampling

Computing the mask loss and matching cost only at a few sampled points rather than at every cell of the mask.

Ch. 13

Progressive shrinking

progressive shrinking

OFA's training order: train the largest network first, then fine-tune while gradually adding smaller subnetworks to the sampling range.

Ch. 12

Early stopping

early stopping

Stopping training early when the validation metric has not improved for a set number of epochs.

Ch. 14

Throughput

throughput

The number of images processed in a given time (images per second). Larger is better.

Ch. 12

Checkpoint

checkpoint

A file saving the weights (and training state) during · at the end of training.

Ch. 14

Search space

search space

The set of all configurations NAS can choose from.

Ch. 12

Pareto optimal · Pareto frontier

Pareto optimal, Pareto frontier

A model not dominated by any other model is Pareto optimal, and the line joining such points is the Pareto frontier (curve).

Ch. 12

Prototype

prototype

A base image shared by all objects. A weighted sum with each object's coefficients gives that object's mask logits.

Ch. 13

Pixel embedding

pixel embedding

A vector of length \(D\), one per cell of the mask grid. Its dot product with a query embedding gives that cell's mask logit.

Ch. 13

Upsampling

upsampling

Increasing the number of cells in a feature map. The values of the new cells are derived from those of the original cells.

Ch. 13

Post-processing

postprocessing

Turning the model's raw outputs (boxes · logits) into detection results. RF-DETR applies a sigmoid and then picks the top K over all (query × class) pairs.

Ch. 14

COCO format

COCO format

A format that writes a detection dataset as JSON with three lists: images · categories · annotations. Boxes are pixel [x, y, w, h] (relative to the top-left corner).

Ch. 14

Detections

supervision.Detections

The container for detection results returned by rfdetr's predict. Holds xyxy (original-image pixels) · confidence · class_id.

Ch. 14

FlexiViT

FlexiViT (Beyer et al., 2023)

A ViT that randomly changes the patch size during training so that one set of weights works at many patch sizes.

Ch. 12

FLOPs

floating-point operations

The number of floating-point operations in one forward pass. A hardware-independent measure of cost.

Ch. 12

FP16 (half precision)

half precision, 16-bit floating point

Floating point that stores each number in 16 bits. Fewer significant digits than FP32, but half the memory and faster on GPUs.

Ch. 14

MaskDINO

Mask DINO

A model that adds a mask branch to DINO, a DETR-family detector. Produces masks from the dot product of query embeddings and a pixel embedding map.

Ch. 13

NAS

Neural Architecture Search

Finding a network's architecture by search over a set of candidates (the search space) instead of having people design it.

Ch. 12

OFA

Once-for-All

Weight-sharing NAS that trains one network supporting many architectures and picks out subnetworks without retraining (Cai et al., 2019).

Ch. 12

ONNX

Open Neural Network Exchange

A file format that describes a network as a graph of standard operations. A common language between runtimes.

Ch. 14

ONNX Runtime

ONNX Runtime (onnxruntime)

An engine that runs ONNX graphs. Choose an execution provider to run on CPU · GPU and so on.

Ch. 14

opset

ONNX operator set version

The version number of ONNX's list of standard operations. The default in rfdetr 1.12.1 is 17.

Ch. 14

PI-resize

pseudo-inverse resize

A transform that changes the patch embedding kernel so that the dot product of a resized patch with the new kernel equals the dot product of the original patch with the original kernel.

Ch. 12

RF-DETR-Seg

RF-DETR-Seg

An instance segmentation model that puts a lightweight segmentation head on the RF-DETR detector and outputs a class · box · mask for each object.

Ch. 13

SAM2

Segment Anything Model 2

A segmentation model that outputs the mask of an indicated object (Ravi et al., 2024). Used to create pseudo-labels for pretraining RF-DETR-Seg.

Ch. 13

TensorRT

NVIDIA TensorRT

NVIDIA's inference optimization tool. Compiles a graph into an ‘engine’ tuned for a specific GPU.

Ch. 14

YOLACT

YOLACT

A real-time instance segmentation model that builds masks by linearly combining \(k\) prototype masks with per-object mask coefficients.

Ch. 13

YOLO format

YOLO format

A dataset format with one text file per image, each line holding ‘class cx cy w h’ (normalized).

Ch. 14

7. Paper map: what to read, and in what order

Rather than reading a paper from start to finish, read it in the order below, and when a formula does not make sense, come back to the relevant chapter of this tutorial.

  1. The abstract and Figure 1
  2. The key formulas in the method section
  3. The main tables in the experiments

Only arXiv numbers whose papers were downloaded and checked while writing this tutorial are listed. The year is the year the paper first appeared on arXiv.

Reading order

Order

Paper

Why read it

Related ch.

0

Vaswani et al., “Attention Is All You Need” (2017), 1706.03762

The original source of attention and the Transformer

05

1

Carion et al., “End-to-End Object Detection with Transformers” (DETR, 2020), 2005.12872

Set prediction, Hungarian matching, object queries. Easy to read and the starting point of every later paper

07

2

Zhu et al., “Deformable DETR” (2020), 2010.04159

Deformable attention, iterative box refinement, the two-stage scheme

08

3

Dosovitskiy et al., “An Image is Worth 16x16 Words” (ViT, 2020), 2010.11929

Turning images into patch tokens and feeding them to a Transformer

06

4

Caron et al., “Emerging Properties in Self-Supervised Vision Transformers” (DINO, 2021), 2104.14294 → Zhou et al., “iBOT” (2021), 2111.07832 → Oquab et al., “DINOv2” (2023), 2304.07193

Self-distillation pretraining, and DINOv2, which became RF-DETR's backbone

06

5

Li et al., “Exploring Plain Vision Transformer Backbones for Object Detection” (ViTDet, 2022), 2203.16527

A plain ViT as a detection backbone: window attention and a few global blocks

09

6

Chen et al., “Group DETR” (2022), 2207.13085 · Cai et al., “Align-DETR” (2023), 2304.07527 · Lin et al., “DETR Doesn’t Need Multi-Scale or Locality Design” (2023), 2308.01904

The three training techniques LW-DETR borrowed: query groups, an IoU-aware classification loss, box reparameterization

09

7

Zhao et al., “DETRs Beat YOLOs on Real-time Object Detection” (RT-DETR, 2023), 2304.08069

The first route to a real-time DETR: a CNN backbone and a lightweight encoder

09

8

Chen et al., “LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection” (2024), 2406.03459

The basis of RF-DETR. ViT encoder + projector + shallow decoder

09

9

Cai et al., “Once-for-All: Train One Network and Specialize it for Efficient Deployment” (OFA, 2019), 1908.09791 · Beyer et al., “FlexiViT: One Model for All Patch Sizes” (2022), 2212.08013

The idea of weight-sharing NAS, and how to switch patch sizes

12

10

Robinson et al., “RF-DETR: Neural Architecture Search for Real-Time Detection Transformers” (2025, ICLR 2026), 2511.09554

The subject of this tutorial. Sections 3 – 4 of the main text and Appendices A (training settings) · K (latency measurement) · L (trends in the discovered architectures)

01 · 10 – 13

11

Bolya et al., “YOLACT” (2019), 1904.02689 · Cheng et al., “Mask2Former” (2021), 2112.01527 · Li et al., “Mask DINO” (2022), 2206.02777

Prototype masks, point sampling, masks made from the dot product of queries and pixel embeddings

13

+

Rezatofighi et al., “Generalized Intersection over Union” (2019), 1902.09630 · Lin et al., “Focal Loss for Dense Object Detection” (2017), 1708.02002 · Milletari et al., “V-Net” (2016), 1606.04797

The original sources of the loss functions: GIoU, focal loss, Dice loss

02 · 04 · 13

+

Lin et al., “Microsoft COCO” (2014), 1405.0312 · Robicheaux et al., “Roboflow100-VL” (2025), 2505.20612

Benchmarks: COCO, and RF100-VL with its 100 datasets

01 · 03

+

Peng et al., “D-FINE” (2024), 2410.13842 · Liu et al., “Grounding DINO” (2023), 2303.05499

The real-time detector and the open-vocabulary detector the RF-DETR paper compares against

01

+

Girshick et al., “R-CNN” (2013), 1311.2524 · Ren et al., “Faster R-CNN” (2015), 1506.01497 · Redmon et al., “You Only Look Once” (2015), 1506.02640 · Liu et al., “SSD” (2015), 1512.02325 · He et al., “Mask R-CNN” (2017), 1703.06870

Two-stage · one-stage detectors before DETR. For when you want to know where anchors and NMS came from

01

Where RF-DETR's components came from

Looking at the lineage from the component side, RF-DETR is an assembly of ideas from many papers. Only those covered in the text are listed.

Component

Source

Form in RF-DETR

Ch.

Set prediction · Hungarian matching · object queries · auxiliary losses

DETR (2020)

Unchanged. Auxiliary losses are what make it possible to drop decoder layers at inference

07

Deformable cross-attention · two-stage query selection

Deformable DETR (2020)

The decoder's cross-attention (1 feature level, 2 sampling points); picking reference boxes from the encoder proposals

08 · 10

Window attention + global blocks

ViTDet (2022) → LW-DETR (2024)

Window blocks inserted into the DINOv2 backbone, implemented with window-major layout

09 · 10

Query groups

Group DETR (2022)

13 groups during training, only the first group at inference

09

Mixed query selection

The detector DINO (Zhang et al., 2022)

Content queries are learned embeddings; reference boxes are proposals picked from the image

09 · 10

IoU-aware BCE loss

Align-DETR (2023)

The default classification loss

09 · 11

Box reparameterization

“DETR Doesn’t Need Multi-Scale or Locality Design” (2023)

Boxes predicted without a sigmoid, in units of the reference box size

09 · 10

C2f projector, shallow decoder, multi-level feature aggregation

LW-DETR (2024)

Batch normalization in the projector replaced with layer normalization

09 · 10

Pretrained backbone

DINOv2 (2023)

Replaces LW-DETR's CAEv2 backbone

06 · 10

Weight-sharing NAS

OFA (2019)

Trains by sampling one configuration uniformly at each step, then runs a grid search afterward

12

Changing the patch size

FlexiViT (2022)

Converts the patch embedding kernel with PI-resize

12

Segmentation head

MaskDINO (2022), interpreted via YOLACT (2019)

Dot product of query-side vectors and pixel embeddings. Multi-scale backbone features are not used

13

Note · papers listed without arXiv numbers
The detector DINO (Zhang et al., 2022) and SAM2 (Ravi et al., 2024) are named in the text, but their arXiv numbers are not given because the original papers were not checked directly while writing this tutorial. Search for them by name. The detector DINO is a different work that merely shares its name with DINO · DINOv2, the self-supervised learning methods in chapter 6.

8. Public implementations · tools

After learning the principles with the small implementations in this tutorial, it is best to start real tasks from a well-tested public implementation.

Implementation

Contents

roboflow/rf-detr

The official RF-DETR implementation and the repository of the rfdetr package. The one used in chapter 14. The license differs by model size, so check the README directly

roboflow/supervision

The Detections returned by predict(), plus drawing tools

Atten4Vis/LW-DETR

The official LW-DETR implementation. RF-DETR's code branched off from here

fundamentalvision/Deformable-DETR

The official Deformable DETR implementation. The original deformable attention module

facebookresearch/detr

The official DETR implementation. The matching · loss code is short and easy to read

lyuwenyu/RT-DETR

The official RT-DETR implementation

facebookresearch/dinov2

DINOv2 code and pretrained weights

cocodataset/cocoapi

COCO's official evaluation tool (pycocotools). The source of the definitions the chapter 3 evaluator follows

ONNX Runtime

A runtime for exported ONNX models. Used with the CPU provider in chapter 14

Tip · moving from this tutorial's code to the official code
The names in the mini implementations follow the official implementation. The correspondence is as follows (file paths are under models/ in the rfdetr package).
  • Matching: Hungarian matching in chapters 7 · 11 ↔ matcher.py
  • Loss: RFDETRCriterion in chapter 11 ↔ criterion.py
  • Deformable attention: the chapter 8 module ↔ ops/modules/ms_deform_attn.py
  • Backbone with windows: chapters 9 · 10 ↔ backbone/dinov2_with_windowed_attn.py
  • Projector: C2f in chapters 9 · 10 ↔ backbone/projector.py
  • Query selection and decoder: chapter 10 ↔ transformer.py, lwdetr.py
  • Segmentation head: chapter 13 ↔ heads/segmentation.py

9. Frequently asked questions

Q. Can I follow along to the end without a GPU?
Every exercise file has a --quick argument that runs a scaled-down version, even on a CPU. Use it first to confirm the code runs. The full runs that produced the numbers in the text were done on a single 8 GB laptop GPU, and each chapter notes the actual time taken (for example, 5 epochs of Nano fine-tuning in chapter 14 took 18.2 minutes, with peak GPU memory of 1.61 GiB). Without a GPU, run the full versions on a free cloud notebook, or reduce the number of epochs and look only at the trends.
Q. I got AP 0.93 in the shape world. Is that a better model than RF-DETR Nano with 48.0 AP on COCO?
No. AP can only be compared within the same dataset · the same evaluation protocol (chapter 3). The shape world is an easy problem with 3 classes, a clean background, and only 1 – 4 objects per image. In fact, in chapter 14, fine-tuning the real Nano on the shape world reaches a test AP of 0.983 in just 5 epochs.
Q. Why does the DETR family not need NMS? Most of the 300 queries are false positives; is that fine?
Because training uses one-to-one matching. If two queries point to the same object, one of them is always penalized toward “no object”, and the queries coordinate by seeing each other through the decoder's self-attention (chapter 7). And as long as unmatched queries carry only low scores, AP loses nothing, because low-score false positives sit at the very end of the ranking (chapter 3). That said, duplicates are not zero. Even the mini DETR in chapter 7 had 45 overlapping pairs at the last layer. They are rare enough that applying NMS gains nothing.
Q. The numbers in the paper and the README differ slightly. Which is right?
Both are values with a source. Table 2 of the paper gives Nano 48.0 AP · Small 52.9 AP, while the README table at the time this tutorial checked it gave 48.4 · 53.0. The models may have been updated, so do not mix the two; cite the source (chapter 1). Even within the paper, the last row of Table 5 (54.6) and Medium in Table 2 (54.7) differ by 0.1, and the paper does not explain why.
Q. When the paper's description and the public code's behavior differ, which should I trust?
If you run inference or fine-tune with the public weights, trust the code's behavior, because those weights were trained with that code. Examples encountered in the text:
  • Window block indices: the notation in Appendix A of the paper differs from the rule in the public code (chapter 10).
  • IA-BCE: matching the paper's formula against the official implementation line by line shows they compute the same thing (chapter 9).
  • Normalization of DETR's classification loss: the paper's formula and the denominator of F.cross_entropy differ (chapter 7).
Q. Are the techniques that showed no effect in the mini experiments (Group DETR, number of windows, focal loss) useless?
No. The shape world with 12 queries · 1 – 4 objects and COCO with 300 queries · dozens of objects are different situations. The LW-DETR paper reported +3.0 mAP from Group DETR on COCO (chapter 9). The mini experiments are useful for checking principles by hand, not for judging the worth of a technique. Conversely, it is also risky to assume that a paper's effect will carry over unchanged to your data. The only way to know is to run ablations on your own data (chapter 11).
Q. Which model size should I choose?
The latencies in the tables are for T4 · TensorRT · FP16, so you need to measure them yourself on the hardware and runtime you will use (chapter 12, chapter 14). The steps are roughly:
  1. Set a latency budget (for example, 30 images per second means under 33 ms per image, including pre- and post-processing).
  2. Measure two or three candidate sizes on your hardware in the same way.
  3. Among those within budget, pick the one with the highest validation AP on your data.
  4. If objects are small, try raising the resolution before the model size (the resolution must be a multiple of patch_size × num_windows).
Q. I have little data. Will fine-tuning work?
In the experiment in chapter 14, with 1,000 images, Nano starting from pretrained weights reached a validation AP of 0.95 in the first epoch. Random initialization under the same conditions reached 0.77 after 3 epochs. The strength with little data comes from pretraining. However, this is a single experiment on easy data. How much your data needs can only be found by holding out a validation set and measuring it yourself.
Q. Do I need to run weight-sharing NAS myself?
Usually not. The published Nano – Large and other sizes are configurations that the paper's search already found on COCO. Pick one of them and fine-tune. Reducing the number of decoder layers or queries at inference can be done without retraining (chapter 12). The paper estimates that its search took about 10,000 GPU hours.

10. What to try next

The best way to make what you read your own is to change things yourself. These projects focus on what the text notes as “not tried”, ordered by difficulty.

Difficulty

Project

What you learn

★☆☆

In the chapter 2 box-fitting experiment, vary the learning rate and coefficients to confirm for yourself that the coefficients are why the combined loss was faster

The relationship between loss coefficients and the learning rate

★☆☆

Feed your own fake predictions (deliberately shuffled scores, added duplicates) into the chapter 3 evaluator and guess the AP beforehand

A feel for what AP responds to

★☆☆

Following chapter 14, turn tens to hundreds of your own images into COCO format and fine-tune Nano

The whole flow from data preparation to inference

★★☆

Rerun the chapter 8 ablations with three seeds to check whether the differences from multi-scale features and the encoder really are within chance

Measuring the resolution of an ablation

★★☆

Rerun the main chapter 11 experiment with a random backbone + a constant learning rate (the text did not measure this at 4,000 images · 60 epochs)

What layer-wise learning-rate decay assumes

★★☆

Redo the chapter 11 focal-loss ablation with coefficient 2 (the text used only coefficient 1)

Separating the type of loss from its magnitude

★★☆

Make the shape world harder: more objects, smaller objects, more queries. Then redo the Group DETR ablation

How a technique's effect changes with data and scale

★★☆

Add a control run to chapter 12 that varies only the resolution while fixing the patch · windows, to separate the share of multi-resolution augmentation in the gain from NAS training

How to set up a control

★★☆

Re-evaluate the configurations picked in the chapter 12 search on the test set, to see how optimistic values chosen on the validation set are

Model selection and overfitting

★★★

Train the chapter 13 segmentation model from scratch without a checkpoint, and see whether the mask term in the matching cost makes a difference then

Interactions when training detection and segmentation together

★★★

Build a TensorRT FP16 engine from the ONNX exported in chapter 14 and measure latency and AP together with the same engine (this tutorial did not build an engine)

The paper's measurement procedure; reporting from the same artifact

★★★

Pretrain the mini RF-DETR backbone with the chapter 6 mini DINO and compare with the masked image modeling pretraining of chapter 11

Differences between pretraining methods; experiment design

Review questions

1. A COCO annotation for a 192×192 image has "bbox": [48, 24, 96, 48]. Convert it to this tutorial's normalized center representation \((c_x, c_y, w, h)\). If you mistakenly read it as center-representation pixel values, where does the box end up?
The first two COCO numbers are the top-left corner. \(c_x = (48 + 96/2)/192 = 96/192 = 0.5\), \(c_y = (24 + 48/2)/192 = 48/192 = 0.25\), \(w = 96/192 = 0.5\), \(h = 48/192 = 0.25\). Misread as the center, the center becomes pixel \((48, 24)\), so the box shifts toward the top left by half its width · height (48, 24 pixels). Its left edge becomes \(48 - 48 = 0\) and its top edge \(24 - 24 = 0\), so it sticks to the corner of the image.
2. An image has 144 classification cells, 2 of them positive. The loss is the sum of BCE divided by the number of positives. Compute the first loss when the classification head's bias is 0 (probability 0.5 in every cell) and when it is initialized to the prior probability \(\pi = 0.01\). If the first loss in the training log is around 50, what should you suspect?
  1. With zero bias: every cell's loss is \(-\log 0.5 = 0.6931\), so \(144 \times 0.6931 / 2 = 49.9\).
  2. Matched to the prior: positive cells give \(-\log 0.01 = 4.6052\) and negative cells \(-\log 0.99 = 0.01005\), so \((2 \times 4.6052 + 142 \times 0.01005)/2 = (9.2104 + 1.4271)/2 = 5.32\).
If the first loss is around 50, prior-probability bias initialization is missing. The first few dozen steps will be spent learning that “almost every cell is negative”.
3. The EMA decay is \(d = 0.993\) and the warmup constant is \(\tau = 100\). Compute the effective decay \(d_n = d\,(1 - e^{-n/\tau})\) at step 100. After warmup, the EMA is an average over roughly how many recent steps?
\(1 - e^{-1} = 1 - 0.3679 = 0.6321\), so \(d_{100} = 0.993 \times 0.6321 = 0.628\). It takes in about 37% of the new weights, so it still follows quickly. After warmup, it is an average over roughly the last \(1/(1 - d) = 1/0.007 \approx 143\) steps.
4. While fine-tuning with rfdetr, cardinality_error stays around 36 throughout training. The data has 3.0 objects per image on average. Is something broken?
Nothing is broken. During training, 13 query groups run together, and each group produces one prediction per object. About 13 × 3.0 = 39 queries score above 0.5 and there are 3.0 ground truths, so the difference is 12 × 3.0 = 36. It is a sign that Group DETR is actually running (in chapter 14, with 2.44 objects per image, it was 29.3). Inference uses only the first group, so it is unaffected by this value.
5. A detector trained on your data gives validation results of AP50 0.97, AP75 0.35, AP 0.48. Drawing the predictions on the images shows that nearly every object was found. What is the problem, and what should you check first?
The objects were found, but the boxes are loose. In the chapter 3 experiment, boxes with a mean IoU of about 0.72 produced numbers of this shape (AP50 0.971, AP75 0.170, AP 0.377). Check in this order:
  1. Whether the ground-truth boxes themselves are loosely drawn (annotation quality)
  2. Whether the box representation · coordinate conversion is off by half a pixel · half a cell
  3. Whether the box losses (L1 · GIoU) are still decreasing
  4. Whether the score ranking follows box quality (in the chapter 3 experiment, the same boxes gave AP75 anywhere from 0.153 to 0.393 depending on the score ranking)
6. A colleague says, “Model A is 4.4 ms in the paper's table, and model B measured 15 ms on my laptop, so A is more than three times faster.” List four things missing from this comparison.
Latency can only be compared between measurements taken under the same conditions. The four missing things are:
  1. Hardware: a T4 in the paper, a laptop for the colleague
  2. Runtime: a TensorRT engine vs. a plain PyTorch module
  3. Precision: FP16 vs. FP32
  4. Scope: forward pass only, or including pre- and post-processing
You should also check whether warmup runs were discarded, whether the timer waited for GPU work to finish, whether many runs were measured and the median used, and whether the machine was busy with other work. A comparison only holds when both models are measured alternately on the same machine in the same way.

Closing the tutorial

In chapter 1, object detection was defined as “the problem of answering, for each object in an image, both where it is (a box) and what it is (a class)”. Starting from that one sentence, we built up boxes and IoU, AP, attention, ViT and DINOv2; went from DETR's set prediction through Deformable DETR and LW-DETR; built RF-DETR's architecture · training · NAS · segmentation ourselves; and finally fitted the real RF-DETR to our own data and exported it. Along the way, three things came up again and again: what to predict and what to pair it with, where to look, and how fast to do it. The detectors were different answers to these three questions. You can now read those answers as formulas, verify them in code, and measure for yourself while keeping a skeptical eye on the numbers.

See also

Favorite site

author MCPBot sitemap