Batch size
lastmod 2026-03-31created 2022-11-2510 edits
Batch size
in Deep learning
- In the Naver English dictionary, batch means "a group (processed together)", "one lot (the amount of food, machines, etc. produced at one time)", and "to group together (for batch processing)".
- In other words, it refers to the size of the data that goes into a single computation.
- A data set corresponding to 1 batch size is called a mini Batch.
- One epoch contains m
($m >= 1$)mini Batches, and ifm = 1, it is called batch learning. - If the batch size is too large, the amount of data to process at once increases, so training slows down and there is a risk of running out of memory.
- Conversely, if the batch size is too small, weights are updated on little data and these updates happen frequently, so training becomes unstable.
Batch size vs Epoch vs Iteration
Meaning of batch size
- When the entire training data set is divided into several small groups, batch size means the number of data items in one small group.
- The entire training set is split into small pieces because feeding all of the training data into the neural network at once uses resources inefficiently and makes training take a long time.
Machine-learning_-_batch_size.jpg
Meaning of epoch
- In deep learning, an epoch means the number of times the entire training set has passed through the neural network.
- For example, 1-epoch means the entire training set has been applied to a neural network and has passed through it once via forward propagation and backpropagation.
Machine-learning_-_epoch.jpg
Meaning of iteration
- Finally, iteration means the number of mini-batches needed to complete 1-epoch.
- In other words, it is also the number of parameter updates needed to complete 1-epoch.
- Since parameters are updated once per mini-batch, iteration is both the number of parameter updates and the number of mini-batches.
- For example, when 700 data items are divided into 7 mini-batches of 100 each, 1-epoch requires 7-iteration, and 7 parameter updates are performed.
How to calculate
Detectron2 has no epoch; you must pass Total Iteration and Batch Size directly. In that case, calculate as follows.
Given the following conditions:
- Number of data items in the training set: 100 images
- Batch Size: 4
- Total Iteration: 1,000
then 100 / 4 = 25 Iteration is 1-epoch, and the total epoch is 1,000 / 25 = 40 Total Epoch.
See also
Favorite site
- Deep learning-6.2. Optimization (3) - Training units (Epoch, Batch size, Iteration)
- Machine learning - The meaning of epoch, batch size, iteration : Naver Blog
- Epochs vs iterations in machine learning: what’s the difference
- (Deep learning terms) Comparing the definitions of Epoch, Batch Size, Step
- [https://blog.naver.com/educoding/223627996678? The core of deep learning: Epochs and Batch Si.. : Naver Blog]