Machine learning

lastmod 2026-10-11created 2014-10-2589 edits

Machine learning category

Machine learning is a field of artificial intelligence that develops algorithms and techniques that enable computers to learn. For example, through machine learning, a system can be trained to tell whether a received email is spam or not.

The core of machine learning lies in representation and generalization. Representation is the evaluation of data, and generalization is the processing of data not yet known. This is also the field of computational learning theory. Machine learning has many applications. Character recognition is the best-known example of its use.

In 1959, Arthur Samuel defined machine learning as "the field of study that gives computers the ability to learn, that is, the ability to perform actions not defined in code".

Model Representation

ML_-Model_Representation-_genericmlatwork.png

~~(He raised his middle finger, training began, and a model was born)~~ ¹

A typical machine learning concept diagram is shown above.

Input Features: input variables
Training Data-Set: training variables
Learning Algorithm
- Cost function: How to find the Parameter (θ).
Hypothesis: A function describing the relationship between the predicted data and the result values
Expected Output: The result of Inferencing with the Hypothesis.

Learning Course

MachineLearning:Course:Basic: Machine learning basics
MachineLearning:Course:Python: Python environment setup (pyenv, pyenv-virtualenv, jupyter, numpy, scipy, matplotlib) and PyCharm
MachineLearning:Course:LinearRegression: Explanation of Hypothesis, Linear regression, and Cost function.
MachineLearning:Course:GradientDescent: Explanation of Gradient descent.
~~Linear regression -> Python implementation~~ Merged into the previous item.
- Advanced Python: Advanced Python course
~~Overfitting~~ To be done later
MachineLearning:Course:Perceptron: About Perceptron & Artificial neuron.
~~Multilayer perceptron~~ Merged into the previous item.
MachineLearning:Course:LogisticRegression: Explanation of Logistic regression.
~~Activation function (Sigmoid function & Euler's number)~~ Merged into the previous item.
Perceptron & Perceptron:Example
~~Feedforward neural network~~
~~Softmax function (Loss function)~~
MachineLearning:Course:Backpropagation: Explanation of Backpropagation.
Convolutional neural network (CNN)
Region-based Convolutional Network (RCNN, R-CNN, SSD)
Fully Convolutional Networks for Semantic Segmentation (FCN)
Genertive Adeversarial Networks (GAN)

What is learning?

Training evaluation

About AP (Average Precision), Mean Accuracy, etc. See Precision and recall.

What is training data? (Dataset)

tykimos.github.io - Talking about datasets

10 best machine learning algorithms

10 Best Machine Learning Algorithms (2024) - Unite.AI ²
- Includes "popular uses" examples for each algorithm.
Korean: 트랜스포머
Korean: 생성적 적대 신경망 (GAN)
SVM
K-means clustering
Korean: 랜덤 포레스트
Naive Bayes
K-nearest neighbors (KNN)
Markov decision process (MDP)
Term frequency-inverse document frequency <- ??
Stochastic gradient descent (SGD)

25 open-source AI tools to cut development time in half

25 Open Source AI Tools to Cut Your Development Time in Half - Jozu MLOps
KitOps - Packaging support used to share and manage models, code, metadata, and artifacts
LangChain - Helps develop custom LLM applications with a modular architecture
Pachyderm - Automates data transformation
ZenML - Helps with data preprocessing, model training, evaluation, and deployment through an abstraction for creating MLOps pipelines
Prefect - Builds machine learning pipelines with the concepts of tasks and flows
Ray - Supports easy scaling of machine learning workloads during model development
Metaflow - Provides a unified API needed to take AI projects from prototype→production
MLflow - Helps data scientists and engineers manage model development and experiments
Kubeflow - Designed to simplify orchestration and deployment of ML workflows on Kubernetes clusters
Seldon core - Simplifies deploying, serving, and managing ML models by converting ML models (TensorFlow, PyTorch, H2o, etc.) or language wrappers (Python, Java, etc.) into production-ready REST/GRPC microservices
DVC - Tracks changes to data and models the way Git does for code, and can run on top of any Git repository
Evidently AI - An observability platform designed to analyze and monitor production ML models
Mage AI - A data transformation and integration framework that builds and automates data pipelines without extensive coding
ML Run - Provides serverless technology that orchestrates end-to-end MLOps systems
Kedro - An ML development framework for creating reproducible, maintainable, modular data science code
WhyLogs - An open-source data logging library designed for ML models and data pipelines
Feast - Solves stakeholders' problems of managing and serving ML features in development and production environments
Flyte - Provides data scientists and data/analytics engineers with a Python SDK for building workflows and deploying them easily to the Flyte backend
Featureform - A virtual feature store that simplifies how data scientists manage and serve features for ML models
Deepchecks - An ML monitoring tool that continuously tests and validates ML models and data from the experimentation to the deployment stage of AI projects
Argo - Provides a Kubernetes-native workflow engine that orchestrates parallel jobs on Kubernetes
Deep Lake - An ML-specialized database tool designed to serve as a data lake for deep learning and a vector store for RAG applications
Hopsworks feature store - Provides an end-to-end solution for managing the ML feature lifecycle, from data ingestion and feature engineering to model training, deployment, and monitoring
NannyML - A Python library specialized in monitoring and maintaining ML models after deployment
Delta Lake - A storage layer framework that brings reliability to data lakes
Optuna - A Python-based hyperparameter optimization framework

Table of Machine learning

Table of Machine learning
Learning Course	Theory	Probability theory → Bayesian probability
	Machine learning	Hypothesis → Loss function → Gradient descent → Overfitting
	Neural network	Artificial neuron → Perceptron → Multilayer perceptron → (Example) → Feedforward neural network → Activation function (Sigmoid function & Euler's number) → Softmax function (Loss function) → Backpropagation (Gradient descent) → (Example)
	Deep learning	Convolutional neural network (CNN) & Region-based Convolutional Network (RCNN, R-CNN)
	ETC	Tutorials
Theory	Basic	Terms
	Probability theory	Bayesian probability, Decision Theory, Probability Density Function (PDF), Radial basis function (RBF), Hyperparameter
	Neuroscience d	Neuron
	Statistics	Covariance, Statistical classification, Confusion matrix, Cross-validation, Root-mean-square deviation (MESD), Mean squared error
	Graph theory
	Matrix	General matrix-matrix multiplication (gemm), Toeplitz matrix, im2col
	Other	Computational learning theory, Empirical risk minimization, Occam learning, PAC learning, Statistical learning, VC theory, Bayesian network, Markov random field, Hidden Markov Model (HMM), Conditional Random Field, Normal Bayes Classifier, Energy Based Model, Occam's razor, Ground truth
Algorithm types	Supervised Learning	Support Vector Machine (SVM), Hidden Markov model, Regression analysis, Neural network, Naive Bayes classifier, K-Nearest Neighbor (K-NN), Decision trees, Ensembles (Bagging, Boosting, Random forest), Relevance vector machine (RVM)
	Unsupervised learning	Clustering, Independent component analysis
	Semi-supervised learning	Generative models, Low-density separation, Graph-based methods, Heuristic approaches
	Other learning	Reinforcement learning, deep learning
By topic	Structured prediction	Graphical models (Bayes net, CRF, HMM)
	Parameter estimation algorithms	Dynamic programming, Expectation-maximization algorithm (EM algorithm)
	Approximate inference methods	Monte Carlo method, AdaBoost
	Approaches	Decision tree learning, association rule learning, genetic programming, inductive logic programming, clustering, Bayesian networks, reinforcement learning, representation learning, similarity metric learning
	Modeling	Neural network, SVM K-NN, decision tree, Genetic Algorithm, genetic programming, Gaussian process regression, linear discriminant analysis, perceptron, radial basis function network
	Recommender system (Korean: 추천 시스템)	Collaborative filtering (Korean: 협업 필터링), Content-based filtering (Korean: 컨텐츠 기반 필터링), Hybrid recommender systems
Data mining		Cross-validation (k-fold), Data set (Training set, Validation set, Test set)
Regression analysis		Linear regression, Logistic Regression, Logit function, Multinomial logistic regression (Softmax Regression)
Clustering		k-means clustering, BIRCH, Hierarchical, Expectation-maximization (EM), DBSCAN, OPTICS, Mean-shift
By type		3D Machine Learning
Artificial Neural Networks (ANN)	Artificial neuron	Perceptron, Sigmoid neuron
	Combination function & Activation function	Sigmoid function, Rectified linear units (ReLU), 1x1 Convolution
	Loss function or Cost function	Softmax, Sum of squares (Euclidean), Hinge / Margin, Cross entropy, Infogain, Accuracy and Top-k
	Algorithms	Multilayer perceptron (MLP), Feed-forward Neural Network (FNN), Long short-term memory (LSTM), Autoencoder, Self-organizing map (SOM), Network In Network (NIN), Adaptive neuro fuzzy inference system (ANFIS)
	Deep learning	Deep neural network (DNN), Convolutional neural network (CNN) & Regions with Convolutional Neural Network (RCNN, R-CNN, SSD), Recurrent neural network (RNN), Gated Recurrent Unit (GRU), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Deep Q-Networks, Deep hyper network (DHN), Deconvolutional Network, Fully Convolutional Networks for Semantic Segmentation (FCN), Generative Adversarial Networks (GAN), Learning Deconvolution Network for Semantic Segmentation, Multi-Scale Context Aggregation by Dilated Convolutions (Dilated Convolutions), Dynamic Routing Between Capsules (Capsules Network), YOLO, Path Aggregation Network for Instance Segmentation (PANet), Image-to-Image Translation with Conditional Adversarial Networks (pix2pix), CycleGAN, BicycleGAN, SlowFast, Kinetics, AVA, MelGAN, EfficientDet, SinGAN, Panoptic Segmentation, U-Net, MONAI, CenterNet, DeepLab, HRNet, OCRNet, ResNeSt
	Models	GoogLeNet, LeNet, AlexNet, ReNet, RCNN, SPPnet, Fast R-CNN, Faster R-CNN, VGG, SqueezeNet, PointNet, Mask R-CNN, MaskLab, OSMN (Video Segmentation), ColorNet, EfficientNet
Problems	Dimensionality reduction	Curse of dimensionality, Factor analysis, CCA, independent component analysis (ICA), LDA, non-negative matrix factorization (NMF), principal component analysis (PCA), t-SNE
	Underfitting & Overfitting	Early stopping, Model selection, Normalization, Regularization (L1, L2, Dropout), Generalization
	Initialization	Xavier Initialization
	ETC	Bias-variance dilemma, Vanishing gradient problem, Local minimum problem, Batch Normalization, Standardization, AutoML
Optimization		Hyperparameter, Simulated annealing, Early stopping, Feature scaling, Normal Equation, Gradient descent, Stochastic gradient descent (SGD), Backpropagation, Convex optimization, Performance Tuning, im2col, Dense-Sparse-Dense Training for Deep Neural Networks (DSD), Deep Compression, Pruning Neural Networks, Shake-Shake regularization, Accurate Large Minibatch SGD
ETC	Libraries	OpenCV, Deeplearning4j (DL4J), Torch, PyTorch, Theano, Caffe, TensorFlow, MMDetection, ConvNetJS, cuDNN, Netscope, Microsoft Azure Machine Learning Studio, OpenPose, DensePose, Keras, tiny-dnn, Detectron, Stanford CoreNLP, Aifiddle, Kubeflow, OpenNMT, alibi-detect, Flashlight (facebook), MediaPipe, Weights and Biases
	Dataset	MNIST (handwriting data), ImageNet, CIFAR-10, TinyImages, PASCAL VOC, COCO, AVSS, YouTube-VOS
	Annotation tools	labelme, cvat, f-BRS, COCO Annotator
	See also	Artificial intelligence, autonomous robots, bioinformatics, computational intelligence, computer vision, data mining, pattern recognition, big data
Unknown keyward		Classification, Anomaly detection, Association rules, Reinforcement learning, Structured prediction, Feature engineering, Feature learning, Online learning, Semi-supervised learning, Unsupervised learning, Learning to rank, Grammar induction, Local outlier factor, Minimum Description Length (MDL), Bayesian MAP, Structural Risk Minimization (SRM), Long Narrow Valley, Word2vec, Autoregressive integrated moving average (ARIMA)

Documentation

Bayesian probability

Bayesian Reasoning and Machine Learning: http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.Online; http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/020217.pdf; 020217_-_Bayesian_Reasoning_and_Machine_Learning.pdf

Deep learning

ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)

http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf

http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

ImageNet_Classification_with_Deep_Convolutional_Neural_Networks.pdf

[Korean translation] ImageNet classification using deep convolutional neural networks

: ImageNet_Classification_with_Deep_Convolutional_Neural_Networks_-_ko.pdf

Going deeper with convolutions (GoogleNet): http://arxiv.org/pdf/1409.4842v1.pdf; Going_deeper_with_convolutions.pdf; Translation: GoogleNet

Feature article: Deep Hypernetwork models (Seoul National University / Byoung-Tak Zhang): Deep_Hypernetwork_models_201508.pdf

[Mocrosoft] Deep Residual Learning for Image Recognition (Winner ILSVRC2015): Deep_Residual_Learning_for_Image_Recognition(Winner_ILSVRC2015)_Microsoft.pdf

[Microsoft] Fast R-CNN, Towards Real-Time Object Detection with Region Proposal Networks (Winner ILSVR2015): Fast_R-CNN,Towards_Real-Time_Object_Detection_with_Region_Proposal_Networks(Winner_ILSVR2015)_Microsoft.pdf

Learning Deconvolution Network for Semantic Segmentation: http://cvlab.postech.ac.kr/research/deconvnet/

Deep EXpectation of apparent age from a single image: https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks.: Inside_outside_net_detecting_objects_in_context_with_skip_pooling_and_recurrent_neural_networks_2015.pdf; A paper on Small Object detection methods.

Cho Dae-hyup's blog - Understanding deep learning and TensorFlow for those who gave up on math: http://bcho.tistory.com/1208; Machine_learning_ebooks_-_Machine_learning_for_those_who_abandon_math.pdf

Densely Connected Convolutional Networks (DenseNets): Deep learning CVPR2017 best paper award

Deep Learning Interviews book: Hundreds of fully solved job interview questions from a wide range of key topics in AI.

Deep learning interview book

A collection of practical questions and solutions for master's/PhD students and job seekers learning machine learning

A printed edition can be purchased, but the full PDF is freely available

https://arxiv.org/abs/2201.00650

[Recommended] "2025 Essential Glossary for the AI Era, Explored Through AI Trends and Issues" published by NIA (National Information Society Agency): Essential Glossary for the AI Era (168p PDF) | GeekNews; https://www.nia.or.kr/site/nia_kor/ex/bbs/View.do?cbIdx=82618&bcIdx=28928&parentSeq=28928; Nia.or.kr_-ai-_2025_terms.pdf

There Will Be a Scientific Theory of Deep Learning: There will be a scientific theory of deep learning. | GeekNews; There Will Be a Scientific Theory of Deep Learning; A paper arguing that a scientific theory characterizing key properties of deep learning, such as the training process, hidden representations, final weights, and performance, is taking shape; Presents five research strands (interpretable idealized settings, tractable limits, simple mathematical laws, hyperparameter theory, universal behaviors) as its key evidence

Tutorials

Getting Started with Deep Learning the Right Way (author: Takayuki Okatani / translator: Sim Hyo-seop): Deep_Learning_-Takayuki_Okatani-2015-_sample.pdf

Introductory guide to machine learning - IDG Deep Dive: http://www.itworld.co.kr/techlibrary/97428; IDG_DeepDive_Machine_learning-20160113.pdf

Understanding Deep Learning (unpublished; 2016-08-22 ver): Understanding_deep_learning_0822.pdf

Fundamental of Reinforcement Learning: https://www.gitbook.com/book/dnddnjs/rl/details; Fundamental_of_Reinforcement_Learning.pdf; Modulabs - Fundamentals of reinforcement learning

Deep Learning Papers Reading Roadmap (deep learning paper roadmap): https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap/blob/master/README.md; Deep_Learning_Papers_Reading_Roadmap.md.zip

[Recommended] Machine Learning-based Web Exception Detection (reference site for the Financial Security Institute project!): https://cloudfocus.aliyun.com/Machine-Learning-based-Web-Exception-Detection-89782?spm=a2c1b.a2c1b4.a2c1b4.16.ZSQoEd; Machine_Learning-based_Web_Exception_Detection_-Insights_and_Trends-_Alibaba_Cloud_Focus.pdf

Machine learning basics, parts 1–57 (Exploring Odds and Ends - Chamstudy ePaiai : Naver Blog): http://sams.epaiai.com/220498694383

Microsoft releases ML for Beginners course: A 12-week, 24-lesson curriculum created by the MS Azure Clouds Advocates team; A classic machine learning course using Scikit-learn (deep learning to come later as a separate AI course); https://github.com/microsoft/ML-For-Beginners

CS146S: The Modern Software Developer (Korean) | GeekNews: [Original] CS146S Korean Edition | The Modern Software Developer; Official Korean version of Stanford's CS146S course

Compress model

Deep Compression and EIE - Deep Neural Network Model Compression and Efficient Inference Engine: Deep_compression_and_EIE_PPT.pdf

Learning bothWeights and Connections for Efficient Neural Networks: Learning_both_weights_and_connections_for_efficient_neural_networks_2015.pdf

Convolutional neural network

A CNN presentation using Reveal.js.: Author - me; Reveal-ml.tar.gz

Gradient-Based Learning Applied to Document Recognition: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf; Gradient-Based_Learning_Applied_to_Document_Recognition.pdf

Visualizing and Understanding Convolutional Neural Networks: http://arxiv.org/abs/1311.2901; 1311.2901v3.pdf

Compressing CNN for Mobile Device (Samsung) - The need for CNN model compression etc ...: [http://mlcenter.postech.ac.kr/files/attach/workshop_fall_2015/삼성전자_김용덕_박사.pdf](http://mlcenter.postech.ac.kr/files/attach/workshop_fall_2015/삼성전자_김용덕_박사.pdf); Samsung_-_Compressing_CNN_for_Mobile_Device.pdf

Using Filter Banks in Convolutional Neural Networks for Texture Classification: https://arxiv.org/abs/1601.02919

Deep belief network

The Applications of Deep Learning on Traffic Identification: Us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf; https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf

Deconvolution neural network

Learning Deconvolution Network for Semantic Segmentation: http://cvlab.postech.ac.kr/research/deconvnet/; https://arxiv.org/abs/1505.04366; 1505.04366.pdf

Segmentation

Learning to Segment (Facebook Research): https://research.fb.com/learning-to-segment/; DeepMask+SharpMask as well as MultiPathNet.

Recurrent Instance Segmentation: https://arxiv.org/abs/1511.08250; 1511.08250.pdf; Slideshare - Single Shot MultiBox Detector and Recurrent Instance Segmentation

vid2vid: https://github.com/NVIDIA/vid2vid; Pytorch implementation for high-resolution (e.g., 2048x1024) photorealistic video-to-video translation.; It can be used for turning semantic label maps into photo-realistic videos, synthesizing people talking from edge maps, or generating human motions from poses.

Learning

Siamese Neural Networks for One-Shot Image Recognition: https://jayhey.github.io/deep%20learning/2018/02/06/saimese_network/; Training a network in deep learning requires a very large amount of training data. One-shot learning overcomes this drawback by training the network so that it can classify even with only one image per label.

NVIDIA AI Developer Newsletter

[Recommended] AI Can Transform Anyone Into a Professional Dancer: https://news.developer.nvidia.com/ai-can-transform-anyone-into-a-professional-dancer/; https://arxiv.org/abs/1808.07371

Transforming Standard Video Into Slow Motion with AI: https://news.developer.nvidia.com/transforming-standard-video-into-slow-motion-with-ai/

NVIDIA SPLATNet Research Paper Wins a Major CVPR 2018 Award: https://news.developer.nvidia.com/nvidia-splatnet-research-paper-wins-a-major-cvpr-2018-award/

AI Learns to Play Dota 2 with Human Precision: https://news.developer.nvidia.com/ai-learns-to-play-dota-2-with-human-precision/

[Recommended] This AI Can Automatically Remove the Background from a Photo: https://news.developer.nvidia.com/this-ai-can-automatically-remove-the-background-from-a-photo/

NVDLA Deep Learning Inference Compiler is Now Open Source: https://devblogs.nvidia.com/nvdla/

Favorite site