Skip to content
Search

Deep learning

lastmod 2026-10-11created 2015-04-24243 edits

Deep learning is defined as a set of machine learning algorithms that attempt high-level abstractions (the task of summarizing the core content or functions within large amounts of data or complex data) through a combination of several nonlinear transformation techniques, and, broadly speaking, it can be described as a field of machine learning that teaches computers the way people think.

Categories

Audio/Music

Mathematical background

Time series (Korean: 시계열)

  • TimesFM - Google's 200M-parameter, 16k-context time series foundation model

Multimodal

  • ONE-PEACE - general representation model for unlimited multimodality
  • ImageBind - Meta's multimodal AI model

Computer Vision Libraries

Training

Format

  • Lance - modern Columnar data format optimized for ML workflows

Tools

  • What-If Tool - Visually probes the behavior of trained machine learning models with minimal coding.
  • Gradio - build a UI for machine learning models in Python in 3 minutes
  • UpTrain - open-source tool for observing ML models
  • Tsai - open-source deep learning library for time series and sequences
  • Netron - visualizer for neural network, deep learning and machine learning models.
  • Preswald - open-source framework for AI data apps

Deep neural network

A deep neural network (DNN) is an artificial neural network (ANN) made up of multiple hidden layers between the input layer and the output layer. Like ordinary artificial neural networks, deep neural networks can model complex non-linear relationships. For example, in a deep neural network architecture for an object identification model, each object can be represented as a hierarchical composition of basic image elements. Here, the additional layers can combine the features of the lower layers that were gradually gathered. This characteristic of deep neural networks makes it possible to model complex data with fewer units (nodes) than a similarly performing artificial neural network.

Comparison of deep model application areas

-

Deep belief network

Convolutional neural network

Deep hyper network

Supervised/Unsupervised

Supervised/Unsupervised

<span style="color:red;">Supervised

Supervised/Unsupervised

Discriminative/Generative model

Generative

<span style="color:red;">Discriminative

Generative

Prediction/Module understanding

Prediction++/<span style="color:red;">Module-

<span style="color:red;">Prediction+++/Module+

Prediction+/<span style="color:red;">Module+++

Reasoning capability

Reasoning++

Reasoning-

<span style="color:red;">Reasoning++++

Connectivity

Full/Compact

Partial/Convolved

Partial/Sparse

Depth

Depth+++

<span style="color:red;">Depth++++

Depth++

Batch/Online learning

Batch

Batch

<span style="color:red;">Online

See DeepStream SDK:

Application

Model

Tracker

Inference resolution

Precision

People detection

PeopleNet-ResNet34 (version 2.5)

No tracker

960x544

INT8

People detection

PeopleNet-ResNet34 (version 2.5)

NvDCF

960x544

INT8

License plate recognition

TrafficCamNet
LPDNet
LPRNet

NvDCF

960x544
640x480
96x48

INT8
INT8
FP16

3D body pose estimation

PeopleNet-ResNet34 BodyPose3D

NvDCF

960x544
192x256

INT8
FP16

Action recognition

ActionRecognitionNet (3DConv)

No tracker

224x224x3x32

FP16

Hardware (GPU) select

See here for notes on hardware selection.

Which GPU should you choose for deep learning?

  • How does a GPU work?
  • The most important GPU specs for deep learning processing speed
    • Tensor cores
    • Memory bandwidth
    • L2 cache / shared memory / L1 cache / registers
  • Estimating Ada / Hopper deep learning performance
  • Advantages and problems of the RTX40 / RTX30 series
  • Raw performance ranking of GPUs
  • GPU deep learning performance per dollar
  • GPU recommendations (view as a recommendation chart)
    • What if you skip the current generation and buy the next one? The future of GPUs
  • Q&A
    • Do I need PCIe 4.0 / PCIe 5.0?
    • Do I need 8x/16x PCIe lanes?
    • If an RTX 4090 / 3090 takes up 3 PCIe slots, how can I fit 4 of them?
    • How do I cool 4 RTX 4090 / 3090s?
    • Can I use several different GPU types?
    • What is NVLink, and is it useful?
    • What do you recommend if I don't have enough money even for the cheapest recommended GPU?
    • What is the carbon footprint of GPUs? How can I use GPUs without harming the environment?
    • What do I need to parallelize across two machines?
    • Do I need an Intel CPU for a multi-GPU setup?
    • Does case design matter for cooling?
    • Will AMD GPUs + ROCm catch up with NVIDIA GPUs + CUDA?
    • When is it better to use the cloud or a dedicated desktop/server?

7 Steps for becoming Deep Learning Expert

7 steps to becoming a deep learning expert

Level 1
The first step is to understand Machine learning, the best resource for which is Andrew Ngs (Ex-Google, Stanford, Baidu), an online course at coursera34. Going through the lectures are enough to understand the basics, but assignments take your understanding to another level.
Level 2
Next step is to develop intuition for Neural Networks. So go forth, write your first Neural Network and play with it.
Level 3
Understanding Neural networks are important, but simple Neural Networks not sufficient to solve the most interesting problems. A variation - Convolution Neural Networks work really well for visual tasks. Standord lecture notes and slides on the same are here: CS231n Convolutional Neural Networks for Visual Recognition (notes), and CS231n: Convolutional Neural Networks for Visual Recognition (lecture slides). Also here and here are two great videos on CNNs.
Level 4
Next step is to get following for running your first CNN on your own PC.
  • Buy GPU and install CUDA
  • Install Caffe and its GUI wrapper Digit
  • Install Boinc (This will not help you in Deep Learning, but would let other researchers use your GPU in its idle time, for Science)
Level 5
Digit provides few algorithms such as - LeNet for character recognition and GoogLeNet for image classification algorithms. You need to download dataset for Lenet and dataset for Googlenet to run these algorithms. You may modify the algorithms and try other fun visual image recognition tasks, like we did here.
Level 6
For various Natural Language Processing (NLP) tasks, RNNs (Recurrent Neural Networks) are really the best. The best place to learn about RNNs is the Stanford lecture videos here. You can download Tensorflow and use it for building RNNs.
Level 7
Now go ahead and choose a Deep Learning problem ranging from facial detection to speech recognition to a self-driving car, and solve it.

Advice for machine learning researchers

This article was written for students who have just started machine learning research and for novice researchers, to help them avoid the various mistakes that come up while doing research. What sets this article apart from many other machine learning tutorials is that its main audience is people doing machine learning research in academia. In other words, it is aimed at people who need to evaluate experimental results rigorously and compare them fairly in order to publish their research as papers.

  1. What to do before building a model
    1. Take a look at your data
    2. Don't look at all of your data
    3. Make sure you have enough data
    4. Talk to domain experts
    5. Do plenty of background research
    6. Consider the setting in which the model will be deployed
  2. How to build reliable models
    1. Don't let information from the test data leak
    2. Try a variety of models
    3. Don't use inappropriate models
    4. Optimize your hyperparameters
  3. How to evaluate models correctly
    1. Use appropriate test data
    2. Use validation data
    3. Evaluate your model multiple times
    4. Don't use accuracy with imbalanced data
  4. How to compare models fairly
    1. A bigger number does not mean a better model
    2. Don't trust community benchmarks
  5. How to report results
    1. Be transparent about your results
    2. Measure performance in several ways
    3. Don't generalize beyond the data

2015

If you had to name the most influential machine learning conferences in the world, NIPS (Neural Information Processing Systems) would be one of them, and lately a great many papers related to deep learning are being published at NIPS. This is a (rough) translation of a post in which one attendee laid out 10 deep learning trends they noticed at NIPS2015.

  1. Neural net architectures are getting more and more complex and sophisticated.
  2. Most of the really awesome work included LSTM.
  3. Attention models have appeared.
  4. The Neural Turing Machine is still interesting but not yet used much for real work.
  5. Vision and natural language processing are no longer separate fields.
  6. Symbolic differentiation has become even more important.
  7. Surprising results come with neural net model compression.
  8. The combination of deep learning and reinforcement learning continues.
  9. If you aren't doing batch normalization, you should be.
  10. Neural net research is progressing in very close connection with commercialization.

2018

On the Developer News Center, we cover everything from AI, robotics, autonomous vehicles, to high-performance computing. This year we highlighted over 300 stories that show how NVIDIA technology can help developers do their life’s work. Here are the most popular stories of the year.

  1. AI Algorithm Can Read Your Mind
  2. NVIDIA Invents AI Interactive Graphics
  3. Transforming Standard Video Into Slow Motion with AI
  4. AI Can Fix Your Grainy Photos (Noise reduction)
  5. New AI Imaging Technique Reconstructs Photos (Image restoration)

2019

2020

2021

2022

  • Deep Learning Is Hitting a Wall - deep learning has hit its limits
    • The limits described in the article: GPT-3 does not understand sentences and is just focused on continuing them naturally. Tesla's autonomous driving model cannot recognize a person holding a STOP sign. It classified a bus buried in snow as a snowplow. The cause is that deep learning does not think; it memorizes and then classifies.
    • So deep learning works remarkably well in areas that are not critical and already determined, where the next result can be predicted. But in areas where that is not possible, there are many problems. The author gives NetHack as an example. AI that plays and clears the game was tested in two forms, classic AI and deep learning AI, and the classic AI performed better.
    • Classic AI, symbolic AI, could understand meaning and proceed. So it responded quickly even when new patterns appeared. Deep learning, however, is useful for classifying the blanks within what already exists, but it did not handle auto-generated dungeons well, because it cannot know what they mean.
    • What the author proposes as an alternative is hybrid AI. (See: Seeing the future of AI through its past - MIT Technology Review) It overcomes the limits of deep learning AI with the symbolic analysis and definable rules of classic AI. This would make it possible to control the causes of internal behavior and to handle part relationships and whole relationships more accurately.
    • This kind of AI does not exist only in the author's argument; AlphaGo and AlphaFold2 are also hybrid AI. Other companies are heading in that direction too.
  • 12 graphs showing the current state of AI in 2022 (2022 AI Index Report)

2023

Review of trends and directions to date, received at work (2024-02-06)

ERTI

AI Algorithm Trends After Deep Learning-ETRI-2018.12.31.pdf
AI_Algorithm_Trends_After_Deep_Learning_-_ETRI-2018.12.31.pdf

NVIDIA DEVELOPER - Top 5 AI Stories of the Week

  • NVIDIA DEVELOPER CENTER - Search Results for: AI Stories of the Week
  • NVIDIA’s Top 5 AI Stories of the Week: 4/22
    • AI Model Can Recommend the Optimal Workout
    • AI Helps Improve the Shopping Experience
    • OpenAI Releases MuseNet: AI Algorithm That Can Generate Music
    • AI Research Could Help Improve Alexa’s Speech Recognition Model by 15%
    • AI Can Interpret and Translate American Sign Language Sentences
  • The Top 5 AI Stories of the Week: 1/28
    • Oak Ridge National Laboratory Installs Two NVIDIA DGX-2 Systems
    • New App Uses AI to Enable Users to Explore Sneakers In AR
    • Video Series: Mixed-Precision Training Techniques Using Tensor Cores for Deep Learning
    • AI Researchers Pave the Way For Translating Brain Waves Into Speech
    • NVIDIA GPUs Help Developers’ Score $1 Million Prize For Improving Zillow’s Zestimate

Deep Learning for Visual SLAM

Deep Learning for RANSAC

3D Deep learning

Image

Deep learning frameworks

Compare

The biggest difference is whether it uses a static (computational) graph or a dynamic (computational) graph.

TF builds a static grpah. At every iteration step, it runs (FP) the same static computational graph built beforehand, via sess.run.

PyTorch uses a dynamic graph. The main difference is that it creates a new computational graph in the FP performed at every iteration.

The final code looks similar, but each has its own pros and cons.

Static Graph

Graph optimization
With a static graph, once the graph is built, the same graph is reused repeatedly, so the framework has the advantage of being able to optimize the graph.
For example, it can achieve efficient computation by reordering code or processing layers at the same time.
Of course, it can be slow at first because of this optimization process, but efficiency is maximized as iterations repeat.
  • As a theoretical example, a graph like conv -> relu -> conv -> relu ... can reportedly be processed as conv + relu -> conv +relu -> through a graph optimization algorithm.
Serialize
Once the graph is built, the entire graph structure is loaded in memory. Because of this, the graph data structure can be serialized to disk. This makes it possible to save the entire structure as a file.
As a result, it can be run using only that file, without access to the original code.
The benefit of serialization is even greater when running code written in Python on top of c++ code. Since the network can be loaded without access to the original Python code, the network can easily be :imported in a c++ environment. (In contrast, a dynamic graph needs access to the original code every time.)

Dynamic Graph

Most of its advantages come from the code becoming cleaner and easier. This shows up most notably in conditional branching.

When branching, you can simply use the conditional statements Python supports, just like numpy code. This works because a new graph is built every time, so execution can take one of the two paths, and the back prop for it can be handled automatically. In contrast, in a static graph like TF, the conditional branch must be explicitly included in the graph-building code inside the tensorflow graph. Also, such code requires separate syntax (tf.cond), and the code becomes complicated.

Similar to conditionals, this advantage also applies to recursive loops. Size problems that arise dynamically during computation are resolved. TF again has separate syntax to handle this. Because of these problems, TF has recently also offered a feature called TF Fold to provide Dynamic Graph functionality. However, at the low level TF is by nature a static graph, and it is only made into a dynamic graph at the high level, so there are still limits. Therefore, TF is bound to be at a great disadvantage when doing functional programming that controls dynamic behavior.

A practical example where Dynamic Graph really matters is the Recurrent Network.

Music Generation

As deep learning is gaining in popularity, creative applications are gaining traction as well. Looking at music generation through deep learning, new algorithms and songs are popping up on a weekly basis. In this post we will go over six major players in the field, and point out some difficult challenges these systems still face. GitHub links are provided for those who are interested in the technical details (or if you’re looking to generate some music of your own).

  • Magenta
  • DeepJazz
  • BachBot
  • FlowMachines
  • WaveNet
  • GRUV
  • MuseNet
  • MusicLM - AI that generates music from text
  • Riffusion - Stable Diffusion fine-tuned to generate music

Reinforcement Learning

Cheat Sheets

Table of Machine learning

Table of Machine learning

Learning Course

Theory

Probability theory → Bayesian probability

Machine learning

Hypothesis → Loss function → Gradient descent → Overfitting

Neural network

Artificial neuron → Perceptron → Multilayer perceptron → (Example) → Feedforward neural network → Activation function (Sigmoid function & Euler's number) → Softmax function (Loss function) → Backpropagation (Gradient descent) → (Example)

Deep learning

Convolutional neural network (CNN) & Region-based Convolutional Network (RCNN, R-CNN)

ETC

Tutorials

Theory

Basic

Terms

Probability theory

Bayesian probability, Decision Theory, Probability Density Function (PDF), Radial basis function (RBF), Hyperparameter

Neuroscience d

Neuron

Statistics

Covariance, Statistical classification, Confusion matrix, Cross-validation, Root-mean-square deviation (MESD), Mean squared error

Graph theory

Matrix

General matrix-matrix multiplication (gemm), Toeplitz matrix, im2col

Other

Computational learning theory, Empirical risk minimization, Occam learning, PAC learning, Statistical learning, VC theory, Bayesian network, Markov random field, Hidden Markov Model (HMM), Conditional Random Field, Normal Bayes Classifier, Energy Based Model, Occam's razor, Ground truth

Algorithm types

Supervised Learning

Support Vector Machine (SVM), Hidden Markov model, Regression analysis, Neural network, Naive Bayes classifier, K-Nearest Neighbor (K-NN), Decision trees, Ensembles (Bagging, Boosting, Random forest), Relevance vector machine (RVM)

Unsupervised learning

Clustering, Independent component analysis

Semi-supervised learning

Generative models, Low-density separation, Graph-based methods, Heuristic approaches

Other learning

Reinforcement learning, deep learning

By topic

Structured prediction

Graphical models (Bayes net, CRF, HMM)

Parameter estimation algorithms

Dynamic programming, Expectation-maximization algorithm (EM algorithm)

Approximate inference techniques

Monte Carlo method, AdaBoost

Approaches

Decision tree learning, association rule learning, genetic programming, inductive logic programming, clustering, Bayesian networks, reinforcement learning, representation learning, similarity and metric learning

Modeling

Neural network, SVM K-NN, decision tree, Genetic Algorithm, genetic programming, Gaussian process regression, linear discriminant analysis, perceptron, radial basis function network

Recommender system (Korean: 추천 시스템)

Collaborative filtering (Korean: 협업 필터링), Content-based filtering (Korean: 컨텐츠 기반 필터링), Hybrid recommender systems

Data mining

Cross-validation (k-fold), Data set (Training set, Validation set, Test set)

Regression analysis

Linear regression, Logistic Regression, Logit function, Multinomial logistic regression (Softmax Regression)

Clustering

k-means clustering, BIRCH, Hierarchical, Expectation-maximization (EM), DBSCAN, OPTICS, Mean-shift

By type

3D Machine Learning

Artificial Neural Networks; ANN

Artificial neuron

Perceptron, Sigmoid neuron

Combination function & Activation function

Sigmoid function, Rectified linear units (ReLU), 1x1 Convolution

Loss function or Cost function

Softmax, Sum of squares (Euclidean), Hinge / Margin, Cross entropy, Infogain, Accuracy and Top-k

Algorithms

Multilayer perceptron (MLP), Feed-forward Neural Network (FNN), Long short-term memory (LSTM), Autoencoder, Self-organizing map (SOM), Network In Network (NIN), Adaptive neuro fuzzy inference system (ANFIS)

Deep learning

Deep neural network (DNN), Convolutional neural network (CNN) & Regions with Convolutional Neural Network (RCNN, R-CNN, SSD), Recurrent neural network (RNN), Gated Recurrent Unit (GRU), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Deep Q-Networks, Deep hyper network (DHN), Deconvolutional Network, Fully Convolutional Networks for Semantic Segmentation (FCN), Generative Adversarial Networks (GAN), Learning Deconvolution Network for Semantic Segmentation, Multi-Scale Context Aggregation by Dilated Convolutions (Dilated Convolutions), Dynamic Routing Between Capsules (Capsules Network), YOLO, Path Aggregation Network for Instance Segmentation (PANet), Image-to-Image Translation with Conditional Adversarial Networks (pix2pix), CycleGAN, BicycleGAN, SlowFast, Kinetics, AVA, MelGAN, EfficientDet, SinGAN, Panoptic Segmentation, U-Net, MONAI, CenterNet, DeepLab, HRNet, OCRNet, ResNeSt

Models

GoogLeNet, LeNet, AlexNet, ReNet, RCNN, SPPnet, Fast R-CNN, Faster R-CNN, VGG, SqueezeNet, PointNet, Mask R-CNN, MaskLab, OSMN (Video Segmentation), ColorNet, EfficientNet

Problems

Dimensionality reduction

Curse of dimensionality, Factor analysis, CCA, independent component analysis (ICA), LDA, non-negative matrix factorization (NMF), principal component analysis (PCA), t-SNE

Underfitting & Overfitting

Early stopping, Model selection, Normalization, Regularization (L1, L2, Dropout), Generalization

Initialization

Xavier Initialization

ETC

Bias-variance dilemma, Vanishing gradient problem, Local minimum problem, Batch Normalization, Standardization, AutoML

Optimization

Hyperparameter, Simulated annealing, Early stopping, Feature scaling, Normal Equation, Gradient descent, Stochastic gradient descent (SGD), Backpropagation, Convex optimization, Performance Tuning, im2col, Dense-Sparse-Dense Training for Deep Neural Networks (DSD), Deep Compression, Pruning Neural Networks, Shake-Shake regularization, Accurate Large Minibatch SGD

ETC

Libraries

OpenCV, Deeplearning4j (DL4J), Torch, PyTorch, Theano, Caffe, TensorFlow, MMDetection, ConvNetJS, cuDNN, Netscope, Microsoft Azure Machine Learning Studio, OpenPose, DensePose, Keras, tiny-dnn, Detectron, Stanford CoreNLP, Aifiddle, Kubeflow, OpenNMT, alibi-detect, Flashlight (facebook), MediaPipe, Weights and Biases

Dataset

MNIST (handwriting data), ImageNet, CIFAR-10, TinyImages, PASCAL VOC, COCO, AVSS, YouTube-VOS

Annotation tools

labelme, cvat, f-BRS, COCO Annotator

See also

Artificial intelligence, autonomous robots, bioinformatics, computational intelligence, computer vision, data mining, pattern recognition, big data

Unknown keyward

Classification, Anomaly detection, Association rules, Reinforcement learning, Structured prediction, Feature engineering, Feature learning, Online learning, Semi-supervised learning, Unsupervised learning, Learning to rank, Grammar induction, Local outlier factor, Minimum Description Length (MDL), Bayesian MAP, Structural Risk Minimization (SRM), Long Narrow Valley, Word2vec, Autoregressive integrated moving average (ARIMA)

Documentation

Bayesian probability

Bayesian Reasoning and Machine Learning
http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.Online
http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/020217.pdf
020217_-_Bayesian_Reasoning_and_Machine_Learning.pdf

Deep learning

ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
ImageNet_Classification_with_Deep_Convolutional_Neural_Networks.pdf
[Korean translation] ImageNet classification using deep convolutional neural networks
ImageNet_Classification_with_Deep_Convolutional_Neural_Networks_-_ko.pdf
Going deeper with convolutions (GoogleNet)
http://arxiv.org/pdf/1409.4842v1.pdf
Going_deeper_with_convolutions.pdf
Translation: GoogleNet
Feature article: Deep Hypernetwork models (Seoul National University / Byoung-Tak Zhang)
Deep_Hypernetwork_models_201508.pdf
[Mocrosoft] Deep Residual Learning for Image Recognition (Winner ILSVRC2015)
Deep_Residual_Learning_for_Image_Recognition(Winner_ILSVRC2015)_Microsoft.pdf
[Microsoft] Fast R-CNN, Towards Real-Time Object Detection with Region Proposal Networks (Winner ILSVR2015)
Fast_R-CNN,Towards_Real-Time_Object_Detection_with_Region_Proposal_Networks(Winner_ILSVR2015)_Microsoft.pdf
Learning Deconvolution Network for Semantic Segmentation
http://cvlab.postech.ac.kr/research/deconvnet/
Deep EXpectation of apparent age from a single image
https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks.
Inside_outside_net_detecting_objects_in_context_with_skip_pooling_and_recurrent_neural_networks_2015.pdf
A paper on Small Object detection methods.
Dae-hyup Cho's blog - Understanding deep learning and TensorFlow for those who gave up on math
http://bcho.tistory.com/1208
Machine_learning_ebooks_-_Machine_learning_for_those_who_abandon_math.pdf
Densely Connected Convolutional Networks (DenseNets)
Deep learning, CVPR2017 Best Paper Award
Deep Learning Interviews book: Hundreds of fully solved job interview questions from a wide range of key topics in AI.
Deep learning interview book
A collection of practical questions and solutions for master's/PhD students learning machine learning and for job seekers
A printed copy can be purchased, but the full PDF is freely available
[Recommended] "Essential Glossary for the AI Era, Explored Through 2025 AI Trends and Issues" published by NIA (National Information Society Agency)
Essential glossary for the AI era (168p PDF) | GeekNews
https://www.nia.or.kr/site/nia_kor/ex/bbs/View.do?cbIdx=82618&bcIdx=28928&parentSeq=28928
Nia.or.kr_-ai-_2025_terms.pdf
There Will Be a Scientific Theory of Deep Learning
A scientific theory of deep learning will emerge. | GeekNews
There Will Be a Scientific Theory of Deep Learning
A paper arguing that a scientific theory is taking shape that characterizes key properties of deep learning such as the training process, hidden representations, final weights, and performance
Presents five research strands as its core evidence (interpretable idealized settings, tractable limits, simple mathematical laws, hyperparameter theory, universal behavior)

Tutorials

Getting Started with Deep Learning the Right Way (author Takayuki Okatani / translator Sim Hyo-seop)
Deep_Learning_-Takayuki_Okatani-2015-_sample.pdf
Introductory guide to machine learning - IDG Deep Dive
http://www.itworld.co.kr/techlibrary/97428
IDG_DeepDive_Machine_learning-20160113.pdf
Understanding Deep Learning (unpublished; 2016-08-22 ver)
Understanding_deep_learning_0822.pdf
Fundamental of Reinforcement Learning
https://www.gitbook.com/book/dnddnjs/rl/details
Fundamental_of_Reinforcement_Learning.pdf
Modulabs - Fundamentals of reinforcement learning
Deep Learning Papers Reading Roadmap (deep learning paper roadmap)
https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap/blob/master/README.md
Deep_Learning_Papers_Reading_Roadmap.md.zip
[Recommended] Machine Learning-based Web Exception Detection (reference site for the Financial Security Institute project!)
https://cloudfocus.aliyun.com/Machine-Learning-based-Web-Exception-Detection-89782?spm=a2c1b.a2c1b4.a2c1b4.16.ZSQoEd
Machine_Learning-based_Web_Exception_Detection_-Insights_and_Trends-_Alibaba_Cloud_Focus.pdf
Machine learning basics, parts 1–57 (Exploring Odds and Ends - Chamstudy ePaiai : Naver Blog)
http://sams.epaiai.com/220498694383
Microsoft releases ML for Beginners course
A 12-week, 24-lesson curriculum made by the MS Azure Clouds Advocates team
Classic machine learning lessons using Scikit-learn (deep learning to come in a separate AI course)
https://github.com/microsoft/ML-For-Beginners
CS146S: The Modern Software Developer (Korean) | GeekNews
[Original] CS146S Korean Edition | The Modern Software Developer
Official Korean version of Stanford's CS146S course

Compress model

Deep Compression and EIE - Deep Neural Network Model Compression and Efficient Inference Engine
Deep_compression_and_EIE_PPT.pdf
Learning bothWeights and Connections for Efficient Neural Networks
Learning_both_weights_and_connections_for_efficient_neural_networks_2015.pdf

Convolutional neural network

CNN presentation using Reveal.js.
Author - me
Reveal-ml.tar.gz
Gradient-Based Learning Applied to Document Recognition
http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf
Gradient-Based_Learning_Applied_to_Document_Recognition.pdf
Visualizing and Understanding Convolutional Neural Networks
http://arxiv.org/abs/1311.2901
1311.2901v3.pdf
Compressing CNN for Mobile Device (Samsung) - the need for CNN model compression etc ...
[http://mlcenter.postech.ac.kr/files/attach/workshop_fall_2015/삼성전자_김용덕_박사.pdf](http://mlcenter.postech.ac.kr/files/attach/workshop_fall_2015/삼성전자_김용덕_박사.pdf)
Samsung_-_Compressing_CNN_for_Mobile_Device.pdf
Using Filter Banks in Convolutional Neural Networks for Texture Classification
https://arxiv.org/abs/1601.02919

Deep belief network

The Applications of Deep Learning on Traffic Identification
Us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf
https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf

Deconvolution neural network

Learning Deconvolution Network for Semantic Segmentation
http://cvlab.postech.ac.kr/research/deconvnet/
https://arxiv.org/abs/1505.04366
1505.04366.pdf

Segmentation

Learning to Segment (Facebook Research)
https://research.fb.com/learning-to-segment/
DeepMask+SharpMask as well as MultiPathNet.
Recurrent Instance Segmentation
https://arxiv.org/abs/1511.08250
1511.08250.pdf
Slideshare - Single Shot MultiBox Detector and Recurrent Instance Segmentation
vid2vid
https://github.com/NVIDIA/vid2vid
Pytorch implementation for high-resolution (e.g., 2048x1024) photorealistic video-to-video translation.
It can be used for turning semantic label maps into photo-realistic videos, synthesizing people talking from edge maps, or generating human motions from poses.

Fire Detection

Covered in Fire Detection#Deep learning based.

Background subtraction

Covered in Background subtraction#Deep learning based.

LSTM

Covered in Long short-term memory.

Learning

Siamese Neural Networks for One-Shot Image Recognition
https://jayhey.github.io/deep%20learning/2018/02/06/saimese_network/
Training a network in deep learning requires a very large amount of training data. One-shot learning overcomes this drawback by training the network so that it can classify even with just one image per label.

NVIDIA AI Developer Newsletter

[Recommended] AI Can Transform Anyone Into a Professional Dancer
https://news.developer.nvidia.com/ai-can-transform-anyone-into-a-professional-dancer/
https://arxiv.org/abs/1808.07371
Transforming Standard Video Into Slow Motion with AI
https://news.developer.nvidia.com/transforming-standard-video-into-slow-motion-with-ai/
NVIDIA SPLATNet Research Paper Wins a Major CVPR 2018 Award
https://news.developer.nvidia.com/nvidia-splatnet-research-paper-wins-a-major-cvpr-2018-award/
AI Learns to Play Dota 2 with Human Precision
https://news.developer.nvidia.com/ai-learns-to-play-dota-2-with-human-precision/
[Recommended] This AI Can Automatically Remove the Background from a Photo
https://news.developer.nvidia.com/this-ai-can-automatically-remove-the-background-from-a-photo/
NVDLA Deep Learning Inference Compiler is Now Open Source
https://devblogs.nvidia.com/nvdla/

Nature

Deep learning of aftershock patterns following large earthquakes
https://www.reddit.com/r/MachineLearning/comments/9bo9i9/r_deep_learning_of_aftershock_patterns_following/
https://www.nature.com/articles/s41586-018-0438-y
https://drive.google.com/file/d/1DSqLgFZLuNJXNi2dyyP_ToIGHj94raWX/view

Cancer

Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening
https://medium.com/@jasonphang/deep-neural-networks-improve-radiologists-performance-in-breast-cancer-screening-565eb2bd3c9f

Text Generation

See Text Generation.

VizSeq - A Visual Analysis Toolkit for Text Generation Tasks
https://arxiv.org/abs/1909.05424

Favorite site

Machine learning

E-book

Data analysis

Artificial neural network

Convolutional neural network

Segmentation

Deep learning

Deep learning libraries

Andrew Ng

Machine Learning Exercises In Python

Guides

Tutorials

Data pool

Service

Sample images useful for proposal documents

Online demo

References

  1. Artificial_Intelligence_6-Deep_Learning.pdf ↩

  2. The_Best_GPUs_for_Deep_Learning_in_2023_-_An_In-depth_Analysis.pdf ↩

  3. [Recommended] Stanford Machine Learning by Andrew Ng ↩

  4. Stanford-machine_learning-andrew_ng-lectures.tar.gz ↩

  5. Machine_Learning_-_OpenCV_ML_algorithms.pdf ↩

  6. S1_nn.pdf ↩

  7. Sigmoid_Function_of_Artificial_Neural_Network.pdf ↩

  8. Aistudy.co.kr_-_theory_oh.pdf ↩

  9. VGG_Convolutional_Neural_Networks_Practical.pdf ↩

  10. ConvNets_24_April.pdf ↩

  11. Forward_and_Backward_Propagation_of_the_Convolutional_Layer.pdf ↩

  12. The_easiest_way_to_understand_CNN_backpropagation.pdf ↩

  13. A_Look_at_Image_Segmentation_using_CNNs_–_Mohit_Jain.pdf ↩

  14. Artificial_Intelligence_6-Deep_Learning.pdf ↩

  15. The_Devil_is_always_in_the_Details_-_Must_Know_Tips_and_Tricks_in_DNNs.pdf ↩

  16. Openresearch.ai-190921.zip ↩

  17. Stanford-machine_learning-andrew_ng-lectures.tar.gz ↩

  18. OLD URL: https://class.coursera.org/ml-005/lecture ↩

  19. Sanghyuk's Github pages blog using octropress ↩

  20. SanghyukChun.github.io-006e2ae.tar.gz ↩

  21. 1ambda.github.io-19962c5.tar.gz ↩

  22. Machine_Learning_Exercises_In_Python_-_Part_1.pdf ↩

  23. Machine_Learning_Exercises_In_Python_-_Part_2.pdf ↩

  24. Machine_Learning_Exercises_In_Python_-_Part_3.pdf ↩

  25. Machine_Learning_Exercises_In_Python_-_Part_4.pdf ↩

  26. Machine_Learning_Exercises_In_Python_-_Part_5.pdf ↩

  27. Machine_Learning_Exercises_In_Python_-_Part_6.pdf ↩

  28. Machine_Learning_Exercises_In_Python_-_Part_7.pdf ↩

  29. Machine_Learning_Exercises_In_Python_-_Part_8.pdf ↩

  30. Samsung_software_membership_-_Neural_network_01.pdf ↩

  31. Samsung_software_membership_-_Neural_network_02.pdf ↩

  32. GPU_-how_much_faster_than_CPU-_NeoBrain.pdf ↩

  33. 1ambda.github.io-master-5c58cab.zip ↩

  34. Gitbook_-leonardoaraujosantos-_artificial-inteligence.pdf ↩

  35. 170915_Tensorflow_Seminar_From_Deep_Learning_Theory_to_Practice_-_POSTECH.pdf ↩

  36. Fueling_the_Gold_Rush_-The_Greatest_Public_Datasets_for_AI-Startup_Grind-_Medium.pdf ↩

author Zer0 sitemap