Skip to content
Search

Kubeflow

lastmod 2026-07-10created 2019-02-1235 edits

Machine Learning Toolkit for Kubernetes.

Categories

  • Kubeflow:Install
  • Kubeflow:Architecture
  • Kubeflow Spark Operator - Can be used for the data preparation and feature engineering stages.
  • Kubeflow Notebooks - Can be used for model development and interactive data science to experiment with AI workflows.
  • Kubeflow Trainer - Can be used for large-scale distributed training or LLM (Learning Leadership Model) fine-tuning.
  • Kubeflow Katib - Can be used for model optimization and hyperparameter tuning with various AutoML algorithms.
  • Kubeflow Model Registry - Can be used to store machine learning metadata and model artifacts, and to prepare models for deployment to production.
  • KServe - Can be used for both online inference and batch inference in the model serving stage.
  • Feast - Can be used as a feature store and can manage offline and online features.
  • Kubeflow Pipelines - Can be used to build, deploy, and manage each stage of the AI lifecycle.
  • ml-metadata

Training Operators

Training Operators support distributed training for various deep learning frameworks such as TensorFlow, PyTorch, and MXNet. Distributed training of machine learning models on Kubernetes can reduce training time. When a user writes a distributed training spec and deploys it to Kubernetes, the Kubeflow Training Operator runs the workload according to the spec. The spec defines the path of the Docker image containing the machine learning model code, the distributed training cluster information, and so on.

Experiments (AutoML)

AutoML is a tool that automates the repeated experiments needed to improve the prediction accuracy and performance of machine learning models. Kubeflow provides AutoML features through Katib. Katib provides Hyper Parameter Tuning and Neural Architecture Search (NAS). Hyperparameter tuning optimizes a model's hyperparameters, and NAS optimizes the neural network architecture, such as the model structure and node weights.

With Katib, you can automate an experiment to find which value of the learning rate hyperparameter, 0.1 or 0.05, gives higher prediction accuracy. You first write an Experiment (AutoML) spec and deploy it to Kubernetes, and Katib then runs the experiments concurrently according to the hyperparameters and parallelism settings defined in the spec to find the best-performing hyperparameters.

KServe

Kubeflow deploys machine learning models to Kubernetes and provides inference through KServe. KServe consists of an Endpoint, Transformer, Predictor, and Explainer. When the Endpoint passes data to the Predictor, the Predictor makes predictions on or classifies the data. The Endpoint can adjust the weight ratio of the data it passes to Predictors, which also makes A/B testing possible.

A Transformer or Explainer can be added as needed and is used connected to a Predictor. The Explainer acts as eXplainable Artificial Intelligence (XAI), giving the reasons behind the prediction or classification results, and the Transformer provides data preprocessing and postprocessing.

Kubeflow Pipelines (KFP)

Kubeflow Pipelines (KFP) is an ML Workflow Orchestration tool for building and deploying machine learning workflows. The goal of KFP is to run diverse experiments quickly and easily by reusing Pipelines and Pipeline Components. Pipelines connect Pipeline Components to compose a Workflow in the form of a DAG (Directed Acyclic Graph), and use Argo Workflow as the Workflow Engine.

API Deployment

Representative open-source inference engines that have been developed include:

Serving Framework

What would it take to do API Deployment with these inference engines in a Kubernetes environment? You need to deploy many Kubernetes resources: a Deployment to deploy the inference engine, a Service to create the Endpoint that receives inference requests, an Ingress to route external inference requests to the inference engine, and so on. Beyond that, operating inference engines comes with many more requirements, such as scale-out when many inference requests come in, monitoring the state of the inference engine, and version updates when an improved model is released.

To handle these many requirements, Serving Frameworks have been developed that add another layer of abstraction over inference engines on Kubernetes.

Open-source Serving Frameworks that have been developed include:

Cons

Unfortunately, Kubeflow turned out to be finicky to set up, unreliable, and hard to configure. It also depended on many outdated components and libraries.

Much of the documentation was broken or outdated, so we could not integrate smoothly with AWS and GitHub without relying on some hacky workarounds.

Initial installation problems

Even before adopting Kubeflow, we already knew there would be a steep learning curve. But because our team had plenty of Kubernetes experience, we thought we could get the initial installation up and running fairly quickly.

Days after our first attempt, we were still struggling with bugs related to the KFDef and Kustomize manifests. The provided manifests failed several times without clear error messages, so we had to check every installation component manually and figure out which component was broken.

Our goal was to integrate Kubeflow with GitHub authentication, but the manifests provided for AWS and OpenID Connect (OIDC) also contained bugs related to the Kubeflow ingress point. To fix this, we had to manually update the ingress to use the required OIDC information.

Overall, although Kubeflow runs on top of Kubernetes and is cloud-agnostic, we ran into many problems running Kubeflow on AWS. Had we used GCP instead, this process would probably have been smoother. Because Kubeflow was built by Google, it often defaults to GCP and does not yet work smoothly with other cloud providers, especially regarding authentication and permission management.

Component integration problems

Kubeflow is made up of various loosely coupled components. In theory this loose coupling is good, because it lets you choose which components to use.

But there are downsides too. Different components depend on different versions of the same dependency, which causes more problems.

While running tests, we found that upgrading one component often broke another. For example, upgrading the KFServing component requires upgrading Istio, the service mesh platform that Kubernetes services use to share data with each other. Because the latest Istio version was not compatible with AWS authentication, this upgrade broke access to the dashboard.

As a result we ended up with an incompatible set of versions, and the only way to recover was to reinstall Kubeflow from scratch.

We also needed to create pipelines directly from notebooks, but even after some hacky workarounds this turned out to be impossible. Kubeflow still had unresolved issues. As one AWS engineer said on GitHub, "In-cluster communication from notebook to Kubeflow Pipeline is not supported in this phase."

Documentation problems

Many documentation pages are labeled 'Out of date', including pages for important Kubeflow components such as Jupyter notebooks.

Worse, some documentation pages written for older versions of Kubeflow do not carry the 'Out of date' label. So while debugging, it was hard to know when to trust the documentation. We kept trying to figure out whether the problem was outdated documentation or something we had done wrong. This slowed everything down.

Many of the links in the documentation returned "Page not found" or "This page does not exist yet." errors, making for a frustrating experience overall.

Kubeflow User Survey 2023

Key takeaways:

  • 84% of users deploy more than one Kubeflow component.
  • The top 3 Kubeflow components are Pipelines (90%), Notebooks (76%), and Katib (47%).
  • The top contributed component used by users is KServe (62%).
  • Documentation (55%) was the biggest gap in Kubeflow, followed by tutorials (50%), with installation (39%) and upgrades (39%) tied for third.
  • In users' ML lifecycle, model monitoring (45%) is the biggest gap, followed by model registry (44%) and initial setup (39%).
  • 52% of users install Kubeflow using raw manifests.
  • The top distribution used to install Kubeflow is AWS (28%), followed by Google Cloud (17%).
  • 74% of users deploy Kubeflow in the cloud and 45% deploy it on-premises.
  • 49% of users run Kubeflow in production.
  • 17% of users contribute back to Kubeflow.
  • 49% of users stay on the latest Kubeflow 1.7 release, and 43% run Kubeflow 1.6 or later.

Kubeflow Alternatives

Data and pipeline versioning

Experiment tracking and meta ML

Training run orchestration

Hyperparameter tuning

Model serving

See MLops for details.

See also

Favorite site

author Zer0 sitemap