Kubeflow Pipelines

lastmod 2025-12-30created 2025-12-302 edits

A tool for defining and automating complex machine learning workflows.

By building a pipeline, you can run a series of tasks such as data preprocessing, model training, validation, and deployment in order.

Pipelines are defined in code, and various components can be reused, making them easy to manage and maintain.

Key concepts

• Component: A reusable unit of work that supports various languages and libraries. Multiple components can be combined to build an experiment.
• Experiment: Connects components to build an entire workflow. You can test various combinations of parameters and data.
• Run: Executes an Experiment and tracks and manages the results of each step. You can rerun failed tasks or reuse the results of previous runs.

Pipeline management

• Pipeline components are visualized as a DAG (Directed Acyclic Graph).
• All pipelines are managed as code, through the SDK or by manually uploading a compressed file.

Pipeline images

Kubeflow pipeline images include various machine learning frameworks such as TensorFlow and Pytorch. You can use custom Docker images as needed.

Pipelines API Reference

• Pipelines API Reference (v2beta1) | Kubeflow

See also

• kubeflow
• DAG

Favorite site

• Github - kubeflow/pipelines
• Kubeflow Pipelines | Kubeflow
• Training a prediction model with Kubeflow Pipelines | KakaoCloud
• Introduction to and experience with Kubeflow Pipeline | Developer Story | SKT Enterprise
• A friend of R and Python analysis and programming (by R Friend) :: (Kubeflow) Overview of pipeline concepts and components (Conceptual overview of Kubeflow Pipelines)