"Software Defined" to Everything

As part of a 2026 book study on machine learning platform engineering (Korean), I have extended to summarize the operational requirements for building an MLOps platform on AWS. MLOps is a set of practices that connects data preparation, training, evaluation, deployment, monitoring, and retraining, making the entire process repeatable. This post explores the four principles and Levels 0, 1, and 2 introduced by AWS, then maps those requirements to the roles and relationships of Kubernetes-based components in an Amazon EKS environment.

MLOps combines ML, short for machine learning, with Ops, short for operations. From an operational perspective, it covers a broad range of activities, including repeated model training, inference, and the associated feature engineering. The book introduces MLOps maturity, and this post brings that discussion together with AWS’s maturity levels while considering how the book’s Kubernetes-based examples translate to Amazon EKS.

1. What Are We Operating?

From an MLOps platform perspective, two questions are central: how to support the machine learning lifecycle—from problem definition and data preparation to training, evaluation, deployment, and monitoring—and how to operate that lifecycle reliably and repeatedly.

Even after training a model that achieves the desired evaluation score, there is still much to check before using it in a service. Is the preprocessing used during training applied consistently at inference time? Which model version is deployed? Can the service handle real requests with the required performance? We also need a process for reevaluating and updating the model when its input data or operating environment changes.

The scope of an MLOps platform is therefore "the entire process through which models are created, used, and improved".

This process begins with defining the problem. The design will differ depending on whether simple rules are sufficient, whether the necessary training data is available, and which types of errors matter most. Model evaluation criteria must also be tied to the problem being solved.

For example, in fraud detection, the cost of incorrectly blocking a legitimate transaction differs from the cost of missing a fraudulent one. In a recommendation system, an improvement in offline evaluation scores does not necessarily mean that real users respond more positively. The platform should retain the data and execution results needed to make these judgments.

After deployment, we need to observe both service health and prediction quality. An API can return HTTP 200 while producing incorrect predictions. Alongside request latency and error rates, there should be a process for evaluating prediction quality once ground-truth labels become available.

The relationships a platform needs to support can be summarized through the following questions:

• Which data and code were used to build the model?
• Which configuration and execution environment were used?
• What was validated before deployment?
• Which model version is currently serving predictions?
• Which version can we return to if something goes wrong?

Being able to answer these questions from execution records makes reproducibility, incident investigation, and recovery possible.

2. Four AWS MLOps Principles for Repeatable Operations

AWS identifies four core principles: Version control, Automation, Continuous X, and Model governance. These principles apply throughout the lifecycle. As maturity increases, both the scope of what is managed and the extent of automation expand.

2.1. Version Control: Record the Conditions That Produced the Model

Knowing the Git commit of the code is not enough to expect the same result if the training data or configuration differs. Each model release should be linked to its dataset identifier, preprocessing method, training configuration, execution environment, model artifacts, and evaluation results. Connecting these records to the inference container image and deployment configuration makes it possible to trace the origins of the model currently serving predictions.

For example, S3 Versioning allows multiple versions of individual objects to be retained. If a training dataset consists of multiple files, a manifest containing the bucket, object key, and version ID of each file can identify the dataset used for a particular run.

Dataset versions and train/validation/test splits should be distinguished. A dataset version represents changes to the collected or processed data. A split determines how data within that version is assigned to different purposes. Recording the split rules and when the split was created provides a basis for comparing evaluation results.

The connections among data, code, executions, and models form their lineage. With lineage, we can rebuild a model, investigate which models were affected by incorrect data, and recover to a previous release. AWS’s ML operations guidance also emphasizes versioning code, configuration, models, and infrastructure alongside maintaining reproducible environments.

2.2. Automation: Define Inputs and Acceptance Criteria

Automation can begin by arranging commands into a sequence. For operational use, however, each stage also needs clearly defined inputs, outputs, and success or failure conditions. Training should stop if data validation fails, and a model that does not meet evaluation criteria should not be promoted to a deployment candidate. It should be possible to identify the failed stage and its cause, adjust the necessary conditions, and run the process again.

When production approval requires human review, reviewers should receive the evaluation results and details of the changes, and the approval decision should be recorded. Keeping automated work and human judgment within the same workflow reduces repetitive tasks while preserving accountable decision-making.

Managing execution environments and infrastructure configuration as code also helps reduce differences among development, validation, and production environments and makes them easier to recreate. Consistent pipelines and environments are the foundation for trusting automated results.

2.3. Continuous X: Connect Operational Results to the Next Change

AWS describes ongoing activities through CI, CD, CT, and CM: Continuous Integration, Continuous Delivery, Continuous Training, and Continuous Monitoring.

These activities form a feedback loop. If input data changes or prediction quality declines in production, the next step is to investigate the cause and consider retraining. A new candidate model proceeds to the deployment process only after passing the evaluation criteria.

Retraining and promotion to production require different acceptance criteria. Training on new data alone does not provide sufficient justification for replacing the existing model.

2.4. Model Governance: Record Responsibilities and Approval Rationale

Model governance manages model ownership, review and approval criteria, access permissions, and audit records.

It should be clear who defines the model’s business objectives, which evaluation results justify deployment approval, and how access to sensitive data and models is controlled. The people responsible for making decisions and responding to operational issues should also be identified.

Operational accountability becomes easier to explain when approval and deployment histories are linked to the following information:

• Model owners and operational maintainers
• Evaluation criteria and validation results
• Approvers and approval times
• Deployed models and their execution environments
• Issues discovered in production and the actions taken

Governance applies throughout the lifecycle, from data usage and model review to deployment, feedback, and retirement.

3. ML Workflow Automation: Levels 0, 1, and 2

The Levels 0, 1, and 2 introduced by AWS describe the degree of automation in an ML workflow. To understand them, it is more useful to examine what is being operated and how changes are handled than to look only at which tools have been installed.

3.1. Level 0: Manually Handing Over a Trained Model

At Level 0, people perform data preparation, training, validation, and the transitions between these stages. The trained model is then handed over to an engineering team.

This can also happen in an EKS environment. For example, a data scientist might train a model in a notebook and hand over the model file. An engineer then integrates it into an API, packages it as a container image, and deploys it.

Deployment to Kubernetes or EKS may already be automated, and the API delivery process may have sophisticated testing and deployment procedures. To assess the automation level of the ML lifecycle, however, we must also examine the preceding data preparation, training, and evaluation stages.

For example, when new data arrives, someone may still need to determine which code to run and in what order, check whether the model matches its preprocessing code, and associate evaluation results with the correct model. These manual connections are potential areas for improvement.

At Level 0, a useful question for reviewing current operations is:

To recreate the model currently in production under the same conditions, whom would we need to ask, and what information would we need?

If the necessary information resides on a particular person’s laptop or in their memory, a practical starting point is to move it into code, configuration, and execution records.

3.2. Level 1: Repeatedly Executing a Training Pipeline

At Level 1, the ML pipeline is automated so that training can be repeated with new data. The operational focus expands from delivering a trained model to operating a repeatable training pipeline.

A defined process runs in response to triggers such as data becoming ready or a scheduled time. The workflow uses consistent pipelines across development, validation, and production environments and delivers the resulting models to prediction services.

The key question at this stage is whether we can explain a single pipeline run. We need to connect the input dataset, validation steps, training configuration, resulting model, and evaluation results.

For an EKS-based setup, Kubeflow Pipelines is one option for defining and executing container-based ML workflows. Dividing a pipeline into components makes it possible to manage each stage’s inputs, outputs, execution order, and status.

Experiment tracking and model management have distinct roles:

• MLflow Tracking can be configured to record each run’s data, parameters, metrics, and artifacts.
• MLflow Model Registry manages registered models, versions, and aliases and links them to training runs.

Together, these records make it possible to trace a selected model back to the experiment that produced it.

Models should be evaluated against predefined criteria using data that was not used for training. When appropriate, candidate models should also be compared with the model currently in production. Managing candidate generation separately from promotion to production preserves the rationale for deployment decisions while allowing retraining to run repeatedly.

3.3. Level 2: Managing the Training Pipeline Itself Through CI/CD

At Level 2, pipeline source development, component builds, testing, deployment, prediction service operations, and monitoring are connected. AWS also describes an orchestrator for coordinating multiple ML pipelines and a model registry for tracking multiple models as components at this level.

The distinction from Level 1 becomes clear when the method of building a model changes. Providing a new dataset to an existing pipeline repeats an already defined procedure. Changing feature calculation logic, replacing a training algorithm, or adding an evaluation stage changes the pipeline itself. Those changes also need testing and release procedures.

The Level 2 workflow can be divided into three parts:

1. Develop and modify pipeline source code based on experimentation.
2. Build and test components and pipelines, then deploy them to the execution environment.
3. Validate and deploy the models produced by pipeline runs, and collect operational metrics.

In an EKS-based platform, training orchestration and serving deployment should be treated as distinct responsibilities. For example, CI can validate Kubeflow Pipelines components and pipeline definitions before registering them. For serving resources, Argo CD can synchronize the deployment configuration stored in Git with Kubernetes.

These two workflows are connected through training run identifiers, model versions, container image versions, and deployment records. This allows us to start with a prediction service running in production and trace back to how its model was created.

The three levels can be summarized by how their operational focus changes:

An orchestrator or model registry may be introduced at Level 1. Rather than determining maturity solely by the presence of tools, we should examine how changes to data and pipelines are validated and brought into operation.

4. Considering Organizational and Platform Growth

As multiple teams begin using the same platform, shared challenges extend beyond automation. We need to decide how to separate data and execution permissions among teams, how to provide environments to new teams, and how to preserve operational accountability for each model.

A useful complementary perspective is described in AWS blog post in 2022: " MLOps foundation roadmap for enterprises with Amazon SageMaker". It describes four stages of foundation building across people, operations, and technology: Initial, Repeatable, Reliable, and Scalable.

The two frameworks offer different perspectives:

• Levels 0, 1, and 2 describe the depth of ML workflow automation.
• The four-stage roadmap describes how organizational and platform operating capabilities expand.

It is therefore more appropriate to use them together than to map them one-to-one.

For an EKS-based platform, this might mean moving from manually creating environments for each team to providing shared templates and standardized deployment procedures. The platform team can provide common execution environments, deployment capabilities, and observability tools, while domain teams retain responsibility for their data, models, evaluation criteria, and operations.

Defining what should be standardized and what should remain a team-level decision allows the platform to grow while accommodating model-specific requirements.

5. What Role Should Amazon EKS Play?

Amazon EKS is a service in which AWS manages the Kubernetes control plane. With EKS Auto Mode, AWS also handles provisioning, scaling, and maintenance of data plane infrastructure, including compute, networking, and storage. Training jobs and inference services can run on this foundation, connected to tools for data storage, experiment tracking, model management, and deployment. (Reference: "What is Amazon EKS? [docs.aws.amazon.com]").

Using the operational requirements discussed above, we can divide the responsibilities as follows:

This is an example configuration intended to clarify responsibilities. In an actual implementation, the balance between managed services and components operated directly by the team should depend on requirements and operational capabilities.

In particular, observing data drift and prediction quality requires the application or a separate evaluation process to produce the relevant metrics and connect them to the monitoring and alerting system.

Decisions about which data to use for training, which evaluations a model must pass, and who approves production deployment should be defined as part of the platform’s operating procedures.

Container image versions and model versions must also be distinguished. A model may be packaged inside an inference image or retrieved from S3 or another location at runtime. Either way, deployment records should identify both the image version and the model version used.

6. What Changes When MLOps Expands to LLMOps?

Version control, automation, continuous improvement, and governance remain important for generative AI and retrieval-augmented generation (RAG) systems. Additional managed assets include prompts, retrieval data, evaluation sets, and model-provider configuration. AWS also emphasizes connecting these assets with evaluation, deployment, and monitoring in generative AI operations.

It should be possible to trace a response produced in operation back to the model, prompt, and retrieved documents that generated it. Evaluation sets are also needed to compare quality before and after a change, along with procedures for rolling back problematic configurations.

7. Closing

When designing an MLOps platform, the first step is to decide how much of the model creation and usage process should be connected.

AWS’s Levels 0, 1, and 2 help explain how the operational focus expands from a trained model to a repeatable training pipeline, and then to CI/CD and operations for the pipeline itself. The enterprise adoption roadmap adds a perspective on the operating structure needed to extend these capabilities across teams and use cases.

Kubernetes provides a foundation for running and managing training and inference workloads. Connecting that foundation to data, code, and model versions—as well as evaluation, approval, deployment, and monitoring records—can turn it into a platform that supports reproducibility and recovery. This post has mapped those operational requirements to the roles and relationships of components in an Amazon EKS-based platform.

A question worth revisiting when building an MLOps or LLMOps platform is:

Can we explain why the model currently in production was approved for deployment? And can we repeat the same process and recover when necessary?

Connecting components and retaining execution records should help us build a platform that can answer that question.