Skip to content

SageMaker AI adds data preparation for model training

AWS has put auto-labeling, synthetic data generation, quality checks, transformation and human review behind Python SDK APIs and an agent skill in Amazon SageMaker AI.

Draftship Editorial3 min read

What AWS announced

AWS announced AI data preparation capabilities in Amazon SageMaker AI on 17 November 2026, aimed at the stage that usually eats the most time before training starts: producing clean, labeled, task-specific data. The set covers auto-labeling, human review, synthetic data generation, data quality evaluation and data transformation. Everything is reachable through Python SDK APIs and an AWS AI-ML agent skill, so the work happens in SageMaker Studio JupyterLab, Kiro, or a local IDE with a coding assistant rather than in a separate toolchain.

The announcement post frames the problem plainly: labeled examples are scarce or expensive, privacy rules constrain what you can use, and defects like duplicates, label noise and answer leakage tend to surface only when a training run disappoints.

The five capabilities, briefly

  • Auto-labeling. You label from an unlabeled dataset, a small seed set, or nothing at all. Built-in task types include classification, named entity recognition, extractive question answering, ordinal rating and summarization, plus custom tasks. You write a short labeling spec, which is validated against your own examples before the full run. For tasks with a fixed answer set, several models label each row and their agreement becomes the row's confidence score.
  • Human review. Rows under your confidence threshold go to a review queue in SageMaker Studio built on the open-source Label Studio Community Edition. Outside subject matter experts get in through a link in the partner app box, and their corrections feed back into auto-labeling. Consensus metrics and labeling reports come out of the same loop.
  • Synthetic data generation. You describe a specification, optionally grounding output in your own seed data and documents, and the service produces a code-based generation pipeline you can read, edit, reuse and scale.
  • Data quality evaluation. Datasets are checked against your business rules, built-in metrics and LLM-as-a-judge metrics, with duplicate and near-duplicate detection and checks for problems you may not have thought to test, including label noise and answer leakage.
  • Data transformation. Reshape data by describing the change in plain language, by using built-in operations such as dedupe, merge, split and filter, or by running your own Python at scale.

How the pieces chain

The source describes two patterns. The first builds a golden evaluation set: auto-label raw data from a seed set, accept high-confidence rows automatically, send the rest to reviewers, and let their corrections improve the next labeling pass. The second goes from raw documents to a trained model in one workflow: generate task-specific examples grounded in your documents, transform them into the format a technique expects (supervised fine-tuning, direct preference optimization or reinforcement learning), validate for duplicates, leakage and rule violations before spending compute, then train with SageMaker AI model customization.

AWS also repeats an argument worth noting: for narrow, well-defined tasks, a smaller model customized on clean task-specific data can match or exceed a larger general-purpose model prompted into shape, at a lower cost per inference. Splunk, a Cisco company, is cited as an early user for its Agent Observability evaluation models, with director of engineering Yash Sheth saying the synthetic generation has helped produce "statistically representative synthetic datasets" while preserving underlying patterns.

Limits and open questions

No pricing is given for the new capabilities, and no regional availability is stated. The post does not name which models sit on the labeling jury, how review seats are licensed beyond the open-source Label Studio edition, or what limits apply to synthetic generation volume. The worked customer support example in the post is cut short in the text available, so the full end-to-end walkthrough is worth reading at the source.

What it means for building apps with AI

Most people shipping an app with an AI builder never fine-tune anything. They prompt hosted models and manage data in the app's own database. This release matters mainly to teams that have decided off-the-shelf model behavior is not close enough and want to own a dataset, and the practical takeaway is that the data pipeline is now describable in code you can inspect. If you are still at the prompting stage, adding AI to your app is the shorter path, and Draftship's built-in database is where that app data lives.

Primary sources

This article was drafted with AI assistance from the primary sources listed below and checked automatically against them. An editor hasn't reviewed it yet.