Amazon Bedrock vs. SageMaker: Which Should You Use?

Bedrock is serverless foundation models via one API — no GPU hosts, pay per 1K tokens. SageMaker is build, train, and deploy for any ML — you pick GPU, pipeline, and ops. They sound similar because both talk to models, but they solve different layers: Bedrock for using frontier LLMs, SageMaker for creating and owning ML — and most teams end up using both together. This in-depth guide compares architecture, pricing, control, and gives a decision tree with hybrid patterns.

What Is Amazon Bedrock — Serverless Foundation Models, One API

Bedrock is AWS's serverless gateway to curated foundation models — Bedrock getting started lists Anthropic Claude 3.5 (Sonnet/Haiku/Opus), Amazon Titan (Text Premier, Embed G1, Image G1), Meta Llama 3 (8B/70B), Mistral Large, Cohere Command/Rerank, AI21 Jurassic — all via one Converse API with IAM, VPC PrivateLink, and CloudWatch. You send modelId: anthropic.claude-3-5-sonnet-20240620-v1:0 and messages; AWS routes to provider inference behind the scenes. No GPU fleet, no container, no scaling policy — AWS handles it.

Pricing is per token on-demand ($3 / $15 per 1M Sonnet), batch 50% off via S3 JSONL in 24h, or provisioned throughput $/hr for reserved Model Units; add-ons Knowledge Bases (S3 + OpenSearch Serverless/Aurora vectors), Agents (tool loops), Guardrails (filter) — each adds tokens or store cost. Swap us.anthropic.claude-... inference profiles for cross-Region. Docs: Bedrock pricing, pricing details.

Add-ons That Make Bedrock More Than an API

• Knowledge Bases: ingest S3 docs → chunk → Titan Embed $0.10/1M → vector store (OpenSearch Serverless or Aurora pgvector or S3 Vectors preview) → RAG via RetrieveAndGenerate.
• Agents: LLM thinks, calls Action Groups (Lambda), observes, loops — each step is another LLM call (1 user request → 3×). See Agents.
• Guardrails: deny topics, PII redaction, groundedness — adds input tokens for filtering.

What Is Amazon SageMaker — Build, Train, and Deploy Your Own ML

SageMaker is the full platform: start in Studio notebooks, submit a Training job on your chosen ml.g5.xlarge / p4d / trn1 with your Docker image, dataset on S3, distributed strategies, spot 60% off; output to Model Registry → deploy to Real-time Endpoint (fully managed, auto-scale), Serverless Inference (pay per invocation like Bedrock), or Batch Transform S3 in/out. Full MLOps: SageMaker Pipelines, Feature Store, Model Monitor (drift, data quality), lineage, and Governance.

You own the stack: pick instance, VPC, spot, pipeline DAG, hyperparam tuning (Automatic Model Tuning), custom metrics, canary traffic shift (10%→100%). JumpStart offers one-click foundation models on SageMaker endpoints too — confusing overlap, detailed below.

JumpStart vs Bedrock — Same Model, Different Hosting

JumpStart deploys Llama 3 or Mistral on your SageMaker endpoint — you pay ml.g5.xlarge $1.41/hr ×24 = $33.84/day plus ops. Bedrock hosts the same Llama via serverless per-token ($2.65/$3.50 per 1M). JumpStart makes sense only if you need that exact Llama with custom vLLM flags, VPC lockdown, or sustained > 500M tokens/day where self-host $/token undercuts Bedrock; otherwise Bedrock is cheaper and zero ops.

Architecture Compared — Serverless API vs Managed Infra

Bedrock: Your VPC → bedrock-runtime:Converse via VPC endpoint → Bedrock managed provider inference → Knowledge Base (S3 + OpenSearch Serverless/Aurora/S3 Vectors) → Agents (Lambda). You manage prompts, IAM, and vector store OCUs; AWS manages GPUs and scaling (provisioned Units abstract them).

SageMaker: Notebook (Studio) → Training job (EC2 GPU, EFS/S3 data, spot, distributed) → Model Registry → Endpoint (auto-scale, shadow/canary) → Pipelines (DAG: data → Feature Store → train → eval → register → deploy) → Model Monitor (drift). You choose ml.trn1.32xlarge vs g5, spot, VPC subnets, pipeline schedule. See How SageMaker works vs What is Bedrock.

Ops Difference

Bedrock: no clusters, no GPU patching, no auto-scale policy — just IAM quotas (default 2K TPM Sonnet, raise via Service Quotas) and provisioned Units if you need guaranteed TPS. SageMaker: you own instance right-sizing, patching (managed but you choose version), endpoint auto-scale (TargetTracking 60% CPU), pipeline schedule, and lineage. Bedrock is FaaS for LLMs; SageMaker is ECS+EKS+CI for ML.

Head-to-Head Table — At a Glance

Pricing Compared — $/Token vs $/Hour and the Break-Even

Bedrock: On-Demand Sonnet $3 in / $15 out per 1M, Haiku $0.80/$4, Titan Text $0.50/$1.50, Embed $0.10/1M, Image $0.04/image, Batch half, Provisioned ~$32/hr/Sonnet unit even idle, fine-tune $8/1M + provisioned hosting. See Bedrock pricing.

SageMaker: ml.g5.xlarge ~$1.41/hr training (spot ~60% off), Real-time endpoint $/hr (auto-scale min 1 → max 10), Serverless endpoint pay per invocation + duration (like Lambda for ML, true scale-to-zero), Batch Transform $/hr. No per-token — cost is instance uptime + storage (S3 + EFS) + pipeline. See SageMaker pricing.

Example — 1M in + 0.5M out / day Sonnet: Bedrock On-Demand = 1×$3 +0.5×$15 = $10.50/day. Same volume self-hosting Llama 3 70B on SageMaker g5.xlarge 24/7 = $1.41×24 = $33.84/day + ops, before spot. At 15M+3M/day (support copilot) Bedrock Sonnet $90/day; SageMaker Llama steady may be $33/day if GPU stays full — self-host wins only if GPU is packed 24/7.

Provisioned vs Real-time: Bedrock provisioned $768/day/unit vs SageMaker real-time g5 at $33/day — different scale. Bedrock provisioned is for frontier model SLA; SageMaker endpoint is for your own model. Don't compare $/hr across them directly — compare total at your load.

When Each Is Cheaper

• Variable/spiky < 60% provisioned cost at peak → Bedrock On-Demand or SageMaker Serverless (scale to zero).
• Steady high (>10M tokens/day or > 500 endpoint invocations/min) → Bedrock provisioned or SageMaker real-time with auto-scale.
• Embeddings 10M tokens: Bedrock Titan Embed $1 total vs SageMaker self-host embed endpoint $33/day idle — Bedrock wins.

Decision Tree — Which Should You Use?

Hybrid Pattern — Most Common in Prod

User → App (Lambda/EC2)
 ├→ SageMaker Endpoint: my-xgboost (ml.m5.xlarge auto-scale) → score=0.92
 └→ Bedrock Converse: anthropic.claude-3-5-sonnet + RAG KB → summary "High risk, block"
→ Shared VPC endpoint, IAM role per service, Cost tags project:fraud

SageMaker trains the tabular/XGBoost/classifier on your private transactions (Bedrock can't), Bedrock adds the LLM layer without you running GPUs. Both log to CloudWatch, both respect VPC endpoints, both taggable for Cost Explorer (project:abc). This is why teams that try to push everything to one often overpay: pushing tabular to Bedrock fine-tune wastes provisioned hours; pushing LLM to SageMaker self-host wastes ops.

Migration Path — No Fork, Layer Gradually

1. Start Bedrock if you have no ML team: prompt + Knowledge Base (S3) → measure InputTokenCount week.
2. Add caching + rerank: prompt cache 90% off cached input, rerank 10K → 3K before LLM.
3. Need proprietary tuning? Bedrock fine-tune Titan first (quick); if base still fails after RAG+prompt+Haiku routing, move heavy training to SageMaker training job with spot.
4. Own endpoint for that custom model → SageMaker Registry → Real-time/Serverless → Bedrock calls it alongside Claude.

This ladder avoids rewriting — your Bedrock app code keeps calling SageMaker endpoint via SDK alongside Converse.

Deep Pricing — Real $ for Real Workloads (Bedrock On-Demand vs SageMaker Endpoint)

Don't compare headline $/token to $/hr — compare total at your load. Three workload shapes:

1. Chatbot 1K requests/day × 2K in + 0.5K out (RAG 10K prompt + short answer): Bedrock Sonnet On-Demand = 2M in×$3/1M +0.5M out×$15/1M = $6+$7.50 = $13.50/day → $405/mo. Same on Haiku = $3.60/day → $108/mo. SageMaker: host Llama 3 70B on ml.g5.xlarge $1.41/hr ×24 = $33.84/day → $1,015/mo even idle, before data transfer — overkill for 1K/day chat. Batch that 1K/day offline via Bedrock Batch 50% off → $6.75/day Sonnet if 24h OK.
2. Support Copilot 10K requests/day × 1.5K in +300 out: Bedrock Sonnet = 15M in×$3 +3M out×$15 = $45+$45 = $90/day → $2,700/mo; Haiku = $24/day → $720/mo. SageMaker Llama self-host g5.xlarge 24/7 = $33.84/day → $1,015/mo plus spot, but you own scaling — if 10K requests burst 8h/day only, SageMaker Serverless endpoint (pay per invocation + ms duration, scale to zero) may be $15/day vs Bedrock $90/day spiky, but steady 10K/day sustained, Bedrock On-Demand still cheaper due to per-token vs whole instance.
3. Embeddings 100M tokens (docs): Bedrock Titan Embed $0.10/1M → $10 total. SageMaker self-host embed endpoint (e.g., ml.m5.xlarge $1.20/hr) 24/7 = $28.80/day → $864/mo — 86× waste for one-time 100M. Never self-host embeddings for batch-only.

Provisioned nuance: Bedrock provisioned ~$32/hr/Sonnet unit (even idle) → $23K/mo if 24/7. At steady 40K input TPM that many tokens on-demand would be ~$35K/mo → provisioned saves ~35%. At 5K TPM spiky, on-demand $2K/mo vs provisioned $23K — stay on-demand. SageMaker provisioned is opposite: real-time endpoint $/hr is always, Serverless is scale-to-zero; pick Serverless for spiky <60% util (Serverless) like Bedrock on-demand. Break-even: measure p50/p95 TPM for a week via CloudWatch InputTokenCount vs SageMaker Invocations.

SageMaker Serverless vs Real-Time vs Batch Transform — Which Inference?

• Real-time endpoint (multi-model or single): ml.g5/m6i $/hr, auto-scale min 2 → max 10 via Application Auto Scaling. Pay even zero invocations if min>0. Best for steady > 200 req/min needing p95 latency <1s.
• Serverless: pay per invocation + compute duration (like Lambda) + concurrency. True scale-to-zero, but cold start 5–10s and max concurrency 200, timeout 60s. Best for spiky <60% util, demos, internal tools. No auto-scale policy — AWS scales.
• Batch Transform: S3 in → S3 out, $/hr for batch cluster hours. For 100K rows nightly scoring (tabular), Batch Transform spot 60% off is far cheaper than real-time idle 24/7. See Batch Transform.

Bedrock has equivalent three: On-Demand (like Serverless pay per token), Batch (like Batch Transform S3, 50% off), Provisioned (like Real-time $/hr). Mirroring helps mental model.

Feature Deep — Knowledge Bases vs Feature Store vs Pipelines

Beyond inference, platform features diverge:

• Bedrock Knowledge Bases: S3 ingestion → chunk (500 tokens) → Titan Embed $0.10/1M → store OpenSearch Serverless (OCU $0.24/hr + storage, min 2 OCUs ~$700/mo always on) / Aurora pgvector / S3 Vectors preview (cheaper per vector). Hidden cost is OCUs, not tokens — many teams switch to Aurora for $300/mo. See Knowledge Bases.
• SageMaker Feature Store: Offline (S3) + Online (low-latency) for ML features, with ingestion, lineage, and time-travel. Bedrock has no feature store — you'd bring SageMaker Feature Store even when LLM is Bedrock (hybrid). Docs: Feature Store.
• Pipelines: Bedrock has no DAG pipelines — you orchestrate via Step Functions/Lambda. SageMaker Pipelines is managed DAG for data → Feature Store → train → eval → register → deploy → monitor (lineage). If you need reproducible training workflows, SageMaker's graph beats Bedrock's ad-hoc scripts.
• Monitoring: Bedrock CloudWatch Invocations, InputTokenCount + Guardrails; SageMaker Model Monitor detects data drift, quality, bias, and feature attribution — deeper. See Model Monitor.

Security, Governance, and Control — Where SageMaker Pulls Ahead

• IAM/VPC: Both use IAM roles + VPC endpoints (Bedrock via PrivateLink, SageMaker via VPC with subnets/security groups). Bedrock's perimeter is simpler (one service), SageMaker's is finer (per Notebook, Training job, Endpoint subnet).
• KMS + Encryption: Both support KMS at-rest for S3/model. SageMaker adds EFS encryption and lineage (who trained what, on which data hash) — audit needs.
• RagGuard: Bedrock Guardrails filters topics/PII at LLM edge; SageMaker needs custom post-processor.
• Data control: Bedrock sends prompts to provider infrastructure (Anthropic/Titan) via AWS — data stays in AWS Region but not your VPC. SageMaker training is fully in your VPC with your ECR container — for strict data residency (no third-party provider), SageMaker wins (you own the container even for JumpStart Llama, which runs in your account).

Learning Curve and Team — Who Should Operate What

Bedrock: a backend engineer can ship RAG in days — add IAM role, call Converse, ingest S3 to Knowledge Base, attach prompt cache (cacheControl: {type: ephemeral} for stable 10K RAG → 90% off input $3→$0.30/1M on Claude). SageMaker: needs ML engineer who knows spot vs on-demand training, distributed strategies (data parallel vs model parallel), Hyperparameter Tuning, and lineage. Estimate: Bedrock PoC 1 week, SageMaker production pipeline 3–6 weeks. For startups with no ML team, Bedrock is the only viable week-one path; SageMaker is for teams with dedicated ML ops.

Observability gap: Bedrock logs to CloudWatch if you enable invocation logging (input/output/latency) — review which template is 10K vs 2K and where to cache/compress (10K → 3K via reranker = 70% off). SageMaker logs every training metric to CloudWatch + Studio Experiments — necessary for hyperparam search.

When to Choose Both — Hybrid Patterns That Actually Ship

In surveys, the most common enterprise pattern is not either/or — it's hybrid:

1. Fraud Platform: SageMaker trains XGBoost on 100M transactions (custom loss, spot ml.m5.4xlarge) → Real-time endpoint /score auto-scale 2→10. Bedrock Claude summarizes alert + drafts analyst email via Knowledge Base over runbooks. Bedrock handles language; SageMaker owns scoring. Same VPC, same EDP commit, tagged project:fraud.
2. Support Copilot: Bedrock Knowledge Base over docs (Titan Embed + OpenSearch) + Claude Agent for tools (Jira, Slack) handles 80% chats. SageMaker custom classifier (fine-tuned BERT on tickets) routes high-risk to humans — trained on private tickets Bedrock never sees.
3. Forecast: SageMaker trains DeepAR on time-series → endpoint; Bedrock Claude explains forecast in natural language for execs.

Why hybrid wins: frontier LLM fine-tune on Bedrock can't touch your proprietary tabular loss; self-hosting that LLM on SageMaker ops heavy ($33/day g5 + your paging). Hybrid keeps each service in the cheaper, lower-ops home. Both share IAM, VPC endpoint, Cost tags, and EDP.

Migration Ladder — Start Bedrock, Layer SageMaker Only If Needed

1. Bedrock only (Week 1): Prompt + Knowledge Base (S3) → measure p50/p95 tokens for 100 samples via CloudWatch.
2. Add caching + rerank: cache stable RAG (90% off), rerank 10K→3K, route Haiku for classify (73% cheaper) vs Sonnet for code — often halves bill before any training.
3. Light Bedrock fine-tune: Titan Text fine-tune $8/1M (5M tokens $40) + provisioned host → test if RAG still fails after prompt wins. Delete provisioned when not A/B testing.
4. Heavy SageMaker: If proprietary training or non-LLM needed, create SageMaker training job (spot 60% off) → Model Registry → Serverless (spiky) or Real-time auto-scale → Bedrock calls it alongside Claude (shared VPC).

This ladder avoids rewriting — your Bedrock app code keeps calling SageMaker endpoint SDK alongside Converse, same IAM role. Teams that jump straight to SageMaker for LLM waste weeks on vLLM tuning Bedrock already solved.

SageMaker Variants You Might Confuse with Bedrock

SageMaker is not one endpoint — three confuse beginners:

• SageMaker Studio Notebooks + JumpStart: UI to launch training and one-click deploy Llama/Mistral to your endpoint — feels like Bedrock but you pay ml.g5 hourly and manage patching. Use when you need JumpStart's custom container or VPC lockdown that Bedrock's abstracted Units don't allow.
• SageMaker Canvas (no-code): drag-drop for business analysts to build forecasts/classifiers without code — Bedrock has no equivalent. If analyst needs forecast, Canvas → SageMaker endpoint, not Bedrock.
• SageMaker Data Wrangler: visual data prep (join, normalize) that feeds Feature Store → training. Bedrock expects you to prepare RAG docs before Knowledge Base ingest — no Wrangler.

Rule: If task is "classify my CSV without code", Canvas beats both Bedrock and raw SageMaker training. If task is "chat with my docs", Bedrock Knowledge Base beats Canvas.

Fine-Tuning Compared — Bedrock Custom vs SageMaker Training Job

Use Bedrock custom only after prompt/RAG/cache/Haiku routing fail and you need Titan adapted slightly. For deep proprietary fine-tune (100M+ tokens) or non-LLM, SageMaker's training job with spot and your container is the only path. Docs: Bedrock custom vs SageMaker training.

Governance — Lineage, Audit, and Who Trained What

SageMaker tracks lineage automatically: dataset S3 version → training job ID → hyperparameters → model artifact hash → endpoint version — query via Lineage for audit. Bedrock logs invocation via CloudTrail + CloudWatch but not dataset lineage — you must version S3 prompts yourself. For regulated industries requiring "who trained this model on which data hash", SageMaker's governance is required; Bedrock's trace is invocation, not provenance.

Common Pitfalls — The $23K Surprises

Latency, Throughput, and Quotas — What Throttles First

Bedrock: On-Demand quota is per model per Region — Sonnet ~2K TPM default, Haiku higher. Burst → ThrottlingException even if budget allows. Raise via Service Quotas weeks before launch. Provisioned reserves TPS — no throttling until Units saturated, then queue. Batch has no real-time SLA (24h). Measure p95 Invocations + OutputTokenCount in CloudWatch before committing.

SageMaker: Real-time endpoint: you set initial instance count and auto-scale policy (TargetTracking 60% CPU or InvocationsPerInstance) — cold scale takes 2–5 min for new instance. Serverless: concurrency 200 max, scale to zero but cold start 5–10s p95. For 100 req/s sustained, Real-time with 5× ml.g5.xlarge is predictable; for 10 req/min spiky, Serverless is cheaper. Neither throttles on tokens — they throttle on concurrent instances. Tune via Endpoint auto scaling.

# Bedrock: ~800ms for Sonnet 2K+0.5K tokens (managed)
# SageMaker JumpStart Llama 70B on g5.xlarge: ~1200ms first token, ~30ms / token after — similar p50 but you own warm-up
# Hybrid: SageMaker XGBoost /score 50ms + Bedrock Claude 800ms = 850ms end-to-end (parallel where possible)

Cost Tags and One Bill — Dont Lose Attribution

Both taggable via Cost Allocation Tags — add project:abc, env:prod to Bedrock inference profiles (via inference profiles) and SageMaker training jobs and endpoints. Filter Cost Explorer by tag to see Bedrock 405 per mo vs SageMaker 1K per mo per project — without tags, Finance sees one Bedrock line and one SageMaker line with no owner. Enable tags on day one; retroactive does not backfill.

Final tip: keep prompts versioned in S3 like code and modelId pinned — 20240620-v1:0 not floating — so pricing and behavior stay reproducible when AWS retires a model version.

Keep modelId version pinned in both — Bedrock anthropic.claude-3-5-sonnet-20240620-v1:0 and SageMaker JumpStart Llama meta-textgeneration-llama-3-70b:2.0.1 — so pricing and behavior stay reproducible across months when AWS retires a version. Test bumps via inference profiles and shadow endpoints before prod.

Example Stacks — Two Real Setups

Chatbot startup (Bedrock-only): S3 docs → Knowledge Base (Titan Embed + OpenSearch) → Claude Agent for Slack/Jira tools → prompt cache 90% off → $405 per mo for 1K requests per day. No SageMaker, no training job, IAM + VPC only.

Fintech platform (Hybrid): SageMaker trains XGBoost fraud model on 100M rows (ml.m5.4xlarge spot, Pipeline + Feature Store) → Real-time endpoint auto-scale 2 to 10, Model Monitor drift → Bedrock Claude via Knowledge Base over runbooks summarizes risk and drafts outreach. SageMaker owns proprietary scorer (data never leaves VPC), Bedrock owns language — total $1K + $405 per mo, same VPC endpoint, same Cost tags.

Both pass audit: lineage for XGBoost via SageMaker Lineage, invocation logs for Bedrock via CloudTrail — regulators see who trained what and who called which model when.

Choose per workload, not ideology — the cheapest correct tool wins.

Frequently Asked Questions

Is Bedrock replacing SageMaker?

No — they layer. Bedrock is serverless foundation-model inference; SageMaker is platform to build/train/deploy any model. AWS invests in both: Bedrock for using frontier LLMs, SageMaker for creating/owning ML. Most enterprises use both via hybrid (SageMaker scorer + Bedrock LLM).

Can SageMaker run Claude or Titan?

Not as managed Bedrock models — Claude/Titan are Bedrock-only curated. SageMaker can host open Llama/Mistral via JumpStart or custom containers, but not Anthropic proprietary. For Claude, use Bedrock; for Llama you have both choices (Bedrock per-token vs SageMaker per-hour).

Which is cheaper?

Depends on load shape: spiky/low (<60% provisioned cost at peak) → Bedrock on-demand or SageMaker Serverless cheaper; steady high >10M tokens/day → Bedrock provisioned or SageMaker real-time auto-scale may win depending on model. Embeddings 10M: Bedrock Titan $1 vs SageMaker self-host $33/day idle — Bedrock wins.

Can I train a custom LLM on Bedrock?

Lightly — Bedrock fine-tunes Titan and select Llama at $8/1M + provisioned hosting. For full pre-training, custom loss, large proprietary data, or non-LLM (XGBoost), use SageMaker training jobs with your code, spot, distributed. Use Bedrock fine-tune first only if RAG+prompt+routing fail.

Which is easier for a beginner?

Bedrock — one Converse API, no GPU/instance to choose, IAM + VPC only. SageMaker needs instance selection, pipeline, Feature Store, Model Monitor — higher curve but full control. Start Bedrock for LLM, add SageMaker when you need to own training.

Can they work together?

Yes — hybrid is common: SageMaker Endpoint for custom model (fraud XGBoost, forecast) + Bedrock Claude for summarization/RAG/agent on top, same VPC endpoint, same Cost tags. No rewrite: Bedrock retains LLM, SageMaker adds your domain model.