Benchmark & Optimize LLM App Performance is a hands-on journey from “it works” to “it flies.” You’ll start by treating speed and cost as product features-defining a baseline with the right metrics (p50/p95 latency, tokens/sec, throughput, determinism, cost per task) and building a lightweight benchmarking harness you can rerun on every change. Next, you’ll learn to hunt bottlenecks across the stack-network, model, prompt, and post-processing-using practical patterns that cut tokens without cutting quality, plus caching strategies for embeddings, RAG, and tool calls. Then you’ll run A/B/C experiments to compare models and prompts on the same dataset, interpret results with simple stats, and choose a winner confidently. Finally, you’ll harden for production with concurrency limits, queues, timeouts, fallbacks, and a 30-day optimization playbook. Expect reusable templates, clear checklists, and realistic demos designed for busy developers and product builders who want measurable gains-not hype.
Benchmark & Optimize LLM App Performance
您将学到什么
Optimize LLM behavior using structured prompting and self-checks to reduce variance and errors.
Design scalable middleware to manage API requests, retries, caching, and token budgets for performance targets.
Build user-centered interfaces that collect feedback and improve LLM accuracy and user trust.
您将获得的技能
要了解的详细信息
添加到您的领英档案
1 项作业
December 2025
了解顶级公司的员工如何掌握热门技能
积累特定领域的专业知识
- 向行业专家学习新概念
- 获得对主题或工具的基础理解
- 通过实践项目培养工作相关技能
- 获得可共享的职业证书
该课程共有3个模块
This module establishes why performance is a product feature, not a backend afterthought. We connect latency, cost, and answer quality to user-perceived speed (p50 vs p95, jitter) and trust. You’ll define a minimal metric set-latency, throughput, tokens/sec, determinism, and win-rate-then build a lightweight benchmarking harness that runs a small eval set, logs prompts/outputs, and exports clean CSVs. By the end, you’ll have a reproducible baseline you can rerun on every change.
涵盖的内容
4个视频2篇阅读材料1次同伴评审
In this module, you'll trace where time actually goes: network hops, model inference, prompt bloat, and post-processing. You’ll learn practical prompt patterns that cut tokens without cutting quality, plus schema-first I/O that improves stability and parsing. We’ll add caching strategies for embeddings, RAG retrievals, and tool calls, including cache keys and invalidation rules to avoid stale answers. Expect clear heuristics for cold vs warm paths and a simple checklist to shave seconds-not just milliseconds.
涵盖的内容
3个视频1篇阅读材料1次同伴评审
The final module turns tuning into a disciplined workflow. You’ll run A/B/C tests across model tiers and prompt variants on the same dataset to compare latency, cost per task, and quality with simple stats - then pick a winner. We’ll cover safe scaling: concurrency limits, queues, backpressure, retries, timeouts, and graceful degradation/fallbacks. You’ll leave with a 30-day optimization plan and a production playbook that keeps your app fast, affordable, and reliable after launch.
涵盖的内容
4个视频1篇阅读材料1个作业2次同伴评审
获得职业证书
将此证书添加到您的 LinkedIn 个人资料、简历或履历中。在社交媒体和绩效考核中分享。
提供方
人们为什么选择 Coursera 来帮助自己实现职业发展
Felipe M.
Jennifer J.
Larry W.
