Applied AIwith Sawan Ruparel

Applied AI Course

Building AI systems that can be evaluated and used

Public self-study edition · Edition 2027.1 (v3.0)

From a model that answers a prompt to a system that can retrieve, act, recover, and be evaluated. A graduate-level course in the engineering of applied AI.

Work through prompting and retrieval, build tool integrations and agent workflows, then evaluate the tradeoffs in quality, cost, latency, and safety. Bring the pieces together in a working capstone.

Taught by Sawan Ruparel at the University of Connecticut in Spring 2025 and Spring 2026. This public edition makes the course materials available for independent study.

Teaching through evidence

The course connects model behavior to engineering decisions: define success, compare a baseline, inspect failures, and explain the tradeoffs. The capstone asks learners to demonstrate those decisions in a reproducible system.

Example: How this course evaluates an architectural decision

Rather than asking “Does the agent work on a demo prompt?”, each phase requires documenting system changes against a reproducible evaluation harness (illustrative capstone decision log):

Architecture stageAnswer faithfulness (n=250)Tool / action successLatency (p95)Cost / 1k queriesPrimary failure mode inspected
0. Baseline (Zero-Shot Frontier LLM)61.2%N/A1.4 s$4.20Unverified hallucination on domain-specific policy rules
1. Hybrid RAG + Reranker (Phase 2)86.4%N/A2.1 s$4.85Multi-hop retrieval miss when entities span two documents
2. MCP Agentic Workflow (Phase 3)92.8%89.0%5.8 s$11.40Infinite tool-retry loop on malformed upstream API schema
3. Routed SLM + Guarded Workflow (Phase 4)91.6%94.5%1.9 s$1.65Router misclassifies edge-case adversarial prompt to fast tier

Every architectural promotion in the capstone must justify its delta in quality, latency, unit cost, and recovery behavior.

For faculty reviewing the course, follow this short path through its design:

  1. Learning outcomes and preparation establish the intended level.
  2. A representative evaluation exercise shows the move from concepts to evidence.
  3. The capstone rubric makes the assessment criteria explicit.

Institutional adaptation brief · Cite this curriculum · Discuss teaching this course

The learning path

  1. Understand model capabilities and inference-time scaling, engineer reliable structured prompts, design deterministic state machines before adding autonomy, and represent domain knowledge with embeddings and hybrid indexes.

    Assignment 1 (End of Phase 1, Week 4): Baseline, Control-Flow & Indexing Spec

  2. Architect hybrid and graph-based retrieval pipelines, build automated evaluation harnesses, and measure faithfulness before scaling complexity.

    Assignment 2 (End of Phase 2, Week 7): Retrieval & Eval Harness Report

  3. Standardize tool integration with the Model Context Protocol (MCP), coordinate multi-step agentic workflows, engineer failure recovery, and design human-in-the-loop escalation.

    Assignment 3 (End of Phase 3, Week 11): MCP Tool Contract & Recovery Trace

  4. Optimize unit economics and latency via small language models (SLMs) and routing, distill or fine-tune deliberately against a baseline, and red-team adversarial failure modes.

    Capstone Stage 1 (End of Phase 4, Week 14): Pareto Frontier & Red-Team Audit

  5. Capstone

    Week 15

    Build a reproducible, end-to-end AI system and defend its architectural decisions with quantitative evaluation evidence.

    Final Capstone Defense (Phase 5, Week 15): End-to-End Reproducible System & Evaluation Defense

Start with a question you can measure

You will compare models and workflows, inspect failure cases, and build a system whose behavior you can explain. Three phase assignments (Weeks 4, 7, and 11) and the capstone guide turn those experiments into a reproducible project.

Begin with Week 1: the AI model landscape, or use the schedule to find a topic.