Home › Tracks › Production AI Products
Elite track
Ship AI products that survive real users and real attackers.
Build the full product around a model: retrieval and context engineering at scale, LLM evaluation and observability, AI red-teaming and security, cost governance, and LLMOps. You leave able to take an AI feature from prototype to a hardened, monitored, publicly hosted product. Bridges to Databases, Software Engineering, and Information Security.
Week by week
Every week unlocks the next. Concepts route you to free, world-class material; projects turn that knowledge into something deployed.
Week 1
The anatomy of a production LLM feature: the model call, structured output, streaming, error handling, and the latency and cost budget you design against from day one.
Bridges to Software Engineering — application architecture and API designBuilds on: nothing, start here
Week 2
Represent meaning as vectors and search it fast: embedding models, similarity metrics, approximate nearest neighbor (HNSW), and the recall-versus-latency tradeoff.
Bridges to Databases — indexing, search structures, and query optimizationBuilds on: LLM Application Foundations
Week 4
Retrieval-augmented generation that holds up: chunking strategy, hybrid and contextual retrieval, reranking, and why retrieval quality dominates generation quality.
Bridges to Databases — query processing, joins, and information retrievalBuilds on: Embeddings & Vector Search
Week 5
Treat eval as engineering: golden datasets, LLM-as-judge with calibration, regression suites in CI, and catching silent quality drift before users do.
Bridges to Software Engineering — automated testing and continuous integrationBuilds on: Production RAG & Context Engineering
Week 6
Build a production-ready RAG platform backed by an agent-first embedded vector lakehouse (LanceDB/pgvector). Implement an automated EDDOps validation suite with custom golden datasets, trace-based latency checks, and LLM-as-a-judge regression tests.
A dockerized, query-active vector service and structured context database with live tracing and a production CI/CD evaluation gate.
Week 7
See inside a non-deterministic system: distributed tracing of LLM calls, token and cost metrics, latency percentiles, and structured logging with correlation IDs.
Bridges to Software Engineering — monitoring, logging, and distributed tracingBuilds on: LLM Evaluation Harnesses
Week 9
Attack your own product before someone else does: prompt injection, jailbreaks, data exfiltration, PII leakage, and building input/output guardrails that actually hold.
Bridges to Information Security — threat modeling, penetration testing, and OWASPBuilds on: AI Observability & Tracing
Week 10
Operationalize model deployments with high availability and resilience. Design dynamic, multi-cloud and multi-model fallback topologies (routing between cloud APIs and local open-weight fallbacks) to mitigate geopolitical risk, single-provider outages, and sudden regulatory export controls.
Bridges to Software Engineering — release management, versioning, and CI/CDBuilds on: AI Security & Red-Teaming
Week 12
Take an AI feature to a real public launch: containerized deploy, health checks, graceful degradation when the model is slow or down, and a launch story that sells the experience.
Bridges to Software Engineering — deployment, resilience, and product engineeringBuilds on: LLMOps, Cost Governance & Geopolitical Fallbacks
An enterprise mandate: the company's AI features are live and the security and reliability teams are flying blind. Build and launch a product with two interlocking systems: an observability pipeline that traces every LLM call with token, cost, and latency telemetry and surfaces silent quality drift; and an automated red-team harness that continuously attacks the AI product with prompt injection, jailbreaks, and data-exfiltration probes, and reports which guardrails held. The deliverable is directly deployable and hyperscalable: real public hosting, CI/CD, a hyper-usable dashboard a security lead reads at a glance, the platform itself secured, and the ingestion path able to absorb high call volume. It ships complete with marketing — a landing page, a pitch, and a demo. Deliver something an enterprise can buy and run on a real product before an incident, not after. Ship it as a real product.
Why it matters: AI security and observability is a board-level concern as AI features ship into regulated industries, and almost no one combines both. A builder who delivers a tracing-plus-red-team pipeline is directly deployable as an AI Security Engineer or LLMOps Lead, a scarce role because it sits at the intersection of security, reliability, and AI.
A publicly hosted product with a stable URL and a hyper-usable security dashboard, plus a public repo: the tracing and observability pipeline, the automated red-team attack suite with a results report, the guardrails it validates, CI/CD on every commit, a marketing landing page, a 10-slide pitch, a recorded demo, and a README documenting the threat model, the drift-detection method, and the scaling design.
Market signal, who wants thisAI security is a proven, acquisition-grade 2026 market: Lakera, which built exactly this guardrails-plus-red-teaming product (Lakera Guard at 98%+ detection, sub-50ms; Lakera Red for automated attack simulation), was acquired by Cisco in May 2025 and folded into Cisco AI Defense. Evaluation leaders like Galileo now ship guardrails that intercept outputs before tool execution. Investors fund AI observability and red-teaming because shipping AI into regulated industries makes pre-incident security a board-level requirement, and almost no product combines tracing and red-teaming in one.
What's next
Each track stands alone, so there's no wrong order. If you want a suggestion, this one pairs well next.