Course Syllabus

Module 1: Foundations of LLMOps (4 h)

• Topics: LLM pipelines, versioning (DVC/MLflow), data workflows, safety & ethics basics
• Hands‑on: Set up MLflow/DVC for dataset & model tracking.
• Outcome: Understand end-to-end LLM lifecycle and governance foundations.

Module 2: Data & Prompt Engineering (6 h)

• Topics: Prompt types (zero/few-shot, Chain‑of‑Thought), parameter tuning, retrieval‑augmented generation (RAG) fundamentals
• Lab: Build optimized prompts for text summarization and QA.
• Outcome: Effective prompting practice to enhance LLM responses.

Module 3: Fine‑Tuning & Domain Adaptation (6 h)

• Topics: Supervised fine-tuning, PEFT/LoRA techniques, multi-domain transfer
• Lab: Fine-tune an open-source LLM on a custom dataset.
• Outcome: Deploy efficient, domain-tuned LLMs.

Module 4: Deployment & Infrastructure (6 h)

• Topics: Containerization (Docker/K8s), serving (TensorFlow Serving, vLLM), cloud deployment (Azure, AWS)
• Lab: Containerize and deploy LLM on Kubernetes with auto-scaling.
• Outcome: Production-grade LLM serving systems.

Module 5: Monitoring & Observability in LLMOps (6 h)

• Topics: Observability tools (Prometheus, Airflow, ML monitoring), safety metrics, anomaly detection
• Lab: Build dashboards to track latency, errors, hallucinations.
• Outcome: Confidence in LLM uptime and safety oversight.

Module 6: CI/CD & Continuous Improvement (4 h)

• Topics: CI/CD pipelines (GitHub Actions), version control, retraining triggers, automated testing pipelines.
• Lab: Create full CI/CD workflow from commit → training → deployment.
• Outcome: Seamless LLM lifecycle integration with dev workflows.

Module 7: Foundations of AgentOps (4 h)

• Topics: Agent architecture, decision-making, planning, multi-agent systems .
• Lab: Build and deploy a simple LLM-driven agent using LangChain or OpenAI Agent SDK.
• Outcome: Grasp agent behavior and orchestration basics.

Module 8: Agent Coordination & Real‑Time Adaptation (4 h)

• Topics: Multi-agent coordination, tool-retrieval chaining, real-time adaptation, human-in-the-loop, observability for agents
• Lab: Create a two-agent workflow with AgentOps monitoring.
• Outcome: Build and manage interactive, scalable agent systems.

Module 9: AgentOps Observability & Safety (4 h)

• Topics: Logging, metrics, anomaly detection, Explainability (LIME/SHAP), policy enforcement
• Lab: Implement monitoring dashboards and trace agent decision logs.
• Outcome: Ensure secure, interpretable agent behavior in production.

Module 10: Security, Ethics & Governance (4 h)

• Topics: Ethical AI constraints, bias mitigation, data privacy/GDPR, secure agent pipelines, compliance .
• Lab: Perform bias checks and secure agent endpoints.
• Outcome: Build secure, compliant LLM and agent operations.

Module 11: Scaling & Continuous Learning (4 h)

• Topics: Scaling infra (Ray, Kubernetes), continual learning for agents, optimizing pipelines.
• Lab: Scale agent deployment and implement retraining pipeline.
• Outcome: Robust systems able to evolve with data.

Module 12: Capstone Project (4 h)

• Task: Design and implement a hybrid LLM + agent pipeline (e.g., RAG-based assistant with task execution and observability).
• Deliverable:
  • Notebook/Repo with code
  • Running service with CI/CD, monitoring dashboards, CI tests, continuous retraining
• Outcome: Showcase end-to-end mastery of both LLMOps and AgentOps.

• Key Features

    Hands-on labs using real tools: Kubeflow, vLLM, LangChain, Ray, Prometheus, MLflow/DVC, GitHub Actions.

    Real-world case studies: e‑commerce chatbots, agent-based workflows.

    Safety & ethics built-in from day 1.

    CI/CD integration, automated tests, versioning.

    Focus on observability—LLM and agent-level metrics, logs, tracing.

    Instructor-led sessions plus self-paced modules.

    Capstone project with peer review and certification.