Challenge 21: Azure AI Foundry
25-30 min | Cost: Free | Domain: Generative AI (15-20%)
Exam skills covered
- Identify features and capabilities of Azure AI Foundry
- Describe the model catalog in Azure AI Foundry
- Identify Azure AI Foundry deployment options
Overview
Azure AI Foundry (formerly Azure AI Studio) is Microsoft's unified platform for building, evaluating, and deploying AI applications. Think of it as the "one-stop shop" for generative AI development on Azure. It brings together model access, prompt engineering, evaluation tools, and deployment — all in a single portal at ai.azure.com.
The platform is organized around hubs and projects. A hub is a top-level container that manages shared resources like compute, connections, and security settings across your organization. A project lives inside a hub and is where individual teams do their AI work — selecting models, testing prompts, building flows, and deploying applications. This hub-project hierarchy enables enterprise governance while giving teams flexibility.
A standout feature is the model catalog — a curated collection of AI models from multiple providers. Beyond OpenAI's GPT models, you can access models from Meta (Llama), Mistral, Microsoft (Phi), Cohere, and others. This lets you compare and choose the best model for your specific use case, considering factors like performance, cost, and licensing.
Explore
Task 1: Understand the Hub + Project model
Azure AI Foundry uses a hierarchical structure for organization:
| Component | Purpose | Analogy |
|---|---|---|
| Hub | Shared infrastructure and governance | An office building |
| Project | Individual team workspace | A team's floor/suite |
| Model deployment | A running model ready to accept requests | A service desk |
| Connection | Link to external resources (storage, APIs) | Network cables |
Task 2: Explore the model catalog
Navigate to: ai.azure.com → Model catalog
The model catalog offers models from multiple providers:
| Provider | Example Models | Strengths |
|---|---|---|
| OpenAI | GPT-4o, GPT-4, GPT-3.5-Turbo, DALL-E | General purpose, strong reasoning |
| Meta | Llama 3.1, Llama 3 | Open-source, customizable |
| Mistral | Mistral Large, Mistral Small | Efficient, multilingual |
| Microsoft | Phi-3, Phi-3.5 | Small models, efficient for specific tasks |
| Cohere | Command R+ | Enterprise search, RAG scenarios |
Model catalog features:
- Model cards — Description, capabilities, limitations for each model
- Benchmarks — Performance comparisons across tasks
- Deployment options — Serverless API, managed compute, or self-hosted
- Licensing info — Open-source vs. proprietary terms
- Try it — Test models directly in the catalog before deploying
Task 3: Understand deployment options
Azure AI Foundry offers different ways to deploy models:
| Deployment Type | Description | When to Use |
|---|---|---|
| Serverless API (MaaS) | Pay-per-token, no infrastructure management | Quick start, variable workloads |
| Managed Compute | Dedicated compute with model hosted for you | Predictable workloads, custom models |
| Azure OpenAI deployment | Via Azure OpenAI Service resource | OpenAI models with enterprise features |
Serverless API (Models as a Service) is especially noteworthy:
- No need to provision compute
- Pay only for tokens consumed
- Models from Meta, Mistral, and others available this way
- Fast to set up — get an endpoint in minutes
Task 4: Explore prompt flow and evaluation
Azure AI Foundry includes tools for building and evaluating AI applications:
Prompt Flow — Visual tool for building LLM application workflows:
- Chain multiple LLM calls together
- Add data processing steps between calls
- Include branching logic
- Connect to external data sources
- Test and debug flows visually
Evaluation — Measure AI application quality:
- Groundedness — Are responses based on provided data?
- Relevance — Do responses answer the question?
- Coherence — Are responses logically structured?
- Fluency — Is the language natural?
- Safety — Does the output avoid harmful content?
Your task: Consider a customer support chatbot. What evaluation metrics would matter most? (Groundedness and relevance — you want accurate answers based on real documentation, not hallucinated responses.)
For the exam, remember that Azure AI Foundry is the platform that brings everything together — model selection, prompt engineering, evaluation, and deployment. It's not a model itself, but the environment where you work with models.
Key Concepts
| Concept | Definition |
|---|---|
| Azure AI Foundry | Microsoft's unified platform for building, evaluating, and deploying generative AI applications |
| Hub | Top-level container for shared resources, compute, connections, and security governance |
| Project | Team workspace within a hub for building AI solutions |
| Model catalog | Curated collection of AI models from multiple providers (OpenAI, Meta, Mistral, Microsoft, etc.) |
| Prompt flow | Visual tool for building multi-step LLM application workflows |
| Models as a Service (MaaS) | Serverless pay-per-token deployment requiring no infrastructure management |
Common Misconceptions
| Misconception | Reality |
|---|---|
| Azure AI Foundry only offers OpenAI models | The model catalog includes models from Meta, Mistral, Microsoft, Cohere, and other providers |
| Azure AI Foundry replaces Azure OpenAI Service | They work together — Azure OpenAI Service provides the models; AI Foundry is the development platform |
| You need a hub for every project | Multiple projects share a single hub; the hub provides shared governance and resources |
| All models in the catalog are free to use | Models have different pricing; some are pay-per-token, others require dedicated compute |
| Prompt flow requires coding | Prompt flow provides a visual, low-code interface for building LLM workflows (though code can be added) |
Knowledge Check
1. What is the purpose of the model catalog in Azure AI Foundry?
2. In Azure AI Foundry, what is the relationship between a hub and a project?
3. A company wants to quickly test a Meta Llama model without provisioning any compute infrastructure. Which deployment option should they choose?
4. What does the "groundedness" evaluation metric measure in Azure AI Foundry?
5. Which statement about Azure AI Foundry is correct?