Skip to main content

Challenge 44: Semantic and Vector Search

Estimated Time

60-75 min | Cost: ~$1.50 (Basic tier Search + OpenAI embeddings) | Domain: Knowledge Mining & Extraction (15-20%)

Tier Requirement

Semantic ranking requires Basic tier or higher for Azure AI Search. Vector search requires Basic tier or higher. The Free tier does not support these features.

Exam skills covered​

SkillWeight
Configure semantic ranking on an indexHigh
Implement vector search with embeddingsHigh
Execute hybrid search queries (keyword + vector)High
Generate embeddings with Azure OpenAIHigh
Configure vector search profiles and algorithmsMedium

Overview​

Semantic ranking​

Semantic ranking re-ranks initial keyword search results using deep learning models that understand the meaning and context of queries and documents. It doesn't change which documents match — it re-orders the top results for better relevance.

Vector search finds documents based on mathematical similarity between vector embeddings (dense numerical representations). Unlike keyword search, it finds semantically similar content even without shared terms.

Combines keyword (BM25) scoring with vector similarity, producing the best of both approaches. A Reciprocal Rank Fusion (RRF) algorithm merges the ranked lists.

ApproachFinds "car" when query is "automobile"Exact phrase matchingBest for
Keyword only❌✅Known-item lookup
Vector only✅❌Semantic similarity
Hybrid✅✅Production RAG scenarios

Prerequisites​

  • Azure AI Search (Basic tier or higher)
  • Azure OpenAI with text-embedding-ada-002 or text-embedding-3-small deployed
  • Python 3.9+ with azure-search-documents>=11.4.0, openai>=1.0.0
  • .NET 8 with Azure.Search.Documents, Azure.AI.OpenAI

Implementation​

Task 1: Deploy an embedding model​

# Create Azure OpenAI resource (if not already done)
AOAI_NAME="aoai-ai102-$(openssl rand -hex 4)"
az cognitiveservices account create \
  --name $AOAI_NAME \
  --resource-group $RG \
  --location eastus \
  --kind OpenAI \
  --sku S0 \
  --yes

# Deploy text-embedding-3-small model
az cognitiveservices account deployment create \
  --name $AOAI_NAME \
  --resource-group $RG \
  --deployment-name "text-embedding-3-small" \
  --model-name "text-embedding-3-small" \
  --model-version "1" \
  --model-format OpenAI \
  --sku-capacity 120 \
  --sku-name Standard

AOAI_ENDPOINT=$(az cognitiveservices account show \
  --name $AOAI_NAME --resource-group $RG \
  --query "properties.endpoint" -o tsv)

AOAI_KEY=$(az cognitiveservices account keys list \
  --name $AOAI_NAME --resource-group $RG \
  --query "key1" -o tsv)

Task 2: Create a vector-enabled index​

from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndex,
    SearchField,
    SearchFieldDataType,
    SimpleField,
    SearchableField,
    VectorSearch,
    HnswAlgorithmConfiguration,
    VectorSearchProfile,
    SemanticConfiguration,
    SemanticSearch,
    SemanticPrioritizedFields,
    SemanticField,
)

index_client = SearchIndexClient(endpoint=endpoint, credential=credential)

# Define fields including a vector field
fields = [
    SimpleField(name="id", type=SearchFieldDataType.String, key=True, filterable=True),
    SearchableField(name="title", type=SearchFieldDataType.String, filterable=True, sortable=True),
    SearchableField(name="content", type=SearchFieldDataType.String),
    SimpleField(name="category", type=SearchFieldDataType.String, filterable=True, facetable=True),
    SearchField(
        name="contentVector",
        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
        searchable=True,
        vector_search_dimensions=1536,
        vector_search_profile_name="vector-profile"
    ),
]

# Configure vector search
vector_search = VectorSearch(
    algorithms=[
        HnswAlgorithmConfiguration(name="hnsw-config"),
    ],
    profiles=[
        VectorSearchProfile(
            name="vector-profile",
            algorithm_configuration_name="hnsw-config",
        )
    ]
)

# Configure semantic search
semantic_config = SemanticConfiguration(
    name="semantic-config",
    prioritized_fields=SemanticPrioritizedFields(
        title_field=SemanticField(field_name="title"),
        content_fields=[SemanticField(field_name="content")]
    )
)

semantic_search = SemanticSearch(configurations=[semantic_config])

# Create the index
index = SearchIndex(
    name="vector-index",
    fields=fields,
    vector_search=vector_search,
    semantic_search=semantic_search
)

result = index_client.create_or_update_index(index)
print(f"Index '{result.name}' created with vector and semantic search")

Task 3: Generate embeddings and upload documents​

from openai import AzureOpenAI
from azure.search.documents import SearchClient

# Initialize OpenAI client for embeddings
aoai_client = AzureOpenAI(
    api_key=AOAI_KEY,
    api_version="2024-06-01",
    azure_endpoint=AOAI_ENDPOINT
)

def get_embedding(text: str) -> list[float]:
    """Generate embedding vector for text."""
    response = aoai_client.embeddings.create(
        input=text,
        model="text-embedding-3-small"
    )
    return response.data[0].embedding

# Sample documents
documents = [
    {"id": "1", "title": "Azure AI Search Overview", "content": "Azure AI Search is a cloud search service with built-in AI capabilities for indexing and querying.", "category": "search"},
    {"id": "2", "title": "Vector Databases Explained", "content": "Vector databases store data as high-dimensional vectors enabling similarity search using mathematical distance.", "category": "database"},
    {"id": "3", "title": "RAG Pattern Architecture", "content": "Retrieval Augmented Generation combines search retrieval with large language model generation for accurate responses.", "category": "ai"},
    {"id": "4", "title": "Natural Language Processing", "content": "NLP enables computers to understand human language through tokenization, embeddings, and transformer models.", "category": "ai"},
]

# Generate embeddings for each document
for doc in documents:
    doc["contentVector"] = get_embedding(doc["content"])

# Upload to index
search_client = SearchClient(endpoint=endpoint, index_name="vector-index", credential=credential)
result = search_client.upload_documents(documents)
print(f"Uploaded {len(result)} documents")

Task 4: Execute vector, semantic, and hybrid queries​

from azure.search.documents.models import VectorizedQuery

# Pure vector search
query_embedding = get_embedding("How do I find similar documents?")
vector_query = VectorizedQuery(
    vector=query_embedding,
    k_nearest_neighbors=5,
    fields="contentVector"
)

results = search_client.search(
    search_text=None,
    vector_queries=[vector_query],
    select=["id", "title", "category"]
)
print("=== Vector Search Results ===")
for result in results:
    print(f"  Score: {result['@search.score']:.4f} | {result['title']}")

# Hybrid search (keyword + vector)
results = search_client.search(
    search_text="search service cloud AI",
    vector_queries=[vector_query],
    select=["id", "title", "category"],
    top=5
)
print("\n=== Hybrid Search Results ===")
for result in results:
    print(f"  Score: {result['@search.score']:.4f} | {result['title']}")

# Hybrid + Semantic ranking
results = search_client.search(
    search_text="How does retrieval augmented generation work?",
    vector_queries=[vector_query],
    query_type="semantic",
    semantic_configuration_name="semantic-config",
    query_caption="extractive",
    select=["id", "title", "content"],
    top=5
)
print("\n=== Hybrid + Semantic Results ===")
for result in results:
    print(f"  Score: {result['@search.score']:.4f} | Reranker: {result.get('@search.reranker_score', 'N/A')} | {result['title']}")
    if result.get("@search.captions"):
        print(f"    Caption: {result['@search.captions'][0].text}")

Expected Output​

=== Hybrid + Semantic Results ===
  Score: 0.0322 | Reranker: 3.42 | RAG Pattern Architecture
    Caption: Retrieval Augmented Generation combines search retrieval with LLM generation...
  Score: 0.0298 | Reranker: 2.87 | Azure AI Search Overview
  Score: 0.0241 | Reranker: 1.95 | Vector Databases Explained
  Score: 0.0189 | Reranker: 1.12 | Natural Language Processing

Break & fix​

#ScenarioSymptomRoot CauseFix
1Vector field dimension mismatchUpload fails: "Vector field dimension mismatch"Document vector has 768 dimensions but index expects 1536Use the same model for indexing and querying; match dimensions in index to model output (ada-002=1536, 3-small default=1536)
2Semantic ranker returns no captionsResults lack @search.captionsqueryAnswer or queryCaption not requestedAdd query_caption="extractive" to search options
3Hybrid search ignores vectorResults identical to keyword-onlyvectorQueries array is empty or fields property doesn't match index field nameEnsure fields matches the vector field name in index (contentVector)
4"Semantic search not available" errorHTTP 400 mentioning semantic configurationSearch service is Free tier — semantic requires Basic+Upgrade to Basic tier or higher
5Poor vector search relevanceIrrelevant results returnedQuery text is embedded with different model than documentsUse the same embedding model for both indexing and querying

Knowledge Check​

1. You implement hybrid search combining keyword and vector queries. How are the two result sets merged into a single ranked list?

2. You configure a vector field with dimensions: 1536 and vectorSearchProfile: 'my-profile'. The profile uses HNSW algorithm. What does HNSW stand for and what does it optimize?

3. What is the key difference between semantic ranking and vector search?

4. You need to generate embeddings for a vector search index. Which Azure OpenAI model is purpose-built for this?

5. You execute a vector query with k=5 and also set top=3 in search options. How many results are returned?

Cleanup​

az group delete --name rg-ai102-search --yes --no-wait

Learn More​