Skip to main content

Challenge 28: Face Detection and Analysis

Estimated Time

45 min | Cost: $1-3 (estimated) | Domain: Implement Computer Vision Solutions (10-15%)

Limited Access

Face identification and verification features require Limited Access approval. This challenge focuses on detection features available without approval.

Exam skills covered​

  • Implement facial detection solutions
  • Detect faces and analyze face attributes
  • Understand responsible AI limitations on face services

Overview​

Azure AI Face service provides face detection with attribute analysis. Detection is available without restrictions; identification/verification require approval.

Detection attributes (available without Limited Access):

  • Face location (bounding box)
  • Head pose (pitch, roll, yaw)
  • Blur level (low, medium, high)
  • Exposure level (underExposure, goodExposure, overExposure)
  • Noise level
  • Occlusion (forehead, eyes, mouth occluded)
  • Accessories (headwear, glasses)
  • Quality for recognition

Restricted features (require Limited Access approval):

  • Face identification (1:N matching)
  • Face verification (1:1 matching)
  • PersonGroup management

Prerequisites​

  • Azure subscription
  • Azure AI Face resource
  • Python 3.9+ or .NET 8
  • Package: azure-ai-vision-face (v1.0+)

Implementation​

Task 1: Create Face Resource​

az group create --name rg-ai102-face --location eastus2

az cognitiveservices account create \
  --name face-ai102 \
  --resource-group rg-ai102-face \
  --kind Face \
  --sku S0 \
  --location eastus2

FACE_ENDPOINT=$(az cognitiveservices account show --name face-ai102 --resource-group rg-ai102-face --query properties.endpoint -o tsv)
FACE_KEY=$(az cognitiveservices account keys list --name face-ai102 --resource-group rg-ai102-face --query key1 -o tsv)

Task 2: Detect Faces and Analyze Attributes​

import os
from azure.ai.vision.face import FaceClient
from azure.ai.vision.face.models import (
    FaceDetectionModel,
    FaceRecognitionModel,
    FaceAttributeTypeDetection03
)
from azure.core.credentials import AzureKeyCredential

client = FaceClient(
    endpoint=os.environ["AZURE_AI_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_AI_KEY"])
)

# Detect faces with attributes
image_url = "https://learn.microsoft.com/azure/ai-services/computer-vision/media/face-detection/face-landmarks-annotated.png"

detected_faces = client.detect_from_url(
    url=image_url,
    detection_model=FaceDetectionModel.DETECTION_03,
    recognition_model=FaceRecognitionModel.RECOGNITION_04,
    return_face_id=False,
    return_face_attributes=[
        FaceAttributeTypeDetection03.HEAD_POSE,
        FaceAttributeTypeDetection03.BLUR,
        FaceAttributeTypeDetection03.EXPOSURE,
        FaceAttributeTypeDetection03.NOISE,
        FaceAttributeTypeDetection03.MASK,
        FaceAttributeTypeDetection03.QUALITY_FOR_RECOGNITION
    ]
)

print(f"Detected {len(detected_faces)} face(s):\n")

for i, face in enumerate(detected_faces):
    rect = face.face_rectangle
    print(f"Face {i+1}:")
    print(f"  Bounding box: left={rect.left}, top={rect.top}, "
          f"width={rect.width}, height={rect.height}")
    
    attrs = face.face_attributes
    if attrs:
        # Head pose
        pose = attrs.head_pose
        print(f"  Head pose: pitch={pose.pitch:.1f}, roll={pose.roll:.1f}, yaw={pose.yaw:.1f}")
        
        # Image quality attributes
        print(f"  Blur: {attrs.blur.blur_level} (value: {attrs.blur.value:.3f})")
        print(f"  Exposure: {attrs.exposure.exposure_level} (value: {attrs.exposure.value:.3f})")
        print(f"  Noise: {attrs.noise.noise_level} (value: {attrs.noise.value:.3f})")
        
        # Mask detection
        print(f"  Mask: type={attrs.mask.type}, covers_nose_and_mouth={attrs.mask.nose_and_mouth_covered}")
        
        # Recognition quality
        print(f"  Quality for recognition: {attrs.quality_for_recognition}")
    print()

Task 3: Detect Faces in Local Image​

# Detect from local file
with open("group-photo.jpg", "rb") as f:
    image_data = f.read()

faces = client.detect(
    image_content=image_data,
    detection_model=FaceDetectionModel.DETECTION_03,
    recognition_model=FaceRecognitionModel.RECOGNITION_04,
    return_face_id=False,
    return_face_attributes=[
        FaceAttributeTypeDetection03.HEAD_POSE,
        FaceAttributeTypeDetection03.QUALITY_FOR_RECOGNITION
    ]
)

print(f"Found {len(faces)} faces in group photo")
for i, face in enumerate(faces):
    quality = face.face_attributes.quality_for_recognition
    print(f"  Face {i+1}: quality={quality} {'✓' if quality == 'high' else '⚠'}")

Expected Output​

Detected 1 face(s):

Face 1:
  Bounding box: left=194, top=98, width=242, height=325
  Head pose: pitch=-2.3, roll=1.5, yaw=-4.2
  Blur: low (value: 0.042)
  Exposure: goodExposure (value: 0.623)
  Noise: low (value: 0.089)
  Mask: type=noMask, covers_nose_and_mouth=False
  Quality for recognition: high

Found 5 faces in group photo
  Face 1: quality=high ✓
  Face 2: quality=high ✓
  Face 3: quality=medium ⚠
  Face 4: quality=high ✓
  Face 5: quality=low ⚠

Break & fix​

ScenarioSymptomRoot CauseFix
No faces detectedEmpty array returnedFace too small (< 36x36 px) or severely occludedEnsure faces are at least 36x36 pixels; use higher resolution
403 Forbidden on identifyAccess deniedFeature requires Limited Access approvalApply at https://aka.ms/facerecognition; use detection only
InvalidImage error400 Bad RequestImage format unsupported or corruptedUse JPEG, PNG, GIF, or BMP; max 6MB
Wrong detection modelAttributes missingDetection_01 doesn't support all attributesUse detection_03 for latest attribute support
Inconsistent resultsDifferent face countsDetection model differencesStick to one detection model consistently

Knowledge Check​

1. Which Face service features require Limited Access approval?

2. What is the minimum detectable face size in Azure Face service?

3. Which detection model should you use for the most complete attribute support?

4. What does 'qualityForRecognition' indicate?

5. What is the difference between face detection and face identification?

Cleanup​

az group delete --name rg-ai102-face --yes --no-wait

Learn More​