metaverseDubai/docs/METAHUMAN_PIPELINE.md

MetaHuman Pipeline - Dubai Metaverse

Overview

This document describes the MetaHuman creation and integration pipeline for the Dubai Metaverse project.

MetaHuman Framework

What are MetaHumans?

MetaHumans are Unreal Engine's high-fidelity digital humans created using the MetaHuman Creator. They provide photorealistic human characters with advanced facial animation capabilities.

MetaHuman Creator

Tool: MetaHuman Creator (web-based or plugin)

Features:

• High-quality human models
• Extensive customization
• Realistic facial features
• Hair and clothing options

---

MetaHuman Creation

Step 1: Access MetaHuman Creator

1. Epic Games Account: Sign in to Epic Games account
2. MetaHuman Creator: Access via web or Unreal plugin
3. Create New MetaHuman: Start new MetaHuman creation

Step 2: Base Character

1. Preset Selection: Choose base preset
2. Gender: Select gender
3. Age: Adjust age
4. Ethnicity: Select ethnicity (Dubai multicultural mix)

Step 3: Facial Features

1. Face Shape: Adjust face shape
2. Eyes: Customize eyes
3. Nose: Customize nose
4. Mouth: Customize mouth
5. Ears: Customize ears
6. Skin: Adjust skin tone and features

Step 4: Hair and Appearance

1. Hair: Select hair style and color
2. Eyebrows: Customize eyebrows
3. Facial Hair: Add facial hair (if applicable)
4. Makeup: Add makeup (if applicable)

Step 5: Clothing

1. Outfit: Select clothing style
2. Colors: Adjust clothing colors
3. Accessories: Add accessories (optional)

Step 6: Export

1. Review: Review MetaHuman
2. Export: Export to Unreal Engine
3. Save: Save MetaHuman asset

---

Import to Unreal Engine

Import Process

1. MetaHuman Plugin: Ensure MetaHuman plugin is enabled
2. Import: Import MetaHuman from MetaHuman Creator
3. Validation: Verify import is successful
4. Placement: Place in level

Asset Organization

Content/Assets/Characters/MetaHumans/
├── MH_Dubai_01/
│   ├── SK_Character_Male_01
│   ├── [Animation assets]
│   └── [Material assets]
├── MH_Dubai_02/
└── ...

---

Animation Setup

Animation Blueprints

ABP_NPC_Walk

Walking animation blueprint

• Locomotion: Walking animations
• Blend Spaces: Speed-based blending
• Transitions: Smooth transitions

ABP_NPC_Sit

Sitting animation blueprint

• Idle Sitting: Sitting idle animations
• Sitting Variations: Different sitting poses

ABP_NPC_Phone

Phone usage animation blueprint

• Phone Usage: Using phone animations
• Variations: Different phone poses

ABP_NPC_Gestures

Gesture animation blueprint

• Gestures: Hand gestures, talking
• Variations: Different gesture types

Animation Assets

• AM_Walking: Walking animation
• AM_Sitting: Sitting animation
• AM_PhoneUsage: Phone usage animation
• AM_Gestures: Gesture animations

---

Behavior Trees

BT_NPC_Wander

Wandering behavior for NPCs

Behavior

1. Random Movement: Move to random locations
2. Wait: Wait at locations
3. Repeat: Repeat wandering pattern

Implementation

• Blackboard: Store target location
• Tasks: Move to location, wait
• Decorators: Conditions for behavior

BT_NPC_Social

Social behavior for NPCs

Behavior

1. Group Formation: Form groups with other NPCs
2. Conversation: Simulate conversation
3. Gestures: Use gestures during conversation

Implementation

• Blackboard: Store group members, conversation state
• Tasks: Group formation, conversation
• Decorators: Social conditions

BT_NPC_Phone

Phone usage behavior

Behavior

1. Phone Usage: Use phone at intervals
2. Walking with Phone: Walk while using phone
3. Stop and Use: Stop to use phone

Implementation

• Blackboard: Store phone usage state
• Tasks: Phone usage, movement
• Decorators: Phone usage conditions

---

NPC Controller

BP_NPCController

AI controller for NPCs

Components

1. Behavior Tree: Reference to behavior tree
2. Blackboard: NPC state and data
3. Perception: AI perception (sight, hearing)
4. Movement: AI movement component

Settings

• Behavior Tree: Assign behavior tree
• Perception: Configure perception settings
• Movement: Configure movement settings

---

ChatGPT Integration (Optional)

BP_DialogueSystem

ChatGPT-powered dialogue system

Implementation

1. API Integration: Integrate ChatGPT API
2. Dialogue Generation: Generate dynamic dialogue
3. NPC Responses: NPC responds to player
4. Context Awareness: Context-aware dialogue

Setup

1. API Key: Store API key securely (not in repository)
2. API Client: Create API client
3. Dialogue Handler: Handle dialogue requests
4. Response Processing: Process API responses

Usage

1. Player Interaction: Player interacts with NPC
2. Dialogue Request: Request dialogue from ChatGPT
3. NPC Response: NPC speaks generated dialogue
4. Continuation: Continue conversation

---

MetaHuman Animator

Facial Capture

Purpose: Capture facial animation for MetaHumans

Setup

1. MetaHuman Animator Plugin: Enable plugin
2. Camera Setup: Set up camera for capture
3. Calibration: Calibrate facial capture
4. Recording: Record facial animation

Usage

1. Record: Record facial performance
2. Process: Process facial capture
3. Apply: Apply to MetaHuman
4. Refine: Refine animation

---

NPC Placement

Placement Strategy

1. Density: Moderate NPC density
2. Distribution: Distribute across district
3. Groups: Place NPCs in social groups
4. Activities: Place NPCs in activity areas

Areas

• Marina Promenade: Pedestrians walking
• Cafes: NPCs sitting, socializing
• Roads: NPCs walking along roads
• Parks: NPCs in parks, green spaces

---

Performance Optimization

NPC Optimization

1. LODs: Use LODs for distant NPCs
2. Culling: Cull NPCs outside view
3. Instancing: Use instanced meshes where possible
4. Animation: Optimize animation complexity

Behavior Optimization

1. Update Frequency: Optimize behavior tree updates
2. Perception: Optimize perception checks
3. Pathfinding: Optimize pathfinding

---

Testing

NPC Testing

1. Behavior: Test NPC behavior
2. Animation: Test animations
3. Performance: Test performance with NPCs
4. Interactions: Test NPC interactions

Dialogue Testing

1. ChatGPT Integration: Test API integration
2. Dialogue Quality: Test dialogue quality
3. Response Time: Test response time
4. Context: Test context awareness

---

Troubleshooting

Common Issues

Issue: MetaHuman not importing

• Solution: Check MetaHuman plugin is enabled
• Solution: Verify MetaHuman Creator connection

Issue: Animations not working

• Solution: Check animation blueprint setup
• Solution: Verify animation assets are assigned

Issue: Behavior tree not executing

• Solution: Check behavior tree assignment
• Solution: Verify blackboard setup

Issue: Performance issues

• Solution: Reduce NPC count
• Solution: Use LODs
• Solution: Optimize behavior trees

---

Documentation

MetaHuman Catalog

Document each MetaHuman:

• Name
• Appearance
• Clothing
• Behavior
• Usage

Behavior Documentation

Document behavior trees:

• Behavior description
• Blackboard variables
• Tasks and decorators
• Usage

---

Version: 1.0 Last Updated: [Current Date]