metaverseDubai/PIPELINE.md

Development Pipeline - Dubai Metaverse

Overview

This document outlines the complete development pipeline for the Dubai Metaverse project, including asset creation workflows, import processes, and iteration cycles.

Pipeline Stages

1. Pre-Production

Concept & Planning

• Input: Art bible, district selection, reference images
• Output: Asset list, technical specifications, style guides
• Tools: Photoshop, Midjourney, Stable Diffusion
• Deliverables: Concept art, moodboards, asset breakdown

Data Acquisition

• Input: OpenStreetMap, GIS data, elevation data
• Output: Geospatial data, terrain data, building footprints
• Tools: Python scripts, GIS software, RealityCapture
• Deliverables: Processed data, terrain mesh, building outlines

---

2. Asset Creation

3D Modeling

Workflow:

1. Blockout: Create basic shape in Blender/Maya
2. High-Poly: Add detail, create high-poly version
3. Low-Poly: Create optimized low-poly (if not using Nanite)
4. UV Mapping: Unwrap UVs, prepare for texturing
5. Export: Export to FBX or direct to Unreal

Tools:

• Blender (free, recommended)
• Maya (professional)
• Houdini (procedural generation)

Standards:

• Hero Assets: High-poly only (Nanite)
• Primary Buildings: High-poly + low-poly with LODs
• Background: Simplified geometry with LODs
• Format: FBX 2020 or newer

Naming Convention: See NAMING_CONVENTIONS.md

---

Texturing

Workflow:

1. Bake Maps: Bake normal, AO, curvature maps from high-poly
2. Texture Creation: Create textures in Substance Painter
3. Material Setup: Set up PBR materials
4. Export: Export texture sets
5. Import: Import to Unreal, create materials

Tools:

• Substance Painter (primary)
• Substance Designer (procedural materials)
• Photoshop (additional editing)

Texture Sets:

• Base Color: Albedo/diffuse
• Normal: Normal map
• Roughness: Surface roughness
• Metallic: Metallic map
• AO: Ambient occlusion
• Additional: Emissive, height (where needed)

Resolution Standards:

• Hero Assets: 8K (8192x8192) - UDIM workflow
• Primary Buildings: 4K (4096x4096)
• Background: 2K (2048x2048)

Format:

• Export: PNG or TGA (16-bit)
• Import: BC7 (DXT5) for color, BC5 for normals

---

Photogrammetry (Optional)

Workflow:

1. Photography: Capture 100-500 photos of building
2. Processing: Process in RealityCapture or Luma AI
3. Cleanup: Clean mesh, fix holes, optimize
4. Texturing: Generate textures from photos
5. Export: Export mesh and textures
6. Import: Import to Unreal, convert to Nanite

Tools:

• RealityCapture (professional)
• Luma AI (simplified)
• Agisoft Metashape (alternative)

Requirements:

• High-quality camera (DSLR or mirrorless)
• Consistent lighting
• Overlapping photos (70-80% overlap)
• Multiple angles

---

3. Procedural Generation

Houdini Workflow

Process:

1. Create HDA: Build Houdini Digital Asset
2. Parameters: Set up exposed parameters
3. Export: Export HDA file
4. Import: Import HDA to Unreal via Houdini Engine
5. Instance: Create instances in level
6. Bake: Bake to static meshes (optional)

Use Cases:

• Modular building generation
• Road network generation
• Terrain sculpting
• Complex procedural geometry

Tools:

• Houdini (Houdini Engine plugin in Unreal)

---

PCG (Procedural Content Generation) Workflow

Process:

1. Create Graph: Create PCG graph in Unreal
2. Define Rules: Set up placement rules
3. Input Data: Provide input (splines, volumes, points)
4. Generate: Execute graph to generate content
5. Iterate: Adjust rules and regenerate
6. Bake: Bake to static (optional, for performance)

Use Cases:

• Building placement
• Road props (lamp posts, barriers)
• Vegetation placement
• Traffic generation

Tools:

• Unreal Engine PCG Framework (built-in)

---

4. Import to Unreal

Static Mesh Import

Process:

1. Import Settings: Configure import settings
2. Nanite: Enable Nanite if applicable
3. Materials: Assign materials or create new
4. LODs: Generate LODs if not using Nanite
5. Collision: Generate collision meshes
6. Validation: Validate mesh and materials

Import Settings:

• Nanite: Enable for high-poly static meshes
• LODs: Generate LODs for non-Nanite meshes
• Collision: Auto-generate or use custom
• Lightmap UVs: Generate if needed

Validation Checklist:

• ✅ Mesh imports without errors
• ✅ Materials assigned correctly
• ✅ Nanite enabled (if applicable)
• ✅ Collision generated
• ✅ Scale is correct
• ✅ Naming follows conventions

---

Texture Import

Process:

1. Import: Import texture files
2. Compression: Set compression settings
3. sRGB: Configure sRGB (color maps only)
4. Mip Maps: Generate mip maps
5. Virtual Texturing: Enable if using virtual textures

Settings:

• Base Color: sRGB enabled, BC7 compression
• Normal: sRGB disabled, BC5 compression
• Roughness/Metallic: sRGB disabled, BC7 compression
• AO: sRGB disabled, BC7 compression

---

Material Creation

Process:

1. Create Material: Create material asset
2. Texture Assignment: Assign texture maps
3. Parameters: Expose parameters for instances
4. Shading Model: Set shading model (Default Lit)
5. Blend Mode: Set blend mode (Opaque, Masked, Translucent)
6. Test: Test material in level

Material Instances:

• Create material instances for variations
• Expose parameters (color, roughness, etc.)
• Use instances for similar materials

---

5. Level Assembly

World Partition Setup

Process:

1. Enable World Partition: Enable in project settings
2. Cell Size: Configure cell size (128m default)
3. Data Layers: Create data layers for organization
4. Streaming: Set up streaming volumes
5. Validation: Test streaming performance

Data Layer Organization:

• Buildings
• Props
• Vegetation
• Lighting
• Vehicles
• NPCs

---

Placement

Process:

1. Manual Placement: Place assets manually in level
2. PCG Placement: Use PCG for procedural placement
3. Spline Placement: Use splines for roads, paths
4. Validation: Validate placement and scale

Placement Tools:

• Unreal Editor (manual)
• PCG Framework (procedural)
• Spline tools (roads, paths)

---

6. Lighting & Atmosphere

Lighting Setup

Process:

1. Sky Light: Set up HDRI sky
2. Directional Light: Configure sun light
3. Lumen: Enable Lumen GI and reflections
4. Day/Night Cycle: Set up time of day system
5. Atmosphere: Configure volumetric fog
6. Post-Process: Set up post-process volume

Lighting Workflow:

• Start with HDRI sky
• Add directional light (sun)
• Adjust Lumen settings
• Set up day/night cycle
• Add atmosphere and fog
• Configure post-process

---

7. Gameplay Systems

Blueprint Creation

Process:

1. Design: Plan blueprint functionality
2. Create: Create blueprint class
3. Implement: Implement logic
4. Test: Test in level
5. Iterate: Refine based on testing

Blueprint Organization:

• Gameplay (player, interactions)
• Vehicles (Chaos vehicles)
• NPCs (AI, behavior)
• Systems (day/night, weather)

---

8. Optimization

Performance Optimization

Process:

1. Profile: Profile performance (Unreal Insights)
2. Identify: Identify bottlenecks
3. Optimize: Apply optimizations
4. Validate: Validate performance improvement
5. Iterate: Repeat as needed

Optimization Techniques:

• LOD generation and assignment
• Nanite clustering
• Material optimization
• Lightmap optimization
• Streaming optimization
• Draw call reduction

---

9. Cinematic Rendering

Sequencer Workflow

Process:

1. Create Sequence: Create Sequencer asset
2. Add Cameras: Add camera actors
3. Animate: Animate cameras and objects
4. Configure: Configure render settings
5. Render: Render with Movie Render Queue

Movie Render Queue:

• Set output format (EXR, MP4)
• Configure resolution (8K, 4K)
• Set up additional passes
• Configure anti-aliasing
• Render sequence

---

Iteration Cycle

Asset Iteration

1. Create/Update: Create or update asset in DCC tool
2. Export: Export to Unreal format
3. Import: Import to Unreal
4. Review: Review in Unreal
5. Feedback: Gather feedback
6. Iterate: Return to step 1 if needed

Level Iteration

1. Place/Update: Place or update assets in level
2. Test: Test in editor or play mode
3. Review: Review visual quality and performance
4. Feedback: Gather feedback
5. Iterate: Return to step 1 if needed

---

Quality Control

Asset Validation

Checklist:

• ✅ Naming follows conventions
• ✅ Scale is correct
• ✅ Materials are assigned
• ✅ Textures are correct resolution
• ✅ LODs are generated (if needed)
• ✅ Collision is set up
• ✅ Performance is acceptable

Level Validation

Checklist:

• ✅ All assets placed correctly
• ✅ Lighting is correct
• ✅ Performance targets met
• ✅ No visual artifacts
• ✅ Interactions work correctly
• ✅ Streaming works correctly

---

Automation

Scripts

• Asset Validation: scripts/validate_assets.sh
• Texture Validation: scripts/texture_validation.py
• Performance Audit: scripts/performance_audit.py
• Documentation Generation: scripts/generate_docs.sh

Tools

• Houdini HDAs: Procedural generation tools
• PCG Graphs: Reusable procedural content
• Material Functions: Reusable material logic

---

Version Control

Asset Versioning

• Use Git LFS for large files
• Commit assets after validation
• Use descriptive commit messages
• Tag major milestones

Level Versioning

• Commit level changes regularly
• Use branches for major features
• Merge after review and testing

---

Best Practices

1. Always Validate: Validate assets before committing
2. Follow Naming: Follow naming conventions strictly
3. Document Changes: Document significant changes
4. Test Regularly: Test in level regularly
5. Optimize Early: Optimize for performance early
6. Iterate Quickly: Short iteration cycles
7. Communicate: Communicate changes to team

---

Version: 1.0 Last Updated: [Current Date]