proxmox/docs/04-configuration/UDM_PRO_BROWSER_AUTOMATION_PROGRESS.md

UDM Pro Browser Automation Progress

Last Updated: 2026-01-14
Status: In Progress - Page Mapping and Button Detection

---

Summary

We've successfully implemented browser automation for UDM Pro routing configuration using Playwright. The script can:

• ✅ Authenticate with UDM Pro
• ✅ Navigate to Routing & Firewall settings
• ✅ Detect page structure and state
• ✅ Find buttons using multiple detection strategies
• ⚠️ Challenge: Finding the correct "Add Route" button

---

Completed Features

1. Authentication ✅

• Successfully logs in with unifi_api account
• Handles SSL certificate issues
• Comprehensive error handling

2. Navigation ✅

• Successfully navigates to Settings → Routing & Firewall → Static Routes
• Handles page redirects
• Waits for page load and API responses

3. Page Structure Understanding ✅

• Maps page sections, tables, and buttons
• Understands button context (toolbar, table header, routes section)
• Detects page state and loading indicators

4. Button Detection ✅

• Multiple detection strategies:
  • Text-based selectors (50+ variations)
  • Aria-label and data-testid selectors
  • Class-based selectors
  • Icon-based detection
  • Position-based detection
  • JavaScript evaluation
  • XPath selectors
• Priority-based button selection
• Theme button filtering

5. Menu/Dropdown Handling ✅

• Detects when buttons open menus
• Finds menu items
• Attempts to click "Add Route" options in menus

---

Current Challenge: Add Button Detection

Problem

The script successfully finds buttons on the routing page, but the button it identifies opens a theme menu (with "Light" option) rather than the Add Route form.

Findings

1. Buttons Found:

   • Button 0: "UDM Pro" (Site Switcher)
   • Button 1: Icon-only, class unifi-portal-nboz5b e10mj2ih2 (theme button)
   • Button 2: Icon-only, class unifi-portal-1bmvzvc eqfginb7 button__qx3Rmpxb but
   • Buttons 4-5: "Submit Support Ticket"
   • Button 6: "Go back to Home"
   • Buttons 7-8: Icon-only, disabled

2. Page Structure:

   • No tables detected on the routing page
   • Route-related text not consistently found
   • Buttons are in toolbars/headers, not near route content

3. Possible Reasons:

   • Add button may be in a different location (sidebar, different tab, collapsed section)
   • Add button may only appear when certain conditions are met
   • Add button may use different UI patterns than expected
   • Page may use a different layout than documented

---

Tools Created

1. configure-static-route-playwright.js

Main automation script with:

• Comprehensive button detection
• Menu handling
• Form detection
• Error recovery
• Screenshot capture
• Pause mode for manual intervention

2. inspect-routing-page.js

Debugging tool to list all clickable elements and identify potential Add buttons.

3. map-routing-page-structure.js

Page structure mapper that analyzes sections, buttons, forms, and tables.

4. find-add-button-comprehensive.js

Comprehensive button finder that tests potential Add buttons to see what they do.

5. comprehensive-page-mapper.js

Full page mapper that:

• Maps page at different scroll positions
• Analyzes button hierarchy
• Finds route-related text
• Identifies most likely Add button candidates

---

Next Steps

Immediate Actions

1. Visual Inspection:

   • Run script with HEADLESS=false to visually see the page
   • Manually identify where the Add button is located
   • Note its exact selector, position, and context

2. Screenshot Analysis:

   • Review screenshots in scripts/unifi/screenshots/
   • Identify the Add button location
   • Document its characteristics

3. Page State Investigation:

   • Check if Add button only appears when:
     • Routes table is empty
     • Specific tab is selected
     • Section is expanded
     • User has specific permissions

4. Alternative Approaches:

   • Check if Add button is in a sidebar or different panel
   • Look for keyboard shortcuts (Ctrl+N, etc.)
   • Check if it's a link instead of a button
   • Verify if it requires clicking something else first

Code Improvements

1. Enhanced Page State Detection:

   // Add logic to detect:
   - Empty vs populated routes table
   - Active tab/section
   - Expanded/collapsed sections
   - Permission-based UI differences
   

2. Better Button Prioritization:

   // Improve priority calculation:
   - Check button's actual function (test click)
   - Verify button opens form vs menu
   - Filter out theme/settings buttons more aggressively
   

3. Alternative Navigation:

   // Try different navigation paths:
   - Direct URL to static routes
   - Click through menu items
   - Use keyboard shortcuts
   

---

Usage

Run Main Script

cd /home/intlc/projects/proxmox
UNIFI_USERNAME=unifi_api UNIFI_PASSWORD='L@kers2010$$' \
  HEADLESS=true node scripts/unifi/configure-static-route-playwright.js

Run with Visible Browser (for debugging)

UNIFI_USERNAME=unifi_api UNIFI_PASSWORD='L@kers2010$$' \
  HEADLESS=false node scripts/unifi/configure-static-route-playwright.js

Run Page Mapper

UNIFI_USERNAME=unifi_api UNIFI_PASSWORD='L@kers2010$$' \
  node scripts/unifi/comprehensive-page-mapper.js

---

Environment Variables

• UNIFI_UDM_URL: UDM Pro URL (default: https://192.168.0.1)
• UNIFI_USERNAME: Username (default: unifi_api)
• UNIFI_PASSWORD: Password (required)
• UNIFI_BROWSER_USERNAME: Override username for browser automation
• UNIFI_BROWSER_PASSWORD: Override password for browser automation
• DRY_RUN: Set to true to test without making changes
• HEADLESS: Set to false to see browser (default: true)
• PAUSE_MODE: Set to true to pause at key points for manual intervention

---

Screenshots

Screenshots are automatically saved to scripts/unifi/screenshots/ at each step:

• 01-login-page.png: Login page
• 02-credentials-filled.png: After filling credentials
• 03-after-login.png: After successful login
• 05-routing-page.png: Routing page
• 06-static-routes-page.png: Static Routes page
• 07-before-add-button.png: Before attempting to click Add button
• 08-after-button-click-menu.png: After clicking button (if menu appears)
• error-state.png: Error state

---

Related Documentation

• UDM_PRO_BROWSER_AUTOMATION_GUIDE.md - Setup and usage guide
• UDM_PRO_BROWSER_AUTOMATION_STATUS.md - Status and known issues
• UDM_PRO_ROUTING_API_LIMITATIONS.md - Why browser automation is needed

---

Last Updated: 2026-01-14