Recording & Replay System

BrowseGenius implements a sophisticated test recording and replay system that captures the first test execution and enables fast, deterministic replays on subsequent runs.

Overview

flowchart LR
    A[First Run] --> B[Record Actions]
    B --> C[Capture Network]
    C --> D[Generate JSON]
    D --> E[Store Locally]
    E --> F[Subsequent Runs]
    F --> G[Load JSON]
    G --> H[Replay Actions]
    H --> I[Validate Network]

Recording Phase

Action Recorder Service

The ActionRecorder (src/services/actionRecorder.ts) captures every action during test execution:

What gets recorded:

• Action type (click, input, select, navigate, wait, assertion)
• Target element with multiple selector strategies
• Input values and interaction details
• Screenshots before and after actions
• Execution method (vision vs DOM)
• Success/failure status and error messages
• Timing information

Element selector strategies:

{
  id: "email",                        // Highest priority
  dataTestId: "login-email",          // Test-specific
  name: "email",                      // Form fields
  css: "input[type='email']",         // Generated selector
  xpath: "//input[@id='email']",      // Alternative path
  ariaLabel: "Email address",         // Accessibility
  text: "Email",                      // Visible text
  coordinates: { x: 450, y: 300 }     // Visual fallback
}

Network Monitor Service

The NetworkMonitor (src/services/networkMonitor.ts) uses Chrome DevTools Protocol to capture all HTTP traffic:

Captured data:

• Request URL and method
• Request headers and body
• Response status code
• Response headers and body
• Timing information (start, end, duration)
• Association with specific actions

Setup:

// Attach debugger
await chrome.debugger.attach({ tabId }, '1.3');

// Enable network monitoring
await chrome.debugger.sendCommand({ tabId }, 'Network.enable');

// Listen for network events
chrome.debugger.onEvent.addListener((source, method, params) => {
  if (method === 'Network.requestWillBeSent') {
    // Capture request details
  }
  if (method === 'Network.responseReceived') {
    // Capture response details
  }
});

Recording Workflow

// 1. Start recording
actionRecorder.startRecording(
  testCase.id,
  testCase.title,
  testCase.flowName,
  startUrl
);

// 2. Start network monitoring
await networkMonitor.startMonitoring(tabId);

// 3. Execute test (actions automatically recorded)
for (const step of testCase.steps) {
  // Find element
  const element = await findElement(step);

  // Record action start
  const actionId = await actionRecorder.recordAction(
    'click',
    step.action,
    {
      target: element,
      url: currentUrl,
      pageTitle: document.title,
      executionMethod: 'dom',
      screenshotBefore: await captureScreenshot()
    }
  );

  // Perform action
  await element.click();

  // Update action with result
  actionRecorder.updateAction(actionId, {
    success: true,
    screenshotAfter: await captureScreenshot(),
    networkRequestIds: networkMonitor.getRecentRequests().map(r => r.id)
  });
}

// 4. Stop and save
await networkMonitor.stopMonitoring();
const recording = actionRecorder.stopRecording();

// 5. Store in state
useAppState.getState().testPlanner.actions.addRecording(recording);

// 6. Optional: Download JSON file
await actionRecorder.downloadRecording(recording);

JSON Recording Format

Structure

{
  "id": "rec_1234567890",
  "testCaseId": "tc_login_flow",
  "testCaseName": "User Login and Profile Verification",
  "flowName": "Authentication",
  "recordedAt": "2025-01-12T10:30:00.000Z",
  "browserInfo": {
    "userAgent": "Mozilla/5.0...",
    "viewport": { "width": 1920, "height": 1080 }
  },
  "startUrl": "https://example.com/login",
  "actions": [
    {
      "id": "action_001",
      "stepIndex": 0,
      "timestamp": "2025-01-12T10:30:01.000Z",
      "actionType": "input",
      "description": "Enter email address",
      "target": {
        "id": "email",
        "css": "input[type='email']",
        "xpath": "//input[@id='email']",
        "coordinates": { "x": 450, "y": 300 }
      },
      "value": "user@example.com",
      "url": "https://example.com/login",
      "networkRequests": [...],
      "executionMethod": "dom",
      "success": true
    }
  ],
  "summary": {
    "totalActions": 7,
    "successfulActions": 7,
    "failedActions": 0,
    "duration": 5000,
    "networkRequestCount": 3
  },
  "allNetworkRequests": [...]
}

See the Data & Storage Model documentation for complete schema details.

Replay Phase

Selector Matching Strategy

The replay engine tries selectors in priority order:

async function findElement(target: ElementSelector): Promise<HTMLElement> {
  // 1. Try ID (most stable)
  if (target.id) {
    const el = document.getElementById(target.id);
    if (el) return el;
  }

  // 2. Try data-testid (test-specific)
  if (target.dataTestId) {
    const el = document.querySelector(`[data-testid="${target.dataTestId}"]`);
    if (el) return el;
  }

  // 3. Try name attribute
  if (target.name) {
    const el = document.querySelector(`[name="${target.name}"]`);
    if (el) return el;
  }

  // 4. Try CSS selector
  if (target.css) {
    const el = document.querySelector(target.css);
    if (el) return el;
  }

  // 5. Try XPath
  if (target.xpath) {
    const result = document.evaluate(
      target.xpath,
      document,
      null,
      XPathResult.FIRST_ORDERED_NODE_TYPE,
      null
    );
    if (result.singleNodeValue) return result.singleNodeValue as HTMLElement;
  }

  // 6. Try aria-label
  if (target.ariaLabel) {
    const el = document.querySelector(`[aria-label="${target.ariaLabel}"]`);
    if (el) return el;
  }

  // 7. Try text content
  if (target.text) {
    const elements = document.querySelectorAll('*');
    for (const el of elements) {
      if (el.textContent?.trim() === target.text) {
        return el as HTMLElement;
      }
    }
  }

  // 8. Fallback to coordinates (vision-based)
  if (target.coordinates) {
    return await findElementByCoordinates(target.coordinates);
  }

  throw new Error('Element not found with any selector strategy');
}

Network Request Validation

async function validateNetworkRequests(
  expectedRequests: NetworkRequest[],
  actualRequests: NetworkRequest[]
) {
  const results = {
    matched: [],
    missing: [],
    extra: [],
    statusMismatches: []
  };

  for (const expected of expectedRequests) {
    const actual = actualRequests.find(req =>
      req.url === expected.url &&
      req.method === expected.method
    );

    if (!actual) {
      results.missing.push(expected);
      continue;
    }

    results.matched.push({ expected, actual });

    if (actual.responseStatus !== expected.responseStatus) {
      results.statusMismatches.push({
        url: expected.url,
        expected: expected.responseStatus,
        actual: actual.responseStatus
      });
    }
  }

  // Find extra requests not in recording
  for (const actual of actualRequests) {
    const expected = expectedRequests.find(req =>
      req.url === actual.url &&
      req.method === actual.method
    );
    if (!expected) {
      results.extra.push(actual);
    }
  }

  return results;
}

Replay Workflow

async function replayRecording(
  recording: TestRecording,
  config: ReplayConfig
) {
  // Start network monitoring
  await networkMonitor.startMonitoring(tabId);

  const results = [];

  // Navigate to start URL
  await chrome.tabs.update(tabId, { url: recording.startUrl });
  await waitForPageLoad();

  // Replay each action
  for (const action of recording.actions) {
    try {
      // Find element
      const element = await findElement(action.target);

      // Perform action
      switch (action.actionType) {
        case 'click':
          await element.click();
          break;
        case 'input':
          element.value = action.value as string;
          element.dispatchEvent(new Event('input', { bubbles: true }));
          break;
        case 'select':
          element.value = action.value as string;
          element.dispatchEvent(new Event('change', { bubbles: true }));
          break;
        // ... other action types
      }

      // Wait for network requests
      if (action.networkRequests?.length > 0) {
        await waitForNetworkIdle(config.networkTimeout);
      }

      // Validate network requests
      const actualRequests = networkMonitor.getRecentRequests();
      const validation = await validateNetworkRequests(
        action.networkRequests || [],
        actualRequests
      );

      results.push({
        action: action.id,
        success: true,
        networkValidation: validation
      });

      // Delay before next action
      await sleep(config.actionDelay);

    } catch (error) {
      results.push({
        action: action.id,
        success: false,
        error: error.message
      });

      if (!config.continueOnError) {
        break;
      }
    }
  }

  await networkMonitor.stopMonitoring();

  return {
    recordingId: recording.id,
    success: results.every(r => r.success),
    results
  };
}

Configuration Options

ReplayConfig

{
  // Selector priority order
  selectorPriority: ['id', 'dataTestId', 'css', 'xpath', 'coordinates'],

  // Timing
  actionDelay: 500,           // ms between actions
  networkTimeout: 5000,       // ms to wait for network

  // Validation
  validateNetworkRequests: true,
  validateScreenshots: false,  // Visual comparison (future)

  // Flexibility
  allowSelectorFallback: true,
  allowCoordinateFallback: true,

  // Error handling
  continueOnError: false,
  maxRetries: 3
}

Best Practices

1. Stable Selectors

Use data-testid attributes for test-specific elements:

<input type="email" data-testid="login-email" />
<button type="submit" data-testid="login-submit">Sign In</button>

2. Network Request Matching

Be specific about which requests to validate:

// Only validate critical API calls
const criticalRequests = recording.allNetworkRequests.filter(req =>
  req.url.includes('/api/auth/') ||
  req.url.includes('/api/user/')
);

3. Screenshot Management

Balance detail with storage:

// Option 1: Screenshots only for assertions
const captureScreenshot = action.actionType === 'assertion';

// Option 2: Key actions only
const captureScreenshot = ['navigate', 'assertion'].includes(action.actionType);

4. Sanitize Sensitive Data

Remove sensitive information before sharing:

function sanitizeRecording(recording: TestRecording) {
  return {
    ...recording,
    actions: recording.actions.map(action => ({
      ...action,
      value: action.actionType === 'input' && action.target?.id === 'password'
        ? '********'
        : action.value
    })),
    allNetworkRequests: recording.allNetworkRequests.map(req => ({
      ...req,
      requestHeaders: sanitizeHeaders(req.requestHeaders),
      requestBody: sanitizeBody(req.requestBody),
      responseBody: sanitizeBody(req.responseBody)
    }))
  };
}

Performance Optimization

Replay is faster than recording because:

1. No LLM calls - Actions come from JSON, not GPT-4
2. Deterministic execution - Direct DOM manipulation
3. Parallel network validation - Check in background
4. Optional screenshot comparison - Can be disabled

Storage optimization:

// Lightweight recording (no screenshots)
const lightRecording = {
  ...recording,
  actions: recording.actions.map(a => ({
    ...a,
    screenshotBefore: undefined,
    screenshotAfter: undefined
  }))
};

// Reduce to ~10-20% of original size

Integration with CI/CD

name: Replay Test Suite

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Replay recordings
        run: npm run test:replay -- recordings/*.json
      - name: Upload reports
        uses: actions/upload-artifact@v2
        with:
          name: test-reports
          path: reports/

Future Enhancements

• [ ] Visual screenshot diffing for validation
• [ ] Smart selector healing (auto-find similar elements)
• [ ] AI-powered element matching when selectors fail
• [ ] Parallel test execution across recordings
• [ ] Recording versioning and diffing
• [ ] Cloud storage for team sharing
• [ ] Integration with Playwright/Selenium for cross-browser