Usage & Configuration

Detailed guide to configuring and using the CoreReporter.

Basic Configuration

Playwright reporters must be specified as file paths, not class instances. Create a reporter file that extends CoreReporter:

Step 1: Create a reporter file


// reporter.ts
import { CoreReporter } from '@lytics/playwright-reporter';
import { FilesystemAdapter } from '@lytics/playwright-adapters/filesystem';
 
class CustomReporter extends CoreReporter {
  constructor() {
    super({
      adapters: [
        new FilesystemAdapter({ outputDir: './test-results' })
      ]
    });
  }
}
 
export default CustomReporter;

Step 2: Reference it in your config


// playwright.config.ts
import { defineConfig } from '@playwright/test';
 
export default defineConfig({
  reporter: [
    ['list'],
    ['./reporter.ts'],
  ],
});

Configuration Options

CoreReporterConfig


interface CoreReporterConfig {
  /** Storage adapters to write results to */
  adapters: ResultAdapter[];
  
  /** Optional function to add environment metadata */
  environmentEnricher?: () => Record<string, unknown>;
}

adapters

Array of adapters that implement the ResultAdapter interface. All adapters run in parallel.


// reporter.ts
class CustomReporter extends CoreReporter {
  constructor() {
    super({
      adapters: [
        new FilesystemAdapter({ outputDir: './test-results' }),
        new SlackAdapter({ webhookUrl: process.env.SLACK_WEBHOOK_URL! }),
      ]
    });
  }
}

environmentEnricher

Optional function that returns environment metadata to include in test runs:


// reporter.ts
class CustomReporter extends CoreReporter {
  constructor() {
    super({
      adapters: [/* ... */],
      environmentEnricher: () => ({
        // CI/CD metadata
        branch: process.env.GITHUB_REF,
        commit: process.env.GITHUB_SHA,
        author: process.env.GITHUB_ACTOR,
        buildNumber: process.env.GITHUB_RUN_NUMBER,
        
        // Runtime info
        nodeVersion: process.version,
        
        // Custom metadata
        environment: process.env.TEST_ENV || 'local',
        triggeredBy: process.env.TRIGGER_TYPE || 'manual',
      })
    });
  }
}

Environment Variables

The reporter uses these environment variables by default:

Variable	Used For
`GITHUB_RUN_ID`	`runId` and `buildId`
`ARTIFACT_BASE_URL`	`reportLink` in test results

You can override these via environmentEnricher.

Creating Custom Adapters

Implement the ResultAdapter interface:


import type { ResultAdapter, CoreTestResult, CoreTestRun } from '@lytics/playwright-reporter';
 
class MyCustomAdapter implements ResultAdapter {
  private connection: DatabaseConnection;
 
  async initialize(): Promise<void> {
    // Called once before any tests run
    // Use for setup: connect to database, create directories, etc.
    this.connection = await Database.connect();
    console.log('MyCustomAdapter initialized');
  }
 
  async writeTestResult(result: CoreTestResult): Promise<void> {
    // Called after each test completes
    // Write individual test result
    await this.connection.insert('test_results', {
      testCaseId: result.testCaseId,
      journeyId: result.journeyId,
      status: result.status,
      durationMs: result.durationMs,
      timestamp: result.timestamp,
      error: result.error ? JSON.stringify(result.error) : null,
    });
  }
 
  async writeTestRun(run: CoreTestRun): Promise<void> {
    // Called once after all tests complete
    // Write test run summary
    await this.connection.insert('test_runs', {
      runId: run.runId,
      totalTests: run.totalTests,
      passed: run.passed,
      failed: run.failed,
      passRate: run.passRate,
      durationMs: run.durationMs,
      timestamp: run.timestamp,
    });
  }
 
  async close(): Promise<void> {
    // Called after writeTestRun
    // Cleanup: close connections, flush buffers, etc.
    await this.connection.close();
    console.log('MyCustomAdapter closed');
  }
}

ResultAdapter Interface


interface ResultAdapter {
  /** Initialize the adapter (connect, create directories, etc.) */
  initialize(): Promise<void>;
 
  /** Write an individual test result */
  writeTestResult(result: CoreTestResult): Promise<void>;
 
  /** Write a full test run summary */
  writeTestRun(run: CoreTestRun): Promise<void>;
 
  /** Cleanup and close connections */
  close(): Promise<void>;
}

Best Practice: Handle errors gracefully in adapters. A failing adapter shouldn’t crash the test run or prevent other adapters from working.

Error Parsing

parseError

Parse raw Playwright errors into structured format:


import { parseError } from '@lytics/playwright-reporter';
 
// In a custom adapter or post-processing script
const error = parseError(result.errors[0]);
if (error) {
  console.log(`Matcher: ${error.matcher}`);     // "toHaveText"
  console.log(`Expected: ${error.expected}`);   // "Welcome"
  console.log(`Actual: ${error.actual}`);       // "Hello"
  console.log(`Locator: ${error.locator}`);     // "getByRole('heading')"
  console.log(`File: ${error.location.file}`);  // "tests/home.spec.ts"
  console.log(`Line: ${error.location.line}`);  // 42
}

TestError Type


interface TestError {
  matcher: string;      // Playwright matcher (e.g., "toHaveText", "toBeVisible")
  expected: string;     // Expected value
  actual: string;       // Actual value
  locator: string;      // Element locator
  location: {
    file: string;       // Test file path
    line: number;       // Line number
    column: number;     // Column number
  };
  message: string;      // Full error message
  snippet: string[];    // Code snippet around the error
}

Multiple Projects

The reporter handles multiple Playwright projects correctly:


// playwright.config.ts
export default defineConfig({
  projects: [
    { name: 'chromium', use: { browserName: 'chromium' } },
    { name: 'firefox', use: { browserName: 'firefox' } },
    { name: 'webkit', use: { browserName: 'webkit' } },
  ],
  reporter: [
    ['list'],
    ['./reporter.ts'],
  ],
});

Each test result includes projectName so you can filter/group by browser.

Troubleshooting

Tests are skipped with “missing annotation” warning

Cause: Tests don’t have required journeyId and testCaseId annotations.

Solution: Add annotations to all tests:


test('my test', async ({}, testInfo) => {
  pushTestAnnotations(testInfo, {
    journeyId: 'YOUR_JOURNEY_ID',
    testCaseId: 'YOUR_TEST_CASE_ID',
  });
  // Test implementation...
});

Adapter initialization fails

Cause: Adapter configuration is incorrect or external service is unavailable.

Solution: Check adapter configuration and credentials. Adapters log errors to console.

Stats don’t match expected values

Cause: Retries are handled automatically.

Explanation:

totalTests = unique tests (deduped by testCaseId)
totalExecutions = all test runs (including retries)

If a test fails and retries, it counts as 1 test but 2+ executions.

Adapters not receiving results

Cause: Adapter threw an error during initialize().

Solution: Check console for initialization errors. Ensure all required configuration is provided.

Best Practices

1. Always Include a Local Adapter

Even in CI, include FilesystemAdapter for debugging:


// reporter.ts
class CustomReporter extends CoreReporter {
  constructor() {
    super({
      adapters: [
        new FilesystemAdapter({ outputDir: './test-results' }),
        // ... other adapters for CI
      ]
    });
  }
}

2. Use Environment Enrichment

Add CI/CD context to make results more useful:


environmentEnricher: () => ({
  branch: process.env.GITHUB_REF,
  commit: process.env.GITHUB_SHA,
  pr: process.env.GITHUB_PR_NUMBER,
  environment: process.env.TEST_ENV,
})

3. Handle Adapter Failures Gracefully

Adapters should catch and log errors without throwing:


async writeTestResult(result: CoreTestResult): Promise<void> {
  try {
    await this.doWrite(result);
  } catch (error) {
    console.error(`[MyAdapter] Failed to write result: ${error}`);
    // Don't re-throw - allow other adapters to continue
  }
}

4. Use Consistent Annotation Patterns

Establish team conventions for annotations and validate them:


// In a shared test setup file
test.beforeEach(async ({}, testInfo) => {
  // Validate annotations exist
  if (!testInfo.annotations.some(a => a.type === 'testSuiteName')) {
    throw new Error('Missing testSuiteName annotation');
  }
});