🚀 Quick Tip: Mastering `test.slow()` and `test.fixme()` in Playwright

Introduction

As your Playwright test suite grows, you'll inevitably encounter tests that are a bit more... particular. Some tests might be inherently slower due to complex UI interactions or reliance on slower third-party services. Others might be temporarily broken or flaky due to ongoing development or known issues you haven't had a chance to address yet.

Playwright provides two incredibly useful annotations to handle these scenarios gracefully: test.slow() and test.fixme(). This quick tip will show you how to use them effectively to keep your test runs informative and your CI green (when appropriate!).

Dealing with Inherently Slow Tests: test.slow()

Sometimes, a test just needs more time. This could be due to animations, complex calculations in the UI, or waiting for a particularly slow API response that you don't control. Instead of globally increasing your test timeout (which can hide other performance issues), Playwright allows you to mark specific tests as slow.

When you annotate a test with test.slow(), Playwright automatically triples the configured test timeout for that specific test.

// This test might involve a lengthy report generation process in the UI
test.slow(); // Mark this test as slow
test('should generate and download a complex annual report', async ({ page }) => {
  test.info().annotations.push({ type: 'info', description: 'This test is marked slow as report generation can take up to 2 minutes.' });
 
  await page.goto('/reports');
  await page.selectOption('#reportType', 'annual_summary');
  await page.fill('#year', '2023');
  await page.click('button#generateReport');
 
  // Expect the download to start, which might take a while
  const downloadPromise = page.waitForEvent('download');
  // The UI might show a loading spinner for an extended period
  await expect(page.locator('.loading-spinner')).toBeVisible({ timeout: 120000 }); // Custom long timeout for one step
  await expect(page.locator('.report-ready-message')).toBeVisible(); // This benefits from test.slow()
 
  const download = await downloadPromise;
  expect(download.suggestedFilename()).toContain('annual_report_2023');
  // You might also save the download to a temporary path if needed
  // await download.saveAs(`/path/to/save/annual_report_${new Date().getTime()}.pdf`);
});

When to use test.slow():

• For tests that are consistently slow due to legitimate reasons and not due to a bug or poor test design.
• When a specific interaction within a test is known to take longer than the default timeout.
• As a temporary measure while investigating and optimizing a slow test.

How to call it:

• test.slow(); on its own line before the test definition.
• test.slow('Optional description why this test is slow'); with a description.
• You can also conditionally call test.slow(condition, 'description'); where condition is a boolean.

Handling Broken or Flaky Tests: test.fixme()

What about tests that are currently failing due to a known bug in the application, an unstable environment, or because the test itself needs refactoring? You don't want these to block your CI pipeline or constantly report failures for issues you're already aware of.

This is where test.fixme() comes in. When you mark a test with test.fixme(), Playwright will skip this test during the run. It won't be executed, and it won't be reported as a pass or a fail. Instead, it's typically marked as "skipped" or "pending" in test reports.

test.fixme(); // Mark this entire test as one that needs fixing
test('should allow users to interact with the new beta feature', async ({ page }) => {
  test.info().annotations.push({ type: 'bug', description: 'Test fails due to APP-1234 - Beta feature API not yet deployed to test env.' });
 
  // Test steps for the beta feature
  await page.goto('/beta-dashboard');
  await page.locator('#enableBetaToggle').check();
  // ... more interactions that are currently broken ...
  await expect(page.locator('.beta-success-modal')).toBeVisible();
});
 
test.describe('User Settings Suite', () => {
  test.fixme(process.env.CI === 'true', 'Skipping settings flaky test APP-5678 in CI only for now.');
  test('should update user profile picture - temporarily flaky', async ({ page }) => {
    // ... test steps for profile picture update ...
    await page.goto('/settings/profile');
    await page.locator('input[type="file"]').setInputFiles('path/to/avatar.jpg');
    await expect(page.locator('.avatar-preview')).toHaveAttribute('src', /.*avatar.*/);
  });
 
  test('should update notification preferences', async ({ page }) => {
    // This test runs as normal
    await page.goto('/settings/notifications');
    await page.locator('#emailNotifications').check();
    await expect(page.locator('#emailNotifications')).toBeChecked();
  });
});

When to use test.fixme():

• When a test is failing due to a known bug in the application that is being tracked.
• When a feature is under active development or refactoring, and its tests are temporarily unstable.
• For flaky tests that you intend to fix but want to temporarily remove from failing the main build.
• When an external dependency required by the test is unavailable in a specific environment.

How to call it:

• test.fixme(); on its own line before the test definition.
• test.fixme('Optional description why this test is marked as fixme');
• test.fixme(condition, 'description'); where condition is a boolean. This is very useful for disabling a test only in certain environments (e.g., test.fixme(process.env.CI === 'true', 'Skipping in CI due to flakiness')).

Benefits

• Cleaner Test Reports: test.slow() correctly allocates more time, and test.fixme() prevents known failures from cluttering your reports.
• Improved CI Stability: Avoids unnecessary CI failures due to tests that are either slow by design or known to be broken.
• Clear Communication: Annotations like test.slow('Reason...') and test.fixme('Bug APP-123...') document the state of these tests for the whole team.
• Focus: Allows the team to focus on legitimate, unexpected failures rather than being distracted by known issues or slow tests timing out.

Conclusion

test.slow() and test.fixme() are simple yet powerful tools in your Playwright arsenal. Use them judiciously to create a more stable, informative, and manageable testing process. By acknowledging that not all tests are perfect or equally fast, you can maintain a healthier and more productive test suite.

Remember to regularly review your test.fixme() tests to ensure they eventually get fixed and re-enabled, and to investigate if test.slow() tests can be optimized!