Introduction to Playwright Testing

Playwright is a powerful open-source testing framework, maintained by Microsoft. It is perfect for end-to-end automated testing of websites.It provides support for multiple browsers and has libraries for multiple programming languages.

The framework offers an extensive set of tools and features, allowing you to run automated tests against websites on the following browsers:

• Chrome (Chromium)
• Firefox
• WebKit
• Microsoft Edge (Chromium)

Should you choose Playwright over Selenium or Puppeteer?

Generally, Playwright will be faster to run automated tests than Selenium. One of the reasons is that Playwright uses a protocol based on Websockets, while Selenium uses multiple HTTP(s) requests during a single test run, the so-called WebDriver API.

While Playwright offers support for multiple browsers, it is unable to automate Safari. Instead, Playwright comes bundled with a custom version of WebKit (the engine behind Safari). Selenium on the other hand is capable of automating Safari, through the use of SafariDriver.

Below is a handy overview, allowing you to compare the advantages and disadvantages regarding Selenium vs Playwright.

What can you do with Playwright that is not possible with Selenium?

Network Interception

Playwright allows you to mock network traffic: you can intercept HTTP(s) requests and change their response. This makes it easy to test various scenarios in your app, for example what would happen if the response contains an error, etc.

Selenium is working on adding Network Interception through the use of WebDriver BiDirectional Protocol, but at the time of this writing this is not fully functional yet.

Visual Comparison

Playwright Test provides Visual Comparison support. This means you can take a base/golden screenshot and compare future screenshots to detect changes. Snapshots are also supported by Playwright Test, allowing you to compare text and binary content.

Selenium does not provide this functionality out of the box. If you want to do this with Selenium, you would need to use a test framework for this, such as WebDriverIO.

Test Downloads

Playwright offers the ability to test file downloads. During a test, you might want to test the download functionality of your app. You can fetch the download url, file system path and payload stream using the Download object from the event emitted by Playwright.

Selenium does not offer this functionality. If you want to do this with Selenium, TestingBot provides the ability to Test File Downloads with Selenium.

API Testing

API Testing is a feature implemented in Playwright Test. This gives you the possibility to write API tests to test REST-API services. You can even send API requests during your UI tests.

This might come in handy when you need to set up a specific state in your API before you start your tests.

Setting up and running your first Playwright Test

To get started with Playwright Test, you can install the package:

npm i -D @playwright/test

Next, you will need to create a configuration file, similar to the example below.

import { defineConfig } from '@playwright/test'
import { getCdpEndpoint } from './testingbot.config'

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({

  testDir: './tests',

  /* Maximum time one test can run for. */
  timeout: 30 * 1000,

  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,

  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,

  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,

  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',

  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://localhost:3000',


    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },

  projects: [
    {
      name: 'playwright-chrome@latest:Windows 10',
      use: {
        connectOptions: { 
          wsEndpoint: getCdpEndpoint({
            browserName: 'chrome',
            browserVersion: 'latest',
            platform: 'WIN10'
          }) 
        }
      },
    }
  ]
})

Also create a file called testingbot.config.js which contains:

exports.getCdpEndpoint = (userCapabilities) => {
    const defaultCapabilities = {
        'tb:options': {
            key: process.env.TB_KEY,
            secret: process.env.TB_SECRET
        }
    }
    const capabilities = Object.assign(userCapabilities, defaultCapabilities)
    const cdpUrl = `wss://cloud.testingbot.com/playwright?capabilities=${encodeURIComponent(JSON.stringify(capabilities))}`
    return cdpUrl
}

All that's left is to create a simple test, which will navigate to the Playwright website, click a link and verify the URL:

test('Sample TestingBot test', async ({ page }) => {
	await page.goto('https://playwright.dev/');

	// Click the get started link.

	await page.getByRole('link', { name: 'Get started' }).click();

	// Expects the URL to contain intro.

	await expect(page).toHaveURL(/.*intro/);
})

To run the test, you can now execute this command:

npx playwright test

Playwright Testing with TestingBot

TestingBot provides a remote grid of browsers, capable to run Playwright tests on all browsers supported by Playwright.

For each test you run, TestingBot will collect a video, screenshots and logs.

For more information, please see the TestingBot Playwright Test Documentation.