githubEdit

[ADR-0014] Choose a testing framework

Switch to Playwright for our one testing framework for e2e tests.

  • Status: accepted

  • Deciders: Development team

Technical Story: Harmonize our tests tooling so ideally we use just one framework of writing tests whenever that’s possible.

Context and Problem Statement

Currently we rely heavily on Codeception for most of our tests. But we use two different syntaxes. The “old” onearrow-up-right and Gherkinarrow-up-right. During the WYSIWYG Epic we also wrote some tests in JSarrow-up-right. Ideally we should use one way of writing all of these tests.

Considered Options

Decision Outcome

Chosen option: Switch to Playwright as our one testing framework for e2e tests.

Pros and Cons of the Options

Cypress

  • Good, because easy to start using it (one binary, simple interface)

  • Good, because assertion librariesarrow-up-right are included

  • Good, because still in active developmentarrow-up-right

  • Good, because large documentation, numerous examples and tutorials

  • Good, because it has a watch mode integrated that allows for easier development

  • Bad, because droppedarrow-up-right by WordPress in favor of Puppeteer, then Playwright

  • Bad, because some complex cases require a succession of promises, which make them less readable

Playwright

  • Good, because easy to use: install/setup is quick and async/await notation + clear function names (using the expect libraryarrow-up-right for assertions) make the tests easy to write and understand

  • Good, because still in active developmentarrow-up-right

  • Good, because it’s the tool that WordPress will soon use too (and they’re working on a npm packagearrow-up-right with various utilities that we might be able to use in the future?) (+ best practicesarrow-up-right)

  • Good, because it has a useful VSCode packagearrow-up-right to easily run and debug tests

  • Good, because it’s faster by default, using concurrent workers

  • Bad, because since Playwright is fairly new, the support from the community is limited and the documentation is sometimes lacking

Last updated