Visual Regression Tests
Catching frontend regression bugs
Last updated
Catching frontend regression bugs
Last updated
In order to catch css regression bugs before they land on production, or ideally before even merged, we use use a tool () for comparing visual changes, test regressions and verify changes.
The way this tool works is that it creates a series of screenshots for predefined urls and device resolutions and it automatically recognizes visual differences caused by our css code changes and reports them back in a way we can easily inspect them on browser and decide if these are within accepted threshold or not.
By default, for all Planet 4 websites, when their staging website is being build/deployed, the following happens:
Before the deployment, a screenshot of certain elements is taken
After the deployment a new screenshot is taken
A report is generated to see if the two screenshots match.
When a backstopjs job fails on staging, the workflow triggers a "Hold" workflow for the developers to visually look at the report, and decide if they want to proceed or not.
If the backstopJS job does not fail the workflow triggers directly the production deploy pipeline.
Go to the CircleCI workflows for your website
Get in the latest production workflow.
Find the job visualtests-compare and get into it (click it so that you go in the details)
Go to the tab artifacts.
Drill down to find the folder html_report
Click on index.html. It will open in a new tab the report with the comparisons
If you are happy with result go back to the workflow and approve the hold-production job.
Developers can run the regression tests locally as they work on a feature branch.
The test get run by a custom Planet 4 docker image that includes all the necessary dependencies for backstopJS and the scripts relevant to the above workflow.
In the readme file of the above repository you can read how you can do modifications and local testing.
You can either mount the same docker image to your local docker environment and modify the json file, or you can go full hardcore and install and configure BackstopJS locally. There is a simple backstop configuration already to get started.
The docker image is generated by the repository and the docker images are stored in .
The default page and viewport settings are controlled by the file in the above repository.
