Planet 4 is setup to run automated acceptance tests on each deployment pipeline.
But you can also add extra scenarios to test other parts of your website's functionality. The purpose of these tests is multiple:
To test if any customisations done by NRO developers are working as they should.
To test if the latest planet4 release works together with the NRO customisations.
To create tests for your site you have to do the following:
In your planet4-nro repository, create a folder called "tests", like the example in the planet4-netherlands repository.
Inside that directory, write tests, following the Codeception syntax, in php files.
Some relevant info:
The tests run on the develop and release sites of your environment. They do not run on self contained site setups, but on your actual develop and staging sites.
The tests run on your develop site in the develop workflow of CircleCI, after the site gets build and deployed.
The tests run on your release site in the release workflows after a successful deployment and before the "hold" job (that triggers production deployments).
If the test fails on the develop workflows, it will still trigger the release site deployment. It does not block the next steps.
If the test fails on the release workflows, it will block the next steps. So, production deployments cannot happen if the tests fail on the release site.
Example of failed tests on the release workflow. The hold-promote job that would usually allow you to deploy to production is not triggered, as it is blocked by the test-release job failing
Example of successful tests on the release workflow. Tests succeeded, so hold-promote is available.