Testing Craft CMS

Illustration by Pixel and Tonic

Illustration by Pixel and Tonic

How to setup PHPUnit and start testing Craft CMS
Updated 13. October 2024

Content:

Intro Overview PHP dotenv setup PHPUnit setup Unit test Feature test Setup script

Craft CMS is a fantastic content management system written in PHP. Unfortunately, it lacks a proper, out-of-the-box testing suite that is easy to configure and use. 

I tried and failed to use the test suite provided by Craft. The main issue I had was that it was designed to be used on a different database than the one used by Craft in my development environment. Because of this, I decided to configure PHPUnit so that I could write my tests using a tool I'm comfortable with.

In this article, I will guide you through the process of configuring PHPUnit on a Craft 5 project to enable automated testing.

Overview

A short introduction to automated testing

Automated testing, sometimes incorrectly called unit testing, is simply the practice of writing scripts or programs that verify that software works as intended. Many philosophies and methodologies exist for writing and organizing automated tests, but two main types of tests are generally written for software projects: Unit tests, and feature tests.

Unit tests verify that the smallest possible unit of code is tested. This is typically a function, or a method in a class. The idea is that if the unit works as intended, all uses of that unit will also work as intended. A unit test of a getter method will assert that the returned output is as expected. A unit test of a method that processes data asserts that the returned output was as expected based on the provided input.

Feature tests are the opposite of a unit test. Unlike unit tests, feature tests assert functionality from a holistic perspective. For example, a feature test can assert that an API endpoint with CRUD functionality works as intended by asserting that all available services provided by that API endpoint correctly create updates, read, and delete data. Therefore, an API endpoint feature test will be performed by sending HTTP requests and asserting that the API endpoint reacts correctly. Because of this, a feature test will assert the functionality of all units (methods and functions) used by that feature. This ensures that not only a single unit of code works as intended, but the whole feature works as intended.

Writing feature tests on web-based projects is probably the most effective way to secure the confidence and sleep of product owners and developers. Unit tests ensure that all the project's building blocks are healthy.

From a stakeholder perspective, I think feature tests and unit tests follow the 80/20 rule. Feature tests will take 20% of the time to write and assert 80% of functionality. Unit tests will take the remaining 80% of the time and assert the last 20% of the functionality. 

Most projects need both unit tests and feature tests. However, in a real-world setting with tight budgets and timelines, I would always prioritize feature tests.

PHP dotenv setup

As stated in the introduction, this testing setup is designed to work on the same database Craft uses in my dev environment and therefore, uses the same .env file as Craft. 

The setup is based on phpdotenv 5.*. If your project uses an older version, you must upgrade before you can proceed with this guide.

composer upgrade vlucas/phpdotenv:^5

Once you have upgraded and resolved any breaking changes, you can start the PHPUnit setup.

PHPUnit setup

Installing and configuring the testing environment

Start by installing PHPUnit as a dev dependency.

composer require phpunit/phpunit --dev

To verify that PHPUnit is installed, run this command.

php ./vendor/bin/phpunit --version
PHPUnit 10.5.35 by Sebastian Bergmann and contributors.

Once PHPUnit is installed, you must create the basic directory structure for your tests.

mkdir -p tests/Unit
mkdir -p tests/Feature

Now, you must add the testing bootstrap script. This script will run before your actual tests and will load environment variables and make Craft methods accessible for your tests.

vim tests/bootstrap.php
bootstrap.php 
<?php
/*
    Written by Stanley Skarshaug 2024.09.29
    https://www.haxor.no
*/

use Dotenv\Dotenv;
use craft\helpers\App;

// Set path constants
define('CRAFT_BASE_PATH', dirname(__DIR__));
define('CRAFT_VENDOR_PATH', CRAFT_BASE_PATH.'/vendor');

// Load Composer's autoloader
require_once CRAFT_VENDOR_PATH.'/autoload.php';

// Load dotenv
$dotenv = Dotenv::createImmutable(CRAFT_BASE_PATH);
$dotenv->load();

// Load Craft
define('CRAFT_ENVIRONMENT', 'testing');
require CRAFT_VENDOR_PATH.'/craftcms/cms/bootstrap/console.php';

Then, you must create a phpunit.xml file, where the basic configuration of PHPUnit is defined.

vim phpunit.xml
phpunit.xml 
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    colors="true"
    testdox="true"
    bootstrap="tests/bootstrap.php"
    requireCoverageMetadata="false"
    cacheDirectory=".phpunit.cache">

    <testsuites>
        <testsuite name="Feature">
            <directory>tests/Feature</directory>
        </testsuite>
        <testsuite name="Unit">
            <directory>tests/Unit</directory>
        </testsuite>
    </testsuites>
</phpunit>

This configuration is very pragmatic and designed for a workflow where only limited tests are needed to ensure that the project is healthy before committing new code. 

Because I don't care about 100% test coverage and only write tests for units and features I consider mission-critical, the requireCoverageMetadata attribute is set to false.

The testdox attribute is set to true because I like to see a verbose output of exactly what tests were run and what succeeded and failed.

Finally, the cache directory is added to the .gitignore file to complete the setup.

vim .gitignore
.gitignore 
...

.phpunit.cache

Unit test

Writing and runing your first Craft unit test

As an example unit test, we will write a test on a site-specific Craft CMS plugin with a service that returns an array of data on all the published articles on my blog.

vim tests/Unit/HaxorArticleServiceTest.php
HaxorArticleServiceTest.php 
<?php
/*
    Written by Stanley Skarshaug 2024.09.29
    https://www.haxor.no
*/

use PHPUnit\Framework\TestCase;
use mesusah\crafthaxor\services\ArticlesService;

final class HaxorArticleServiceTest extends TestCase
{
    public function test_can_get_all_articles(): void
    {
        $articlesService = new ArticlesService();
        $articles = $articlesService->getAll();

        $this->assertIsArray($articles);
        $this->assertTrue(count($articles) > 0);
    }
}

This unit test verifies that the method returned value is an array with at least one item. 

To run this test and all other test classes in the "Unit" test suite, run the following command:

php ./vendor/bin/phpunit --testsuite=Unit
PHPUnit 10.5.35 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.15
Configuration: /var/www/html/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:01.401, Memory: 22.00 MB

Haxor Article Service
 ✔ Can get all articles

OK (1 test, 2 assertions)

Feature test

Writing and runing your first Craft feature test

As a simple feature test, we will check if all published articles in a specific section of my blog are in working order. Since we want to check the health using web requests, we first need to install the guzzle package.

composer require guzzlehttp/guzzle --dev

Then, we can write our feature test.

vim tests/Feature/ArticlesTest.php
ArticlesTest.php 
<?php
/*
    Written by Stanley Skarshaug 2024.09.29
    https://www.haxor.no
*/

use PHPUnit\Framework\TestCase;
use craft\elements\Entry;
use craft\helpers\App;
use GuzzleHttp\Client;

final class ArticlesTest extends TestCase
{    
    public function test_can_access_all_articles(): void
    {
        $client = new Client([
            'base_uri' =>  App::env('PRIMARY_SITE_URL'),
            'timeout'  => 4.0,
        ]);
        
        $entries = Entry::find()
            ->section(["articles"])
            ->all();

        foreach ($entries as $entry) {
            $route = $entry->getUrl();
            $response = $client->request('GET', $route);

            $this->assertEquals(200, $response->getStatusCode());
        }
    }
}

This test loops through all published entries in my blog's "articles" section and checks if the HTTP response code is 200 (OK).

To run this test, and all other test classes in the "Feture" test suite, run the following command: 

php ./vendor/bin/phpunit --testsuite=Feature
PHPUnit 10.5.35 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.15
Configuration: /var/www/html/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:06.972, Memory: 12.00 MB

Articles
 ✔ Can access all articles

OK (1 test, 75 assertions)

Setup script

For your convenience

To make it simple to install PHPUnit on all Craft projects, I have created a simple setup script that creates all the necessary files and directories and installs the necessary dependencies.

Navigate to your project root, and run this command:

wget -O - https://raw.githubusercontent.com/mEsUsah/craft-phpunit/refs/heads/master/setup.sh | bash

After completing the script, all the steps defined in this article are automatically completed, and a couple of simple example tests are added. 

To make it pass if you are running Craft Solo, you must change the assertion from CmsEdition::Pro to CmsEdition::Solo in the ExampleUnitTest.

vim tests/Unit/ExampleUnitTest.php
ExampleUnitTest.php 
<?php
/*
    Written by Stanley Skarshaug 2024.09.29
    https://www.haxor.no
*/

use PHPUnit\Framework\TestCase;
use Craft;
use craft\enums\CmsEdition;

final class ExampleUnitTest extends TestCase
{
    public function test_check_craft_edition(): void
    {
        $this->assertEquals(CmsEdition::Solo, Craft::$app->edition);
    }
}

To run these test, and verify that the setup was successful, run this command:

php ./vendor/bin/phpunit
PHPUnit 10.5.35 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.15
Configuration: /var/www/html/phpunit.xml

..                                                                  2 / 2 (100%)

Time: 00:01.006, Memory: 10.00 MB

Example Feature
 ✔ Can access homepage

Example Unit
 ✔ Check craft edition

OK (2 tests, 2 assertions)

Thank you for reading this article! 🎉 I hope it inspired you to write automated tests using PHPUnit on your Craft CMS projects.