Fast database tests with PHPUnit, Nette and Doctrine
Nette + PHPUnit + Doctrine – Efektivní práce s databází v testech. Tests that touch a real database are the most useful tests you can write, and the slowest. Here is how I made them fast with one small PHPUnit extension.
The problem
We had a PHP application built on Nette and Doctrine. I wanted to write PHPUnit tests, and the testing strategy involved working with the database. Not mocks, a real database.
A test like that always needs the same things:
- a database,
- its structure (tables),
- fake data (rows),
- … the test itself …,
- and a cleanup afterwards.
I looked at what was out there. The Nette world mostly talks about Nette Tester, its own testing tool, and the articles about testing models with a database didn’t fit PHPUnit. Laravel has the RefreshDatabase trait that resets the database after each test. Symfony documents a separate test database and points to DAMADoctrineTestBundle, which wraps each test in a transaction. Nette + Doctrine + PHPUnit had nothing like that ready, so I built it.
The naive version
The first version does everything inside the test: drop the schema, create it, run migrations, load fixtures, then test.
class MyServiceTest extends TestCase
{
public function testSomething(): void
{
passthru(sprintf('php "%s/../bin/console" --env=test orm:schema-tool:drop', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test orm:schema-tool:create', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test migrations:migrate', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test doctrine:fixtures:load', __DIR__));
// We need container to work with the application ...
$container = Bootstrap::getTestsContainer();
// ... the service we are testing ...
$service = $container->getByType(MyService::class);
// ... check state of the app before ...
$this->assertCount(0, $service->countSomething());
$service->doSomething();
// ... make sure the app changed into expected state ...
$this->assertCount(1, $service->countSomething());
}
}The container comes from the application’s Bootstrap, with one extra config file for tests and a static variable, so it’s built only once per run:
private static null|Container $testsContainer = null;
public static function getTestsContainer(): Container
{
// Local caching
if (self::$testsContainer === null) {
$_ENV['APP_ENV'] = 'test';
$configurator = self::boot();
// Customize the app for tests
$configurator->addConfig(__DIR__ . '/../config/tests.neon');
self::$testsContainer = $configurator->createContainer();
}
return self::$testsContainer;
}It works. Tada! But it rebuilds the whole database for every single test, and that gets slow very quickly.
What we really want
- Run PHPUnit.
- Only once, at the start, prepare the database structure and data.
- Before each test, start a transaction.
- Run the test.
- Roll the transaction back after each test.
There are two ways to hook into PHPUnit. A bootstrap file is the easy way, but it runs only once before all tests. An extension uses PHPUnit’s event system (since PHPUnit 10). It’s more robust, and you can hook into any stage of the test suite. That’s what we need.
Step 0: create the extension
An extension implements one interface:
namespace PHPUnit\Runner\Extension;
use PHPUnit\TextUI\Configuration\Configuration;
interface Extension
{
public function bootstrap(Configuration $configuration, Facade $facade, ParameterCollection $parameters): void;
}and gets registered in phpunit.xml:
<extensions>
<bootstrap class="Omnicado\Tests\PHPUnit\TestingDatabaseExtension" />
</extensions>Inside bootstrap() you register subscribers on the $facade. Each one reacts to one event.
Step 1: prepare the database only once
$facade->registerSubscriber(new class () implements TestRunnerStartedSubscriber {
public function notify(TestRunnerStartedEvent $event): void
{
passthru(sprintf('php "%s/../bin/console" --env=test orm:schema-tool:create', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test migrations:migrate', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test doctrine:fixtures:load', __DIR__));
}
});Step 2: start a transaction before each test
$facade->registerSubscriber(new class () implements TestStartedSubscriber {
public function notify(TestStartedEvent $event): void
{
Bootstrap::getTestsContainer()
->getByType(EntityManagerInterface::class)
->beginTransaction();
TestingDatabaseExtension::$transactionStarted = true;
}
});Step 3: roll back after each test
A test can end in more ways than one. It can finish, be skipped, or error out, so we roll back on all three events:
$facade->registerSubscriber(new class () implements SkippedSubscriber {
public function notify(Skipped $event): void
{
TestingDatabaseExtension::rollBack();
}
});
$facade->registerSubscriber(new class () implements TestFinishedSubscriber {
public function notify(TestFinishedEvent $event): void
{
TestingDatabaseExtension::rollBack();
}
});
$facade->registerSubscriber(new class () implements ErroredSubscriber {
public function notify(Errored $event): void
{
TestingDatabaseExtension::rollBack();
}
});The flag makes the rollback safe to call more than once:
public static function rollBack(): void
{
if (!self::$transactionStarted) {
return;
}
Bootstrap::getTestsContainer()
->getByType(EntityManagerInterface::class)
->rollback();
self::$transactionStarted = false;
}Now the test itself is clean. No setup, no cleanup, only the behaviour:
class MarketplaceStockUpdateLogRepositoryTest extends TestCase
{
public function testGetMostRecentStockQuantities(): void
{
$container = Bootstrap::getTestsContainer();
$repository = $container->getByType(MarketplaceStockUpdateLogRepository::class);
$this->assertCount(0, $repository->getMostRecentStockQuantities());
$entity = new MarketplaceStockUpdateLog($repository->nextId(), 1, 1);
$repository->save($entity);
$this->assertCount(1, $repository->getMostRecentStockQuantities());
}
}Step 4 of 3: “cache” the database
One more thought. If the structure hasn’t changed, and the data hasn’t changed, and every test rolls back its changes, why create a fresh database on every run?
The structure comes from migrations and the data from fixtures. So I hash those two directories and store the hash in a file. The database gets rebuilt only when the hash differs. It’s also skipped completely when only unit tests run, because they don’t need a database at all.
$facade->registerSubscriber(new class () implements TestRunnerStartedSubscriber {
public function notify(TestRunnerStartedEvent $event): void
{
$cacheFilePath = __DIR__ . '/../.database.cache';
$currentDatabaseHash = TestingDatabase::calculateDirectoriesHash(
__DIR__ . '/../../migrations',
__DIR__ . '/../DataFixtures',
);
if (
TestingDatabaseExtension::isRunningUnitTestOnly() === false
&& TestingDatabase::isCacheUpToDate($cacheFilePath, $currentDatabaseHash) === false
) {
passthru(sprintf('php "%s/../bin/console" --env=test orm:schema-tool:create', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test migrations:migrate', __DIR__));
passthru(sprintf('php "%s/../bin/console" --env=test doctrine:fixtures:load', __DIR__));
file_put_contents($cacheFilePath, $currentDatabaseHash);
}
}
});The hash is just the MD5 of every PHP file in those directories:
public static function calculateDirectoriesHash(string ...$directories): string
{
$finder = Finder::find('*.php')->in(...$directories);
$files = array_keys(iterator_to_array($finder->getIterator()));
$hash = '';
foreach ($files as $file) {
$hash .= md5_file($file);
}
return $hash;
}The check compares the stored hash with the current one, and also looks at the age of the cache file against a validity window:
public static function isCacheUpToDate(string $cacheFilePath, string $currentDatabaseHash): bool
{
if (!file_exists($cacheFilePath)) {
return false;
}
$cachedDatabaseHash = file_get_contents($cacheFilePath);
$lastModificationTimestamp = filemtime($cacheFilePath);
$cacheValidUntil = time() + self::CACHE_VALID_FOR_SECONDS;
if ($cacheValidUntil < $lastModificationTimestamp) {
return false;
}
return $currentDatabaseHash === $cachedDatabaseHash;
}The result: the database is built once, and only again when migrations or fixtures change. Every test starts from the same clean state, and tests stay independent of each other. The full working example is on GitHub.
