· HUBBR Developer Day 2018 · Havlíčkův Brod · given in Czech

How we write REST APIs with TDD

Jak píšeme REST API pomocí TDD. Write tests. Lots of tests. Here’s how we got from zero tests to one for every endpoint of our API, and why we now write the test first.

Who tests?

I opened with three questions to the room: who writes tests, who writes REST APIs, and who tests their REST API. For a long time I wasn’t in that last group.

Zero tests

Four or five years before the talk I built Odbav.to. Two years of development, a one-man show, on Nette. The number of tests: zero. The tests directory in that repository contains a .htaccess and a web.config, and nothing else.

Entrydo was a different setup: a team with two backend developers, two web developers and one mobile developer, all depending on the same API. That doesn’t work without tests. Today there are tests everywhere: a test for every presenter in the API.

What we test

Our API endpoints are Nette presenters. Each one is thin: it takes the request, hands it to a use case, and returns a response.

final class BillingInfoDetailPresenter extends UserAwarePresenter implements IPresenter
{
    /** @var BillingInfoDetailUseCase */
    private $useCase;

    public function __construct(BillingInfoDetailUseCase $useCase)
    {
        $this->useCase = $useCase;
    }

    public function run(Request $appRequest): IResponse
    {
        $response = $this->useCase->handle(
            new BillingInfoDetailRequest($this->userId)
        );

        return new OKResponse($response);
    }
}

So we test the whole thing from the outside: send an HTTP request, check the HTTP response.

How we used to test

The first version was one test class per presenter. Make a request, assert the status code:

class BillingInfoDetailPresenterTest extends TestCase
{
    use DatabaseSetup;

    /** @test */
    public function responseShouldBeValid(): void
    {
        $httpClient = $this->createHttpClient();
        $httpClient->request(IRequest::GET, RouterFactory::BILLING_ROUTE);

        $response = $httpClient->getResponse();

        $this->assertSame(IResponse::S200_OK, $response->getStatus());
    }
}

It works, but you write the same class over and over, and a 200 status says very little about what came back.

How we test now

Now a single test goes through a data provider. Each case is a name, a URL, a JSON file with the request body, a JSON file with the expected response, and the values to put into the URL:

public function getPresenters(): array
{
    return [
        'Digital Queue - Pair ticket with phone' => [
            '/digital-queues/tickets/<ticketId>/pair-with-phone',
            __DIR__ . '/PostRequestData/digital-queue/pair-ticket-with-phone/request.json',
            __DIR__ . '/PostRequestData/digital-queue/pair-ticket-with-phone/response.json',
            ['ticketId' => TicketFixture::getTicketId()],
        ],

        'Digital Queue - Queue reservation - Reserve' => [
            '/digital-queues/tickets/<ticketId>/queues/<queueId>/reservation',
            __DIR__ . '/PostRequestData/digital-queue/queue-reservation/reserve/request.json',
            __DIR__ . '/PostRequestData/digital-queue/queue-reservation/reserve/response.json',
            [
                'ticketId' => TicketFixture::getTicketId(),
                'queueId' => EventQueueFixture::QUEUE_ID,
            ],
        ],
    ];
}

The test sends the request through an in-process HTTP client and compares the JSON it gets back with the expected file. Adding a test for a new endpoint means adding two JSON files and one array entry. The examples in the talk repository show the rest: the test client is built on NBrowserKit, the clock is mocked so responses with dates stay stable, and every run gets a fresh database with its schema created from the Doctrine entities and data loaded from fixtures.

The workflow

This is where the TDD comes in. For a new endpoint we go in this order:

  1. Expectations: a new entry in the data provider, with the JSON request, the JSON response and the URL.
  2. Presenter: write the code until the response matches.
  3. Router: wire up the route.

The challenges

Why we test

GitLab CI pipeline: 9 jobs from master in 6 minutes and 17 seconds, queued for 6 seconds. Build stage: composer. Test stage: code-checker, integration-tests, lint, migrations up, phpstan, unit-tests. Deploy stage: dev and production. All jobs passed.
The whole pipeline, from build to production, in 6 minutes 17 seconds.

The API test suite alone was 243 tests with 480 assertions, running in 2.4 minutes in CI.

The rest of the talk was a live demo. It’s in the recording, and the example code is on GitHub.

All 26 slides
  1. Slide 1: Title slide: TDD + REST API, Jan Mikeš, 20 Oct 2018, Entrydo logo and speaker photo
  2. Slide 2: Czech text: write tests, lots of tests
  3. Slide 3: Meme: a plain bug next to the same bug wearing a suit, labelled Bug and Feature
  4. Slide 4: Audience question: who tests?
  5. Slide 5: Audience question: who writes REST APIs?
  6. Slide 6: Audience question: who tests their own REST API?
  7. Slide 7: Logos of Odbav.to, Eventgers and Entrydo
  8. Slide 8: Odbav.to: 4–5 years ago, 2 years of development, One Man Show and Nette logos
  9. Slide 9: Odbav.to GitLab repository with a tests folder last touched 4 years ago
  10. Slide 10: Odbav.to: zero tests
  11. Slide 11: Cartoon trash can saying "I'm full of garbage!"
  12. Slide 12: Entrydo team: two backend, two web and one mobile developer
  13. Slide 13: Long list of presenter test files in the Entrydo codebase
  14. Slide 14: Toy Story meme: tests everywhere
  15. Slide 15: What we test: a BillingInfoDetailPresenter class calling a use case, PHP code
  16. Slide 16: How we used to test: PHPUnit test sending a GET request, asserting status 200
  17. Slide 17: How we test now: data provider mapping routes to JSON request and response files
  18. Slide 18: Workflow: 1. expectations data provider with JSON request, response and URL, 2. presenter, 3. router
  19. Slide 19: Challenges: start writing tests, find the right approach, demo data, speed, automation
  20. Slide 20: Why we test: Xdebug, full automation with 6 minutes from push to production, sleeping well
  21. Slide 21: PHPUnit API test suite output: 243 tests, 480 assertions in 2.4 minutes
  22. Slide 22: GitLab pipeline: composer build, six test jobs, deploy to dev and production in about 6 minutes
  23. Slide 23: Workflow recap: data provider with JSON request, response and URL, then presenter, then router
  24. Slide 24: Demo time, with a cheering Minion
  25. Slide 25: Demo code at github.com/janmikes/hubbr-2018
  26. Slide 26: Thank you, questions? Homer Simpson, with links entry.do, janmikes.cz and email

Related talks