18 May 2019 · Jobs Dev 2019 · Prague · given in Czech

Migrating a legacy PHP project to PSR-4 with scripts and Rector

Migrace vašeho legacy projektu na PSR-4 automatickými nástroji. Renaming classes across a thousand files sounds like a search-and-replace job. It isn’t, when the rest of the team keeps committing about 100 times a day to the same code while you do it.

The starting point

The project was a big legacy PHP application at GLAMI, built on Nette. PSR-4 is the PHP standard that maps a namespace to a directory, so Glami\Cache\Apc lives in src/Cache/Apc.php and Composer can find it without any index. This code didn’t follow it. It relied on Nette RobotLoader instead.

RobotLoader’s killer feature is that it doesn’t care: it scans your directories and loads a class from whatever file it’s in, whatever the file is called. You can have 20 classes in one file and nothing complains. That’s convenient for years, and then it’s a mess. Our bootstrap had hundreds of lines of require __DIR__ . '/../app/model/...' and a list of single files added to RobotLoader by hand.

Why do it with scripts

I didn’t want to do this by hand, for three reasons:

• Legacy code grows really fast. While you migrate, others keep adding code the old way.
• A script can be run again, and again. If the first run fails or the branch gets stale, you throw it away and rerun it on fresh master.
• Manual know-how stays in the head of whoever did the work. A script stays in the repository.

Step one: RobotLoader to Composer classmap

The path had two stages: RobotLoader, then Composer classmap, then Composer PSR-4. The classmap is the easy first step. Composer scans the listed directories once and writes a lookup table of every class:

"autoload": {
    "classmap": ["app", "libs", "src"]
},
"autoload-dev": {
    "classmap": ["tests", "tests_db", "tests_selenium"]
}

Is there any real performance gain from leaving RobotLoader? People in the Nette community said they never noticed a difference. Our monitoring did show one.

The catch with a classmap: add a new class and forget to regenerate it, and you get Class 'Glami\MyClass' not found. PSR-4 doesn’t have that problem, which is why the classmap was only a step.

Step two: classmap to PSR-4

The target configuration is short:

"autoload": {
    "psr-4": {
        "App\\": "app",
        "Glami\\": "src"
    },
    "classmap": ["libs"]
},
"autoload-dev": {
    "psr-4": {
        "Tests\\": "tests",
        "TestsDB\\": "tests_db",
        "TestsSelenium\\": "tests_selenium"
    }
}

The naive plan is three bullets: rename class and namespace, replace all usages, profit. Too naive, you are.

The requirements

• Non-breaking for the team, which made about 100 commits a day
• Repeatable
• Repeatable by someone else
• Bug-free

Plan properly

Most of the work was the plan, written as a numbered checklist before touching any code. In short:

1. One class per file. Split files that contain more than one class, with Rector.
2. Name every file after its class.
3. Rename directories so they start with a capital letter (except templates).
4. Decide the correct namespace for every class.
5. Generate a map of old class name to new class name for Rector.
6. Migrate the classes with Rector.

Moving libs/klarka to src had its own list: fix every __DIR__ and ../ path, then a full-text search and manual fixes in .gitlab-ci.yml, the gettext extract command, the translation scripts, composer.json, phpstan.neon and the entry-point PHP files. One line in the plan was in bold: watch out for how presenters and cron jobs are mapped. The plan also links phpcs-psr4-sniff, a CodeSniffer rule that checks whether files follow PSR-4.

The tools

Rector does instant upgrades and refactoring of PHP code. It parses the code into an abstract syntax tree, so a rename is a real rename, not a string replace. Around the same time we used it to migrate a whole application from Nette to Symfony, which I talked about in a separate talk.

Around Rector I wrote a few small scripts. The first renames files to match the class inside:

$files = Finder::findFiles('*.php')->from(__DIR__ . '/../src');
$renames = [];

foreach ($files as $file) {
    $filePath = $file->getPathname();
    $content = NetteFileSystem::read($filePath);
    $match = Strings::match($content, '/^\w*\s*(class|interface|trait) *(\w*)/m');

    // File does not contain class
    if ($match === null) {
        continue;
    }

    $fileName = $file->getFilename();
    $className = $match[2];
    $correctFileName = $className . '.php';

    // File is already same as class name
    if ($fileName === $correctFileName) {
        continue;
    }

    $renames[$fileName] = $correctFileName;
    $newFilePath = str_replace($fileName, $correctFileName, $filePath);
    NetteFileSystem::rename($filePath, $newFilePath);
}

echo Json::encode($renames, Json::PRETTY);

The second one finds files with multiple classes. It uses RobotLoader itself, because RobotLoader already knows which class lives in which file, and runs Rector in Docker on each of those files to split them, followed by composer dump-autoload.

Then the Rector config, generated from the old-to-new map:

Rector\Rector\Class_\RenameClassRector:
    Glami\libs\Statistics\NormalDistribution: Glami\Statistics\NormalDistribution
    Glami\libs\PHPStan\PHPStan: Glami\PHPStan\PHPStan
    Glami\libs\PageTimeApc: Glami\PageTimeApc
    Glami\libs\Tools\Date\Range: Glami\Tools\Date\Range
    Glami\libs\Cache\Apc: Glami\Cache\Apc
    # ...

And the last script searches the whole project, outside vendor and a few other directories, for any old class name that still appears as a string. Those are the places Rector can’t see: config files, templates, strings.

90% of the time: edge cases

The renames were the easy part. Most of the time went into things like these:

- /** @var  */
+ /** @var */
  private $collation;

- $mockSoapService = $this->getMockBuilder('\Klarka\GpWebPay\SoapService');
+ $mockSoapService = $this->getMockBuilder(SoapService::class);

- $this->expectException('MMDB_Exception');
+ $this->expectException(\Klarka\MMDB\MMDB_Exception::class);

- class DownloaderResult extends Klarka\HTTP\Downloader\Result {}
+ class DownloaderResult extends \Klarka\HTTP\Downloader\Result {}

Broken docblocks, class names hidden in strings, and relative names that only worked because of where the file used to live.

Code review?

The resulting change touched 1,128 files. GitHub stops showing a diff at 1,000 files. Nobody can review that line by line. What you can review is the script that produced it, and the commit message says exactly which script ran. If the script is right, the diff is right, and if master moved in the meantime, you just run it again.

Satisfaction

Rector processed around 10,000 files and changed 1,128 of them. The test suite, 1,371 tests and 3,873 assertions, passed in just over two minutes.

Related talks

• How we migrated from Nette to Symfony like ninjas · 2019
• Build your own GitHub bot · 2019

Rector on GitHub phpcs-psr4-sniff Nette to Symfony talk All talks