TL;DR: use the -h option when extracting a tarball if you wish for symlinks at the target extraction destination to be treated as actual directories.

Read on for details…

So, it turns out that extracting a tar file that contains a directory that already exists at the target extraction location produces different behavior depending upon whether the existing target directory exists actually or is a symlink.

Now, having made such a statement, I’m sure you’re thinking, “ahh, there must be a story behind this revelation.” And you would be right to think that. So let me tell you…

I’ve been putting off using Debian bookworm as the base image in my various Docker builds because it was causing build errors. Specifically:

runc run failed: unable to start container process: error during container init: exec: "/bin/sh": stat /bin/sh: no such file or directory

That error made no sense to me. How can /bin/sh be missing? This started happening when I updated to a specific PHP version Docker image — I don’t remember which version, or when, but it must have been shortly after the release of Debian bookworm and the default base image when pulling, say, php:8.2.28-fpm became bookworm.

Back at that time, after about 5 minutes, I gave up and specified the Debian bullseye tag: php:8.2.28-fpm-bullseye. The build worked, and I’ve been carrying on that way ever since — kicking the can down the road.

Well, today, as I was updating a project to PHP 8.4, I initially specified php:8.4.5-fpm in the Dockerfile, and of course, lo and behold, it was based on bookworm, and it failed with the error above. I confirmed, briefly, that the build would work if I specified the bullseye tag. But then I decided it was time to stop kicking this can down the road.

I will cut out the fumbling intermediate debugging steps and get to the juicy part. As it turns out, one of the first things my Docker builds do is extract a tarball from the S6 Overlay into the root of Docker image’s file structure. Among other things, this tarball contains a bin directory with some executables.

What I discovered was this: In the bullseye base Debian image, /bin is an actual directory.

The output of docker run -it --rm --entrypoint "" php:8.4.5-fpm-bullseye bash -c "ls -lah /" is like so:

You can see that looks normal. Particularly, bin is a directory, not a symlink.

Ahh, but in bookworm, it is a symlink to /usr/bin. Observe the result of docker run -it --rm --entrypoint "" php:8.4.5-fpm bash -c "ls -lah /":

We now see in the bookworm based image that bin is a symlink. This is at the root of the problem. I discovered this because I noted, based on the error, that sh was not in the bin directory after running my tar extraction. In fact, the only things in the bin directory were the things zin the bin directory in the S6 tarball. I looked at the contents of bin before running the tar extraction and saw that it contianed all the things it should. And that is also when I noticed it was a symlink. Shortening my sleuthing, once again, that eventually led me to this answer on StackExchange. After a bit more research, sure enough, adding the -h option to my tar extraction command resulted in an additive merge of the contents of /bin from the S6 file, just as I wanted.

The Docker build stopped failing, and life could go on with the more recent bookworm based Docker image, rather than continuing to kick the can down the road with bullseye until we come to the point in the future where Docker PHP images aren’t built and tagged for bullseye at all.

Mystery solved. And hopefully, if anyone else runs into this edge case behavior, I’ll have enough Google-foo search juice to lead them here.

My friend, Kevin Smith, posted a thing on Twitter not too long ago that got my attention regarding this subject.

Why deploy frequently, even as the feature is still being developed?

“If any change can potentially affect any part of the system, then introducing two changes at once is much more complicated to debug than introducing one change. Was the problem change A? Change B? Some…

— Kevin Smith (@_KevinSmith) December 27, 2024

He linked to the Kent Beck article he quoted so naturally I clicked on through and I thought it was excellent.

I’ll quote some things and then comment.

A lesson I learned from my officemate at Oregon, David Meyer (now a director at Cisco), is that as systems grow in complexity, every element is potentially coupled to every other element. This suggests that systems be made as simple as possible to keep that N^2 from blowing up too far, and it suggests that changes be made one at a time. If any change can potentially affect any part of the system, then introducing two changes at once is much more complicated to debug than introducing one change. Was the problem change A? Change B? Some interaction of A and B? Or was it just a coincidence? Introducing one change at a time keeps the cost of maintenance in check.

At the same time, systems need to grow rapidly. You need many changes but you can only introduce one change at a time. One way to reconcile the conflicting needs is to reduce the cycle time. If you can change a system in tiny, safe steps but you can make those steps extremely quickly, then it can look from the outside like you are making big changes.

This is really good. As software engineers, when we release often, even multiple times during feature development, from our perspective we’re making small incremental changes. But from the outside, when we’re finished, we’ve released big changes with no ill effects.

Timothy Fitz, formerly of IMVU, told a story that brought this lesson home to me. The discipline they came to was that as soon as they said to themselves, “That change couldn’t possibly break anything,” they deployed immediately. If you weren’t at least a little worried, why would you even say that? By making the overhead of deployment vanishingly small, they could create value with every deployment. Either the deployment was fine, in which case the engineer regained confidence, or the deployment broke, in which case the engineer learned something about the sensitivities of the system.

I didn’t realize I was “discovering the sensitivities of the system” at the time, but early on in the first stages of an API transition for a project I was working on, I ran a deployment which took the app down in production. I had, of course, tested this new API endpoint extremely thoroughly in my local dev environment, a UAT environment, and a Staging environment. This new API endpoint performed exactly to spec in every test environment. And so, I deployed to production.

It broke spectacularly.

What I had done was actually deployed 2 things at once, though I didn’t think about it that way at the time. I deployed the new API endpoint, AND I deployed the change to the front-end that consumed the new API endpoint. The API endpoint was for the main menu of the application, which is an endpoint every page in the app touches. So when the endpoint failed after deploy, it took the whole thing down.

What I learned that day (again) was to deploy things more incrementally. From then on, I would deploy new API endpoints first. And then after testing that API endpoint, I would run another deployment to switch over the React app to consume that endpoint. That allowed testing the JSON response of the new endpoint before relying on it.

So, while there may be multiple deployments during the process for a given API endpoint, development can actually move faster and not be afraid of breaking things.

Kent Beck goes on with an analogy of code as inventory.

In Toyota Production System, Taiichi Ohno makes an analogy between inventory and water in a river. By lowering the water level in the river (reducing inventory), you can uncover previously hidden rocks (identify bottlenecks). Undeployed software is inventory. By deploying in smaller batches, you can identify bottlenecks in your software process.

When I used to work at a film company, we did a filmed lecture series with an entrepreneur. One of the things he talked about was how inventory is a liability. What you want is to optimize the supply chain so that inventory flows and you aren’t storing things in big warehouses or for too long. You always want as little inventory as possible.

While I’ve never been in that sort of business, I thought that was interesting and filed it away in the back of my head.

That’s why I find this analogy fascinating. I now have a way to apply that knowledge. Unreleased code is inventory. Holding it back (storing it in the warehouse) is not a zero-cost effort. Code gets stale as the application moves forward around it.

Additionally, have you ever built up a succession of branches that have to go out in a certain order? The more of that you have, the higher the cost of development.

Startups have a vital need to eliminate waste. Because many of the assumptions on which a startup are based are likely to prove false, most startups need to iterate to be successful.

He talks here about startups, but I think this is generally true across any good organization.

If the team can learn to learn from each iteration and can make those iterations short and cheap, then the whole enterprise has a greater chance of succeeding overall. Startups have the initial advantage of no code and no customers, so establishing a rapid deployment rhythm is relatively easy, although maintaining that rhythm through growth can be a challenge.

He talks about the “no code and no customers” bit, but my thinking is, startup or not, code or not, the principles he’s talking about are the goal and something all organizations should strive for.

Now, of course, you have to be careful when you have established code, and customers relying on said code. But the principles of rapid deployment are no less useful. The problem-set to solve is the same problem-set.

What’s the cash value?

On my team, we don’t ask permission to deploy of non-technical people, and I would encourage all software teams and engineers to try to be in a position to manage and run deployments at will.

Of course this means 3 things (at least):

1. You need to have and maintain a track record of not breaking things when you deploy.

If your deployments break often, non-technical stake-holders are going to want to put controls on deployments. Of course this is the wrong instinct, because study after study shows software gets more stable the more often a team deploys and/or feels comfortable deploying. But the non-technical stake-holders aren’t in a position to know that. All they know is that things break a lot, and when they dig in to find out why, it’s related to a deployment. So, be studious with your development practices and think through all the effects that your deployments will have on the software.

Maintain ways of testing, or environments to test in, that mirror production to the extent possible. And then always run your code and deployments through those before releasing to production.

Of course, deploying often is also part of not breaking things. Because the fewer changes that are deployed at once, the fewer things there are to go wrong. AND if something does go wrong, where to go to fix it is narrowed down considerably.

2. You need to make sure you have a way of rolling back a deployment swiftly in case of failure.

Failures can and do happen. You cannot avoid the occasional failure. So make sure you have a plan in place for rolling back. And go over it in detail occasionally. With your team. In most cases, if the system is down for a minute, you’re not going to catch too much flack. If it’s down for 10 minutes, you’re going to get lots of messages. If its down for longer than that due to a bad deploy, you can kiss frequent deployments goodbye.

3. You need to have systems like feature flags in place that let you, the software team, deploy code whenever you want, even though the product team or other stake-holders may not be ready to “release” a feature.

On my team, we make copious use of feature flags to determine when features go live. We decouple deployment from release. You’ll probably want to go in after release and remove conditionals and feature flags, and yes, this is a trade-off with having more code-paths in your software (at least, temporarily). But I promise you it’s well worth it to keep the code inventory low.

Be the software expert

I tell engineers not to ask for permission to deploy. If you ask for permission to deploy from non-technical stake-holders, you are eroding their confidence in your ability to do your job. Don’t do that. Instead, be confident. And of course, as mentioned, be right. Don’t break things.

When announcing releases, I encourage my team to use language like, “this release adds the groundwork for ‘x’” rather than saying, “I’m releasing feature ‘x’.” This removes the need to ask for permission. Then the conversation with the product team, or whoever controls when features are released goes more like this: “when would you like to make feature ‘x’ available?”

Perhaps what I’m saying about confidence and not asking for permission to deploy applies only to mid-level and above engineers. Or maybe it only applies to senior-level and above engineers. I would encourage teams to still keep “permission” to deploy within the software engineering team. The engineers are the ones in the best position to know what’s best for the software.

And for the junior and mid-level engineers, let this be an encouragement to level up your skills. This is part of becoming a senior engineer. As the engineer, you need to be the expert in your field. Half that battle is acting like the expert you are. Trust me, I know for many like myself, confidence is a learned behavior, but it’s well worth learning.

I am in the “no” camp on that question. Strong types make for systems that are easier to reason about and have fewer bugs. And I’d like to use an illustration from an experience I’ve had recently on why that is. Keep in mind this is but one illustration among many of how weak typing causes bugs.

In a particular system my team is working on, the front-end is powered by React and Typescript. Of course, we chose Typescript for it’s stronger typing in the Javascript ecosystem. Even so, it apparently is not strong enough to save us from the following scenario (and I wish it would).

Our front-end React app makes API requests to a PHP back-end. The authentication of the request for the user happens with Tokens. The relevant code for this illustration looks about like this:

interface Token {
    accessToken: string;
    accessTokenExpires: number;
    refreshToken: string;
}

export async function makeApiRequest (
    {
        uri,
        queryParams = new URLSearchParams(),
        payload = {},
        token,
    }: { 
        uri: string;
        queryParams?: URLSearchParams;
        payload?: Record<never, never>;
        token: Token;
    },
) {
    const baseUrl = getConfigStringServerSide(ConfigOptions.API_BASE_URL);

    return sendApiRequest({
        url: new URL(`${baseUrl}${uri}?${queryParams.toString()}`),
        payload,
        headers: new Headers({
            Authorization: `Bearer ${token.accessToken}`,
            Accept: 'application/json',
            'Content-Type': 'application/json',
        }),
    });
}

That’s fine, but the problem with this function is that it requires an access token to be present. We have one or two API requests that we need to be able to make unauthenticated. But we need all the rest of the processing to happen on the request so the only thing we need to do is make the token optional.

So here’s what the code was updated to:

interface Token {
    accessToken: string;
    accessTokenExpires: number;
    refreshToken: string;
}

function createHeadersFromToken (token: Token | undefined): Headers {
    const headers = {
        Accept: 'application/json',
        'Content-Type': 'application/json',
    } as Record<string, string>;

    if (token) {
        headers.Authorization = `Bearer ${token}`;
    }

    return new Headers(headers);
}

export async function makeApiRequest (
    {
        uri,
        queryParams = new URLSearchParams(),
        payload = {},
        token,
    }: {
        uri: string;
        queryParams?: URLSearchParams;
        payload?: Record<never, never>;
        token?: Token;
    },
) {
    const baseUrl = getConfigStringServerSide(ConfigOptions.API_BASE_URL);

    return sendApiRequest({
        url: new URL(`${baseUrl}${uri}?${queryParams.toString()}`),
        payload,
        headers: createHeadersFromToken(token),
    });
}

We made the token optional, and we moved the headers creation out to a factory method to only add the Authorization header if the token was present.

But, can you spot the bug? I’ll bet you can’t (not quickly), and we didn’t either. And no alarms are going off. No linters are complaining. Typescript isn’t crying about it. Nothing is saying there’s something wrong.

And yet, now authenticated API requests are broken in our local dev. Can you guess why yet?

Because the error is not easy to spot, and there’s no underlines in the editor from the linter, and Typescript is compiling just fine, we start combing through the code, console.logging here, step debugging in PHP and/or dumping and dying there. After tracing the request through the front-end and heading over to the PHP side, we finally got to the part in PHP that validates the Authorization header. And here’s what the debugging output said the token was:

"Bearer [object Object]"

Let me be clear, that’s a “string” of [object Object] which was sent as an Authorization header in the API request to the PHP application. And because it’s a string in a header, there are no errors in the PHP application about types (the typing error would have needed to happen in Javascript/Typescript). So the PHP app looks at that header, tries to authenticate it as a token, and, what do you know, there is no valid [object Object] token, so the PHP app correctly returns an HTTP 401 status code response.

So by now you’ve spotted the error in the refactored code. No? Let me help. It’s this bit right here:

if (token) {
    headers.Authorization = `Bearer ${token}`;
}

Remember, if you read the code, that the Token type is an object:

interface Token {
    accessToken: string;
    accessTokenExpires: number;
    refreshToken: string;
}

So what happened? What went wrong here? Weak typing is what went wrong. Typescript is apparently not strongly enough typed to flag this as an error. And the underlying Javascript will happily type coerce an object to a string as [object Object] when you put it into a string context rather than raising an error like strong typing advocates think it should.

An object cannot reasonably become a string and to try to do so should be a program error.

Strong typing solves this problem. In PHP, if you try to concatenate or interpolate an object into a string, PHP isn’t going to be happy. And it will be very obvious how, why, and where it isn’t happy.

In fact, here is an example in PHP. This is using a keyed array (which is really a hash map or dictionary in most languages) because that is the closest analog to a Javascript object.

<?php

$testArray = ['foo' => 'bar'];

$testString = "test: $testArray";

echo $testString;

To be fair, PHP does type coerce the array into a string (I don’t think it should do that at all). But, in PHP 8, it outputs a Warning ^[1] (in PHP 5.6 and 7.x it outputs a Notice which isn’t as great) that looks like this:

Warning: Array to string conversion in /path/to/file.php on line 5
test: Array

And, perhaps most importantly, the offending code gets highlighted and underlined by PHPStorm as a program error, “Array to string conversion” so that the developer can see right away, without even running the program, that there’s a program error.

Back to Javascript.

Javascript’s type coercion in this case (and nearly every case I can think of) was an uncaught program error. It’s a mistake any one of us on the team might have made during that refactor and which we would normally expect our tooling to catch so we could pull the correct property out of the object to put into that string. Here’s what that code should look like:

if (token) {
    headers.Authorization = `Bearer ${token.accessToken}`;
}

The fact that no part of the program was able to catch this program error is a failure of language and tooling. Yes, it was a mistake by a developer (a mistake the likes of which every single developer reading this article has made), but it was much more a failure of the language.

Now, perhaps there’s a failure of our tooling to catch this, but man we’ve got as many dials as we can on Typescript and ESLint cranked up as high as they’ll go, and yet nothing caught this.

What really prevents things like this is strong typing. This is one of many illustrations on why I am very much pro-strong typing.

In this most recent case, my team and I are building a new application that will need to have some knowledge from a third-party application, which is currently being implemented, but the implementation is not complete. This application provides data feeds that we can “subscribe” to, to get information from that system. That’s all very well and good, but how does our application access it? Well, that’s a little up in the air. For a while the thinking was that there would be an ETL that would load up a MySQL database, and our new application will query the database.

However, more recently, there’s been a question of whether that MySQL database should be a data lake instead. And the problem is, of course, development on our new app cannot be stalled by access to this source knowledge.

Here’s the thing: we know what data we’ll have access to, but we don’t yet know how we’ll access it. How the data gets into our system is implementation detail that therefore needs to be abstracted. The good news is, I’ve been a proponent of abstracting the data source from the system for a while. And one of the ways I do this is with repository interfaces. Those repository interfaces have implementations — usually to MySQL. But what if you have a different data persistence mechanism you want to swap in? Repository interfaces make this very simple. The rest of your software doesn’t need to change, because it’s based on an interface. And the implementation detail of the persistence mechanism remains just that, an implementation detail.

But, enough of my yammering, let’s get to some sample code.

The appliation we’re building needs to access customer data provided by another application. And so I created a repository interface that returns entities with the information we know about that customer from the other system:

interface CustomerRepository
{
    public function findCustomerById(string $id): CustomerResult;

    public function getCustomerById(string $id): Customer;
}

readonly class Customer
{
    public static function createEmpty(): self
    {
        return new self(
            '',
            '',
            '',
            '',
        );
    }

    public function __construct(
        public string $id,
        public string $firstName,
        public string $lastName,
        public string $fullName,
    ) {
    }
}

readonly class CustomerResult
{
    public bool $hasCustomer;

    public Customer $customer;

    public function __construct(Customer|null $customer)
    {
        if ($customer === null) {
            $this->hasCustomer = false;

            $this->customer = Customer::createEmpty();

            return;
        }

        $this->hasCustomer = true;

        $this->customer = $customer;
    }
}

To keep development of this application moving forward, I created a MySQL database in our local dev Docker environment, and implemented that interface with a concrete class named CustomerRepositoryForPdo. Then I wired up the DI to serve that concrete implementation anytime the interface is requested.

The upside is, if we do end up going with a MySQL database as the gateway to the customer information for this application, my concrete implementation is probably 90% there. And if we go a different direction, I’ll just create a different concrete implementation.

The point I’m making is this: consider using interfaces in strategic locations. In this case, consider interfaces for your data repository implementations. Your application is more than the data it accesses.

In addition to making the data source implementation swappable, creating this separation also encourages you to think about your application in a much more robust way. Rather than designing it strictly to the data model of the underlying persistence mechanism, and then upending everything when that changes, you design your application as separate objects working together as a cohesive whole. Some might call that object-oriented programming 🙂.

And, after all, separation of concerns is fundamental to building high-quality software

Of late in the React world in general, SSR (server-side rendering) has been a big topic of discussion. And specifically in Next’s App Router, SSR is the default mode, and you have to take steps to make a component client-side rendered (CSR).

While in some ways, SSR feels like getting back to my roots as a PHP engineer, building the page on the server and shipping it to the client, it also causes some problems. Particularly when using SSR combined with HTTP Streaming, which is what Next does — particularly if you’re using suspense boundaries. What happens is, when you are waiting on an async data load, React/Next will deliver an initial payload, and stream the rest when the async process finishes. So what’s the problem? My approach to OAuth token refreshes has had to change to accommodate SSR.

The Problem with OAuth Token Refreshes and SSR/HTTP Streaming

So, it turns out, you can’t set cookies when streaming after the initial payload, and so Next just disallows trying to set cookies at all in server-side rendered components. That’s just as well because any time I would need to update a cookie from a token refresh, it would be after the suspense boundary and initial payload anyway.

The Old Way in CSR

So the first application my team and I built in Next was started before server-side components and streaming were things in Next. So it’s entirely CSR. And it initially began life as a stateless front-end ^[1]. As such, combined with the fact that we’re using NextAuth to create the initial sign-in and token exchange, we stored the JWT in the NextAuth cookie (encrypted with a server-side secret).

This is the NextAuth JWT callback config we set up:

jwt: async (
    {
        token,
        user,
        account,
    }: {
        token: JWT;
        user: User;
        account: Account;
    },
) => {
    // Initial sign in
    if (account && user) {
        token.user = user;

        return {
            accessToken: account.access_token,
            accessTokenExpires: account.expires_at,
            refreshToken: account.refresh_token,
            user,
        };
    }

    return token;
},

That stores the access token and refresh token in the cookie (encrypted).

NextAuth doesn’t have a means of token refresh built in, and the recommended approach in their docs just flat-out doesn’t work. So we built our own refresh system into our API Request Mechanism.

The long and short of it is, when an API request returns a 401 status, we use a Redis backed locking mechanism to pause any other request that might be in flight also trying to refresh the token, and do a token refresh, then release the lock and make the request with new token.

Most relevantly for this discussion, because the front-end makes requests by hitting a Next “back-end” endpoint (and the back-end endpoint makes the actual API request), we can send back cookie headers with that response to the client. Here’s the relevant refresh code

import { NextApiRequest, NextApiResponse } from 'next';
import { encode } from 'next-auth/jwt';
import { serialize } from 'cookie';
import NextAuthJwt from '../NextAuthJwt';
import { SetAccessTokenToTempStore } from './RefreshedAccessTokenTempStore';
import getRefreshAccessToken from './GetRefreshAccessToken';

export default async function RefreshAccessToken (
    req: NextApiRequest,
    res: NextApiResponse,
    token: NextAuthJwt,
) {
    const newToken = await getRefreshAccessToken(token);

    // If we were unable to get refreshed token, there's nothing else we can do
    if (newToken === null) {
        return;
    }

    // Encode the new token as a string for the cookie
    const newTokenString = await encode({
        token: newToken,
        secret: "wouldn't you like to know",
    });

    const cookieName = '__Secure-next-auth.session-token';

    req.cookies[cookieName] = newTokenString;

    res.setHeader(
        'Set-Cookie',
        [
            serialize(
                '__Secure-next-auth.session-token',
                String(newTokenString),
                {
                    httpOnly: true,
                    sameSite: 'lax',
                    path: '/',
                    secure: true,
                    // @see node_modules/next-auth/jwt/index.js DEFAULT_MAX_AGE
                    maxAge: 30 * 24 * 60 * 60,
                },
            ),
        ],
    );

    /**
     * Set the new access token to the temp store so other in-flight requests
     * can access it
     */
    await SetAccessTokenToTempStore(token.accessToken, newToken.accessToken);

    // Update the current token object with the new access token
    token.accessToken = newToken.accessToken;
}

SSR Streaming Can’t Set Cookies

Obviously none of that will work in the new SSR Streaming world, where these requests are happening after the initial payload, so what do we do?

The first approach was to create a dedicated back-end endpoint that could set cookie headers in the response when a new token was found in the Redis store. While this is not where I ultimately landed, it does work, and seems to work well. It just felt a tad “hacky” to me, which is why I ultimately rethought this whole thing. You may note that in the code above from the CSR methodology, we are making use of a TokenTempStore (backed by Redis) to store the token for 90 seconds, with the old token as the key, so other requests that may already be in-flight can pick up that token. So this first approach leverages that temporary token store.

Here’s the updated RefreshAccessToken function:

import NextAuthJwt from '../auth/[...nextauth]/NextAuthJwt';
import { SetTokenToTempStore } from './RefreshedAccessTokenTempStore';
import GetRefreshedAccessToken from './GetRefreshedAccessToken';

export default async function RefreshAccessToken (
    token: NextAuthJwt,
): Promise<NextAuthJwt | null> {
    const newToken = await GetRefreshedAccessToken(token);

    if (newToken === null) {
        return null;
    }

    await SetTokenToTempStore(
        token.accessToken,
        newToken,
    );

    return newToken;
}

Then I created an API route that the front-end could hit with an AJAX request that would check for a new token in that temp store and set a cookie in the response if needed:

import { NextResponse } from 'next/server';
import { encode } from 'next-auth/jwt';
import GetTokenFromCookies from '../GetTokenFromCookies';
import { GetTokenFromTempStore } from '../RefreshedAccessTokenTempStore';

export async function GET () {
    const token = await GetTokenFromCookies();

    /**
     * This really shouldn't happen because we only call this endpoint from
     * authenticated pages, but…
     */
    if (!token) {
        return NextResponse.json({ status: 'OK' });
    }

    const newToken = await GetTokenFromTempStore(token.accessToken);

    // If new and existing are the same, we've already set the cookie
    if (!newToken || newToken.accessToken === token.accessToken) {
        return NextResponse.json({ status: 'OK' });
    }
    
    // Now we can send the new token in a cookie header

    const newTokenString = await encode({
        token: newToken,
        secret: "wouldn't you like to know",
    });

    const response = NextResponse.json({ status: 'OK' });

    response.cookies.set(
        '__Secure-next-auth.session-token',
        newTokenString,
        {
            httpOnly: true,
            sameSite: 'lax',
            path: '/',
            secure: true,
            // @see rxeffect/web/node_modules/next-auth/jwt/index.js DEFAULT_MAX_AGE
            maxAge: 30 * 24 * 60 * 60,
        },
    );

    return response;
}

And here’s the JSX component I included in the layout of any authenticated page:

'use client';

import { useEffect } from 'react';

// This should be called from a server component
export default function CheckForNewTokenAndSetCookie () {
    const check = () => {
        fetch('/api/request/check-store-for-new-token', { cache: 'no-store' });
    };

    useEffect(() => {
        const timeout = setTimeout(check, 2000);

        const interval = setInterval(check, 60000);

        return () => {
            clearTimeout(timeout);
            clearInterval(interval);
        };
    // We actually don't want this to re-render ever, so, here we go
    // eslint-disable-next-line react-hooks/exhaustive-deps
    }, []);

    return null;
}

As I said, this worked, but it really felt a bit hacky, and like there must be another way. As it turns out, there is.

Storing it all in Redis, full time

You see, the problem is we were still thinking about this front-end app as a stateless app, when in reality, we had already introduced a state storage mechanism via Redis. In the back-end world, in PHP, we store state in persisting mechanisms all the time. Why can’t we do that here? What would that look like?

The solution I arrived at was to create a session ID, store that in the cookie, and then store the token under that session ID. Then you use the ID in the cookie to look up the storage. Here’s what that looks like.

First, I updated the NextAuth JWT callback to store a session ID on the cookie, instead of the token values.

jwt: async (
    {
        token,
        user,
        account,
    }: {
        token: JWT;
        user: User;
        account: Account;
    },
) => {
    // Initial sign in
    if (account && user) {
        const sessionId = await CreateTokenStore(account, user);

        return { sessionId };
    }

    return token;
},

As you can see, we have a new function we’re calling that creates the token store and returns the session ID. Here’s what that looks like:

import { Account } from 'next-auth';
import { v4 as uuid } from 'uuid';
import { User } from './User';
import getRedisClient from '../../../cache/RedisClient';

/**
 * 4800 seconds is 80 minutes, which is how long refresh tokens are set for in
 * this application. Unfortunately this knowledge isn't available in the token,
 * we just have to know it as a magic value here, which means if we ever change
 * it there… ¯\_(ツ)_/¯
 */
const redisTokenExpireTimeInSeconds = 4800;

export async function CreateTokenStore (
    account: Account,
    user: User,
): Promise<string> {
    const redis = getRedisClient();

    const id = uuid();

    await redis.set(
        `user_token:${id}`,
        JSON.stringify({
            accessToken: account.access_token,
            accessTokenExpires: account.expires_at,
            refreshToken: account.refresh_token,
            user,
        }),
        'EX',
        redisTokenExpireTimeInSeconds,
    );

    return id;
}

Now the refresh function can just update the token in the Redis store:

import {
    GetTokenFromCookies,
    SetTokenStoreBasedOnCookieKey,
} from '../../auth/[...nextauth]/TokenStore';
import { AcquireLock, ReleaseLock } from './RefreshLock';
import GetRefreshedAccessToken from './GetRefreshedAccessToken';

export default async function RefreshAccessToken () {
    const token = await GetTokenFromCookies();

    // To ensure that only one request is refreshing the token we await a lock
    await AcquireLock(token.accessToken);

    /**
     * Now we check the token in the store again to make sure the token wasn't
     * already refreshed by another request
     */
    const tokenCheck = await GetTokenFromCookies();

    // It looks like the token was already refreshed while we awaited a lock
    if (tokenCheck.accessToken !== token.accessToken) {
        await ReleaseLock(token.accessToken);

        return;
    }

    const newToken = await GetRefreshedAccessToken(token);

    // If there is no token, the refresh was unsuccessful, and so we won't save
    if (!newToken) {
        await ReleaseLock(token.accessToken);

        return;
    }

    // WE HAVE A NEW TOKEN! YAY! Now set it to the token store
    await SetTokenStoreBasedOnCookieKey(newToken);

    await ReleaseLock(token.accessToken);
}

Presto. Less hacking, less code, sleep better at night.

For completeness, I’ll drop the entire TokenStore code, which contains some of the functions you see in the code above:

import { cookies } from 'next/headers';
import { Account } from 'next-auth';
import { v4 as uuid } from 'uuid';
import { decode } from 'next-auth/jwt';
import { User } from './User';
import getRedisClient from '../../../cache/RedisClient';
import { NextAuthJwt, NextAuthJwtSchema } from './NextAuthJwt';

/**
 * 4800 seconds is 80 minutes, which is how long refresh tokens are set for in
 * auth0: see Applications > Quest > Settings > Refresh Token Expiration
 * Unfortunately this knowledge isn't available in the token, we just have to
 * know it as a magic value here, which means if we ever change it there… ¯\_(ツ)_/¯
 */
const redisTokenExpireTimeInSeconds = 4800;

export async function CreateTokenStore (
    account: Account,
    user: User,
): Promise<string> {
    const redis = getRedisClient();

    const id = uuid();

    await redis.set(
        `user_token:${id}`,
        JSON.stringify({
            accessToken: account.access_token,
            accessTokenExpires: account.expires_at,
            refreshToken: account.refresh_token,
            user,
        }),
        'EX',
        redisTokenExpireTimeInSeconds,
    );

    return id;
}

export async function GetTokenFromStore (
    sessionId: string,
): Promise<NextAuthJwt | null> {
    const redis = getRedisClient();

    const tokenString = await redis.get(`user_token:${sessionId}`);

    if (!tokenString) {
        return null;
    }

    const token = JSON.parse(tokenString) as NextAuthJwt;

    try {
        NextAuthJwtSchema.parse(token);

        return token;
    } catch (error) {
        return null;
    }
}

const getIdFromCookie = async (): Promise<string | null> => {
    const cookie = cookies().get('__Secure-next-auth.session-token');

    if (!cookie) {
        return null;
    }

    const cookieDecoded = await decode({
        token: cookie.value,
        secret: "wouldn't you like to know",
    });

    return cookieDecoded?.sessionId as string | null;
};

export async function SetTokenStoreBasedOnCookieKey (token: NextAuthJwt) {
    const sessionId = await getIdFromCookie();

    if (!sessionId) {
        return;
    }

    const redis = getRedisClient();

    await redis.set(
        `user_token:${sessionId}`,
        JSON.stringify(token),
        'EX',
        redisTokenExpireTimeInSeconds,
    );
}

export async function FindTokenFromCookies (): Promise <NextAuthJwt | null> {
    const sessionId = await getIdFromCookie();

    if (!sessionId) {
        return null;
    }

    return GetTokenFromStore(sessionId);
}

export async function GetTokenFromCookies (): Promise<NextAuthJwt> {
    const token = await FindTokenFromCookies();

    if (!token) {
        throw new Error('Unable to find token');
    }

    return token;
}

And here’s the Redis lock code:

import sleep from 'sleep-promise';
import getRedisClient from '../../../cache/RedisClient';

export async function AcquireLock (accessToken: string) {
    const redis = getRedisClient();

    let resp = null;

    let tries = 0;

    let acquiredLock = false;

    do {
        // eslint-disable-next-line no-await-in-loop
        resp = await redis.set(
            `refresh_token_lock:${accessToken}`,
            'true',
            'EX',
            60,
            'NX',
        );

        if (resp !== null && resp.toLowerCase() === 'ok') {
            acquiredLock = true;

            break;
        }

        tries += 1;

        // eslint-disable-next-line no-await-in-loop
        await sleep(1000);
    } while (tries < 65);

    if (acquiredLock) {
        return;
    }

    throw new Error('Could not acquire lock');
}

export async function ReleaseLock (accessToken: string) {
    const redis = getRedisClient();

    await redis.del(`refresh_token_lock:${accessToken}`);
}

I hope this may be helpful to anyone struggling with any of this.

You should never use the HTML event handler attributes — those are outdated, and using them is bad practice. (emphasis theirs)

This is, in my polite opinion, completely wrong. The novices are right on this one. MDN is a tremendous resource, and I understand why they recommend the second form, but combating this particular ideology is essential to rehabilitating HTML’s full functionality, and building durable applications with it. I’ll explain.

Overall, I found this to be a fantastic read with many great points that I mostly agreed with and only had slight quibbles and mild disagreements. What’s fascinating is that his thinking about making HTML more robust and inlining behavior is exactly what I love about using React as I discussed in my post/presentation on co-location.

Perhaps if we were to make HTML more robust my need and like of React would be diminished (although, pessimistically, I don’t see that happening).

Still the thinking behind this post I heartily agree with. Inline your behavior and your styling. They’re not separate concerns, and the way we’ve tried to make them all separate concerns has really distorted HTML over the years.

A traditional array in software is an indexed collection of elements. And traditionally they are at least thought of as the same type of elements in any given array. Some languages don’t really enforce that they’re the same type, and some do (guess which category PHP falls into with its already non-conforming arrays).

Now, what PHP calls an array is actually a data structure known as a hash map, a hash table, or a dictionary. That’s because PHP’s “array” keys can be pretty much any scalar value (here’s PHP’s documentation on “arrays”).

PHP’s “arrays” will mostly behave like arrays as long as you don’t assign keys with the values. PHP will happily increment a numerical index starting at 0 as you assign items to the array.

What we don’t have in PHP (to be fair we don’t have it in JavaScript either, but I don’t know if that’s the gold standard we want to be comparing to) is enforcing types in “arrays”. And of course, the “array” can be a hash map at any time by simply assigning a key of type string, say, yourself. Any array will accept that. There’s nothing you can do about it.

As I said, PHP is very array heavy. Avoiding PHP’s “arrays” is pretty much impossible. And I’d also like to note that while I dislike the name because they are not actually arrays, there’s nothing wrong with a dictionary data type. It’s actually quite useful. The real problem is we don’t have real arrays in PHP so we must use a dictionary to accomplish the same thing.

And, I’d also argue that even if we had traditional arrays, we shouldn’t be passing them around as transports for data (I’d make the same argument for dictionary types). When you need to pass a collection of data, I think you should use a collection. In PHP, we implement collections as classes. There are libraries that you can use to create collection classes, but I’ve found every single one of them to be very heavy and overly complicated to use, and yet still don’t support all the use cases I often have — at least not easily.

Additionally, most collection libraries want to implement ArrayAccess so that they can be treated as arrays. I object to this on a fundamental level. They are not arrays, just call the methods on the collection. That makes things much clearer and easier to read anyway.

And the good news is, building a collection class is pretty trivial. The collection class I’ll demonstrate here will utilize PHP “arrays” (or a dictionary) internally. That’s what I’ll call an internal implementation detail that the outside world need know nothing about.

So, let me show you some code:

readonly class SomeEntity
{
    public function __construct(
        public SomeValueObject $someValueObject,
        public AnotherValueObject $anotherValueObject,
    ) {
    }
}

readonly class SomeEntityCollection
{
    /** @var SomeEntity[] */
    private array $entities;

    /** @param SomeEntity[] $entities */
    public function __construct(array $entities)
    {
        $this->entities = array_map(
            static fn (SomeEntity $entity) => $entity,
            $entities,
        );
    }
}

Alright, let me stop there.

Obviously this collection doesn’t do anything yet except collect some entities. Once it’s collected them, you can’t really do much yet. But I do want to point out something I’m doing to work around PHP “arrays” lack of types. We want the collection to only be of one type. The only way the collection gets filled is through the constructor. So immediately in the constructor, we validate the types by iterating over the incoming array and typing the items in the array_map callable’s typehint. If something is not of the right type, PHP will throw a type exception with that code 👍.

Now, I’m not a fan of writing code that will not yet be in use. That’s a little something I like to call premature optimization. So bear that in mind as you write collection classes. But let’s say that I have a need to know how many items are in the collection. That’s quite simple to add:

readonly class SomeEntityCollection
{
    /** @var SomeEntity[] */
    private array $entities;

    /** @param SomeEntity[] $entities */
    public function __construct(array $entities)
    {
        $this->entities = array_map(
            static fn (SomeEntity $entity) => $entity,
            $entities,
        );
    }

    public function count(): int {
        return count($this->entities);
    }
}

Simple.

Now let’s see what it might look like to output some json for an API endpoint from a collection.

readonly class SomeEntity
{
    public function __construct(
        public SomeValueObject $someValueObject,
        public AnotherValueObject $anotherValueObject,
    ) {
    }
}

readonly class SomeEntityCollection
{
    /** @var SomeEntity[] */
    private array $entities;

    /** @param SomeEntity[] $entities */
    public function __construct(array $entities)
    {
        $this->entities = array_map(
            static fn (SomeEntity $entity) => $entity,
            $entities,
        );
    }

    public function asJson(): string {
        return (string) json_encode(
            array_map(
                static fn (SomeEntity $entity) => [
                    'someValue' => $entity->someValueObject->toNative(),
                    'anotherValue' => $entity->anotherValueObject->toNative(),
                ]
                $this->entities,
            ),
        );
    }
}

This is the beauty of typed, custom collections. When you build it, you know the data, and you can locate the logic to deal with that data where it belongs; with the classes designed to deal with the data. You can put all kinds of methods on these custom collection classes as you need them. You might need to filter, or get the first item, or get the last item, etc. etc. The options are limitless. And it means you write these methods once on the collection, and you can use them anywhere, and they’re safe to use because they’re well typed and well-defined.

As you can see, PHP’s array/dictionary type is useful in implementations. And, as you can see, I typically use it most in collection classes. If you’ve never considered this before — as I hadn’t many years ago when someone introduced me to the idea of not passing arrays around, but using collections — I hope this will inspire you.

We recently came across a need to get field values from a Drupal database table to make decisions about the app. The short story is, we have entities in Drupal to which Drupal fields are attached. Those fields can be set to values that control the behavior of that entity on the front-end. So when we get that entity, we need to check the field values for various aspects of the front-end and API output.

Initially in this new PHP app, we created an object-oriented way of getting the values that we needed into a custom collection class which was returned by the repository. That collection class looked something like this:

class InstanceVariableCollection
{
    /** @var InstanceVariableRecord[] */
    private array $records;

    /** @param InstanceVariableRecord[] $records */
    public function __construct(array $records = [])
    {
        $this->records = [];

        foreach ($records as $record) {
            $this->records[$record->field_variables_key] = $record;
        }
    }

    public function getValue(string $key): string {
        return $this->records[$key]->field_variables_value;
    }
}

This is fine as far as it goes. That class is populated with the appropriate data and returned from our repository. And we can get values out by key. Thus, we have replicated the functional way that Drupal 7 does it in a more object-oriented way.

But we weren’t completely satisfied with this. There are a number of issues here that we can address to make this class both easier to use and more safe.

First of all, we know what the keys are because they’re pre-determined field names. But the caller doesn’t know what those names are in our code. What happens if we call a key that doesn’t exist. We’re going to have an unrecoverable exception. Naturally, we hope we don’t run into that. Surely we’d test this in dev, run into an exception for a field that doesn’t exist, and fix it. But applications do not run on hope. Suppose that data isn’t in the database in production? There’s nothing saying it HAS to be there.

With those things in mind, there are two things we can do to make this code safer and easier to use at the same time.

Let’s start with enumerating our known keys. There’s no reason to use a string as the key of known possible values. That just makes the caller guess at what a legal value is. We can modify our code to use enums in this manner:

enum InstanceVariableKeys: string
{
    case DISABLE_FEATURE_1 = 'disable_feature_1';
    case DISABLE_FEATURE_2 = 'disable_feature_2';
    case INSTANCE_NAME = 'instance_name';
    case INSTANCE_PER_PAGE = 'instance_per_page';
}

class InstanceVariableCollection
{
    /** @var InstanceVariableRecord[] */
    private array $records;

    /** @param InstanceVariableRecord[] $records */
    public function __construct(array $records = [])
    {
        $this->records = [];

        foreach ($records as $record) {
            $this->records[$record->field_variables_key] = $record;
        }
    }

    public function getValue(InstanceVariableKeys $key): string {
        return $this->records[$key->value]->field_variables_value;
    }
}

We now have an enum that enumerates the possible known values that the key can be. This both takes the guess work out of calling the getValue method and allows us to inspect a list of values to know what we might be after.

But we haven’t yet solved the problem of what to do about trying to get a key that may not exist in the databse but our application thinks it should. Since we know all the keys, we can also know what the defaults should be, at least in this case. There may be cases where you would want to throw an exception if a value was missing from the database, but I will not demonstrate that here because that was not the case in the code we were working with. I will just note that you’ll want to modify your code to throw a custom exception with relevant information about what went wrong rather than letting PHP throw an exception about a missing array key that will be harder to debug and track down due to its generic-ness.

In this case, we decided to provide some defaults for cases where a value might not exist in the database. Here’s what that looks like:

enum InstanceVariableKeys: string
{
    case DISABLE_FEATURE_1 = 'disable_feature_1';
    case DISABLE_FEATURE_2 = 'disable_feature_2';
    case INSTANCE_NAME = 'instance_name';
    case INSTANCE_PER_PAGE = 'instance_per_page';

    public function defaultValue()
    {
        return match($this) {
            InstanceVariableKeys::DISABLE_FEATURE_1 => false,
            InstanceVariableKeys::DISABLE_FEATURE_2 => false,
            InstanceVariableKeys::INSTANCE_NAME => '',
            InstanceVariableKeys::INSTANCE_PER_PAGE => 20,
        }
    }
}

class InstanceVariableCollection
{
    /** @var InstanceVariableRecord[] */
    private array $records;

    /** @param InstanceVariableRecord[] $records */
    public function __construct(array $records = [])
    {
        $this->records = [];

        foreach ($records as $record) {
            $this->records[$record->field_variables_key] = $record;
        }
    }

    public function getValue(InstanceVariableKeys $key): string
    {
        return $this->records[$key->value]->field_variables_value ?? $key->defaultValue();
    }
}

With that update, we should never get a thrown exception from our getValue method because we know all the values that the keys can be, and we’ve made sure they all have defaults. But there’s one further refinement we can make here. You may have noticed that all the variables are string values coming from the database, but we have things like disable_feature_1. The string that comes back from the database is (string) 0 or (string) 1. What we really want to do in those cases is convert that to boolean. Right now it’s on the caller to then do a string compare against '0' or '1' or cast to boolean. And we also have possible integer values coming out of the database as strings.

To make our collection a little smarter, let’s break it up a little like this:

enum InstanceVariableKeysString: string
{
    case INSTANCE_NAME = 'instance_name';
    
    public function defaultValue(): string
    {
        return match($this) {
            InstanceVariableKeys::INSTANCE_NAME => '',
        }
    }
}

enum InstanceVariableKeysBoolean: string
{
    case DISABLE_FEATURE_1 = 'disable_feature_1';
    case DISABLE_FEATURE_2 = 'disable_feature_2';

    public function defaultValue(): bool
    {
        return match($this) {
            InstanceVariableKeys::DISABLE_FEATURE_1 => false,
            InstanceVariableKeys::DISABLE_FEATURE_2 => false,
        }
    }
}

enum InstanceVariableKeysInteger: string
{
    case INSTANCE_PER_PAGE = 'instance_per_page';

    public function defaultValue(): int
    {
        return match($this) {
            InstanceVariableKeys::INSTANCE_PER_PAGE => 20,
        }
    }
}

class InstanceVariableCollection
{
    /** @var InstanceVariableRecord[] */
    private array $records;

    /** @param InstanceVariableRecord[] $records */
    public function __construct(array $records = [])
    {
        $this->records = [];

        foreach ($records as $record) {
            $this->records[$record->field_variables_key] = $record;
        }
    }

    public function getString(InstanceVariableKeysString $key): string
    {
        return $this->getValue($key);
    }

    public function getBoolean(InstanceVariableKeysBoolean $key): bool
    {
        return (bool) $this->getValue($key);
    }

    public function getInteger(InstanceVariableKeysInteger $key): int
    {
        return (int) $this->getValue($key);
    }

    private function getValue(
        InstanceVariableKeysString|InstanceVariableKeysBoolean|InstanceVariableKeysInteger $key
    ): string|bool|int {
        return $this->records[$key->value]->field_variables_value ?? $key->defaultValue();
    }
}

Now we have solid types coming out of our collection class because we already know what each value’s type should be.

You could argue that there’s a better database design, and you’d be right. This is not the way I’d design the data in persistence. But this is the system we have. And what we’ve done is encapsulated the responsibility of dealing with the data idiosyncrasies as best we can, in the safest manner we can for the system we’re working with. We now have very safe code giving out properly typed values that we can work with on the other side. The caller need not know all the silly stuff. It can get the value of one of the enumerated keys and be on its merry way.

And we’ve made it really easy to know what the legal values are. This makes the system easy to use, and easy to maintain. And we made the intention of each key very clear with types.

I hope therefore that this inspires you to think about ways to make your code clear and future use easy with enumeration.

What is technical debt, anyway?

It’s a salient question and one that I think is a little muddy. Is technical debt a thing that happens when software stagnates and the world around it moves on? Or is it something that happens when poor decisions are made in software design? Or is it something that occurs because business needs change and those changing needs weren’t fully taken into account when building the original software? Or, to consider further, is technical debt what happens when an engineer or a team of engineers take shortcuts (warranted or not) to meet deadlines, get applications out the door, or are lazy?

The problem is that I’ve seen all these types of things referred to as technical debt.

In addition, there’s a general conception that technical debt results from bad code. The trouble is, the frustrations and head-desking™ that are generally associated with technical debt are not always the result of bad code. But calling it technical debt tends to be associated with that mindset and tends to push us to re-write applications that just need some standard maintenance care. It also makes us think that the previous developers on the project were not good developers and that us current developers know what’s up, yo! But that is rarely the case. It also tends to give us this idea that if we just write the code correctly in the first place there will be no frustrations in the form of what we call technical debt.

Now, I’m not here to say that technical debt, as a phrase or an idea, isn’t valid or doesn’t have a place in our vocabulary. Indeed, I quite like the way we can talk about how technical debt needs to be paid off just like any other kind of debt. And I enjoy talking about the interest rate of a given type of technical debt and how it can crush you. And how, if you keep taking out those tech debt loans, you will eventually reach a point where you can’t even pay the interest, much less make any meaningful progress on the debt itself.

All those things are valid, but I don’t think they really get at the heart of the issue. Technical debt is, in some ways, the wrong abstraction. It doesn’t really answer the real question that we need to be asking, which is: what is the maintenance load of what we’re working on?

Maintenance load

In the majority of cases, I think we’d be better off to talk about maintenance load. How much time does it take to maintain this application? What’s the cost of change? Maintenance load is what we’re really after when we talk about technical debt anyway. What does it take to keep the lights on? How hard is it to change or add features?

Maintenance load also implies that the work is necessary and unavoidable. And that’s good because we’re not starting out with a combative mindset toward our software. Running any application carries with it a certain amount of maintenance. You can write the cleanest of clean code all day every day, and you will still have maintenance load.

Referring to the maintenance load of software reminds us that software is not free to run and that it does have a cost associated with it. And as you add features, or change features, if you approach it with the question and mindset, “how much load is this adding to our team?” you will be much better positioned to run the app and add the features.

It also gives you something much more concrete to talk about to stakeholders asking for features or updates. And when you can evaluate what your current maintenance load is, and how much you would be adding, and weigh that against your team’s capacity, you then know something much more solid than, “well, there’s a lot of technical debt and things are kind of icky.” When you know the maintenance load, then you can identify solutions such as refactoring or reworking the parts of an application that are giving you trouble, or hiring more engineers. Or perhaps the decision can be made to postpone or rethink features. It’s the difference in talking about things being kind of hard to manage, vs thinking more concretely about capacity. From there you can decide whether capacity is being eaten up by maintenance load that can be reduced or not.

In some ways it’s a subtle shift in thinking, but I deem it a valuable one and I hope maybe you can see how it is as well.

I had the privilege of presenting on co-location. What is co-location? Well, I’m glad you asked. Embedded is a video of the presentation I gave. Below the video is my presentation in text form (adapted slightly to fit the text format). So you can pick your preferred format, or do both!

Let’s Be Better Engineers

I’ve made it my goal for as long as I’ve been a software engineer to become better and better at my craft and what I do. I am not content with reaching a certain level and staying there. I don’t like the feeling of coasting. I hope that’s a goal everyone shares. That’s really what these concepts are about. I’ve spent the last ~20 years honing my craft and I have some thoughts, ideas, opinions, and suggestions. This post represents but a handful of those things.

Code Comments

As we start walking down this path, I want to start off by thinking about code comments.

Now, I could talk about the purpose of code comments. I could say that you should comment the “why” of your code and not the “how.” I could spend time talking about how you shouldn’t clutter your code with needless comments. I might then dive into code comment philosophy and talk about keeping your code comments fun and casual, but brief and to the point. But I don’t want to talk about any of that (although, as you can see, I have opinions on the subject).

Instead, I want to talk about where code comments belong and why they belong there.

I suspect you probably haven’t given a lot of thought to why you put code comments where you do because they’re pretty intuitive to be honest. But let’s interrogate that for a minute.

Do you put code comments in one big documentation file? Or do you put them in a docs directory with a file structure that mirrors the file structure of the class or function or whatever that you’re working on? Do you put all code comments on the company wiki, Jira, Confluence, your ticketing system, what have you?

I do hope those thought experiments sound silly to you. They’re supposed to.

Code comments are best and most helpful when they are co-located with the code being commented upon. “Of course,” you say. It’s silly to think otherwise. And you’re right! But why is that? It’s so intuitive we don’t stop to think about it. But here are a few reasons:

1. Ease of use while writing or updating code. The code comments are right there and can’t be missed.
2. If the code comments aren’t right there with the code, how would we know they exist at all?
3. If we have to maintain a mental model of where code comments go for a particular bit of code, what comments might be available, where to go to write comments, we’re really just adding a mental tax to our ability to reason about our code. And then, even if we know comments exist somewhere, how do we go about finding them when we go to update the code?
4. How would separated code comments get maintained over time when you can’t see them right with the code you’re working on?

Code comments inline with the code makes things simple, easy to use, and easy to maintain.

Organizing By Technical Concern

Given the title of this post, I hope you’re starting to get a sense of where I’m taking this. Things that go together, should, well, go together. But before we talk more about that, I want to explore the more common convention in PHP and discuss what I see as the shortcomings of that approach.

By far, the most common convention for code organization that I’ve seen over the last 15 or so years is to organize by technical concern. I’ve inherited many PHP code-bases over the years, and they’ve all been organized in this fashion.

And if you look at the common conventions of PHP projects and framework recommendations over the last many years, you can see that developers have had a tendency to organize their code by technical concern. And why is that? It’s become those technical concerns are quickly apparent to us and to framework designers when no other concerns are known about an application and its needs.

I would also say this is particularly true for those who know less about systems design and maintenance. But we sure do know a lot about language features, classes, services, etc.

So often, PHP projects will have a namespace for controllers, a namespace for models, a namespace for services (whatever those are), a namespace for views (again, whatever those are, though they tend to be template files in, say, the Laravel world).

Of course, the dividing line between a model, a service, or a controller can all get pretty fuzzy and vague. I love to draw boundaries around vagaries, but we can’t start doing that until we understand that the conceptual model around which we’re trying to draw boundaries in this paradigm is an unhelpful one. I believe that it pushes us in directions that make our code harder to understand, more prone, and harder to change overtime — which is the biggest problem when considering this pattern.

Organizing by technical concern encourages us to cross boundaries in our namespaces all the time to get what we need to get the job done. And this is because the boundaries we’ve drawn are meaningless, architecturally speaking. I’d wager, in fact, that you don’t even think about the organization of your code as drawing boundaries. You simply think of it as some way to put things somewhere in the system. So when a “controller” needs to get a service or 10 from the “services” namespace to do its job, we cross that boundary and get it without batting an eye. And in this way, we’ve trained ourselves that boundaries are meaningless.

Organizing by technical concern is as if you went into a library and all the books were organized by color. If you want a red book, you have to go to the south wing of the library. If you want a blue book, those are on the east wing. And gray books are on the upper level.

Those are obviously not very helpful ways to organize books ^[1], so what if, instead, we organized books by size? 4x8 books are on the lower level and 6x9 books are on the upper level.

That’s also obviously unhelpful. When you’re looking for a book, you care about concerns other than color or size. You might be interested in genre, author, title, etc. The same is true for our code. When you need to work on a feature, you’ll have a hard time finding all the components of that feature if they’re all organized by a different concern.

Houston, we have a problem

As I started coming up through the ranks as a software engineer, I, like many, become quite opinionated about how code should be organized. And that was along those technical lines, because that’s what I knew and that’s what I was taught. But over time, as I started working on larger and larger projects more often, I found it became more and more difficult to work on isolated features of systems.

And so I started asking, why are we organizing our code this way? Why do we separate our code along technical lines? What’s the advantage of doing so? How is this helping us? And I could not come up with satisfactory answers to those questions.

That’s a difficult position to be in. Everyone I looked up to and respected in the PHP community organized by technical lines and I don’t like to go against the grain. So I started looking around at other communities and other ways of doing things.

So, I’m not the first person to talk about this, and none of these ideas originated with me specifically. I’d even say, these days, more people in the PHP world are asking these questions. But at the time I didn’t know of any.

Locality of Behavior

So what can we do instead? If we’re not happy organizing by technical concerns, what principle should guide us on this journey into the unknown? What can we possibly do instead?

The primary feature for easy maintenance is locality: Locality is that characteristic of source code that enables a programmer to understand that source by looking at only a small portion of it. — Richard Gabriel

Richard Gabriel is a retired software engineer who worked in Lisp, primarily. I believe this quote captures the heart of what I’m getting at. Locality is the thing that I didn’t have in the codebases I was working on that I desperately wanted. How do we get it?

The way forward is to organize our namespaces by “use case” or “business case” rather than technical lines.

• This allows for the locality Richard Gabriel described by creating logical, well-defined boundaries
• “Use cases” are about how users interact with a system
• “Use cases” allow the system to be broken into logical components
• “Use cases” are designed around features and system capabilities
  • Again, from the standpoint of the end-user of the system
• It should be noted that the end-user can be
  • a human (computer or mobile devices, etc.)
  • a machine (a cron job, a scheduled job, etc.)
  • or both

So how can we become better engineers? That’s the question before us. I’d argue that in order to become great software engineers — in order to level up and be a cut above — we need to learn how to draw good boundaries.

I started out drawing those unhelpful boundaries around technical concerns because I could easily understand those sorts of distinctions (or so I thought) in my inexperience with how systems work and what’s needed to maintain them.

But I found that in order to grow as an engineer, I had to start thinking deeply about the flow of code and how applications actually do what they need to do. To be great at what I do, I had to learn how to draw good boundaries.

I found that drawing boundaries around technical lines actually became a huge hindrance to writing good software because it’s an unhelpful abstraction just like organizing books by color or size would be for libraries. It does not help with the mental model of how an application actually works.

Good Patterns

Now, I’m a PHP guy, but I want to take a step back for a minute and talk about some front-end code. I am a full stack engineer and I write code all over the stack. Specifically, I want to talk about React.

I avoided React for many years, despite its rising popularity. I finally had to cave when the technology we chose to re-build the front-end of a flagship application was React. It turns out React is really amazing for web applications. Further, I find React has revealed some patterns that may be relevant to us all.

It turns out that keeping related things as close together as possible reduces complexity and decreases mental overhead. And with modern Javascript frameworks, a byproduct has been that related things are kept together.

But let me back up even further to when I started using Tailwind CSS. I was already well down this road of thinking with PHP, but I hadn’t fully made the turn nor did I completely grasp the significance of this paradigm shift to apply it to my thinking on the front-end as well.

I balked at Tailwind at first. “Isn’t this just inline CSS? And, after all, we all know inline CSS is bad!” But as I slowly gave in, I discovered something. It became apparent to me that what Tailwind provides is a way to keep all the related bits of a component in one place — co-located together. With Tailwind, there’s no more hunting down CSS for the HTML you’re working on, it’s right there. There’s no more worrying about whether I can or should delete CSS declarations because it’s all right with the component inline.

I had the “eureka” moment with Tailwind that it was exactly what I was pursuing on the PHP side. So as you can imagine, when I started using React, I used it with Tailwind and everything clicked into place. Now ALL the things related to each other were in the one place: HTML, CSS, and Javascript.

Of course, with React, you can import components from anywhere in your filesystem, so you could still get this wrong, but that’s not the encouraged way in React. And certainly on my team, when we build the pieces of the front-end that make up the apps, we keep everything together. If we need a custom hook, or a subcomponent, we’ll create those files in the same directory as the parent. If something ends up being useful in more than one place, perhaps we’ll carefully consider and move it up a level. But even that is done carefully (we don’t want to introduce complexity through hasty abstractions to de-duplicate incidental duplication). So in this way, we maintain locality of our front-end components.

Coming up through the front-end world in the mid to late 2000s, we all got quite a dose of the “separation of concerns” mantra. Specifically, that HTML, CSS, and Javascript should all be kept separate because you don’t want to mix those concerns. Ever. And I believe this. I even taught it to new hires and younger engineers in the early 2010s.

The problem is, we have to think more carefully about what concerns should be separated, and what are the concerns. When you start approaching your code from a “use case” or “business case” point of view, you suddenly realize that what you have with HTML/CSS/JS are not 3 separate concerns, but 3 technical aspects of the same concern where HTML provides the structure, CSS provides the styling, and Javascript provides the behavior, all the with the goal of empowering the use case that’s in view.

So what I and my team discovered when we started building front-ends in React is that thing Richard Gabriel called “locality.”

Applying a heuristic

How can we apply all this specifically to the more back-end side of the code? How can we re-think how we organize our code along usage or business lines and not technical lines?

I’m going to give you a simple heuristic that I have found immensely helpful in this journey. It still helps me. I still apply this heuristic whenever I’m adding or updating a feature. I push this heuristic as the leader of my team as my predecessor did.

You may have heard of the idea that “things that change together go together.” And that’s a perfectly fine idea. But I think to get at the heart of it, the heuristic we can apply is, “is this thing easy to delete?”

In other words, does it have tendrils so deep into the rest of the codebase that deleting it would be a real slog? Whether you would ever actually delete the thing you’re building or not is immaterial. What we’re after here is the mental exercise, which is, I would argue, the approach that leads to far better architecture and code quality.

Another way you can ask the question is, “what is the blast radius of the thing we’re adding to this app?”

The result is carefully encapsulated and contained pieces, with 1 or 2 tendrils into the outside world that make up the application. This is the co-location of related things.

Microservices?

You probably remember microservices? That old idea that took off in the 2010s? I jest… a little. But it really was a craze that has died down into a more proper placement by now. You may be employing them to one degree or another. I survived the craze, and I currently work on an app that started down the microservices route. I would argue that it was split up along unhelpful lines, but the thinking is not entirely without merit.

Now, I would argue that most apps don’t need 50 microservices. I find that to be way too much separation for the vast majority of cases. Most things need 2 or 3 larger services at most. And when you have only 2 or 3 services for your application, you don’t have to deploy 8 microservices to make a single coordinated change. And if you need to scale performance, there’s load balancing, primary and secondary mysql servers, etc. But I digress…

The point is, some of the thinking behind microservice is worth exploring. A lot of what we were doing there wasn’t a bad thing.

One of the ideas was to draw boundaries and co-locate related things together in one place, and then to contain them to that one place.

And what I’ve been describing is a way to do that, but within one application, not many microservices. What we want to do is draw good boundaries within the application or project.

So having laid out a lot of the philosophy behind co-location rather than technical concern, I don’t want to just leave it there. I’d like to dive into some more specific examples.

A Specific Example

Organizing by use case is sort of like microservices within one application. What we’re doing here is creating boundaries around a use case for the application. In PHP, I tend to draw those use case boundaries at the top level of my src directory.

When we organize this way, it should make us think very deeply about boundaries and dependencies ^[2] as we design these “microservices” within our application in a way that makes sense together.

And when we organize this way, we should be able to jump into a project and start to comprehend it right away easily and quickly at a high level.

For the purposes of this topic, I’ve created a small, simple little notes/todo application demo. It’s not a feature-rich app. I would not even call it deployable in its current state. It lacks niceties. I created it in about 1.5 days. But I wanted to create something simple to demonstrate the thinking behind co-location that you could easily dive into. This is just to give you a taste and to help you get the gears turning.

It’s a PHP app with a little tailwind sprinkled on top. You can clone it and run it on your own computer, inspect the code, etc.

There are two main features in this app: todos and notes.

As we start to dive in to how this application is organized, note the comprehensibility right way. You can see a couple of technical concerns at a high level: Authorization and Persistence. And that’s okay. Apps are technical, and they’re going to have technical use cases. Note that I’m treating them as use cases and that their related things aren’t scattered across the known universe.

Now let’s imagine you’ve been put on this project, and you’ve never seen this application before. You’ve been given a bug fix for the Notes feature. Where do you go to work on such a feature?

I can’t pause for dramatic effect in a text post the way I can when presenting this to a live audience, but pretend I did…

As you can see, it’s not a trick question. You go to the Notes directory. Everything to do with the Notes feature is right there in the Notes directory. And it’s pretty self-contained. The tendrils to and from the world outside the Notes directory are limited. Somewhat repeatedly to my footnote on dependencies, they are also provided. Nothing in this directory reached out to get what it needs via service location or anything like that. Dependencies are declared in constructors to be provided by the DI Container, or via routing and request objects. You might recognize this as the “inversion of control” principle.

Let’s dive in just a smidge and look at the class that receives the incoming HTTP request to list the notes: \App\Notes\GetNotesListAction.

As we look over the class, we can easily see the tendril coming in from the outside world as the $request object in the __invoke method argument. In that way, we see that it accepts an incoming HTTP request. This is an entry point from the outside world — a tendril, if you will. Because we’re declaring that we need that request object and we know we need to know a little about the outside world here, we get from that request the things we need.

This HTTP action collects that information, dispatches a request to the more inner workings of the Notes feature, gets back what it needs, passes some information to a template to be rendered by a template engine, and returns that output as the ResponseInterface body.

Because everything else is happening within the Notes boundary, you have just, at a very high level, comprehended the Notes Listing feature of the application.

So then, what if you needed to work on adding a note, where would you go in that directory structure? Here it is again for reference:

Once again, it’s not a trick question. In the way I’ve organized this application, it’s very clear that where you would go is the Add directory. I’ve considered that a sort of “sub” use case of the Notes feature.

I won’t go over every little detail of this little demo app, you are welcome and encouraged to do that on your own. But I think you can start to see the organizational principles at work as well as the intentionality of the boundaries. Very little in the Notes namespace depends on the outside world.

And to line this up with our heuristic: if you would like to delete the notes feature, just remove that namespace and everything in it, then remove that one little tendril leading to it by removing the HTTP route. Then, poof, it’s gone.

You may also want to drop the database table that stored the notes, but that’s entirely up to you.

Separation of Concerns

I’m not normally a mind-reader, but I can almost hear all the thoughts of my readers out there emanating in my direction. And those thoughts are going something like this, “what about separation of concerns?”

Separation of concerns is a very good thing to think about. It is, I would say, a fundamental, core principle of good software design. However, as you might guess by now, I think we need to revisit what concerns need to be separated. What are the useful separations that drive better behavior, easier maintenance, and higher code quality? And what are the useless and even harmful separations that, at best, serve no purpose, and at worst drive poor software design?

If we diagram separating “concerns” by their technical nature, I think we can get a clear sense of just how meaningless these separations are. Conceptually, things are pretty messy here. Boundaries mean nothing because the organizing principle is an unhelpful one, just like organizing books by color in a library.

And look at just how often those boundaries are crossed. We jump to a new boundary at pretty much every conceivable point in the application flow. And because we’ve drawn these meaningless distinctions, we can’t actually draw meaningful distinctions. We don’t even see the lines we’ve drawn as boundaries and they’ve become distinctions without a meaning.

For the next diagram, we’re going to, instead, draw the lines of communication and let that reveal the architecture to us. In order to draw good boundaries, we have to ask, “what elements are constantly talking to each other to get the job done?”

In this diagram, everything gets a lot clearer. You can see those tendrils into the boundary, then all the communication happens within that boundary. All the related elements are kept together with that namespace drawn around them. And we can see the other boundaries floating around out there, but we don’t care about those right now because we’re working within the relevant namespace to our feature or use case.

And in this model, because we’ve drawn good boundaries, the need to cross the boundary should now stop and make us think, “should I really cross that boundary?”

I hope you remember this proposition from the beginning: that we want to become better engineers. So the question is, can all this make us better engineers? Isn’t this just technical preference stuff? Suzie prefers to organize her code along usage lines, but Bobby prefers to organize his code along technical lines. And isn’t that all it is? A preference?

It should be obvious by now that I think boundaries and encapsulation are a big deal. I’d say, in fact, that learning to draw good boundaries is when I went from a mid-level engineer to a senior+ engineer.

You know the old joke that there are 2 hard things in computer science, 1. naming things, 2. cache invalidation, 3. off by one errors. Well, naming things is hard, and what is naming things except declaring what something is and what something is not. It’s drawing clear boundaries. And what I’m proposing here is that we need to expand our concept of naming things beyond just that final segment of the namespace. We need to be declaring good boundaries by naming things well with all the segments of the namespace.

As a mid-level engineer, I threw all the classes of a certain technical category into a namespace for that category. But that is far from scalable and doesn’t encourage us to think about our software and how it’s organized for long-term stability and maintainability.

An additional heuristic we can apply as we look at those diagrams is, “can a non-technical person understand the boundaries by their names?” Whether a non-technical person ever would look at the code is immaterial to the exercise. What we’re after here is comprehensibility. There’s a sense in which the machines running the code don’t care about the organization of our code. That’s why we talk about comprehensibility. Source code is about us and how we can read it and maintain it as humans. And so it’s about making us better engineers.

Drawing boundaries in unhelpful places encourages thinking about our code in the wrong kinds of units — units that don’t make sense to the business of the application or the real world use cases.

Conversely, drawing boundaries around use cases is a much more helpful way to think about your code and its flow. We’re drawing distinctions in functionality and purpose in a way that aligns more closely with the work we’re doing at the moment, it encourages encapsulation of responsibilities that make sense, and makes features easy to maintain and easy to delete.

So ask these instigating questions the next time you start a new feature:

1. Where is the real, meaningful boundary for this thing?
2. Is it easy to delete?
3. Can a fresh developer walk into the project and understand the codebase?

Diagram those lines of communication, even if it’s just in your head. Elements that are constantly talking to each other to get their jobs done probably go together.

You can find the sample app here.

I want to highlight three problems with this approach, directly caused by the use of a service locator.

• There’s a bigger chance of runtime errors.
• The code is obfuscated to the outside.
• It increases cognitive load.

Let’s look at these problems, one by one.

While he focuses on cognitive load the most heavily perhaps, in my estimation, the runtime errors ^[1] issue is the biggest one for me. Being able to know from a static analysis standpoint whether your code should run okay or not is huge. This also factors into my love of strong typing as well — which is yet another topic for yet another day.

The Big Bad Monolithic Re-Write

A full, all at once, hard cut-over re-write is almost never advisable. Granted, I’ve done it once or twice. The most notable was my own website where I market and sell my add-ons for ExpressionEngine and Craft, BuzzingPixel.com. I did it for various reasons that are uninteresting, and it worked well for the most part. However, I did run into two or three cases that I forgot to account for, that were accounted for in the old, janky system. And it really pissed off a couple of customers that found their software was disabled or their experience degraded in some way by the re-write.

The bottom line is it may have been inadvisable for me to do a monolithic re-write. And, guess what? It’s been a few years now, and I’ve moved on from the ways of thinking in which I wrote that codebase and now I hate working on it. It’s all very much legacy code.

I’m fairly confident that my way forward with that codebase from here out, if I need to implement major things or make major changes, will be the strangler fig method.

One of the problems with monolithic re-writes is that it’s nearly impossible to account for every case the old and difficult to understand system was covering. Not everything will have a place in the new re-write, and something — often many somethings — will be missed. This can be true for patterns other than the monolithic re-write, but it is most true for this pattern. The old has to be cut off and the new put in its place. A switch has to be flipped. This means testing is hard, and once the switch is flipped, every case that wasn’t accounted for has to be immediately found and remedied.

I recommend not doing a monolithic re-write nearly every time. So, what then? What can we do?

Strangler Fig

I keep referencing this thing. I’ve even named my post after this thing. But what is it? Martin Fowler has written about it. He, in fact, has pioneered the use of this nomenclature in software design, so let’s see what he has to say:

When Cindy and I went to Australia, we spent some time in the rain forests on the Queensland coast. One of the natural wonders of this area are the huge strangler figs. They seed in the upper branches of a tree and gradually work their way down the tree until they root in the soil. Over many years they grow into fantastic and beautiful shapes, meanwhile strangling and killing the tree that was their host.

He goes on to describe this as a really good way to approach a re-write. And he describes a strategy:

In particular I’ve noticed a couple of basic strategies that work well. The fundamental strategy is EventInterception, which can be used to gradually move functionality to the strangler fig and to enable AssetCapture.

This is not a bad strategy, but based on my experience, I believe we can think more broadly about how to apply a strangler fig mindset.

1. ABC - Always Be Correcting

Okay, okay, I’d rather say, “Always Be Improving”, but then it’s not an alphabetical acronym. This can be a very broad application of a strangler fig mindset. As you constantly work on a system, always let your new code and the approaches and lessons you’ve been learning over time strangle out the old. Don’t stick slavishly to the old way of doing things.

I recently heard someone say something to the effect of, “just pick a convention for your project and STICK TO IT.” It’s hard for me to state just how strongly I disagree with this approach. Since you’re always writing legacy code, it’s already a big enough battle to keep things up to date and modern. Don’t complicate it by overburdening yourself with a dogged commitment to paths you chose with less knowledge and experience. Never be afraid to update your priors. ^[1]

So please do apply your experience, knowledge, and wisdom as you work on older codebases. ^[2] And remember that knowledge is not static, it grows and changes over time.

2. Release Immediately

In some ways, this could have perhaps been option 1.5, but I wanted to emphasise it:

When you see parts of the system that need to be tended, refactored, or updated, and you have time, do it and release it. This is particularly easy if it’s pretty straightforward nothing else needs to be coordinated, and there are no user-facing changes.

As a software engineer, you are more than just a vicarious mouse and keyboard for project stakeholders. You are one of the stewards of the codebase(s) and code quality therein. It falls on your shoulders to maintain the code and keep it in good shape. Pay a few cents now for the many dollars worth of future return of being able to move quickly and easily in the future as the product and business needs evolve. Remove your old, harder to understand, less effective code by immediately replacing it with newer, more up-to-date, easier to understand code.

3. Feature Flags (or Feature Toggles)

We’re getting closer here to the Event Interception that Martin Fowler talked about. Of course, you can use feature flags to hide new parts of the system (and you should where applicable). But you can also use it to route code from an old code path to a new one. When you have some new part of the system that is replacing an old part of the system, you can use feature flags to do this “strangling”.

There are two main reasons for doing so:

1. You’re not entirely confident that the new code you’re about to release covers all the use cases of the old code. The feature flag is a safety valve. When users aren’t getting what they need out of your new application code, you can switch back and go do some more work on the new code. ^[3]
2. You need to coordinate the code changes with some other external factor — be it with another team in your company, perhaps coinciding with the announcement and availability of new features, or some service that your new code needs to talk to, etc.

While feature flags have a very broad applicability beyond the “strangler fig” adjacent use case, they are a powerful tool in your tool box for accomplishing a strangler fig mindset on an on-going basis. You can strangle out the old code at a time of your choosing with your new code.

4. Event Interception

Regardless of what system you’re using or how you accomplish it, the idea behind a full blown event interception is that you have something sitting between the consumer and your new and old application, service, or code intercepting the call. ^[4] That something determines how to route the call to the code based on whatever criteria you determine. If the interceptor determines that the new system can handle the request, it routes the request to the new system. Otherwise, it sends the request to the old system. The consumer doesn’t need to know anything else but to continue consuming in the same way as before.

When a full re-write of your application is required, rather than employing a monolithic approach, I would argue that the strangler fig approach is most often your best path forward. It allows you to re-write the system a bit at a time, release early and often to catch mistakes and missed logic from the old system for just one particular use case, and incorporate it as you go.

Case Study

I think a good case study is an application that I worked on and continue to work on. The application was originally built in Drupal many years ago (long before I was on the team). Drupal made a big transition with Drupal 8 that left the app without a good upgrade path. Knowing everything we know now, we would choose different platforms for building this application. However, Drupal powered this application and the business for years. It literally enabled the success of the company. Contained within the custom modules we added to that Drupal application over its many years of service is all the knowledge needed for the application to serve the business in the way it needs to.

So, for this app, we chose to take a hybrid approach. We determined that we would build a new front-end in React, and initially power it with API calls to Drupal. That meant in turn that we would update Drupal to serve JSON at its page URLs when a request was made to it from the new React application. Then, we would begin a strangler fig approach of updating all the API calls to a new PHP application. This is why I call this a hybrid approach. The React front-end was entirely new, including a fresh coat of paint. But the primary business logic was still contained in Drupal.

We rolled the React front-end app out with hardly a hiccup, after which we began writing the new PHP back-end. I came up with a list of every endpoint the new React app requests from Drupal, and made a todo task out of each one so we have a list. And now we pick them off one by one.

For the strangler fig configuration, we actually were very fortunate that the Drupal application’s URLs are powered entirely by query strings. So in React, our configuration is set up so that if there’s no path on our API call, it calls the Drupal powered API. And if there is a path, it sends the call to the new API. Once we’ve completed a re-write of an endpoint, we simply adjust the parameters of the call to go to the new API’s URI path and the configuration of our API call takes care of the rest.

This allows us to write endpoints in the new API one at a time, and roll them out when they’re ready. We can take plenty of time combing through the old business logic of an endpoint to find all the ways it could respond, and all the edge cases, and determine the best way to write that in our modern PHP app.

All told, it’s been a remarkably positive experience. We’re actually not done with the new PHP API yet, but we’ve converted several end-points over and it’s going very well.

The Long Haul

As we consider strangler fig, it’s adaptations and permutations, I think we find that it’s a mindset in which we create and re-create software for the long haul. Software is ever-evolving, ever-changing. We constantly find new and better ways to write software. Re-writing an application from scratch always has unforeseen consequences. In writing a new application alongside the old and strangling it out, we find a far easier route to stability and maintainability.

AHA (pronounced “Aha!” like you just made a discovery) is an acronym I got from Cher Scarlett which stands for

Avoid Hasty Abstractions

The way I think of this principle is beautifully described by Sandi Metz who wrote:

prefer duplication over the wrong abstraction

This is such a solid golden nugget of wisdom that I want you to read it again, then read Sandi’s blog post on the subject: The Wrong Abstraction. It’s fantastic.

Kent also has a really great presentation, which is embedded in the post if you follow the link.

DRYing things out

I am now primarily a back-end engineer working most often in PHP, but I started out in HTML and CSS. While creating sites with HTML and CSS, I quickly discovered that I had to do a lot of HTML code duplication across those static HTML files. This prompted me to look into PHP, where I could use PHP’s include feature to abstract the duplicated parts of pages into single files and then include them on the various pages. That provided a single source of truth for, say, the main navigation of the site, or the footer.

By pulling content from some single source files into several other pages, I had created a very basic abstraction that de-duplicated the knowledge and code that was previously duplicated.

After that experience, I learned more about the principles of abstraction, coupled with learning about the DRY principle. Being the opinionated extremist I can sometimes tend to be, I wanted to apply both to EVERYTHING. Any code that looked like “duplicate code” was up for grabs to be abstracted to a “single source of truth.”

Parched?

The problem is, just because code looks like it’s being duplicated doesn’t necessarily mean that it will stay duplicated, or that one part of the system using a de-duplicated abstraction will continue to evolve in the exact same way as another part of the system that same abstraction is being used in. The problem I’m describing here is what seasoned software engineers refer to as “incidental duplication.” This duplication happens all the time. Why? Because software implements similar patterns everywhere. And because these patterns look so similar, it is tempting to apply a blanket de-duplication approach.

De-duplicating incidental duplication has the opposite effect of what is intended. As we de-duplicate the code into various abstractions, we tend to cling to these abstractions, writing mitigations that handle more and more use cases as disparate parts of the system evolve in their separate ways. This makes the code powering the software hard to reason about, hard to understand, and even harder to change.

“Technical Debt” and the way out

Often, when you hear engineers talk about technical debt ^[1], what they’re really dealing with is hasty abstractions born out of a desire to de-duplicate incidental duplication. Over time, as hastily abstracted de-duplication grows, the burden of maintaining these abstractions, making simple changes to various parts of the software without affecting other parts of the software, becomes overwhelming. If something isn’t done, the software will eventually become so prone to bugs that it will be impossible to support.

There are 3 ways out of this problem:

1. Re-write the application
2. Refactor to re-duplicate the inappropriate abstractions
3. Don’t hastily abstract in the first place

1. Re-writing your application

This is most often the wrong call, though it can be necessary for this and many other reasons. Even then, it should be done carefully, and probably not all at once ^[2]. Often your new application will be missing business logic or features that the old system had. This is because the old and gnarly system, difficult to work on though it may be, contains within its inscrutable logic, a wealth of information necessary for the correct operation of the business, service, or use case it powers.

The reason for the re-write is the reason this logic and these features weren’t fully understood when moving to the new system. The engineers found the old application too difficult to understand. They probably would have had better luck de-tangling the old system first before a re-write, or even just sticking with the current system and refactoring it over time.

And if the reasons for the failure of the first application are not well understood — which is very often the amalgamation of many years of bad abstractions — then the new application will wind up in the same situation as the old application.

2. Refactoring to re-duplicate

When you find yourself in the sticky situation of poor and hasty abstractions, re-duplicating is your most useful way forward. Push the conditional branches of logic and mitigations that are applicable in the bad abstraction back out to where they’re being used. In doing this de-tangling, you’ll be able to find and understand the business logic of the application and preserve it.

3. Just say no to bad abstractions

Don’t create the wrong abstractions in the first place, and back out of them early and often when you do.

I realize bad abstractions are unavoidable. We’re all human, we all make mistakes. Even the most senior of us make bad calls and inappropriate abstractions. One of the keys to becoming a more senior engineer is to recognize these code smells very early in the process and back out of them quickly. Constant vigilance!

And of course, as we endeavor not to write these hasty abstractions in the first place, we should not be wary of code duplication. After you duplicate code, live with it for a while to determine if an abstraction is actually needed. ^[3]

Examining DRY

In my estimation, the principle of DRY has been widely misunderstood and misapplied. Let’s take a closer look.

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
— Andy Hunt & Dave Thomas - DRY—The Evils of Duplication - The Pragmatic Programmer

(emphasis is mine)

Many engineers hear about DRY and think about it only in terms of their code. That was me. I was “many engineers” at one time. But the DRY principle is really about creating single sources of truth for knowledge, not just code. I’d argue that DRY needs to be applied to code much more carefully than to knowledge within a codebase. As a simple (and even silly) example: you probably shouldn’t be manually writing out your application’s name in every place it’s seen or used. You should instead have a source of truth from which that knowledge is pulled. There may be many places where the code to consume that knowledge is duplicated, but the knowledge is not duplicated.

An important bit from the afore quoted chapter in the second edition of The Pragmatic Programmer:

Let’s get something out of the way up-front. In the first edition of this book we did a poor job of explaining just what we meant by Don’t Repeat Yourself. Many people took it to refer to code only: they thought that DRY means “don’t copy-and-paste lines of source.”

That is part of DRY, but it’s a tiny and fairly trivial part.

DRY is about the duplication of knowledge, of intent. It’s about expressing the same thing in two different places, possibly in two totally different ways.

I also encourage you to read the part in that chapter (available as a free PDF) under the heading, “Not All Code Duplication is Knowledge Duplication”.

Keep yourself hydrated, don’t be too DRY

So as you work on your applications, websites, etc., remember these principles:

1. Don’t be wary of code duplication. Duplicating code costs almost nothing. Inappropriate abstractions grow into monsters over time that significantly increase maintenance load and cost a great deal in the long run.
2. But do use the principles of DRY programming when considering the knowledge of your system.

If you bear these things in mind, I believe you’ll find the job of both writing and maintaining software a much lighter burden in the long run.

So, this is a sort of fresh start for my site and my “brand” (whatever that is I guess). I hope you enjoy my ramblings about software.