Aggressive Caching with Symfony HTTP Client

Section intitulée the-symfony-cachinghttpclientThe Symfony CachingHttpClient

The HttpClient component comes with a client that can cache responses when possible. It means the client won’t issue another request to the server if the response is already available in the cache. It works like your browser by reading some headers like cache-control or expires

Unfortunately, this client is not enabled by default. You must configure it manually.

If you are using the component directly:

$httpClient = new Symfony\Component\HttpClient\CachingHttpClient(
    Symfony\Component\HttpClient\HttpClient::create()
    new Symfony\Component\HttpKernel\HttpCache\Store('/path/to/cache')
);

The CachingHttpClient acts as a decorator for the regular HttpClient. It also implements the HttpClientInterface, so you can use it as a regular client.

And if you are using the Symfony full stack framework, you must configure it in the framework.http_client section, then decorate the client in the services section:

framework:
    http_client:
        scoped_clients:
            myClient:
                base_uri: "%env(MY_CLIENT_ENDPOINT)%"

services:
    Symfony\Component\HttpKernel\HttpCache\Store:
        arguments:
            $root: '%kernel.cache_dir%/http-cache'

    Symfony\Component\HttpClient\CachingHttpClient:
        decorates: myClient
        arguments:
            $client: '@.inner'
            $store: '@Symfony\Component\HttpKernel\HttpCache\Store'
            $defaultOptions:
                base_uri: '%env(MY_CLIENT_ENDPOINT)%'

Side notes:

• There is a Symfony issue to be able to configure the cache directly in the configuration framework.http_client;
• There is a plan to decouple the http-client cache from the http-kernel cache;
• Caching responses involves consuming more disk space. You must monitor this to avoid critical issues when the disk is full. And if the disk is really slow, it may lead to bad performance too.

Section intitulée beyond-the-default-behavior-aggressive-cachingBeyond the default behavior – Aggressive caching

We work on an application that must download a huge amount of data before booting. During the development phase, we don’t care if the data is very fresh or not. Since we don’t really like to wait, we wrote a small piece of code that we want to share with you.

Basically, the upstream server does not ship the Cache-Control or Expire header, or it’s not big enough. So we’ll set or increase this value. Obviously, we should not do this in production, and we should ensure that it won’t break the application. All responses are not cacheable, so you must use this carefully.

Usually, to add more behavior to a Symfony component, the best way is to decorate it. Let’s do it! Since we already use the CachingHttpClient, we’ll add a decorator to fake the cache-control header. Symfony will see the new value and cache the response accordingly.

If you use the component directly, you can configure it this way:

$httpClient = new Symfony\Component\HttpClient\CachingHttpClient(
    new App\Bridge\Http\Client\FakeCacheHeaderClient(
        Symfony\Component\HttpClient\HttpClient::create()
    ),
    new Symfony\Component\HttpKernel\HttpCache\Store('/path/to/cache')
);

If you use Symfony with the full stack framework, you can configure it this way:

# the same configure as before, but with the new decorator:

when@dev:
    services:
        App\Bridge\Http\Client\FakeCacheHeaderClient:
            decorates: myClient
            decoration_priority: 10 # Should be before the CachingHttpClient
            arguments:
                $client: '@.inner'

What looks like the decorator?

Section intitulée the-implementationThe implementation

It’s pretty straightforward, we overdrive the header cache-control with a big value. we chose 1 month (We really don’t like to wait 😅), but it’s up to you to choose the best value.

use Symfony\Component\HttpClient\DecoratorTrait;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;

final class FakeCacheHeaderClient implements HttpClientInterface
{
    use DecoratorTrait;

    public function request(string $method, string $url, array $options = []): ResponseInterface
    {
        $response = $this->client->request($method, $url, $options);
        // Break Async: we don't care here, but we need all headers to be able to update them
        $response->getStatusCode();

        return new class($response) implements ResponseInterface {
            public function __construct(
private ResponseInterface $response,
            ) {
            }

            public function getStatusCode(): int
            {
                return $this->response->getStatusCode();
            }

            public function getHeaders(bool $throw = true): array
            {
                $headers = $this->response->getHeaders($throw);

                // One month
                $headers['cache-control'] = 'public, max-age=2592000, s-maxage=2592000';

                return $headers;
            }

            public function getContent(bool $throw = true): string
            {
                return $this->response->getContent($throw);
            }

            public function toArray(bool $throw = true): array
            {
                return $this->response->toArray($throw);
            }

            public function cancel(): void
            {
                $this->response->cancel();
            }

            public function getInfo(?string $type = null): mixed
            {
                return $this->response->getInfo($type);
            }
        };
    }
}

And that’s all!

However, there is a little drawback by doing that, we broke the asynchronous capabilities of the Symfony HTTP Client, because we need all headers in order to update them. But in our situation it is okay since we don’t use them.

Section intitulée conclusionConclusion

If your application makes a lot of calls to external API, and the API is cacheable, you should consider using the CachingHttpClient. It will save a lot of time and resources.

And if you want to cache more aggressively, you can use the FakeCacheHeaderClient to fake the cache-control header. If you want to grab this code, please do. But you must remember that it may break your application because… it caches aggressively and dumbly!