Custom 404 handling for images

I'm looking for a way to handle 404's on images in Drupal. I have created several subscribers, but none seem to get triggered. My first try was to create subscriber that extends RouteSubscriberBase. The alterRoutes method looked like:

protected function alterRoutes(RouteCollection $collection) {
    if ($route = $collection->get('system.404')) {
      $route->setDefault('_controller', '\Drupal\my_module\Controller\ImageRouteController::load');
    } else {
     (..)
    }
  }

This works perfectly for all 404's except for images (ending in .jpg or .png). This will still generate a default 404 page. It's actually a light-weight version, not the same as a 404 on a non-image non-existing page. When I debug through the Drupal code (starting in index.php) I see it goes through many steps before generating that light 404. So it does reach the Drupal code.

I found one particular interesting bit of code (which is reached when visiting a non-existing image):

$this->dispatcher->dispatch(KernelEvents::EXCEPTION, $event);

This can be found in the handle method of Symfony\Component\HttpKernel\HttpKernel. With that knowledge I managed to find this page: https://www.drupal.org/node/2331613 . Again, it doesn't seem to work. What I have now:

my_module.services.yml:

my_module.exception_image_subscriber:
    class: Drupal\my_module\EventSubscriber\ExceptionImageSubscriber
    tags:
      - { name: event_subscriber }

src\EventSubscriber\ExceptionImageSubscriber.php (removed comments):

<?php
namespace Drupal\my_module\EventSubscriber;

use Drupal\Core\EventSubscriber\HttpExceptionSubscriberBase;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\Event\GetResponseForExceptionEvent;

class ExceptionImageSubscriber extends HttpExceptionSubscriberBase {

  /**
   * {@inheritDoc}
   */
  protected function getHandledFormats() {
    return ['svg','jpg','gif','png','webp'];
  }

  /**
   * {@inheritdoc}
   */
  protected static function getPriority() {
    return 0;
  }

  /**
   * Handles a 404 error for images
   */
  public function on404(GetResponseForExceptionEvent $event) {
     dd('test'); // <- not firing?!
  }
}

This doesn't seem to do anything, not for non-images or for images.

I start to believe the problem is that modules are not loaded when images are requested. Is there any way around this issue?