Access Control for Entity List Builder

This maybe a very obvious and basic question, but I can't seem to find it asked anywhere else.
I have a custom entity which has its list builder defined in the annotation and not is routing.yml. The list builder class extends Drupal\Core\Entity\EntityListBuilder.

What I need to do is to implement some custom logic around who can access that page, however there is no access() method on that class as I would expect there to be, and there doens't seem to be a method in the defined access handler class (extends EntityAccessControlHandler) either.

Am I missing something very obvious?

The entity list builder doesn't provide routes. For this you have another annotation entry:

*     "route_provider" = {
*       "html" = "Drupal\Core\Entity\Routing\AdminHtmlRouteProvider",

Which by default points at DefaultHtmlRouteProvideror in this case at a subclass changing some routes to admin routes.

You can do the same, extend DefaultHtmlRouteProvider and override getCollectionRoute():

  protected function getCollectionRoute(EntityTypeInterface $entity_type) {
    // If the entity type does not provide an admin permission, there is no way
    // to control access, so we cannot provide a route in a sensible way.
    if ($entity_type->hasLinkTemplate('collection') && $entity_type->hasListBuilderClass() && ($admin_permission = $entity_type->getAdminPermission())) {
      /** @var \Drupal\Core\StringTranslation\TranslatableMarkup $label */
      $label = $entity_type->getCollectionLabel();

      $route = new Route($entity_type->getLinkTemplate('collection'));
      $route
        ->addDefaults([
          '_entity_list' => $entity_type->id(),
          '_title' => $label->getUntranslatedString(),
          '_title_arguments' => $label->getArguments(),
          '_title_context' => $label->getOption('context'),
        ])
        ->setRequirement('_permission', $admin_permission);

      return $route;
    }
  }

For details how to add a custom access() method to a route see https://www.drupal.org/docs/8/api/routing-system/access-checking-on-routes