How do I grant to a role the permission to flush the cache?

You can alter the permission used to access a route, as described in Altering existing routes and adding new routes based on dynamic ones. In the specific case, the code used in alterRoutes(RouteCollection $collection) would be the following one.

Since the administer site performance permission is a custom one, it needs to be added as new permission by the module that implements the route subscriber class, as described in How to create a custom permission on Drupal 8 - 9.

For a module whose machine name is mymodule, the code would be similar to the following one.

mymodule.permissions.yml

administer site performance:
  title: 'Administer site performance'
  description: 'Access the site performance settings page.'
  restrict access: true

mymodule.services.yml

services:
  mymodule.route_subscriber:
    class: Drupal\mymodule\Routing\RouteSubscriber
    tags:
      - { name: event_subscriber }

src/Routing/RouteSubscriber.php

namespace Drupal\mymodule\Routing;

use Drupal\Core\Routing\RouteSubscriberBase;
use Symfony\Component\Routing\RouteCollection;

/**
 * Listens to the dynamic route events.
 */
class RouteSubscriber extends RouteSubscriberBase {

  /**
   * {@inheritdoc}
   */
  protected function alterRoutes(RouteCollection $collection) {
    if ($route = $collection->get('system.performance_settings')) {
      $route->setRequirement('_permission', 'administer site performance');
    }
  }

}

If the users with that role should not access the settings in that page (Caching and Bandwidth optimization), but only be able to clear the cache, I would implement a custom route that shows a confirmation form and clear the cache (which means calling drupal_flush_all_caches()) when the user confirms the operation.