Adding Middleware::retry() to a custom httpClient effects the default http_client

We have two http_clients in our custom service file on __construct, one default from Drupal's http_client service, another custom new Client created by the use of http_client_factory and http_handler_stack services following formOptions function.

The custom client was used to implement guzzle's retry mechanism by pushing Middleware::retry into the current handler_stack in our custom services. But this middleware seems to be consistent in the default httpClient as well, causing all client calls going through the retryDecider(). gist of reference

I want to seperate Middleware effect of both clients, what should I do? Thanks for providing your share of knowledge!

  /**
   * Constructs a new service object.
   */
  public function __construct(EntityTypeManagerInterface $entity_type_manager, ClientInterface $http_client, ClientFactory $http_client_factory, HandlerStack $stack, TimeInterface $datetime_time, ConfigFactoryInterface $config_factory) {
    $this->entityTypeManager = $entity_type_manager;
    $this->datetimeTime = $datetime_time;
    $this->configFactory = $config_factory;
    $this->httpClient = $http_client;

    // Utilise retry middleware to retry 2 times, base on timeout of 20sec.
    $stack->push(Middleware::retry($this->retryDecider(), $this->retryDelay()));
    $this->customClient = $http_client_factory->fromOptions([
      'handler' => $stack,
      'timeout' => 20,
    ]);
  }

  /**
   * {@inheritdoc}
   */
  public function apiRequest($type, $method, array $data = [], $retry = FALSE) {
    $client = $retry ? $this->customClient : $this->httpClient;
    switch ($method) {
      case "POST":
        $result = $client->request('POST', $this->requestUrl . '/' . $type, [
          'form_params' => $data,
          'headers' => [
            'Accept'     => 'application/x-www-form-urlencoded',
          ],
        ]
        );
        break;

      case "GET":
        $result = $client->request('GET', $this->requestUrl . '/' . $type, [
          'query' => $data,
          'headers' => [
            'Accept'     => 'application/json',
          ],
        ]
        );
        break;

      case "PATCH":
        $result = $client->request('PATCH', $this->requestUrl . '/' . $type, [
          'form_params' => $data,
          'headers' => [
            'Accept'     => 'application/x-www-form-urlencoded',
          ],
        ]);
    }

    $apiRequest = $result->getBody()->getContents();

    return json_decode($apiRequest);
  }

  /**
   * Boolean decider of retry attempts for Guzzle.
   */
  protected function retryDecider() {
    return function (
      $retries,
      Request $request,
      Response $response = NULL,
      RequestException $exception = NULL
   ) {
      if ((0 < $retries) && ($retries <= 2)) {
        $this->getLogger('API')->info('%uniqid SystemAction retryDecider msg="Retrying %retries"', [
          '%uniqid' => $this->uniqid,
          '%retries' => $retries,
        ]);
      }
      // Limit the number of retries to 3.
      if ($retries >= 2) {
        $this->getLogger('API')->info('%uniqid SystemAction retryDecider msg="Attempt on Retrying for Guzzle Client"', [
          '%uniqid' => $this->uniqid,
        ]);
        return FALSE;
      }

      // Retry connection exceptions.
      if ($exception instanceof ConnectException) {
        return TRUE;
      }

      if ($response) {
        // Retry on server errors.
        if ($response->getStatusCode() >= 500) {
          return TRUE;
        }
      }

      return FALSE;
    };
  }

  /**
   * Delay of each retries between request attempts.
   */
  protected function retryDelay() {
    return function ($numberOfRetries) {
      return 1000 * $numberOfRetries;
    };
  }