The values returned from hook_node_access_records() aren't considered to allow access

I am implementing a solution in Drupal 9 where users are allowed to see only the nodes that belongs to their same group. If a node doesn't belong to any group, t hey should be able to see it. I've used hook_node_grants() to assign the right grants to the current user and hook_node_access_records() to set the view grant per node, as suggested in How do I filter a view by the results of hook_node_access?

When visiting a node a user isn't supposed to see, the user gets a 403 error, as expected result. On a view, the node is visible, though.

As workaround, I could implement hook_views_query_alter() and add an access condition, but I prefer more generic solution.

This is the code I am using.

 function mymodule_node_access_records(NodeInterface $node) {
  if ($node->hasField('field_group') && !empty($term = $node->field_group->getValue())) {
    $term = reset($term);
    $grants[] = [
      'realm' => 'mymodule_realm',
      'gid' => $term['target_id'],
      'grant_view' => 1,
      'grant_update' => 0,
      'grant_delete' => 0,
      'priority' => 0,
    ];

    return $grants;
  }
}

 function mymodule_node_grants(AccountInterface $account) {
  $grants = [];
  if ($group = checkGroupHelper($account)) {
    $grants['mymodule_realm'] = $group;
  }
  return $grants;
}

I've managed to get this working by adding Query tag node_access under my view's Query settings. It started filtering results based on the node_access table. Never seen it as a requirement, so that's a bit strange, but hopefully it can help others wondering like me.