Set local actions - disabled/inactive

I am currently searching for a possibility to set an action to "inactive" depending on something. I have found the hook_menu_local_actions_alter(&$local_actions), but do not know how I could use it to set a task to disabled/inactive.

What I have tried for now is:

function my_module_menu_local_actions_alter(&$local_actions) {
  $local_actions['entity.my_module.add_form']['options']['attributes']['disabled'] = 'disabled';
}

This adds disabled="disabled" to the action html element, but the element is not disabled.

I have posted a Workaround: https://drupal.stackexchange.com/a/303740/9586

This is not the perfect solution, but since I did not found another way to solve this I did the following:

function my_module_menu_local_actions_alter(&$local_actions) {
    $local_actions['entity.my_module.add_form']['options']['attributes']['class'] = ['is-disabled'];
    $local_actions['entity.my_module.add_form']['route_name'] = 'entity.my_module.collection';
}

What I did here is that I added a class to the action item which shows it as disabled. Just by doing this does not disable it, but just adds a "grey" style.

In addition to that I have overwritten the route to point to the same route as the action is shown on so that the user just gets redirected to the same url as he/she has been on before.