Display field label (not key) as views page title in contextual filter

Finally figured it out! Posting my answer for future generations.

All we need is YOUR_TEMPLATE.theme file for this.

Below is my code with comments. It might not be perfect, so if anyone will correct some things I will appreciate that.

use \Drupal\views\ViewExecutable;

// Changing views title. It does not change HEAD <title> tag!
function YOUR_THEME_NAME_views_pre_render(ViewExecutable $view) 
{
  // checkRoute() and checkArg() are custom functions and described below
  if(checkRoute('view.VIEW_ID.VIEW_DISPLAY_ID') && checkArg())
  {
    // This changes view title, as well as setting $variables['page']['#title'] to the same value.
    // gamesGenreTitle() is custom function and described below.
    $view->setTitle(gamesGenreTitle($view->args[0]));
  }
}

// Here we can change HEAD <title>
function YOUR_THEME_NAME_preprocess_html(&$variables)
{
  // checkRoute() and checkArg() are custom functions and described below
  if (checkRoute('view.VIEW_ID.VIEW_DISPLAY_ID') && checkArg() && isset($variables['head_title'])) 
  {

    // Unnecessary piece, tokens are needed for my purpose
    $token = Drupal::token();

    // $variables['page']['#title'] was changed in views_pre_render(), so we can change HEAD <title>.
    // I use [current-page:page-number] token to uniqueize the title
    // To improve this, we can add logic to inlcude current page number only if there is more than 1 page
    $variables['head_title'] = $variables['page']['#title'] . " | Page " . $token->replace('[current-page:page-number]');
  }
}

// In case you're using current page title in breadcrumbs, might want to change it too.
function YOUR_THEME_NAME_preprocess_breadcrumb(&$variables)
{
  // checkRoute() and checkArg() are custom functions and described below
  if(checkRoute('view.VIEW_ID.VIEW_DISPLAY_ID') && checkArg() && isset($variables['breadcrumb']['2']['text']))
  {
    // You might want to var_dump($variables) before doing this trick in order to find your piece of breadcrumb.
    $variables['breadcrumb']['2']['text'] = gamesGenreTitle(getArg_0());
  }
}

// Getting allowed values of field (labels) used to make title
function gamesGenreTitle($val)
{
  $options = options_allowed_values(\Drupal::service('entity_field.manager')->getFieldStorageDefinitions('taxonomy_term')['field_NAME']);

  // Check if we are passing correct data
  if(is_numeric($val) && $val <= count($options))
  {
    return $options[$val];
  }
}

// Check if needed argument is set and numeric.
function checkArg()
{
  $arg = Drupal::routeMatch()->getParameter('arg_0');
  // In my case argument from views was passed to arg_0
  // It may vary, so try to var_dump($arg) or ksm($arg) to check if it is what you need

  return isset($arg) && is_numeric($arg);
}

// Checking current route, we need exact view and display, otherwise title will be changed on every page.
function checkRoute($check)
{
  $route = \Drupal::routeMatch()->getRouteName();
  return $route == $check;
}

// Getting arguments, passed to views.
function getArg_0()
{
  // And again, arg_0 might not be your argument, double check it with var_dump() or ksm()
  return Drupal::routeMatch()->getParameter('arg_0');
}

And another method, if you don't have a lot of field labels, you can use this in views title override field (twig is supported there):

{% if arguments.field_NAME_value == 1 %} YOUR TITLE 1 {% elseif arguments.field_NAME_value == 2 %} YOUR TITLE 2 {% endif %}

Hope this will help someone!