Is there any way to add JS to form only when loaded using ajax-api in modal?

I have one page on which there is a button. On click of that button I am opening a webform in modal popup (Ajax dialog). What I need is to add a GA script to the form. I have tried using hook_preprocess_page and hook_form_alter to attach the script but none of these 2 works. However script gets attached to form if opened directly with URL.

   /**
 * @param $form
 * @param FormStateInterface $form_state
 * @param $form_id
 */
function custom_module_form_alter(&$form, FormStateInterface $form_state, $form_id) {
  /**
   * Custom javascript for the search form
   */
  if($form['#id'] == 'webform-submission-contact-us-add-form') {
    if(isset($_GET['x']) && ($_GET['x'] == 1)) { //just to check the path
      $form['#attached']['library'][] = 'custom_module/ga_popup';
    }
  }
}

In my template twig file I have the link to open this form like:

<a href="{{ contact_us_url }}"
            class="property-contact-us-btn webform-dialog webform-dialog-narrow button">{{'Contact Us' | t}}</a>

Any idea on this?

Thanks.