Score:1

Why might a batch initiated in hook_install() sometimes not process?

in flag

I have two modules, module_alpha and module_beta.

Both modules install a database table with hook_schema() which they use to track information about published nodes. Data is logged in the table with service methods executed from hook_node_insert() and hook_node_update() hooks. This all works.

But, I need to process all existing nodes when the module is installed to backfill the custom tables with information about the already existing published nodes. Each module implements hook_install() and uses a batch process to process all existing published nodes. However, I am seeing strange inconsistencies. Sometimes the batch process executes, sometimes it does not. A scenario may work for one module, but not the other. All of this even though the code is nearly identical. Here are the scenarios I've tested:

Module install method Module Alpha Module Beta
drush en MYMODULE Batch executes Batch does not execute
drush cim Batch does not execute Batch does not execute
UI, enable module from /admin/modules Batch executes Batch executes
UI, sync config from /admin/config/development/configuration Batch executes Batch executes

Batch executes indicates that the batch executes completely as expected, with progress messages, and my table is filled with data. Batch does not execute indicates that the batch is not executed at all. No error messages shown or in logs. The weirdest part about this data is that drush en module_alpha always succeeds and drush en module_beta always fails.

What could be causing the batch process to work in some cases but not in others? How can I consistently execute this batch process across all 4 of these methods, whether the module is enabled via UI or drush? To be honest, the only method I'll use is drush cim

Code:

In MYMODULE.install:

function MYMODULE_install() {
  $batch = new \Drupal\Core\Batch\BatchBuilder();
  $batch
    ->setTitle(t('Bulk processing existing items.'))
    // This operation references a different function per module.
    // In both cases, the function lives in MYMODULE.module and the code is nearly identical.
    ->addOperation('_MYMODULE_initialize_items', []);

  batch_set($batch->toArray());
}

In MYMODULE.module:

function _MYMODULE_initialize_items(&$context) {
  $batch_size = 25;
  $entity_type = 'node';
  // I target different bundles in module_alpha and module_beta.
  $bundle = 'MY_TARGET_BUNDLE';

  if (!isset($context['sandbox']['processed'])) {
    $context['sandbox']['processed'] = 0;
  }

  if (empty($context['sandbox']['entity_ids'])) {
    $context['sandbox']['entity_ids'] = \Drupal::entityTypeManager()
      ->getStorage($entity_type)
      ->getQuery()
      ->condition('type', $bundle)
      ->condition('status', 1)
      ->execute();

    if (is_array($context['sandbox']['entity_ids'])) {
      $context['sandbox']['total'] = count($context['sandbox']['entity_ids']);
    }
  }

  if (!empty($context['sandbox']['entity_ids'])) {
    $current_batch_ids = array_slice($context['sandbox']['entity_ids'], $context['sandbox']['processed'], $batch_size);
    $current_batch_entities = \Drupal::entityTypeManager()
      ->getStorage($entity_type)
      ->loadMultiple($current_batch_ids);

    foreach ($current_batch_entities as $entity) {
      // I use a different services and methods in module_alpha and module_beta.
      \Drupal::service('MYMODULE.my_service')
        ->processEntity($entity);

      $context['sandbox']['processed']++;
    }
  }

  if (!empty($context['sandbox']['total'])) {
    $context['finished'] = $context['sandbox']['processed'] / $context['sandbox']['total'];
    $context['message'] = t('Processed @processed of @total items.', [
      '@processed' => $context['sandbox']['processed'],
      '@total' => $context['sandbox']['total'],
    ]);
  }
  else {
    $context['finished'] = 1;
  }
}

Example service method called in the batch process:

public function processEntity(EntityInterface $entity) {
  // $this->db is injected `@database` service, i.e. \Drupal::service('database');
  return $this->db
    ->insert('MYMODULE_mytable')
    ->fields([
      'entity_type' => $entity->getEntityTypeId(),
      'entity_bundle' => $entity->bundle(),
      'entity_uuid' => $entity->uuid(),
      // etc...
    ])
    ->execute();
}
leymannx avatar
ne flag
I experienced something similar a while ago and found little information on DO. I tried to create some dummy terms on `hook_install`. This worked when the module got enabled via the UI or `drush en`. But it didn't work when the module got enabled during `drush cim`. After finding no solution I moved the logic from `hook_install` into a custom form in the end, to be triggered manually from the UI.
leymannx avatar
ne flag
Just found this one https://www.drupal.org/project/drupal/issues/2906107. And then there's this one https://www.drupal.org/project/currency_taxonomy/issues/3133817. Same problem.
sonfd avatar
in flag
Interesting. Could this be a bug with `drush config:import`?
leymannx avatar
ne flag
Yeah, I think so. Something's inconsistent somehow. And has not yet been untangled. Do your modules contain any config?
sonfd avatar
in flag
Nope, no config.
sonfd avatar
in flag
I’m voting to close this question because I now think this is basically a bug with `drush cim`. I think my case where module_beta isn't batching with `drush en` is just some kind of weird user error on my part. Throw that out and the issue is exclusively with `drush cim` https://github.com/drush-ops/drush/issues/5105
leymannx avatar
ne flag
Not sure but I think the question might get deleted after being closed for a while. Maybe get that well-documented post content into the issue as well instead of only linking here.
sonfd avatar
in flag
@leymannx Good call. Thank you.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.