Migration entity_lookup not picking up repeating taxonomy terms

Chandra

4/22/24, 2:06 AM

Background: I am creating new articles in Drupal 9.5 from a source database using the Migrate API. The articles optionally have references to one or more taxonomy terms. I have already created the taxonomy terms in Drupal manually, they look good. I then use the entity_lookup process plugin to lookup the taxonomy IDs as part of the article migration. But when the taxonomy terms repeat, they don't seem to be getting picked up right.

Problem:

Minimal migration YAML snippet is below.

Somehow only 'term1' in data row '1' keeps getting set in Drupal correctly when the migration is run. Nothing is set for row '2' (which is correct behavior) and nothing for row '3' either (incorrect behavior). I narrowed down the issue to repeating taxonomy terms, but cannot figure out what is going wrong. I have used the same construct with the explode plugin along with entity_generate and they worked well (but I don't want to go the route of entity_generate).

These links (this and this) referred to in past threads don't help much as the examples and explanations appear incomplete. If anyone can throw better light on this and where I am going wrong in the YAML, would be grateful. Thank you!

source:
  plugin: embedded_data
  data_rows:
    -
      unique_id: 1
      title: 'xxx'
      description: "xxxxx"
      taxoterms: 'term1'
    -
      unique_id: 2
      title: 'xxxx'
      description: "xxxxxxx"
      taxoterms: ''
    -
      unique_id: 3
      title: 'xxxx'
      description: "xxxxxxxxxxxx"
      taxoterms: 'term2,term3'
  ids:
    unique_id:
      type: integer
process:
  title: title
  body/value: description
  body/format:
    plugin: default_value
    default_value: restricted_html
  taxoterms:
    - 
      plugin: skip_on_empty
      source: taxoterms
      method: process
      message: 'Article does not contain taxomony terms'
    - 
      plugin: explode
      delimiter: ','
      strict: FALSE
      source: taxoterms
    - 
      plugin: callback
      callable: trim
    - 
      plugin: entity_lookup
      entity_type: taxonomy_term
      value_key: name
      bundle_key: vid
      bundle: my_taxo_vocab
destination:
  plugin: 'entity:node'
  default_bundle: my_article

2 + 0

migrations

taxonomy-terms

Score:0

Drupal

Hudri

4/22/24, 10:31 AM

entity_lookup does not create new taxonomy terms, it only checks for existing terms and returns null if they don't exist yet.

You need to replace entity_lookup with entity_generate instead. It will do a lookup first, and generates the entity if it doesn't exist yet. On subsequent rows it will return the entity it just generated itself.

+ 1

Chandra

4/22/24, 1:44 PM

Like I mentioned in the question, I am creating articles using the migration API after manually creating the taxonomy terms. I need the term 'id's of the terms inside the article which is why I am using entity_lookup. I believe thus far is correct. My question is on the use of entity_lookup when it has to lookup multiple taxonomy terms. Appreciate any input on that.

Score:0

Drupal

Chandra

4/23/24, 3:01 AM

UPDATE:

The yaml I posted in the question is actually perfectly fine, and it just works! Let me explain what happened.

The problem was an extraneous 'source' property I had set for the 'entity_lookup' plugin. Specifying a source is not necessary because the source for the plugin is of course the output of the previous plugin in the pipeline. Curiously, however, in my question, I posted a version of the YAML file that I was tinkering with, and as part of which tinkering I removed the 'source' property but never actually tested it. As it turns out, the extraneous 'source' property is THE mistake.

Sidenote: I would imagine a plugin in the pipeline would ignore the 'source' since it is not required, but it appears the entity_lookup seems to read it and behave differently. A minor bug? Maybe.

+ 3

Hudri

4/23/24, 7:26 AM

Not a bug, working as intented: If you omit the source property, any process plugin takes the output of the previous plugin as source/input (which was the exploded, trimmed array `['term2', 'term2']`). But `source: taxoterms` always is the original, unchanged string `'term2,term3'`. Process plugins do not call by reference.

Chandra

4/23/24, 9:13 AM

Ok @Hudri if I understood this, if a process plugin is provided a 'source' property, it would treat the value of this property as the source, and ignore the output from the previous plugin in the pipeline?

Hudri

4/23/24, 12:38 PM

Yes (...15 chars...)

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Migration entity_lookup not picking up repeating taxonomy terms

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.