Using migrate (also) to update nodes that already exists in Drupal

I am in the process of porting a non Drupal database to Drupal 9. The original database contains, among other things, a table of Books (with the usual infos) and and separate table (don't ask me why) for Travel Books. The former is 10K records, and was successfully migrated from a books.csv to Drupal. The latter (travelbooks.csv) is just 200 records.

What is interesting (sic!) is that most of Travel Books are also in the Books table, with some new fields in Travel Books table, notably a country and the number of pages (sic again!). In Drupal we have a taxonomy for type_of_book with things like romance, adventure, true stories, crime, poetry etc.. as well as travel book.

Books from books.csv are already in Drupal, with new Nodes and the correct taxonomy terms applied to the field field_book_type which, of course, it's a multi-value field.

Now I want to migrate travelbooks.csv in Drupal but:

• if a book already exists (looking up with title, since the IDs in the original tables were different!), just update the node in Drupal, adding the taxonomy term travel_book
• if a book does not exist in Drupal, create it and just set the taxonomy term travel_book

So far I've been able to do this via a custom Destination plugin, and not just with a migration configuration YAML. THis is what I hoped to use:

uuid: d66124cf-232b-4080-911c-1d26aefd0246
langcode: en
status: true
dependencies: {  }
id: quote_fornitore_voucher_csv_import
class: null
field_plugin_method: null
cck_plugin_method: null
migration_tags: null
migration_group: books
label: 'Import travel books'  
source:
  plugin: csv
  path: /Users/walter/travel_books.csv
  delimiter: ','
  enclosure: '"'
  header_offset: 0
  ids:
    - id
  fields:
    -
      name: id
      label: 'Unique Id'
    -
      name: book_title
    -
      name: country
    -
      name: pages
    -
      name: synopsis
.....
  constants:
    travel_book: 'Travel Book'
process:
  type:
    plugin: default_value
    default_value: book
  field_pages: pages
  field_country: country
  title: book_title
  field_book_type:
    plugin: entity_lookup
    entity_type: taxonomy_term
    bundle: book_type
    bundle_key: vid
    source: constants/travel_book
  nid:
    plugin: entity_generate
    entity_type: node
    bundle: book
    bundle_key: type
    value_key: title
    source: book_title
    access_check: 0
destination:
  plugin: 'entity:node' 
  overwrite_properties:
    - field_pages
    - field_country 
migration_dependencies: null

I have two problems with this migration:

1. Even if entity_generate finds the correct existing node for the current book, the Migration Engine still tries to CREATE a new node, with the same NID, and of course fails with a big red SQL Error

23000]: Integrity constraint violation: 1062 Duplicate entry '2662' for key 'PRIMARY': INSERT INTO "node" ("nid", "vid", "type", "uuid", "langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array

2. I overcome this, only not to be able to add to an existing Book the correct taxonomy term to the list of field_book_type.

I traced the problem with 1 to be that the Entity has enforceIsNew set to True. Or at least that's what I've found. I've written a destination plugin that, based on a new configuration, can force the newly created entity with enforceIsNew = FALSE and it seems to work as expected. Travel books not found in the Books i nDrupal are created, while the one already existing are updated, with the fields pages and country updated.

This is the relevant code, I think.

abstract class EntityBase implements EntityInterface {
...
  public function isNew() {
    return !empty($this->enforceIsNew) || !$this->id();
  }
...

Even though there is an ID, which is being set by the nid process, enforceIsNew is set to TRUE and so isNew() returns True.

<?php

namespace Drupal\migrate_books\Plugin\migrate\destination;


use Drupal\Core\Entity\Entity;
use Drupal\Core\Entity\EntityInterface;
use Drupal\migrate\Plugin\migrate\destination\EntityContentBase;
use Drupal\migrate\Plugin\MigrationInterface;
use Drupal\migrate\Row;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * @MigrateDestination(
 *   id = "update_node"
 * )
 */
class UpdateNodeDestination extends EntityContentBase {

  /** @var string $entityType */
  public static $entityType = 'node';
 
  protected function updateEntity(EntityInterface $entity, Row $row) {
    $entity = parent::updateEntity($entity, $row);
    //check parameter..
    if(!empty($this->configuration['prefer_update']) && $this->configuration['prefer_update']){
      // force NOT to enforce NEW. This will avoid inserting twice nodes already present (which will result
      // in an integrity violation from MySQL
      $entity->enforceIsNew(FALSE);
    }
    return $entity;

  }

}

and the YAMLS is changed with my plugin and the new param prefer_update

destination: 
  plugin: update_node
  prefer_update: true
  overwrite_properties:
    - field_pages
    - field_country

but it seems to me that it should be simpler than that. I am obviously missing something. I am sure this is a pretty standard use case, so it could/should be doable entirely in YAML.

Secondly, I am still not able to add the correct book_type to the list of existing tags. If I add field_book_type to overwrite_properties, of course it gets overwritten and we lose all the previous terms.

I don't seem a way to update by adding a value to a multi value field.

Any help? Thanks W