I've got a pretty large migration set of roughly 200k users. The first time I run the migration (via drush), or after rolling it back and starting again, the rollback+import starts immediately. By this I mean that the progress bar starts showing progress on importing items right away.
I know there's no way around the migration itself taking a long time due to the number of items, but I'm running into an issue on subsequent runs where, before the data actually starts importing into the destination, the migration just sits there, seemingly without doing anything, for an insane amount of time. By subsequent runs, I mean any run of the migration that is not the first run, or the first run after a rollback. So, a migration that is executed either to pull in additional users or is run after the initial one hits an error.
If I add
--feedback=x, I do see a console message of
Processed 0 items (0 created, 0 updated, 0 failed, 0 ignored) - continuing with 'upgrade_d7_user' every so often, so I know it must be doing something and updating after that number of items, but I don't know what it is. It seems like we are just waiting for it to "take a look" at every item before processing it, which does not happen on an "initial" run, and I'd guess is basically doubling how long the run will take. I guess my questions are:
- What exactly is the migration even doing at this point? Is it just doing some sort of data verification?
- Is there some way to bypass this step and just jump directly into processing the data itself? We're already looking and days to run this single migration and this extra time is pretty debilitating.