Handling Low Quality Data in Drupal Migrations

So, you are migrating a site to Drupal – or just importing some data whether it is a one off or regular repeating process – ideally, you have perfectly regular data. Unfortunately, the world is rarely ideal, so what can you do when your data is very much not nice? When importing through the Migrate API, there is a lot you can do.

Drupal 7 to Drupal 8 often has nice data – predictable structure, most pieces of any one kind of content type are mostly the same. To some degree, on older Drupal sites though, data is often less comparable between dates. Spreadsheets of data from external sources out of your control are the often distinctly unreliable – or from many other external systems such as Dotetnuke – and be very challenging.

We’ll discuss various methods to resolve this ranging from simple migration plugin configuration adjustments to full pledged custom migration plugins. After starting we'll initially focus on yml migration plugns before moving on to using both source and process plugins to solve some real world examples of messy data.