In the first part of this series, we showed you how to perform a very basic migration without any customisation. In this part, we will show you how to modify some fields during migration. 

With the upcoming Drupal 8.1.0 release on April 20th, Migration will become a non-experimental part of Drupal core, with it’s powerful UI and rollbacks provided out-of-box. Some parts of these blog posts should become outdated as well, but we will do our best to keep updating them.

Let’s start from the beginning. The whole Migrate API was totally rebuilt for Drupal 8. For writing or customising your migrations, you should be familiar with the Plugin API, the OOP paradigm and the YAML file structure. 

Migrate API consists of multiple plugin types and one YAML file called the migration template. Plugins process data in each part of the migration process and the migration template glues these plugins together. For each migration, you will probably use at least three plugins: source, destination and process.

The source plugin, as its name implies, is used for describing the original data source. In this plugin you will need to implement several functions:
function fields() - in this function, you will need to define fields, which will be available for mapping in the process section in the migration template. You need to define an associative array of machine names and human-readable names. Machine names are used for mapping. Human-readable names can be translated, but to be honest, I’ve never seen this during migration execution.
function query() - in this function you need to build a database query for the original database. You will use the methods defined in SelectInterface in Drupal core.
function prepareRow(Row $row) - this function can perform operations on a single record obtained using the defined query. You can modify and fill some custom fields in this function.
getIds() - in this function you will define which fields are unique and their types.

The destination plugin describes new data sources. You don’t need to define this plugin for your custom entity if you don’t want any additional parameters. Examples of custom destination plugins are User and File.

The process plugin is a plugin you will probably never need to define. The Migrate module in Drupal core provides several migrate plugins covering all standard use cases. All of these plugins are described quite well in the official Drupal documentation. You will use these plugins for performing some operations, or setting the default value of a single field.

As I mentioned in the beginning, you need to glue these plugins together in the migration template. Let’s have a look at the migration template in some use cases.

The most common use case will probably be related to a change of the entity structure. First of all, you will need to find the YAML migration template. It is usually located in MODULE_NAME/migration_templates. Let’s use the most common entity in Drupal - node. You will find migration templates for nodes in core/modules/node/migration_templates. As you can see, there are multiple migration templates, depending on which version of Drupal you are performing the migration from.


You can clearly see sections where plugins are called.

Pic 2

Now, let’s assume that we need to add one more custom field in our new Drupal 8 website, but this field doesn’t exist in the old Drupal 6 website because it was represented as two different fields. There are two issues you could encounter. If you run migration with the default migration template, you will get an error saying that those two fields don’t exist. If you run content type migration as a dependent, you will get two additional fields in your Drupal 8 content type.

To solve either problem, first you need to create a virtual field for the migration purposes. You can do that by defining your own source plugin. You need to create a new class inside your custom module in \Drupal\MODULE_NAME\Plugin\migrate\source\ namespace. This custom class will extend the source plugin class provided by Drupal core.

  1. <?php
  3. /**
  4. * @file
  5. * Contains \Drupal\MODULE_NAME\Plugin\migrate\source\Node.
  6. */
  8. namespace Drupal\MODULE_NAME\Plugin\migrate\source;
  10. use Drupal\migrate\Row;
  11. use Drupal\node\Plugin\migrate\source\d6\Node as D6Node;
  13. /**
  14. * Custom Drupal 6 node source from database.
  15. */
  16. class Node extends D6Node {
  18. }

You need to implement the fields() function:

  1. /**
  2. * {@inheritdoc}
  3. */
  4. public function fields() {
  5.  $fields = parent::fields();
  6.  $fields += array(
  7.    'merged_body' => $this->t('Merged body'),
  8.  );
  9.  return $fields;
  10. }

When you have your field defined, you need to store some data in it. You can do this by implementing the prepareRow() function. In the prepareRow() function, you get data from other fields using the getSourceProperty/getDestinationProperty functions and you put it back using the setSourceProperty/setDestinationProperty functions:

  1. /**
  2. * {@inheritdoc}
  3. */
  4. public function prepareRow(Row $row) {
  5.   $field1 = $row->getSourceProperty("field_field1");
  6.   $field2 = $row->getSourceProperty("field_field2");
  7.   $row->setDestinationProperty("merged_body", $field1 . " " . $field2);
  8.   return parent::prepareRow($row);
  9. }

The most important thing you need to do to get your plugin recognised is to include annotation just before the beginning of your class:

  1. /**
  2. * Custom Drupal 6 node source from database.
  3. *
  4. * @MigrateSource(
  5. *   id = "MIGRATION_SOURCE_ID"
  6. * )
  7. */
  8. class Node extends D6Node {

Your complete class should now look like this:

  1. <?php
  3. /**
  4. * @file
  5. * Contains \Drupal\MODULE_NAME\Plugin\migrate\source\Node.
  6. */
  8. namespace Drupal\MODULE_NAME\Plugin\migrate\source;
  10. use Drupal\migrate\Row;
  11. use Drupal\node\Plugin\migrate\source\d6\Node as D6Node;
  13. /**
  14. * Custom Drupal 6 node source from database.
  15. *
  16. * @MigrateSource(
  17. *   id = "MIGRATION_SOURCE_ID"
  18. * )
  19. */
  20. class Node extends D6Node {
  21.   /**
  22.   * {@inheritdoc}
  23.   */
  24.   public function fields() {
  25.     $fields = parent::fields();
  26.     $fields += array(
  27.       'merged_body' => $this->t('Merged body'),
  28.     );
  29.     return $fields;
  30.   }
  32.   /**
  33.   * {@inheritdoc}
  34.   */
  35.   public function prepareRow(Row $row) {
  36.     $field1 = $row->getSourceProperty("field_field1");
  37.     $field2 = $row->getSourceProperty("field_field2");
  38.     $row->setDestinationProperty("merged_body", $field1 . " " . $field2);
  39.     return parent::prepareRow($row);
  40.   }
  41. }

After we have our custom source plugin ready, we need to use it in the migration template. The migration template is a configuration entity defined inside a YAML file. You need to store this YAML file in the /modules/custom/YOUR_MODULE/config/install directory. The file should be named migrate.migration.MIGRATION_NAME.yml. For this use case, you can copy core/modules/node/migration_templates/d6_node.yml. You need to change the ID here to your MIGRATION_NAME for it to recognise your migration template. The other thing you need to change is the source plugin ID from d6_node to your custom ID, which you defined in the annotation.
Now, you can map your field defined in the source plugin to a real entity field in the process plugin. You can process the field before using some process plugins of course, but it is not necessary this time. Your final code should now look like this:

  2. label: Nodes
  3. migration_tags:
  4.  - Drupal 6
  5. builder:
  6.  plugin: d6_node
  7. source:
  9. process:
  10.  nid: nid
  11.  vid: vid
  12.  type: type
  13.  langcode:
  14.    plugin: default_value
  15.    source: language
  16.    default_value: "und"
  17.  title: title
  18.  uid: node_uid
  19.  status: status
  20.  created: created
  21.  changed: changed
  22.  promote: promote
  23.  sticky: sticky
  24.  'body/format':
  25.    plugin: migration
  26.    migration: d6_filter_format
  27.    source: format
  28.  'body/value': merged_body
  29.  'body/summary': teaser
  30.  revision_uid: revision_uid
  31.  revision_log: log
  32.  revision_timestamp: timestamp
  33. destination:
  34.  plugin: entity:node
  35. migration_dependencies: {}

After you enable your module, and run drupal migrate:debug in your terminal, you should see your custom migration. The migration template is imported only during the installation of your module. If you need to change it afterwards, do it through configuration management in a single config import or through a drush config-edit migrate.migration.MIGRATION_NAME in your terminal. 

In this two part series, we have covered the most basic use cases of Migration framework. If you think we missed anything, please comment and let us know. There are a lot of other possible use cases for migrating to Drupal 8, we would love to hear about your experience with it.