PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column ‘field_xxx_display’ cannot be null:

If you are using Features module to import/export sample data like for kickstart data for your new feature. If you are using the newest version dependent modules – uuid and uuid_featurs(7.x-1.0-alpha or above) of the Feature module, then you will get the following error at the time importing sample data that contains field referencing asset such as image, video,etc:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column ‘field_xxx_display’ cannot be null:

The issue is that there appears to have a bug in uuid_features module of the newest version that strips away attribute ‘display’ from the sample import. One solution is to reattach back the missing attribute just before the sample data is saved. To do so, we use hook_node_presave() as following:

function MODULE-NAME_node_presave($node) {
    //fix  for sample data import where 'display' is striped away for field_xxx causing db syntax error at node_save
    if($field_instance = field_get_items('node', $node, 'field_xxx')){
        foreach($field_instance as $key => $field_ins){
            $node->field_xxx['und'][$key]['display'] = 1;
        }
    }

Here, in line 3, the field instance is retrieved for the node being imported. If it exists, then in line 4, we go through all of the instances of that field and add the attributed ‘display’ that was striped away by uuid_features module. Note, that the word ‘display’ is removed from ‘field_xxx_display’ as mentioned in the error message because its added by Drupal to store the data

Handling Assets for Custom Module in Drupal

[UPDATE Aug 22, 2014]There is newer version of this article “Import Export Sample Data And Assets For Kickstart All Via Features” and a Drupal module samle_data at Drupal site. It is also in the git repo DesignsSquare Lib Sample Data at branch ‘7.x-1.x’

You have custom module a plugin that you have made exportable via Features module. This custom module has sample kickstart data that contains assets such as images, video, etc. In this post, we cover how to manage these assets, so when user enables your custom module, those are copied accordingly and automatically. Furthermore, we provide a custom module on github to handle assets for you so you don’t have to ever worry about assets import/export for you sample kickstart data. At last, the assets referenced directly from content is also packaged for auto transfer whenever a custom module is enabled.

Env: DRupal:7.26, Features:7.x-2.0, UUID:7.x-1.0-alpha5+17-dev, Feature_UUID:7.x-1.0-alpha3+15-dev

About uuid_features_file_path

Each artifact exported via Features module receives an attribute – uuid_features_file_path. This attribute specifies where are the artifact located which is used at the time of import(when module is enabled). We are going to set this attribute at the time of export, so it is pointing to our artifact location. Its important to note, we like to keep our assets part of module dir that way all plugin info is in one dir easier to deliver and import for user.

Set uuid_features_file_path At Export

We are going to hook into feature export process via MODULE-NAME_uuid_node_features_export_render_alter(&$export, $node, $module) as following:

function MODULE-NAME_uuid_node_features_export_render_alter(&$export, $node, $module){
    if($module == 'MODULE-NAME'){
        //setting artifact source location so it imports accordingly
        $field_instance = field_get_items('node', $node, 'field_ARTIFACT-FIELD-NAME');
        $export->field_ARTIFACT-FIELD-NAME['und'][0]['uuid_features_file_path'] = drupal_get_path('module','MODULE-NAME').'/imports/'.$field_instance[0]['filename'];
...
}

Here, we first ensure the code is executed for our module by having IF condition. Afterward, we retrieve field containing artifact for particular Node instance. Next, we update the export attribute uuid_features_file_path with a path to the custom module and path to artifact

This will export the custom module sample date into file with uuid_features_file_path specifying assets located in our custom module folder. There is nothing else to do, because at the import, features module will take care of coping the assets from location specified in uuid_features_file_path to the public dir of that Drupal instance in folder assets “uri” is set to

Forget About Assets…

In the above solution, a user is constrained to have the module placed in the exact location and named accordingly as specified in the uuid_features_file_path, otherwise, it will break. How about forgetting about assets of the sample kickstart data, so you can focus building features. Here is a Drupal module that does just that:
https://github.com/kapasoft-config-scripts/designssquare-lib-assets

Once enabled, this module does the following:

  • Handles all assets of the sample data being exported/imported via Features module
  • Besides Drupal core fields, it also works for custom fields referencing assets
  • It stores the assets relative to the module, so user can name the exportable feature or place it as they wish
  • The assets need to be handled by ‘file_managed’ functionality in order for the module to work

With this module enabled, you should never need to worry about assets any more.

How About Assets Referenced From Content …In Sample Data

So far, we have handled asset export/import for cases where the asset is referenced by some field in a node, however. Most of the time, your sample data will have content that will also reference assets(i.e.

<img src="/sites/default/files/img/some.jpg" alt="thumbnail">

). We also like to export/import those assets, so the sample data doesn’t break on clients Drupal instance. Here are steps to accomplish this

1. Make Part of Custom Module

Make sure the assets is part of the custom module directory…let, say the assets are in ‘modules/CUSTOM-MODULE/assets/img’ directory

2. Generate Array of Assets To Import

Run the following script in the ‘PHP Executable’ block to generate an array of assets to export/import:

include_once DRUPAL_ROOT . '/includes/utility.inc';
include_once DRUPAL_ROOT . '/includes/file.inc';

$MODULE_NAME = 'CUSTOM-MODULE-NAME';
$PATH_TO_MODULE = drupal_get_path('module', $MODULE_NAME);
//source dir or location of the files to import relative to the module specified by variable - $MODULE_NAME
$BASE_SOURCE_REL_PATH = '/assets/img';
//the destination folder relative to the public dir of Drupal instance
$DESTINATION_DIR = '/img';

$output = "function _assets_list(){ \n";
$output .= "\$module_dir = drupal_get_path('module', '" . $MODULE_NAME . "');\n";
$output .= "     return array (\n";

$output .= write_file_array('', $MODULE_NAME, $PATH_TO_MODULE, $BASE_SOURCE_REL_PATH, $DESTINATION_DIR);

$output .= "     );\n";
$output .= "}\n";
drupal_set_message("<textarea rows=100 style=\"width: 100%;\">" . $output . '</textarea>');

function write_file_array($rel_path = '', $MODULE_NAME, $PATH_TO_MODULE, $BASE_SOURCE_REL_PATH, $DESTINATION_DIR)
{

    $output = '';
    $full_path = $PATH_TO_MODULE . $BASE_SOURCE_REL_PATH . $rel_path;
    if ($handle = opendir($full_path)) {
        while (false !== ($file = readdir($handle))) {
            if (is_dir($full_path . '/' . $file) && $file != '.' && $file != '..') {
                $output .= write_file_array('/' . $file, $MODULE_NAME, $PATH_TO_MODULE, $BASE_SOURCE_REL_PATH, $DESTINATION_DIR);
            } elseif ($file != '.' && $file != '..') {
                $uri = file_build_uri($DESTINATION_DIR . $rel_path . '/' . $file);
                $output .= "     '" . $uri . "' => array(\n";
                $output .= "     'uri' => '" . $uri . "',\n";
                $output .= "     'filename' => '" . $file . "',\n";
                $output .= "     'source' => \$module_dir.'" . $BASE_SOURCE_REL_PATH . $rel_path . '/' . $file . "',\n";
                $output .= "),\n";
            }
        }
        closedir($handle);
    }
    return $output;
}

In line 4, we specify our custom module that contains the assets to export. In line 7, we specify the exact directory of assets to export relative to our custom module. In line 9, we specify the destination folder to import the assets relative to the public directory of the Drupal instance. In another words, if the sample content references image via tag

<img src="/sites/default/files/img/some.jpg" alt="thumbnail">

then the destination dir is ‘/img’ assuming the Drupal public dir is ‘sites/default/files’. After running this script, it will generate function ‘_assets_list()’ that contains all the assets to export/import and is going to be used part of the installation script as described next step

3. Transfer assets via module installation script

Next, we configure our Custom Module installation script CUSTOM-MODULE-NAME.install to transfer assets whenever the module is enabled

function CUSTOM-MODULE-NAME_install(){
    $t = get_t();

    //looping through the files and coping to the destination
    foreach(_assets_list() as $file){
        $uri = $file['uri'];
        $dir_name = drupal_dirname($uri);
        file_prepare_directory($dir_name,FILE_CREATE_DIRECTORY);
        (!file_exists(drupal_realpath($uri))) ? file_unmanaged_copy($file['source'], $uri) : '';
    }
    drupal_set_message($t('CUSTOME MODULE NAME assets transferred'));
}

/************COPY _assets_list() HERE*******/

NOTE: Please, copy the function _assets_lists() from the step 2 in the same file CUSTOM-MODULE-NAME.install

This script is run when custom module is enabled. It runs through each asset and transfers from the CUSTOM-MODULE/img into the Drupal/public/img directory. Afterwards, those assets are available when referenced from the sample content

4. Clean on Uninstall

You may also like to remove these assets whenever the custom module with sample data is uninstalled, then add the following in CUSTOM-MODULE-NAME_uninstall() in the CUSTOM-MODULE-NAME.install file:

function CUSTOM-MODULE-NAME_uninstall(){
    $t = get_t();

    //looping through the files and removing one by one
    foreach(_assets_list() as $file){
        $uri = $file['uri'];
        $filename = drupal_realpath($uri);
        (file_exists($uri)) ? file_unmanaged_delete($filename) : '';
    }
    drupal_set_message($t('CUSTOM-MODULE-NAME assets removed'));
}

This script is run on module uninstall and it runs through the same list of assets from _assets_list() and removes each asset from the public dir

Summarize

The attribute uuid_features_file_path specifies the location from where to transfer assets at the time of import, so we hooked into the export and updated this attribute to a location of our assets. We keep it in the same custom module dir for easy delivery and import by user. Furthermore, there is a module developed in github that will handle all assets of the sample data kickstart for you. At last, we also go over how to export/import assets referenced by sample data from the content itself

References

http://drupalcontrib.org/api/drupal/contributions!features!features.api.php/7

Make Custom Field with File Exportable in Drupal Features

In this post, we cover a case where we have a custom field that contains a file and we would like to be able export/import it with a Features module in Drupal

Env: Drupal 7, UUID Features 7.x-1.0-alpha3+1(dev), UUID 7.x-1.0-alpha5+1(dev)

Intro

We have content type with a field field_elastic_slide that contains a file among other things(see our other post for “How To Implement Custom Field Types In Drupal”). We are exporting Nodes with this field using features module. Without doing anything, the export will contain FID(File ID) of the file which is not going to be relevant in another Drupal instance since there is completely different FIDs. So, we will alter the export to store all info of a file in the export instead only FID of the file. Afterwards, we will hook into the import process to use this file info to import the file in the new Drupal Instance

Alter Export To Include All File Info

We are going to use hook_uuid_node_features_export_render_alter(&$export, $node, $module){..} to alter the export as following:

function MODULE-NAME_uuid_node_features_export_render_alter(&$export, $node, $module){
        if($module == 'MODULE-NAME-DOING-EXPORT'){
        $field_instance = field_get_items('node', $node, 'field_elastic_slide');
        foreach($field_instance as $key => $instance){
            $file = file_load($instance['image']);
            $export->field_elastic_slide['und'][$key]['image'] = (array)$file;
            ...
        }
    }
}

In line 5, we load all the file by its FID. Afterwards, we alter the export for our custom field to store array of file info instead only FID. The Feature module will take from there and write the file info in the file ending .features.uuid_node.inc part of the export. So far, so good.

Alter Import To Import File

When importing custom fields containing file, we have to save the file in the table file_managed by ourselves. The feature module will do the work. It will only do the work of importing files for the default field types(file, text,etc). So, we will save the file in the table file_managed and that will generate the necessary FID to be set for our custom field.

To alter import, we use hook_uuid_node_features_rebuild_alter as following:

function MODULE-NAME_uuid_node_features_rebuild_alter(&$node, $module){
    if($module == 'MODULE-NAME'){
        $field_instance = field_get_items('node', $node, 'field_elastic_slide');
        uuid_features_load_module_includes();
        foreach($field_instance as $key => $instance){
            $uuid = $instance['image']['uuid'];
            unset($instance['image']['fid']);
            unset($instance['image']['timestamp']);
            $file_to_save = (object)$instance['image'];
            $final_file = file_save($file_to_save);
            $node->field_elastic_slide['und'][$key]['image'] = $final_file->fid;
            
            //transfer file into public dir available 
            $source = _get_source_file($file_to_save->uri);

            file_unmanaged_copy($source, $file_to_save->uri);
...
}

function _get_source_file($uri){
    $source = drupal_get_path('module','MODULE-NAME').'/imports/public/'.substr($uri, 9);
    if(!file_exists($source)){
        watchdog(WATCHDOG_NOTICE, 'MODULE-NAME Import: Image does not exists at '.$source);
    }
    return $source;
}

Here, the export is retrieved and available in the $node variable. So, in line 3, we retrieve our custom field containing file info we saved in the export in the previous step. Afterwards, we remove some info among it the FID that is irrelevant in the new Drupal instance and is not needed. Next, in highlighted line, we use File API to save the file that makes entry in table file_managed. Then, we set the new FID to our custom field leaving rest for Features module to continue with the import process. At last, the file is copied from predefined location to a Drupal public dir.

Clean Afterwards

When an instance with our custom field is deleted, we want to ensure to delete the file from the file_mage and file_usage tables. Otherwise, there is going to be an error – PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 next time re-enabling the module or uploading the file with the same name.

//clean file_manage database on field instance delete
function MODULE-NAME_node_delete($node){
    if($node->type == 'CERTAIN-TYPE'){
        //removes files from file_manage/file_usage tables
        $field_instance = field_get_items('node', $node, 'field_FIELED-NAME');
        foreach($field_instance as $key => $instance){
          if(isset($instance['image']) && $instance['image']){
            $file = file_load($instance['image']);
            file_delete($file);
            file_usage_delete($file, 'MODULE-NAME');
          }
  ...
}

The hook_node_delete($node) is called every time a node is being deleted. So, in line 3, we check that the particular node of our interest is being deleted. If it is, then we retrieve the field containing artifact. At last, in line 7, we load the artifact by its FID before removing from file_managed/file usage table and file system as well

This also removes the files from Drupal public directory(by default sites/default/files) that was copied when module enabled as described above

Summary

In short, for exporting and importing custom fields containing files, there is an extra work of physically saving the file that needs to be done by ourselves and is not done by Features module. To do the task at hand, we altered the import so that, instead only the FID, it stores all the info of the file in the export. Afterwards, we used this info to physically import the file(save in managed_file table) and set the new FID to our custom field containing the file in the New Drupal Instance that import takes place

Troubleshoot

The specified file [..] could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions

This was coming up when trying to copy artifacts via file_unmanaged_copy but the destination directories were missing.

//ensure dir is present
                $directory = drupal_dirname($final_file->uri);
                file_prepare_directory($directory, FILE_CREATE_DIRECTORY);
                
                file_unmanaged_copy($source, $final_file->uri);

By adding the file_prepare_directory() in front of file_unmanaged_copy(), creates a directories if missing. As result, the file_unmanaged_copy() executes without any errors

References:
http://www.phase2technology.com/blog/making-your-module-features-exportable/
https://origin.previousnext.com.au/blog/how-handle-embedded-images-when-migrating-content-plone-drupal