Caching in Drupal

The $variable[‘messages’] is not always available and, thus, it fails in situations where you need to have messages be available anytime and everywhere in the code during single request. As solution to my problem, I utilized Drupal Caching

In search for solution, I started by creating the following function:

function get_messages(){
    $status_messages = array();
    //loop through each message type
        foreach (drupal_get_messages(null, false) as $type => $messages) {
               foreach($messages as $key => $message){
                   $status_messages[$type]['enabled'] = true;
                   $status_messages[$type]['messages'] = array(
                       'message' => $message
                   );
               }
        }
return $status_messages;
}

This works well, however. The messages don’t get to be reset. As a result, messages build up from previous requests something that is unacceptable. We could try to reset messages by calling Drupal API function drupal_get_messages with parameter ‘true’, but then the messages got to be reset on the first time execution of our function – get_messages() and all other times it returns no messages. My challange

I had two ideas on how to approach this problem:

  1. Keep & Reset at the End of Request

    One option is to clear messages at the last point of the request

  2. Cache messages and Reset at the Start of Request

    At first time calling drupal_get_messages(), we would cache the messages and then reset at the same time by calling the function with parameter “TRUE”

For the first solution, it wasn’t clear where exactly is the last point of request good for reset messages. I was also concerned having to touch code in two places to solve single problem
For the second solution, I liked it because it would kill two birds with one stone. We would solve our problem and improve the performance by caching messages and, thus, minimizing execution.

To solve this problem, I turn to Drupal caching mechanism.

Cache During One Request

Drupal provides reference to static storage that is permanent during single execution. To utilize it as cache, here is common pattern of the function as follows:

function some_function(){
 $my_data = &drupal_static(__FUNCTION__);
  if (!isset($my_data)) {
    // Do your expensive calculations here, and populate $my_data
  }
  return $my_data;
}

Here, in the second line we got the static reference with name of our function(i.e. __FUNCTION__). For the first time, this reference will not be set, so we populate it with the data that we like to cache. For all other executions, the static reference is holding our data that we return

Here, is how our message function looks with the Drupal cache mechanism:

function get_messages(){
    $status_messages = &drupal_static(__FUNCTION__);

    if(!isset($status_messages)){
        //loop through each message type
        foreach (drupal_get_messages(null, true) as $type => $messages) {
               foreach($messages as $key => $message){
                   $status_messages[$type]['enabled'] = true;
                   $status_messages[$type]['messages'] = array(
                       'message' => $message
                   );
               }
        }
     }
     return $status_messages;
}

Here, we get the static reference first and then check if its already set. if not, we retrieve the messages and reset at the same time by calling Drupal API function drupal_get_messages with “true” parameter. As result, we reset the messages that solves our problem and, in addition, our performance is improved due to caching

Cache Over Multiple Requests

Perhaps, there is going to be time when you will need to cache over multiple requests instead only one. To do so, Drupal provides cache functions(i.e. cache_get, cache_set, cache_clear_all) that store cache into Database. There is an excellent article on caching in including Drupal cache functions at Beginners Guide Caching Data Drupal 7

References

  • https://www.lullabot.com/blog/article/beginners-guide-caching-data-drupal-7
  • https://api.drupal.org/api/drupal/includes%21bootstrap.inc/function/drupal_static/7

How to Pass Variables Horizontally and Vertically in Drupal

There are some variables($node, $page,etc) you depend more than on anothers in Drupal. Its important to have access them when you need. In the following, we describe two ways -“top to down” and “left to right” to pass and accesses any variable at any part of the execution in Drupal

Top to Down …or Vertical

A good example, would be passing some variable from page scope to the node scope

function THEME_preprocess_page(&$vars, $hook){
...
   $vars['node']->bazar_product = $prod_default;
}

Since page scope calls theme function ‘node’ to render the node, it takes the variable – vars as parameter(see theme function for node). You can utilize that by storing your custom variables into $vars making it accessible in the node scope – hook_preprocess_node().
For passing between any other scope, first find what theme function is called for rendering the children and look at this theme’s function declaration to see exact variables being passed down to children

Left To Right …or Horizontal

Often times we need to know what node type is set for the current request in the html scope – hook_preprocess_html(), however the node type is processed and set during page scope. Since there are no variables passed between hook_preporcess_page and hook_preporcess_html, the variables such as node type is not shared between, however. We need the node type in the html scope, so we can render particular template. To pass variables from one scope to another horizontally, we utilize cache as following:

//get the current content from cache
$content = drupal_set_page_content();
//indicate that you are not adding content but just reading it by setting static variable 'system...added' to False 
$system_main_content_added = &drupal_static('system_main_content_added');
$system_main_content_added = FALSE;

The variable content will have variables from hook_preproccess_page scope. Its important to note, that cache may not always work because you may be asking too early.

Scrap Emails From Gmail via Drush

In the following are steps to scrap emails from gmail account. We first archivete all emails into a file(.mbox). Then we run our custom drush command to scap all emails into output-email.csv file. At last, we use tool such as BriteVerify.com to filter only valid emails.

Archive All Emails

Go to Google Takeout and create archive file(.mbox) from your gmail account

Scrap with Drush

Once you have archive(.mbox) file, run the following drush command:

drush scrap-email --file-name=path/to/name-of-file.mbox

This will create file output-email.csv in your current directory with all the emails
Please, see Appendix below for scrap-email drush command, so you can install on your own machine

Verify Emails

Like with many things, it is also the case with our custom Drush scrap command that it is not perfect and it scraps some bad emails. To clean out the bad emais, we used tool BriteVerify.

Appendix

Here is full Drush command for scraping emails. Please, ensure to put it in file named scrap.drush.inc. For how to install, please, see post – Implementing Custom Drush Commands


<?php
// Same as error_reporting(E_ALL);
//ini_set('error_reporting', E_ALL);
ini_set('memory_limit', '850M');
set_time_limit(0);

function scrap_drush_command()
{
    $items = array();
    $items['scrap_email'] = array(
        'description' => "Scraps all emails from google archive(.mbox) and stores it in output-email.csv in current dir",
        'arguments' => array(//            'type' => 'The type of the smile (half_moon, polity, etc.)',
        ),
        'options' => array(
            'file-name' => 'path to name of the google archive file(.mbox). It can be relative to current dir',
        ),
        'examples' => array(
            'drush scrap_email --file-name=my-gmail.blox' => 'scraps all emails from my-gmail.mbox and stores emails in output-emails.csv in current dir',
        ),
        'aliases' => array('semail'),
        'bootstrap' => DRUSH_BOOTSTRAP_DRUSH, // No bootstrap at all.
    );
    return $items;
}

function drush_scrap_email()
{
    $filepath = drush_get_option('file-name');
    if (!file_exists($filepath)) {
        $filepath = getcwd() . '/' . $filepath;
        if (!file_exists($filepath)) {
            drush_die('File - ' . $filepath . ' doesn\'t exist', 0);
        }
    }

   drush_log('begin scraping...','ok');

    $chunk = 10 * 1024 * 1024; // bytes per chunk (10 MB)

    $f = fopen($filepath, 'rb') or die("Couldn't get handle for " . $filepath);
    $data = '';
    if ($f) {
        while (!@feof($f)) {
            $data .= fgets($f, 4096);
        }
        fclose($f);
    }

    drush_log('done reading string of size: ' . mb_strlen($data, '8bit') . '... start searching','ok');

    $pattern = "/([a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9._-]+)/";
    preg_match_all($pattern, $data, $matches);

    $all_emails = array_unique(array_values($matches[0]));
    $all_emails_filtered = array_filter($all_emails, 'filter_bad_emails');

    print_r($all_emails_filtered);
    drush_log('Count:' . count($all_emails_filtered),'ok');

    drush_log('writing...','ok');

    $date = date('m-j-y');
    $filename = 'output-emails-'.$date.'.csv';
    $filepath = getcwd() . '/' . $filename;

    $file = fopen($filepath, "w") or die("Couldn't get handle for " . $filepath);
    if ($file) {
        foreach($all_emails_filtered as $email){
            fputcsv($file, array($email));
        }
    }

    fclose($file);
    drush_print('done');
}

function filter_bad_emails($email)
{
    $char = $email[0];
    $email_tokens = explode('@', $email);
    $domain_name = array_pop($email_tokens);
    $ext_tokens = explode('.', $domain_name);
    $ext = array_pop($ext_tokens);
    if ($char == '-' || $char == '_' || $char == '.' || is_numeric($char) || (strlen($email) > 30) || (strlen($ext) > 4) || is_numeric($ext) || ($ext == 'c') || ($ext == 'n') || ($domain_name == 'mail.gmail.com')) {
        return false;
    } else {
        return true;
    }
}

Implementing Custom Drush Commands

In this post, we cover how to install and create your first custom Drush command

Install Custom Drush Commands

There are two steps to install any drush command:

  1. a) copy drushrc.php from /path/to/drush/example to your $HOME/.drush/ directory if not already present
  2. b) In the drushrc.php specify directory containing your drush cusotm commands
    ($options[‘include’]=/path/to/my/drush/commands
    

This will import the directory where your custom drush function will reside. Lets create one

Implementing Custom Drush Command

Imlementing drush command can be broken into 3 steps:

  1. Create File
    After DRUSH is aware of the location of our custom commands, then we can create any file by extension ‘.drush.inc’, because Drush will load all files with the extension ‘.drush.inc’. Lets, say we have file smile.drush.inc
  2. Declare Command
    Next, lets declare command in smile.drush.inc as following:

    function FILE-NAME_drush_command() {
        $items = array();
        $items['make-me-smile'] = array(
            'description' => "Makes a Happy Smile.",
            'arguments' => array(
                'type' => 'The type of the smile (half_moon, polity, etc.)',
            ),
            'options' => array(
                '--time' => 'specify time of smile (e.g. 10,30 in sec)',
            ),
            'examples' => array(
                'drush smile polity --time=10' => 'Make a great smile that cheers you up for rest of the day.',
            ),
            'aliases' => array('smile'),
            'bootstrap' => DRUSH_BOOTSTRAP_DRUSH, // No bootstrap at all.
        );
        return $items;
    }
    

    This declares our custom Drush command – make-me-smile. Important to note, the file-name is the hook in the hook_drush_comand() declaration. The ‘boostrap’ specifies the level of Drush to boot in. Some of other bootstrap levels are ‘DRUSH_BOOTSTRAP_NONE’, ‘DRUSH_BOOTSTRAP_DRUPAL_ROOT’, etc…see /path/to/drush/includes/bootstrap.inc for full list and description

  3. Implement Command
    After we declare the custom command, lets implement it:

    function drush_make_me_smile($type = 'polite'){
        drush_print('Smiling....type:'.$type);
    }
    

    Here the first ‘smile’ is from the file name smile.drush.inc following the name of the drush command – make_me_smile, so the function name is combined into drush_COMMAND_NAME where all the ‘-‘ part of COMMAND-NAME need to be replaced to ‘_’

  4. Test Run

    ○  drush smile --help                                                                                      
    ○  drush smile fake                                                                                       
    Smiling....type:fake
    ○  drush make-me-smile test                                                                               
    Smiling....type:test
    

    here, we run our new custom command in two ways – by alias and by full name

    Troubleshooting

    1. Location of Drush installation

    run:

    which drush
    

    This will display executable of drush. Afterwards, see what directory is linking to

Importing and Exporting Products in Drupal Commerce

In prevous post Drupal Commerce Install and Setup, we went over on incorporating commerce functionality with Drupal and creating/displaying Products. In this post, we go over how to import/export Drupal commerce “products” as well as “product displays”. Important to note, there are “products” and then there are “product displays”. The “product display” is displaying one to many products referenced by the display. It is plain Drupal node with an extra field of type – product reference. The “product” is specific type provided by Commerce module.

Initially, I was thinking to use features for import and export products in drupal commerce store but it doesn’t have the capability for products to be featured, so the solution for us was using Views data export to export products and Commerce_feeds for importing products

Exporting Commerce Products

So, you have several products that you created with Drupal Commerce product installed and its UI. To export those products, we first create a view containing the products and then export this view via Views data export into CSV file

a) Install views_data_export and dependent modules
drush dl views_data_export
drush dl image_url_formatter
drush en views_data_export image_url_formatter
b) Create a view with products you like to export

Create a view with products you like to export. The view has to list each product field you like to export, so add each product field. Here are other specifics of the view:

  • ‘size’ is set to unlimited
  • give a display name in the Administration settings(in our example ‘theme_export_display’)
  • select image fields with field formatter ‘image_url’ (see module image_url_formatter) with setting ‘nothing’ for linking
    • Image url and ‘URI Path’ for Uri path.
  • ensure price is of type “raw amount”(i.e 8000 for $80.00)
  • ensure product status output format is ‘0/1’
  • Do not include Product ID.
  • keep field labels, so there is header in the export file
  • Format type for taxonomy fields are ‘link’
b) Configure View for export
  1. Add a new “Data export” display to your view.
  2. Change its “Style” to the desired export type. e.g. “CSV file”.
  3. Configure the options (such as name, quote, etc.). You can go back and do
    this at any time by clicking the gear icon next to the style plugin you just
    selected.
  4. Give it a path in the Feed settings such as “path/to/view/csv”.
  5. Attach this view to the view containing the products you like to export by
    updating the “Attach to:” option in feed settings.
  6. Ensure “Items to Display” is set to “Display All”
c) Generate the file via drush command
drush views-data-export --format=csv VIEW-NAME VIEW-DISPLAY-NAME exports/product-exports/theme_export_Jan11_2014.csv --quote-values --strict=0 --header-row

This will take the view and create csv file from it.

Import Commerce Products

Once you have the csv file with commerce products an export generated as described above, we going to use Commerce_feeds for import this products in another Drupal instance

a) install commerce_feeds module and its dependent modules
drush dl feeds
drush dl feeds_ui
drush dl commerce_feeds
drush en feeds feeds_ui commerce_feeds

This install all the necessary artifacts for using commerce_feeds to import Drupal commerce products

b)Create New Feed Importer

Create a new feed importer named “Product Importer” at Administration -> Structure -> Feeds Importer -> Add Importer. Here are specifics of this importer:

  • Change the parser to “CSV Parser”
  • Change processor to “Commerce Product Processor”
  • In “Commerce Product Processor” settings use product type “product” (or whatever your product is) and change the “Author” to your username.
  • ensure Fetcher is of type ‘file upload’
  • In “mapping”, map:
    • SKU -> Product SKU
    • Title -> Product Title
    • Price -> Price: Amount
    • Image -> Image
    • Set ID as unique target.
    • …//so on with other product fields in the .csv file

c) Run import

Go to ‘/import’ url, use the Product Importer and import your products
Note: before importing, make sure you have copied the artifacts into the same location you importing from if there is any custom fields with artifacts

Exporting Product Displays

Everything would be the same if there wouldn’t be product reference field that links the multiple products to the particular display it is for. To import/export ‘product’ field, we will utilize modules ‘feeds_temper’ and ‘feeds_tamper_ui’ modules.

drush dl feeds_tamper
drush en feeds_tamper feeds_tamper_ui

Their responsibility is to take list of SKUs part of ‘products’ field and create multiple product references for each display at the time of import

a)Create a view

Create view with displays to export(i.e. view ‘theme_display_export’ with display name – ‘display_export’). Here are some important details about the view:

  • ensure result size of node is set to unlimited
  • ensure the output formatter for field ‘product’ is set ot ‘SKU no link’
  • ensure label is attached, so it generates csv with a header row

b) Configure View for export
  1. Add a new “Data export” display to your view.
  2. Change its “Style” to the desired export type. e.g. “CSV file”.
  3. Configure the options (such as name, quote, etc.)
  4. Give it a path in the Feed settings such as “path/to/view/csv”.
  5. Attach this view to the view containing the product displays you like to export by
    updating the “Attach to:” option in feed settings.
  6. Ensure “Items to Display” is set to “Display All”
c)Run Export

Export Product Displays into .csv file by calling drush command as following:

drush views-data-export --format=csv theme_display_export display_export exports/display-exports/display_export_Jan11_2014.csv --quote-values --strict=0 --header-row

This will generate .csv file with product displays

Import Product Displays

Once you have the .csv of product displays, we import them in another Drupal instance. Here are the steps:

a)Create Product Display Importer

First, create product display Importer by going to ‘Admin’->’Structure’->’Feeds Importer’ -> ‘Add New Importer’. Some important things to watch for:

  • ensure this is Nodes Processor under the Process
  • ensure Fetcher is set to ‘file upload’
  • create mappings
b)Create Rule for ‘product’ field

Create Rule for ‘product’ field by clicking on the ‘Tamper’ tab and then ‘add plugin’ under the field ‘product’ to ‘Product:SKU’ mapping or however you named the relationship mapping between display to products.
Note: ensure the rule is of type ‘list/explode’

c)Run importer

At last, run importer by going to ‘/importer’ url and selecting the product display importer following the UI

Troubleshooting

1. Unknown option: –header-row. See `drush help views-data-export` for available options.To suppress this error, add the option –strict=0

Make sure to add –strict=0 to the command or in the ‘alias’ configuration file

2. Exported file’s header comes up empty(“”,””,””…)

Make sure to include label for each field, otherwise the header is empty

3. The Images field is coming up empty when exported into csv file

This because there is need to have a special field formatter to format image into url. Module image_url_formatter does exactly that

4. Price is not importing accordingly all zeros

-ensure price is of type “raw amount”(i.e 8000 for $80.00) in the exported CSV file

5. Product Status is always importing into ‘Disable’ State.

Ensure the exported value of product status is in format of ‘1/0’ that can be configured in the view under ‘output format’

6. Images are not displayed in Admin UI.

Ensure that the exporter is set to export image fields into ‘URI path’ output format that is provided by a separate module ‘image_url_formatter’

7. “A product with SKU some_sku could not be found. Please check that the product exists or import it first”(resulting on display only pointing to first product but not all)

Answer: ??? I am not sure…still figuring this out

8. ‘PDOException: SQLSTATE[42S02]: Base table or view not found:’

Make sure cache is cleared after view created/updated. Also ensure your view returns results

9. Missing Feeds plugin FeedsCommerceProductProcessor

This error come up when there was missing ‘commerce_feeds’ module. After installing and enabling it, the error goes away

10. User by id ‘864’…

This happens when running importer with user configured that doesn’t exist on current Drupal instance. Go to admin/structure/feeds/NAME_OF_YOUR_IMPOTER/settings/FeedsNodeProcessor and update user with any user currently present

11. Warning: is_dir(): Unable to find the wrapper “private” – did you forget to enable it when you configured PHP? in file_prepare_directory()

Ensure that the private dir is configured for your Drupal instance (see “Private file system path” at admin/config/media/file-system)

12. Target is missing for Node Processor of Importer

Make sure the field is present fro the content type you are importing

13. Exports exactly 10 items only no matter what the pager is set to

This is happening when the view exported is not of type ‘Data Export’. Ensure exporting view is of type “Data Export”

Reference

  • https://drupal.org/project/commerce_feeds
  • http://drupal.stackexchange.com/questions/87039/how-can-i-import-and-export-commerce-products-in-drupal-7
  • https://drupal.org/node/622698
  • https://drupal.org/project/commerce_feedsmulti
  • http://www.drupalcommerce.org/node/467

Import/Export Sample Data and Assets for Kickstart All via Features

This is an overview of the custom module – Sample Data developed and used at DesignsSquare.com that was published at Drupal.org for anyone that finds it useful. It is also at github – designssquare-lib-sample-data branch ‘7.x-1.x’

Issue: Widget or Theme needs sample data for kickstart. There is no automatic, easy and standard way to package Sample Data with all of its assets(i.e. images, videos,etc) part of single artifact deliverable for easy install in Drupal.

Solution: Use Features module along with Sample Data module for sample data exports/imports in an automotive manner without another dependency

The common practice to package widgets and themes at DesignsSquare.com delivered to client is by separating the deliverable into 3 parts – code, Structures&Configurations and Sample Data for KickStart all part of one deliverable via Features. The Sample Data part contains sample nodes, menu instances and assets referenced from content, fields and variables.

To automate the packaging process for Sample Data part of the deliverable, we have created Sample Data module. The Sample Data module does the following:

      1. Export/Imports nodes with Alias path
      2. Export/Imports menus based on Alias path
      3. Handle the assets – images, videos, etc for sample data. Specifically:

          A) assets referenced from sample content or other modules located in default public directory(i.e. sites/default/file)
          B) assets referenced by fields(both core type or custom type)
          C) assets referenced by variables
      4. Overrides for context and StormArm variables

Note: assets referenced by custom fields and variables need to use file_managed functionality to work

1. Export/Imports nodes with Alias path

In order to have alias path, you will have to install and enable alias path module – https://www.drupal.org/project/pathauto. Once the sample data have alias path, then export it from features(admin/structure/features). The Sample Data module will hook into the export and store alias path for each node later to restore at the time of import

2. Export/Imports Menus based on Alias path

Once you have the pathauto module enabled, each node will have the alias path used by Sample Data to export and import Menus. Go to Features(admin/structure/features) and look for section “MENU ALIAS” with options to select menues you like to import/export. It will store the menus in the export and then import to the new instance based on alias path. Since it uses alias path, each menu has to have unique links(alias path) to work

3A) Assets referenced from sample content or other modules

Your sample content may be referencing assets. It is also possible that other modules like “imce” for editor is storing assets. The Sample Data module lets you export those assets. Go to Features(admin/structure/features) and look for section “Content Assets”. In this section, it will display all assets(files and directories) from the public directory. Select the ones you like to export and it will store them for later import in the new instance. If you select directory, then it will store the whole tree of the assets

3B) Assets Referenced by Fields

Your sample data(nodes) may have fields of type ‘file’ that is going to reference assets. The Sample Data module will automatically store those assets when you are exporting the sample data(nodes). It will automatically transfer the assets at the time of import in the new instance. The Sample Data module does the same for custom fields referencing assets and not only the Drupal default such as ‘file’. There is one requirement. The ‘file_managed’ functionality needs to be handling the asset in order for the import/export to work

3C) Assets Referenced By Variables

You may have chosen to store asset references in form of FID in the Drupal variable. The Sample Data module will export/import this assets as long as the name of the variable is stored part of the entry in the file_usage table for field(column) – ‘type’:

 //we also add variable name as the 'type' parameter, so we can export via Sample Data module
        file_usage_add($file, 'module_name', 'VARIABLE_NAME', '1');
        // Save.
        file_save($file);

As you see, the variable name is saved part of the usage entry. This is done, so Sample Data module exports the asset

Once the variable is holding reference to the asset and the variable name is stored part of entry in the file_usage table, then in section ‘VARIABLE ASSETS’ in the Features page(admin/structure/features) there is listed available references for you to select to export/import those assets.

4. Overrides for context and StormArm variables

The “CONTEXT OVERRIDE” provides a solution to the fact that features doesn’t let you export something that is already part of another feature. This is an issue if you say have a widget with context that needs to be overridden by site specific context. So by select the context from “CONTEXT OVERRIDE”, it will store those context to import in the new instance

The “STRONGARM OVERRIDE”, besides the ability of overriding another conflicting feature, it also provides a solution to be able export home page, error 404, 403 that is impossible otherwise because it uses the hard coded path(node/id) specific to each Drupal instance. The Sample Data utilizes alias path to import/export home page, error 404 and 403 pages

To summarize, the Sample Data module is one full solution for exporting/importing Sample data for Kickstart data. It exports/imports nodes and menus using alias path. It handles all the assets referenced by content, fields or variables. It also provides ability to override contexts as well as export/import home page, error 404, 403 pages. It does all for you, so you can focus on more important aspects than handling sample data/assets when building widgets,themes and other cool things

Troubleshooting

I am unable to export Content(nodes) via futures

Ensure you have the correct versions for features(7.x-2.0 or above), feature_uuid(7.x-1.0-alpha4…the dev version on July 2014) and uuid(7.x-1.0-alpha5…dev version on July, 2014). Afterwards, enable the content to by exportable at admin/config/content/uuid_features

The specified file public://artifact-name.jpg could not be copied, because the destination directory is not properly configured

Ensure the export feature module has write access and recreate the feature.

Making It Permanent for File_Managed

The file_managed functionality comes with ajax capability to upload and remove file, however. While, it stores the file, it stores it temporarily,so unless we make it permanent, the files just uploaded will be removed after certain time. In this post, we demonstrate one of the many ways how to make the file upload via file_managed a permanent.

Hook Into File_Managed

First, we specify the callback for the form element ‘file_managed’ as following:

$form['backgrounds']['ds_theme_custom_background'] = array(
        '#title' => t('Custom Background'),
        '#type' => 'managed_file',
        '#description' => t('Upload a your custom background image, allowed extensions: jpg, jpeg, png, gif'),
        '#default_value' => isset($form_state['value']['ds_theme_custom_background']) ? $form_state['value']['ds_theme_custom_background'] : theme_get_setting('ds_theme_custom_background'),
        '#upload_location' => DESIGNSSQUARE_THEME.'/theme-settings',
        '#upload_validators' => array(
            'file_validate_extensions' => array('jpg jpeg png gif'),
            // Pass the maximum file size in bytes
            'file_validate_size' => array(MAX_SIZE_LIMIT_DS_THEME*1024*1024),
        ),
        '#process' => array('our_custom_callback'),
    );
}

By specifying ‘#process’, the custom callback – our_custom_callback is being called when ‘upload’, ‘remove’ button is clicked

Make it Permanent

Next, we will ensure file is stored permanently by updating it status as following:

function our_custom_callback($element, &$form_state, $form){
    $element = file_managed_file_process($element, $form_state, $form);

    //file upload
    if (isset($form_state['input']['_triggering_element_name']) && $form_state['input']['_triggering_element_name'] == $element['upload_button']['#name'] && !empty($element['#file'])) {
        //make file permanent
        $file = $element['#file'];
        // Change status to permanent.
        $file->status = FILE_STATUS_PERMANENT;

        //all permanent files need an entry in the 'file_usage' table
        //we also add variable name as the 'type' parameter, so we can export via Sample Data module
        file_usage_add($file, 'designssquare_lib', $element['#name'], '1');
        // Save.
        file_save($file);
    }

    //file removed
    if (isset($form_state['input']['_triggering_element_name']) && $form_state['input']['_triggering_element_name'] == $element['remove_button']['#name']) {
        //ensure file is removed
        if(isset($element['#file']->fid) && $file = file_load($element['#file']->fid)){
            //file exist, lets remove
            //the file is removed despite the count of references present
            db_delete('file_managed')->condition('fid', $file->fid)->execute();
            db_delete('file_usage')->condition('fid', $file->fid)->execute();
            entity_get_controller('file')->resetCache();
        }
    }

    return $element;
}

Here, we check whichever button is clicked – ‘upload’ or ‘remove’. If it is ‘upload’, then we change the status of the file and add entry in the table ‘file_usage’ both needed to make file permanently stored. If button ‘remove’ is clicked, then we ensure the file is removed from both tables – file_managed as well as file_usage. The last step shouldn’t be needed, however. It happens that file_managed functionality, sometimes, adds more than one reference to the same file into file_manage table(see ‘count’) resulting into situation that it doesn’t remove the file because it sees more than one reference. So, we ensure the file is removed no matter how many references listed in the table.

Troubleshooting

1. Fatal error: Call to undefined function – our_custom_callback

For Drupal 2.26,2.28 and, probably, many other version, there seems to be a bug on how the form processing functions are called before all the dependencies are loaded resulting in the error “Fatal error: Call to undefined function – our_custom_callback”. To avoid it, we end up hooking into the Drupal bootstrap process to load our custom function before callback is called via hook_init() as following:

function MODULE-NAME_init(){
    //load for processing file_managed on Ajax calls(i.e. upload, remove)
    module_load_include('inc', 'MODULE-NAME', 'inc/file_defining_custom_callback');
}

Here, we load the file containing the definition of our callback – our_custom_callback. Since the hook_init is called before Drupal is executed, our custom callback is defined that way solving the above problem.

Contextual Links – Editable Blocks

One of the awesome things about Drupal is the contextual links. The contextual links provide a way to attach links to any block on the page for easy editing via UI or any purpose for that matter. In this post, we cover how to specify a region for which the context links should show up, then how to add the links and, at last, how to render them in the template.

1. Set the Region

First, we define a region in page that you like the context links show up when user hovers on top. If you have the default wrapping around the elements added by Drupal(not using block–no-wrap.tpl.php template) then this step is not necessary, because Drupal handles it for you. If you have custom element without any Drupal wrapped html, then you have to specify region for contextual links to show up. To do so, you add the class attribute with name ‘contextual-links-region’ to the outer html element as following:

<div class="contextual-links-region">
    //context links are rendered here
    //rest of the html comes here
</div>

The event on hover is used by context module to make contextual links visible. So by adding class name ‘contextual-links-region’ you are setting the region of the contextual links

2. Create and Add Context Links

To create contextual links, we use context module API:

$contextual_links = contextual_element_info();

This creates the renderable array without the links. The next, we add links we like to show up

    $contextual_links['contextual_links']['#contextual_links'] = array(
           'menu' => array('admin/structure/menu/manage', array('navigation')),
           'block' => array('admin/structure/block/manage', array('system', 'navigation')),
             'views' => array('admin/structure/views/view/'.$vars['view']->name, array($vars['view']->current_display)),
            );

Here, we have listed three different examples with already declared menu links referencing menu, block and views edit form. The index is the module name that declared the path in the menu system via hook_menu. The first parameter of contextual link is the base path of the link. The second parameter is for passing dynamic value that together combines the end path to the form. So, here are the possible paths from the above example:

admin/structure/menu/manage/navigation // for the 'menu' example
admin/structure/block/manage/system or admin/structure/block/manage/navigation //for the 'block' example\
admin/structure/views/view/VIEW_NAME/VIEW_DISPLAY_NAME //for the 'view' example

If you look into, you will find the paths – admin/structure/menu/manage/%/edit, admin/structure/block/manage/%/edit already declared via hook_menu in each of the modules – block, menu. The view path is also declared but in a little bit more dynamic format

2A – Adding Custom Menu Links

Sometimes you may need to add your custom form, then first you will have to do is register the path in the menu system:

//MENU FOR CONTEXT
function MODULE-NAME_menu() {
    $items['custom-path/%/edit'] = array(
        'title' => 'Edit',
        'type' => MENU_LOCAL_ACTION,
        'context' => MENU_CONTEXT_INLINE,
        'page callback' => 'ds_block_process',
        'page arguments' => array(1),
        'access callback' => TRUE,
    );
    // To use local task menu items, there must be a parent page.
    $items['custom-path'] = array(
        'title' => 'The contextual example page',
        'page callback' => 'ds_block_process',
        'page arguments' => array(1),
        'access callback' => TRUE,
    );

    return $items;
}

Here, we register path with pattern ‘custom-path/%/edit’ and the callback function builds the form. Please, see more on this in post – Add and Manipulate Pages with Menu System in Drupal

Afterwards, you add the path to the contextual links:

$contextual_links['contextual_links']['#contextual_links'] = array(
        'MODULE-NAME' => array('custom-path', array('some')),
  );

Here, the contextual link ‘custom-path/some/edit’ is added to the region. The second parameter(i.e. array(‘some’)) is the page arguments passed to the menu as declared above. Overall, adding custom link is just like adding predefined menu links, but only with our custom url and you have to specify module name that registers the path in the menu system.

4. Rendering Links

The last, the contextual menu is rendered in template:

    <?php print render($contextual_links['contextual_links']);?>

This will look up the theme function to context links. Now, for any authorized users who hover over the region, the contextual links will show up on the right corner of the region

Make it Overlay

Perhaps, you like to have your custom contextual links pull the page into overlay page, then the easiest is to declare the custom path to be an admin path:

function MODULE-NAME_admin_paths() {
    $paths = array(
        'custom-path/*/edit' => TRUE,
    );
    return $paths;
}

Now, your custom path for contextual link is an admin path and it automatically applies the overlay

Troubleshoot

1. Views contextual links doesn’t show up

Ensure the Views_UI module is enabled. If this doesn’t solve the problem, see whether the url you you specify in the contextual renderable array exists(i.e.’views’ => array(‘admin/structure/vie..[]’))

References

https://drupal.org/documentation/modules/contextual
https://api.drupal.org/api/drupal/modules!system!theme.api.php/function/hook_preprocess/7
http://dominiquedecooman.com/blog/drupal-7-tip-add-contextual-links-anything
https://api.drupal.org/api/drupal/modules%21system%21system.api.php/function/hook_menu_contextual_links_alter/7
https://api.drupal.org/api/drupal/modules%21contextual%21contextual.module/function/contextual_preprocess/7
http://bleen.net/blog/easier-way-add-contextual-links-drupal-7
http://deglos.com/blog/2011/02/06/hacking-contextual-links-drupal-7

Working with Menus for Themeing in Drupal

In this post, first we look at how to render menu in the template. Afterwards, we look how to manipulate menu html layout and its attributes so the menu can be themed per your design. At last, we go over on having custom menu that is customized in similar way via hooks

1. Build Menu

We like to have menus in the Html scope(THEME_html_process) instead the page scope(THEME_page_process). This is because menu are shared across the page layouts in general, so it is not specific to page per say, however. Drupal comes with the menu already available in the page scope under ‘main-menu’. So, if we want the menu to be available in the html scope we have to create one:

function THEME_preprocess_html(&$vars){{
...
        // Primary nav build links.
        $vars['primary_nav'] = menu_tree(variable_get('menu_main_links_source', 'main-menu'));
}

This builds the render array of main menu and make it available in the html scope. To render the menu itself in html.tpl.php, we would:

    <?php print render($primary_nav); ?>

This take array of main menu generated in preprocess function and renders into html to display in page

2. Overwrite UL Element

Next we overwrite the default ul element and its classes with our custom via hook_menu_tree() as following:

function THEME_menu_tree(&$variables) {
    return '<ul class="nav nav-justified">' . $variables['tree'] . '</ul>';
}
3. Overwrite LI Element

At last, lets overwrite the menu elements themselves. To do so, we use hook_menu_link() as following:

function THEME_NAME_menu_link(array $variables) {
      $element = $variables['element'];
    $sub_menu = '';

    if ($element['#below']) {
        // Prevent dropdown functions from being added to management menu so it
        // does not affect the navbar module.
        if (($element['#original_link']['menu_name'] == 'management') && (module_exists('navbar'))) {
            $sub_menu = drupal_render($element['#below']);
        }
        else if ((!empty($element['#original_link']['depth'])) && ($element['#original_link']['depth'] == 1)) {
            // Add our own wrapper.
            unset($element['#below']['#theme_wrappers']);
            $sub_menu = '<ul class="dropdown-menu">' . drupal_render($element['#below']) . '</ul>';
            // Generate as standard dropdown.
//            $element['#title'] .= ' <span class="caret"></span>';
            $element['#attributes']['class'][] = 'dropdown';
            $element['#localized_options']['html'] = TRUE;

            // Set dropdown trigger element to # to prevent inadvertant page loading
            // when a submenu link is clicked.
            $element['#localized_options']['attributes']['data-target'] = '#';
            $element['#localized_options']['attributes']['class'][] = 'dropdown-toggle';
            $element['#localized_options']['attributes']['data-toggle'] = 'dropdown';
        }
    }

    $element['#localized_options']['attributes']['class'][] = 'agri-nav-item';
    $element['#localized_options']['attributes']['class'][] = 'nav-space';

    //set the parent active when child is currently selected
    if(in_array("active-trail", $element['#attributes']['class'])){
        $element['#attributes']['class'][] = 'active';
        $element['#attributes']['class'][] = 'open';
    }
    // On primary navigation menu, class 'active' is not set on active menu item.
    // @see https://drupal.org/node/1896674
    if (($element['#href'] == $_GET['q'] || ($element['#href'] == '<front>' && drupal_is_front_page())) && (empty($element['#localized_options']['language']))) {
        $element['#attributes']['class'][] = 'active';
    }
    $output = l($element['#title'], $element['#href'], $element['#localized_options']);
    return '<li' . drupal_attributes($element['#attributes']) . '>' . $output . $sub_menu . "</li>\n";
}

Here, we overriding everything from boostrap_menu_link() and made few changes. Added extra classes to the link element, removed caret and ensure parent of selected element is highlighted.

What If You Have Another Menu?

Say you have another menu at the bottom with different markup and class attributes. This menu id is ‘bottom-menu’

A)Build Menu

Build custom menu similar to the Main menu as following:

function THEME_preprocess_html(&$vars){
...
        $vars['bottom_nav'] = menu_tree('menu-bottom-menu');
        $vars['bottom_nav']['#theme_wrappers'] = array('menu_tree__bottom');
}

In the first line, the menu is build via menu_tree() function that takes the menu id and builds the array to render. In the second line, we specify hook for overriding UL element. There is one by default(hook_menu_tree__menu_MENU_NAME), so instead of overriding, we may have used the default one(i.e. hook_menu_tree__menu_bottom_menu)

B) Overwrite UL Element

With the override function specified, we can overwrite the UL element:

function THEME-NAME_menu_tree__bottom(&$variables) {
    return '<ul class="nav-agri nav-tabs-agri nav-justified-agri">' . $variables['tree'] . '</ul>';
} 

Here we use the hook specified, however. You may as well used the default hook(i.e. hook_menu_tree__menu_bottom_menu)

C)Overwrite LI Element

By default the override hook for LI element is hook_menu_link__MENU_ID() plus the menu id. In our case, the override function hook is hook_menu_link__menu_bottom_menu. So, we overwrite the links as following:

function THEME-NAME_menu_link__menu_bottom_menu(array $variables) {
    $element = $variables['element'];
    $output = l($element['#title'], $element['#href'], $element['#localized_options']);
    return '<li' . drupal_attributes($element['#attributes']) . '>' . $output . "</li>\n";
}

There is no dropdown sub-menu to worry, so we create a link and wrap into the LI element before returning

A Word or Two on User Menu

Its very likely your main menu is different than the user menu, so how to handle the html layout for user menu while main menu is using the default hooks for structuring menus
1. User Tree Menu
The user menu has a custom hook – THEME_menu_tree__user_menu available to structure menu tree as following:

function THEME_menu_tree__user_menu(&$variables){
    return '<ul class="dropdown-menu">'.$variables['tree'].'</ul>';
}

2.User Menu Links
There is custom menuhook – THEME_menu_link__user_menu available to structure the menu item list

3.Menu Template file
There is also custom template file – block–system–user-menu.tpl available for you to override block html as you wish for user menu as well

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column ‘field_xxx_display’ cannot be null:

If you are using Features module to import/export sample data like for kickstart data for your new feature. If you are using the newest version dependent modules – uuid and uuid_featurs(7.x-1.0-alpha or above) of the Feature module, then you will get the following error at the time importing sample data that contains field referencing asset such as image, video,etc:

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column ‘field_xxx_display’ cannot be null:

The issue is that there appears to have a bug in uuid_features module of the newest version that strips away attribute ‘display’ from the sample import. One solution is to reattach back the missing attribute just before the sample data is saved. To do so, we use hook_node_presave() as following:

function MODULE-NAME_node_presave($node) {
    //fix  for sample data import where 'display' is striped away for field_xxx causing db syntax error at node_save
    if($field_instance = field_get_items('node', $node, 'field_xxx')){
        foreach($field_instance as $key => $field_ins){
            $node->field_xxx['und'][$key]['display'] = 1;
        }
    }

Here, in line 3, the field instance is retrieved for the node being imported. If it exists, then in line 4, we go through all of the instances of that field and add the attributed ‘display’ that was striped away by uuid_features module. Note, that the word ‘display’ is removed from ‘field_xxx_display’ as mentioned in the error message because its added by Drupal to store the data