Externalizing Configurations in Drupal, Node.js and Grails

In this post, we cover how configurations can be externalize for platform Drupal and frameworks Node.js, Grails. By externalizing configurations, it becomes easer to deploy and move around applications and libraries.

Drupal

There are two approaches
1. Configurations in file
In the sites/default/settings.php add the configurations as follows:

$conf['global_driver_url'] = 'localhost';
$conf['global_driver_port'] = '3000';

The $conf is Drupals global variable array as described here
Then to access it anywhere as follows:

global $conf;
var driverPort = $conf['global_driver_port'];
var driverUrl = $conf['global_driver_url'];

So, here you reference the global variable $conf and then access any variables you wish

2. Configurations from User
We create a content type that contains the configurations. For example, we have content-type name ‘Interface Configurations’ with machine name ‘interface_configurations’

Then in preprocessor in template.php file, we load it as follows:

$nid_config = db_select('node', 'n')
        ->fields('n', array('nid'))
        ->fields('n', array('type'))
        ->condition('n.type', 'interface_configurations')
        ->execute()
        ->fetchCol();
    //load the configurations
    $configurationNode = node_load($nid_config);
    $interfaceConfig = array();
    $interfaceConfig['driver_url'] = $configurationNode->field_driver_url['und'][0]['value'];
    $interfaceConfig['driver_port'] = $configurationNode->field_driver_port['und'][0]['value'];
    $vars['interfaceConfig'] = $interfaceConfig;

Here, we first retrieve the Id of the configuration node containing the configurations. Afterward, it is loaded and values retrieved. At last, it is put in $vars array, so it is available in the html template

Node.js

There are several ways doing it in Node.js and there are some modules such as nconf for the that purpose. In the following, is one way of externalizing configurations in Node.js framework:
1. we create config.js that contains configurations and place it in root folder as follows:

var config = {}

config.controller_url = process.env.CONTROLLER_URL || '/minnehaha';
config.controller_host = process.env.CONTROLLER_HOST || 'localhost';
config.controller_port = process.env.CONTROLLER_PORT || 8010;

module.exports = config;

Here we first declare variable config that will hold the configurations. Then assign our configurations. The ‘process’ contains the command line Node.js variables. So, if you like to have environment based configurations such as dev, prod, etc, then you will have to if/switch condition on ‘process.env.NODE_env’ variable.

Important to note, that the ‘process.env’ queries all of the command environment variables(i.e. PATH,GRAILS_HOME,etc.), so you have access to all of them.

At last, we make or configurations available anywhere in Node.js applications in the next step

2. To make these configurations available, we import the module and then access the configurations as follows:

var config = require('./../config');
...
   var options = {
        host: config.controller_host,
....

Here, the config module is imported and assigned to ‘config’ variable after which the configurations are accessed

Grails

For external configurations in Grails framework,please, see post Externalizing Configurations Per Grails Application

Useful Links:

  • http://stackoverflow.com/questions/5869216/how-to-store-node-js-deployment-settings-configuration-files
  • http://devincarlson.ca/blog/configuring-drupal-use-proxy-server-when-making-external-http-requests

Upgrading Drupal Core and Modules via Drush

There may be a time when it is a time to upgrade your drupal core and modules. It may also be a situation, when some feature or functionality doesn’t work as it should be. In that case, it may be that one of the modules are out of date causing it malfunction.

Its recommended to back up the database and the code of the Drupal site before upgrading

Step 1. Check The Status

To check status on what is out of date or which module has an upgrade available run:

drush cron

This will update the ‘Status Report’. To see the report, login as Admin and go to Reports->Status Report

Step 2. Upgrade Modules and the core

To upgrade the modules and the core, run

drush upc

This will start an interactive mode and walk through the process upgrading each module and eventually the core

To see available options ‘drush upc –help’ in situations where you don’t want to update core but only modules, for example

drush upc --no-core

This will only update modules and skip the core update.

Step 3. Updating Database

To update database, run

drush updb

This will run all database updates
To see available options run ‘drush updp –help’

Step 2 & 3: To upgrade code and database at once

To upgrade modules, core together with database udpates, run:

drush up

Its combines the step 2 and 3. For finding available options, run ‘drush up –help’

Other

1. Find Current project info

drush status

This will display current Drupal and Drush vers, database driver, php configuration, etc.

Useful Lins

Updating drupal core and modules

Locally Untracked Files in Git While Making a Grails Plugin In-Place

At my current workplace, the grails application is divided into application and an extra plugin. To avoid package, maven-install every single time new change made in plugin that need to be seen in application, I decided to make this plugin in-place. To make it in-place plugin, i had to update BuildConfig.groovy as follows:

...
grails.plugin.location.nameOfPluginPlugin = '../../pluginLocation'
...
plugins {
    ...
    //uncomment plugin dependency
}
...

At the same time, i don’t want these changes go into others developers local repos, so i wanted to untrack BuildConfig.groovy file. Here are steps to untrack file from git repo.

Ignoring File in Git

To ignore file globally,here are steps

Step-1: Create global git ignore file, not tracked in repository that is user-specific

git config --global core.excludesfile pathTo/.gitignore_global
 

Note: You can find different sample git ignore files per technology here

Step-2: Added BuildConfig.groovy in my local global git ignore file as follows

...
/relative/pathToApp/BuildConfig.groovy
...

This will keep it ignored, but we still need it make it be untracked before it can be ignored

Step-3: Untrack the file itself since it is already tracked

git rm --cached BuildConfig.groovy

This makes the file to be ignored and also untracked, however. By pushing to the shared repo others will make this file to be untracked as well, which we don’t want

Ignoring vs Untracked

Ignore will only apply to untracked files. So, in our case, where we need the file to still be tracked, the solution of ignoring described above will not work. Instead, we set the file to be assumed unchanged by Git:

git update-index --assume-unchanged appname/grails-app/conf/BuildConfig.groovy  

By ‘assume unchanged’, the file is ignored and no changes appear in git repo. This makes the grails plugin in-place only for me since the changes are not tracked anymore and ,thus, pushed into the shared main repository

Useful links:

Install Node from Source Ubuntu

In this post, we log the steps to install Node.js on Ubuntu from source (the alternatives are many to install from package manger with one good source of information here )

Step 1 – Get the source from Nodejs.com

Go to to http://nodejs.org and copy link of the source tar file of your interest. Afterwards, open console and move, unpack, extract the file as follows:

wget https://nodejs.org/download/release/v0.10.6/node-v0.10.6.tar.gz
tar xvzf node-v0.10.6.tar.gz
cd node-v0.10.6

Make sure your grab the right one source package not the binary archive packages, otherwise you will miss necessary files such as .configure to install it in next step

Step 2 – Compile and Install Node JS

From the console, run the following commands:

cd node-v0.10.6
./configure
make
make install

Firs, the configuration is created created in line 1. Then, the ‘make’ command compiles Node.js after. Afterwards, the ‘make install’ does the installation including copying files accordingly

Step 3 – Test the Installation

Check the version:

node -v

Lets create file test.js that contains the following line:

console.log('hello world');

Afterwards, run this node.js script:

node test.js

This should display the ‘hello world’ text

NOTE: Make sure you check Node Version Manager tool that will automatically grab Node.js and let you install version you wish.

Uninstall Nodejs

First, lest find out which executable used for nodejs

which node

This will return something like /path/bin/node.
Next, go to the path dir:

cd /path

Once in the /path dir, to remove:

rm -r bin/node bin/node-waf include/node lib/node lib/pkgconfig/nodejs.pc share/man/man1/node.1

This will remove Nodejs but not npm. If you install npm again into a custom path that starts off empty, then you can see what it adds and then you will be able to make a list for npm similar to the above list I made for node.

How To Install Drush

Drush is command line utility for installing, maintaining and troubleshooting Drupal platform. This post logs the steps for installing drush on Mac(OS X 10.7.5)/Linux Ubuntu and Windows as well

Linux & Mac

Install Drush

To install the Dev or the most current version for Drush:
1. Clone Drush git repository

sudo git clone https://github.com/drush-ops/drush.git /root/tools/drush

2. Put Drush executable in the search path:

sudo ln -s /root/tools/drush/drush /usr/local/bin/drush

If you don’t know the search locations, then look it up variable $PATH that lists all the search locations for executables:

echo $PATH

Drush is using composer to deploy, so lets install composer as following:

sudo curl -sS https://getcomposer.org/installer | sudo php

this will download the composer. It displays the location that you will need next step

mv dir/downloaded/composer.phar /usr/bin/composer

Here, you move the composer into a path that looks for executables as specified by $PATH, so it can be found. At last, lets install drush

cd /path/to/drush
composer install

DONE! Test it by running ‘drush –v’ which should display current version

Upgrade Drush From Legacy Install

1. Clone Drush git repository

sudo git clone https://github.com/drush-ops/drush.git drush

2. Find where the executable is currently used:

which drush

This will display path of current drush executable. Go to that directory, rename or delete it and create new link to the new version of Drush cloned in Step 1 as following

sudo ln -s /Users/margots/DevTools/drush/drush drush

Here, the path is to the new version of drush cloned in step 1

Done!!


Old Way

Prerequisites

  • wget -or-
  • unzip -or-
  • git -or-

Step-1: Installing Prerequisites.

Verify Unzip installed by running ‘unzip’ from command line. If it isn’t installed then:

sudo apt-get install unzip

Verify wget is installed by running ‘wget’ from command line. If it isn’t installed then:

curl -O http://ftp.gnu.org/gnu/wget/wget-1.14.tar.gz
sudo tar -xzf wget-1.14.tar.gz
cd wget-1.14
sudo ./configure --with-ssl=openssl
sudo make
sudo make install
Stept-2: Installing Drush with PEAR

To verify PEAR(PHP Extension and Application Repo) is installed type ‘pear version’ in command line. If it doesn’t exist then:
For Mac:

cd /usr/local
sudo wget http://pear.php.net/go-pear.phar
sudo php -d detect_unicode=0 go-pear.phar
sudo pear upgrade --force pear
sudo pear upgrade --force Console_Getopt Console_Table
sudo pear upgrade-all

For Ubuntu:

sudo apt-get install php-pear
sudo pear upgrade --force pear
Step-3: Installing Drush

To install drush:

sudo pear channel-discover pear.drush.org
sudo pear install drush/drush
which drush
drush

If you see the following message – ‘Drush needs to download a library from [..]Console_Table-1.x.x.tgz[..]’ with error, then:

sudo rm -Rf ~/.drush
Upgrade drush

To upgrade drush:

sudo pear upgrade drush

Windows

Prerequisites

  • cygen

Installing Drush

Run Windows installer for Drush listed at http://drush.ws/drush_windows_installer

Configure Drush for Cygen

To run drush from Cygen, we mount the path and then create shell alias. To mount path to drush in the Cygen, add the following in the /etc/fstab

C:\ProgramData\Drush\drush.php /cygdrive/c/ProgramData/Drush/drush.php binary,posix=0,user 0 0

Next, we create alias – drush by adding the following in .bashrc in your home dir

alias drush='/cygdrive/c/ProgramData/Drush/drush.php'

Afterwards, reload Cygen shell and run ‘drush –version’. It should display the drush version which is good way to verify if drush is working

Troubleshooting

1. Tip

If you run into any issues while trying to install some package, make sure you run ‘sudo apt-get update’

2. Unable to load autoload.php. Drush now requires Composer in order to install

This happen after upgrading drush. Solution is to install composer as following:

sudo curl -sS https://getcomposer.org/installer | sudo php

this will donwload the composer. It displays the location that you will need next step

mv dir/downloaded/composer.phar /usr/bin/composer

Here, you move the composer into a path that looks for executables as specified by $PATH, so it can be found. At last, lets install drush

cd /path/to/drush
composer install

The This should solve the problem

MS-DOS style path detected: C:\path\drush.php
Preferred POSIX equivalent is: /cygdrive/c/path/drush.php

You have to mount and then refer the drush in the Cygen. See step – Configure Drush for Cygen above

Warning: The lock file is not up to date with the latest changes in composer.json. You may be getting outdated dependencies. Run update to update them.

For details, please, see post – . To fix it, run:

composer update --lock

This will update the lock file to get rid of the error message,so you can proceed with the installation

[RuntimeException] vendor does not exist and could not be created

This message is given when the drush directory(or dir you try to run the composer installer) doesn’t have write permissions. To fix this:

sudo chmod -R 777 DIR
Unable to send e-mail drupal

To turn off drupal sending email add the following line to the php.ini

sendmail_path = /bin/true
[UnexpectedValueException] Could not parse version constraint ^2.6.3: Invalid version string

To solve this error, update the composer currently installed:

composer self-update
the requested PHP extension pcntl is missing from your system drush

The error came when installing drush via composer “composer install”. To solve, I deleted the composer.lock and rerun the installer “composer install –dev”

References

http://duntuk.com/how-install-drush-github-after-drupal-project-removal
https://drupal.org/node/1674222

Drupal install with Drush

In this posts, we list steps to install Drupal site utilizing Drush

Tested: Drush 5.8, Mac 10.7.5, MySQL 5.0.10

Step-1. Download and Install

1)Download Drupal
To download Drupal, run the following in command line:

drush pm-download drupal

This creates folder drupal-x.ver and downdloads drupal. if need specific version, then replace ‘drupal’ to ‘drupal-7.18’ for example

drush pm-download drupal-7.18

2) Rename default folder
After downloading drupal, the drupal code is in default folder – drupal-x.ver. Perhaps, you like to move into name of your website.

sudo mv drupal-7.18 nameOfMySite

3)Create and Set up Database(DB)
Here is how to setup database from the command line interface:

mysql -u USER -p
mysql>create database NAME_OF_NEW_DB;
mysql>GRANT ALL PRIVILEGES ON NAME_OF_NEW_DB.* TO db_user@'localhost' IDENTIFIED BY 'db_passwd';
mysql>flush privileges;

In line one, we login the mysql server run locally. In line two, the new database is created with name ‘NAME_OF_NEW_DB’. In line three, a new user is crated for the new database and all privileges are granted for that user. In last line, the cache is flushed for our changes to take an effect.

4) Install Drupal
Install Drupal as follows:

cd nameOfMySite
sudo drush si --db-url=mysql://dbUser:dbPass@localhost:3306/nameOfDb --account-pass=adminPassword

Here the dbUser, dbPass and nameOfDb are the credentials created in step 3

Besides installing Drupal, it also configures database. The ‘–acount-pass=adminPassword’ specifies password for admin user for the Drupal website.

5. Configure Local DNS
To rout the request to your local drupal instance, update the hosts file. For Unix based systems, it is located at /etc/hosts. If your local domain name is going to be test.com, then insert the following line in the /etc/hosts file:

127.0.0.1       test.com

6. Configure Virtual Host
At last, configure the virtual host for your new site in Apache web server. Depending where is your Apache Web server installed and what version, the location for virtual host settings my differ. In our case, it is located at apacheRoot/conf/extra/httpd-vhosts.conf. So, to configure the virtual host for our new drupal instance, we would insert the following:

 
...
<VirtualHost *:80>
    DocumentRoot "/path/to/drupal_root_dir"
    ServerName test.com
    <Directory /path/to/drupal_root_dir>
        Options FollowSymLinks
        AllowOverride All
    </Directory>
</VirtualHost>
...

In line 3 and 5, the directory of our new drupal site is installed. In line 4, the local domain that we configured in previous step is specified.
Afterwards, Restart Apache server to reload virtual hosts configurations

service apache2 restart

or

sudo /root/To/Apache/dir/bin/apachectl restart

This will restart Apache and load the new configurations
!Some Apache Web Server versions need specifically enable the virtual host, then

sudo a2ensite yoursite

7. Test it!
Open browser and point it to the test.com. If it doesn’t work, see the Troubleshooting section below for help.

Step-2 Set up Root directory

If your web server is configured to service the drupal site from the subfolder than:
a)Specify directory from where the drupal site is serviced from.
In  the .htaccess(in site root folder) uncomment line “ReriteBase/drupal” and updated it to the location of your site:

RewriteBase /PathToSite

b)Configure the Drupal base Url
To configure Drupal base, uncommenting $base_url in settings.php file( usually in sites/default/settings.php) as follows:

$base_url = 'http://www.example.com/pathToSite'; 

Media Modules

For making it possible to insert images into text area using WYSIWYG, there are two modules if using WYSIWYG module for your editor solution:

If you are using CKEditor Module for editor solution instead, then the IMCE module is sufficient

The CKEditor works with the above modules. There is currently issue with WYSIGWYG default module to work with CKEditor, but solution is to use the dev(7.x-2.x-dev) module of WYSIGWYG

Step-3 Install other Useful modules

a) JQuery Update – updated jquery module for drupal core (http://drupal.org/project/jquery_update)

drush dl jquery_update-7.x-2.x-dev -y
drush en jquery_update-7.x-2.x-dev -y

Note:As you see, we chose the dev version of jquery_update to ensure it contains lates jquery lib version, however, in your case it may be different. Afterwards, select jquery version you wish your drupal site uses by going into admin->modules->jquery_update->Configure and select version

b) Twitter Bootstrap – see post ‘Installing Twitter Bootstrap with Drupal

c) WYSIWYG [wysiwyg] – install necessary module before downloading and configuring one of the wysiwyg editors. To install:

drush dl wysiwyg
sudo drush en wysiwyg

Once module wysiwyg installed, then enabled it in Admin->Modules->Check ‘wysiwyg’ module and save. Next, press ‘configure’ to see all the available wysiwyg editors with instructions how to install and enable any from the list. Our favorite – WYMEditor

!Alternative to WYSIWYG module is CKEditor Module. While it is limited to CKEditor, it is slick and can be configured well.

d) Admin Menu [admin_menu]- makes the admin menue items to drop down in hierarchal manner instead one level
To install:

drush dl admin_menu
sudo drush en admin_menu

e) Path Auto [pathauto] – provides an option to alias url to whatever you choose or auto generate alias url from title field.

drush dl pathauto
sudo drush en pathauto

Note: the module ‘pathauto’ dependes on module ‘token’. If installed via drush, then the dependency is installed automatically

f) Date [date] – adds field type of Date with nice date picker UI. To install:

drush dl date
sudo drush en date

Afterwards, enable the time pop-up picker in the Admin->Modules section

g) Views[views] – enable to create views that is an excellent feature to aggregate content in custom way for front page or any section  of the page for that matter. To install:

drush dl views
sudo drush en views

Note: You may be interested in better_exposed_filters module for exposing more enhanced filters to user
Note: the module ‘views’ has dependency of module ‘ctools’ that is auto installed with drush

h)Database Migration – for backing up and migrating database, the backup_migrate is good choice

i)Botcha – to stop spam at form submission including user registertion

Step-4 Developers Modules

a)Theme Developer[devel_themer] – helps to see templates used any page, profiling db queries, generate content and other cool stuff. To install:

drush dl devel_themer
drush en devel_themer

!If you install not via drush then make sure your install also the dependencies. With Drush, it does automatically
Few tools from Theme Developer:

  • Go to Structure->Blocks and enable Develpers block. In the Developers blog you can select ‘devel setting’ to enable profiling DB queries
  • Go to Configuration->Development->Generate Content. This generates some content to work with
  • Select ‘Themer info’ right bottom corner to see what templates called, events triggered and variable renedered

b) Block Class IU[block_class] – it enables to add css styling to any block from UI never need to go into code
To install:

drush dl block_class
drush en block_class

c) Install firebug for Drupal. In Firefox select Tools->Add On->Search with keyword ‘firebug’ and select Drupal Firebug add on. Afterward, restart firefox.
That installs for the browser, but then for firebug support in your drupal development site:

drush dl drupalforfirebug
drush en drupalforfirebug

d) Advanced Help [advanced_help] – To access advanced help index (good for developing views). To install

drush dl advanced_help
drush en advanced_help

e)Examples For Developers – examples for custom module development

drush dl examples
drush en examples

f) Coffee[coffee] – is great UI feature to get around quickly. By pressing AlT+D, it opens the input box where you can type the module or admin functionality you want to get/edit or configure. Afterwards, click the link to go there

g)Module Filter [module_filter] – for assistance on navigating around the modules, the module_filter module is great by filtering and adding extra menu in modules section.

h) Module Instructions[module_instructions] – pulls up all ReadMe files easy available to read in the Modules section

Transform for Versioning and Exporting

Drupal has a way to convert all site or parts of the sites functionality into features for ability to version it into code as well as easy migrating. Here are all the modules that will do the work for Drupal 7:
a)Features – makes it easy for you to export features that is content types, views, etc all of which can be called configurations and content Structure. It packages into a custom module that you can transfer on another Drupal instance

b) uuid features – is integration for Features module that adds a capability for exporting content itself not only the content structure. It also supports file exports, services. Make sure you specify which content structures enabled for the feature module at ‘admin/config/content/uuid_features’. Make sure also create missing UUIDs at admin/config/system/uuid

c)StrongArm – It provides ability to feature variables and configurations

d)commerce_features – it provides ability to feature commerce customer profiles, products, tax rates and types.

d)features_extra – One of the features_extra submodules is fe_blocks. This provides an ability to convert block configurations. This especially useful if your site is using context for organizing blocks and you need disable all the blocks in the ‘structure/blocks’

SEO Modules

Besides ‘autopath’ module for url naming, here are other useful SEO related modules

a) XMLSiteMap
[xmlsitemap] – generates xml site map that is submitted to search engines

b) Site Verify
[site_verify] – provides an easy way to verify your site with search engines

c) SEO Checklist
[seo_checklist] – provides a check list for SEO of your site

c) Google Analytics
[google_analytics] – enables google analytics in site

Other

a) Add This
[addthis] – enables your site users to vote and plus(+1) your products

b) Add To Any
[addtoany] – helps readers share, bookmark, and email your pages and articles (aka stories) using any service, such as Facebook, Twitter, Google+, Pinterest, and other.

c)Enable Menu Title in HTML
Twitter Bootstrap and other CSS styling frameworks often use icons class to insert icons in front of title. By default, Drupal doesn’t permit the title contain html. There is a moduel menu_html that allows easy enable title be html

d)Reference Entities
Entity Reference helps to reference other elements. Really good for building some complex data structures vi UI like photo gallery,podcast,etc.

Troubleshooting

1. Error:

Drupal requires you to enable the PHP extensions in the following list (see the system requirements page for more information):gd
Looks like  missing php5-gd2.so lib. Run:

sudo port install php5-gd2
sudo port activate php5-gd2

2. Error:
Command site-install needs a higher bootstrap level to run - you will[error] need to invoke drush from a more functional Drupal environment to run this command.
Make sure you are in the directory of drupal that was downloaded in step 1.

3. Error:

...SQLSTATE[HY000] [1045] Access denied for user 'someuser'@'localhost...

Apparently, the mysql user provided to drush unable connect database. Ensure the user is remote localhost enabled. Here is how to test:

mysql -u someuser -p -h localhost

If not then, you have to enable it as remote user as follows:

mysql>GRANT ALL PRIVILEGES ON *.* TO someuser@'localhost' IDENTIFIED BY 'db_passwd';
mysql>FLUSH PRIVILEGES;

This will make user ‘someuser’ as remote access user from ip of ‘localhost

Rewrite Fails:
If you able to load from root but unable to go any other pages of the drupal website then it likely an issue with rewrite. To troubleshoot, first check that appache is loading mod_rewrite:

apachectl -t -D DUMP_MODULES

This displays all modules loaded by apache web service. Look to see if ‘rewrite_module’ is present. If it is, then ‘rewrite’ loads, but, apparently, it hasn’t been configured correctly. One way is to have virtual host configuring your drupal portal including rewrite as well:

...
<VirtualHost *:80>
    DocumentRoot "/Users/user/Sites/drupal_root"
    ServerName drupal-test.com
    <Directory /Users/user/Sites/drupal_root>
        Options FollowSymLinks
        AllowOverride All
    </Directory>
    ErrorLog "/private/var/log/apache2/drupal-test.com-error_log"
    CustomLog "/private/var/log/apache2/drupal-test.com-access_log" common
</VirtualHost>
...

The highlighted lines configure the overwrite. The DocumentRoot is folder where your drupal site is serviced from. The ServerName is your configured local DNS name in /etc/hosts file. Put this in your /etc/apache2/extra/httpd-vhosts.conf and uncomment the following line in http.conf:

...
Include /private/etc/apache2/extra/httpd-vhosts.conf

This will make apache load this virtual host configuration.

4. Error:
After login as admin, there is following error:

  file_put_contents(temporary://devel_themer/: failed to open stream: DrupalTemporaryStreamWrapper::stream_open" call failed in devel_themer_store_krumo() 

Apparently, Drupal(devel_themer module) uses local file system for storing temporary files. Make sure the temp folder for drupal got permissions to write. Go to Administration >> Configuration >> Media >> File system and look ‘Temporary Directory’ to find out where is the folder located. Afterwards, enable write permission

5. PDOException: SQLSTATE[HY000] [2002] Connection refused in lock_may_be_available() [..] includes/lock.inc drupal

This error is because Drupal unable to connect to MySQL database. You can double check your DB configurations in settings.php by connecting manually from command line as follows:

mysql -u userName -p -h hostName

Where the userName and hostName is the value from the db configuration in settings.php. Most likely, you don’t have a user with that hostname in the database. To check, run:

SELECT user FROM mysql.user;

And look for the username as well as the hostName. If it doesn’t exist then to create one:

GRANT ALL PRIVILEGES ON db_name.* TO db_user@'hostname' IDENTIFIED BY 'db_passwd';

Installing MySQL on Mac

Despite the popular virtualization solutions like Vagrant, Micro Cloud Foundry and others all of which are perfect solutions for your development environment that would include mysql, there may be reason you need to install Mysql server locally on your machine. This covers steps how to install mysql via macports packaging distribution.

Environment: MySQL 5.5, Mac 10.7

1- Install MacPorts

http://www.macports.org/install.php

2- Update MacPorts

sudo port selfupdate

3- Install MySQL server and client binaries

sudo port install mysql55

4- Install MySQL server startup scripts

sudo port install mysql55-server

5- Initialize the Database:

sudo /opt/local/lib/mysql55/bin/mysql_install_db --user=mysql

6A- Trick to get root user work

sudo chown -R mysql:admin /opt/local/var/db/mysql55
sudo chmod u+rwx,go= /opt/local/var/db/mysql55

6B- Starting MySQL

sudo /opt/local/share/mysql55/support-files/mysql.server start

7- Setting Root password

/opt/local/lib/mysql55/bin/mysqladmin -u root password 'changepasswordhere'

8- Enable the startup script so mysql starts on boot

sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql55-server.plist

– or —

sudo port load mysql55-server

9-Creating user that can access DB from any IP address

GRANT ALL PRIVILEGES ON *.* TO db_user@'%' IDENTIFIED BY 'db_passwd';
FLUSH PRIVILEGES;

10 – Verify all working

shell> mysqladmin version
shell> mysqladmin variables
shell> mysqladmin -h `hostname` version variables
shell> mysqladmin -h `hostname` --port=3306 version
shell> mysqladmin -h host_ip version
shell> mysqladmin --protocol=SOCKET --socket=/tmp/mysql.sock version

11-Reinstalling

If you need to uninstall/reinstall for whatever reason, remove the database directory when you do the uninstall

sudo port uninstall mysql55-server
cd /opt/local/var/db/
sudo rm -rf mysql55/

Troubleshooting

1. ERROR 2003 (HY000): Can’t connect to MySQL server on ‘127.0.0.1’ (61)
I wasn’t able to connect via client such as MySQL Workbench. Part of troubleshooting was running:

mysql -u makapacs -p -h localhost
mysql -u makapacs -p -h 127.0.0.1

The last, displayed the error ‘ERROR 2003 (HY000): Can’t connect to MySQL server on ‘127.0.0.1’ (61)’. Next:

○  telnet localhost 3306
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
Trying fe80::1...
telnet: connect to address fe80::1: Connection refused
telnet: Unable to connect to remote host

Clearly, something doesn’t let connect via TCP connection
Solution:Make sure ‘skip-networking’ is commented out in configuration file(our case /opt/local/etc/mysql55/my.conf) or when starting mysql server, it does’t start with ‘–skip-networking’

In skip-networking mode, the mysql instance doesn’t “listen for TCP/IP connections at all. All interaction with mysqld must be made using named pipes or shared memory (on Windows) or Unix socket files (on Unix)” from MySql Docs

2. ERROR 2003 (HY000): Can’t connect to MySQL server on ‘127.0.0.1’ (111)
This error was because, mysql server was binded on particular ip as follows (/etc/mysql/my.cnf):

# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address = 173.255.xx.xx

So, when mysql binded on specific ip then it only listens remote connection from localhost or the ip specified. Solutions was to comment out ‘bind-address’ and restart mysql server. Afterwards, any remote connection worked

3. Warning: PDO::__construct() [pdo.–construct]: [2002] No such file or directory (trying to connect via unix:///var/mysql/mysql.sock)
This error came out suddenly one day. It cannot find the mysql socket file. To locate

mysqladmin variables | grep 'socket'

This will display where the mysql.socket is created and placed every time mysql starts. Next, create symbolic link so php can find it

sudo ln -s /opt/local/var/run/mysql55/mysqld.sock mysql.sock

Important to note, to find out where Apache looks for mysql.sock, run in browser and look under ‘pdo_mysql’ section

4. Warning: mysql_connect(): No such file or directory
Solved by changing mysql host from ‘localhost’ to ‘127.0.0.1’. The reason for this error is because the “localhost” is a special name for the mysql driver making it use the unix socket to connect to mysql instead of the a tcp socket.

Fatal error: Uncaught exception ‘PDOException’ with message ‘SQLSTATE[HY000]: General error: 2006 MySQL server has gone away’ /includes/database/database.inc

The solution was to add following line in mysql55/my.conf file:

[mysqld]
max_allowed_packet=100M

This gave the necessary extra space for the communication between db and the app

Useful Links:

 

  • http://jackal.livejournal.com/2160464.html
  • http://stackoverflow.com/questions/4798464/installing-mysql-on-snow-leopard-using-macports
  • http://dev.mysql.com/doc/refman/5.0/en/resetting-permissions.html#resetting-permissions-unix
  • http://dba.stackexchange.com/questions/15004/how-to-initiate-mysql-on-mac-os-x-10-6-8
  • http://dev.mysql.com/doc/refman/5.0/en/flush.html
  • http://www.razorsql.com/docs/mysql_connection_troubleshooting.html

 

Moving from Mercurial and BitBucket to Git and GitHub

While skeptical at first, after trying it out github has won my favor as my favorite version control. This post records steps made to transfer from Mercurial and BitBucket to Git and GigHub while keeping history

Step 1 – Set up Password caching following article ‘Set Up Git’

Step 2 – Install git-hg module. This allows to clone a mercurial repository from BitBucket and then converts it to git repository that you can push into the GitHub.com

Followed article ‘Moving Your Mercurial To Rep To Git’

Step 3 – Clone the existing GitHub repository and push it to GitHub as following:

git-hg clone http://bitbucket.org/some/repo name-git-repo
cd name-git-repo
git remote add origin http://github.com/some/repo.git
git push origin master

Potential Issues

1. “fatal: remote part of refspec is not a valid name in .”
This happens when you created new branch that you would like to push into github repo while the github repo doesn’t contain the new brach. Solutions:

git config push.default current

This will change default setting for ‘push’ to create new branch in the github repo if doesn’t exist before pushing. Here are other options:

  • nothing : Do not push anything
  • matching : Push all matching branches (default)
  • tracking : Push the current branch to whatever it is tracking
  • current : Push the current branch

Useful Links

Upgrading PHP on Mac and Ubuntu

In this post we list the steps of installing php 5.4, however, the steps are the same for any version. You may have to choose different binary names but for those you can look up in macports.com. In addition, once multiple versions of php binaries installed, it is easy to switch between the versions. It involves only two steps – activate proper version binaries all of which covered in this post

Mac

Step-1 Installing PHP 5.4
We use macports packaging management system to fetch artifacts

sudo port selfupdate
sudo port install php54
sudo port install php54-gd
sudo port install php54-mysql
sudo port install php54-apache2handler
sudo port activate php54
sudo port activate php54-gd
sudo port activate php54-mysql
sudo port activate php54-apache2handler

The activate commands may not need to be run since its auto activate when running ‘port install’, however. We list here for information

Step-2 Enable PHP in Apache
When installing php54-apache2handler module via MacPorts, it downloads php54.so binary for apache and places it in the /path/to/apache2/modules folder(in our case it is /opt/local/apache2)

So, you have to tell Apache load the new php54 version binary by replacing following lines in apahce configure file http.conf (/path/to/apache2/http.conf) as following:

LoadModule php5_module libexec/apache2/libphp5.so

–with–

LoadModule php5_module modules/mod_php54.so

Afterward, restart Apache service. This ensure that the php module is loaded.

Note: you may want to confirm that the new module ‘mod_php54.so’ with proper permissions is present (for mac 10.7 in /path/to/apache2/modules)

Step-3 Customize PHP

cd /opt/local/etc/php54
cp php.ini-development php.ini

Step-4. Verify Php Version
To verify, run:

php -v

The version may be different. This happens if you have php already installed, then you have to remove the default PHP configuration:

which php
/usr/bin/php
sudo rm /usr/bin/php
sudo ln -s /path/to/php/executable php

Here, we removed the pointer to the old php executable and create reference to the new one. MacPorts downloads the new php executable in /opt/local/bin directory and name it ‘php55’

Aftewards, the ‘php -v’ should match with version you installed.

Switching PHP Version
To switch between versions you would run:

sudo port select --set php php54

And then updating /path/to/apache2/http.conf to load proper *.so module for the particular PHP

...
LoadModule php5_module modules/mod_php54.so
...

To see what php modules available and their exact naming see ‘/path/to/apache2/modules’

To see MacPorts logs

To see MacPorts logs

 /opt/local/var/macports/logs/

Uninstalling before upgrading
You may have to uninstall before you can install newer version. If so, make sure first deactivate/uninstall dependencies i.e. php5-gd and php5-mysql. Please see more
How to Deactivate One Macport Variant and Switch to Another One

Ubuntu

For ubuntu

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:ondrej/php5-oldstable
sudo apt-get update
sudo apt-get install php5
sudo apt-get install php5-mysql
sudo apt-get install php5-curl
sudo apt-get install php5-gd

The ‘php5-oldstable’ is for PHP 5.4. For the php5.5 there is ppa:ondrej/php5, however. The repository ppa:ondrej/php5 doesn’t seem to have lucid Ubuntu distribution as well it is upgrading to Apache2.4 which a lot of changes.

The ‘python-software-properties’ is installed in order to use ‘add-apt-repository’ command, however. you as well manually can add the repository at /etc/apt/sources.list as following:

deb http://ppa.launchpad.net/ondrej/php5-oldstable/ubuntu lucid main
deb-src http://ppa.launchpad.net/ondrej/php5-oldstable/ubuntu lucid main

If you may need to uninstall php on ubuntu:

sudo apt-get -y purge php*

or more conservative approach:

sudo apt-get -y purge libapache2-mod-php5 libapache2-mod-php5filter libexpect-php5 libgv-php5 libow-php5 php5 php5-adodb php5-auth-pam php5-cgi php5-cli php5-common php5-curl php5-dbg php5-enchant php5-exactimage php5-ffmpeg php5-fpm php5-geoip php5-gmp php5-idn php5-imagick php5-imap php5-interbase php5-intl php5-json php5-lasso php5-ldap php5-librdf php5-mapscript php5-memcache php5-memcached php5-mhash php5-midgard2 php5-ming php5-mssql php5-mysql php5-mysqlnd php5-odbc php5-pgsql php5-ps php5-pspell php5-radius php5-recode php5-remctl php5-rrd php5-sasl php5-snmp php5-sqlite php5-suhosin php5-svn php5-sybase php5-tidy php5-tokyo-tyrant php5-uuid php5-xcache php5-xdebug php5-xmlrpc php5-xsl

You can also search for all php packages as following:

sudo aptitude search php5|awk {'print $2'}|grep -v i386|grep -v "^A"|tr "\n"  " "

Other

1. How to locate a dir from which the Apache is currently running on your machine

ps -ef | grep apache

2. How to locate a dir from which the current installation of PHP is running from on machine
Create php config info file – phpinfol.php and place the following.

<?php phpinfo(); ?>

Afterwards, open this file in the browser

Issues

1. Page Not Found

After doing everything and pointing to phpinfo.php, the browser returns page not found. the issue was that it was loading the php module from the wrong dirrectory. The http.conf was containing:

...
LoadModule php5_module modules/mod_php54.so
...

But needed to be:

LoadModule php5_module libexec/apache2/mod_php54.so
2. Exception: Stripe needs the CURL PHP extension. in require_once()[…]

One of the libraries(Stripe) is using CURL, so i forgot to install. To do so:

sudo port install php54-curl

For Ubuntu

sudo apt-get install php5-curl

This also activates the php-curl, so all is left to restart the apache server. Depending on you PHP Version, there may be different php-curl module to install. Please see macports.com for list of packages and versions

3. Stripe needs the Multibyte String PHP extension

Here the Stripe module is using MultiByte String lib that is missing. To install

sudo port install php54-mbstring  

Depending of your PHP Version, you may have to install a different version of mbstring lib. Afterwards, make sure to restart the apache server

3.PDOException: SQLSTATE[HY000] [2002] No such file or directory in lock_may_be_available()

(line 167 of /Users/maxit/Sites/drupal/commerce/first-commerce/includes/lock.inc)

By default, mysql is configured not to use TCP connection(particular for php55), so your sites will break. For solution see post Installing Mysql On Mac for ‘ERROR 2003 (HY000): Can’t connect to MySQL server on 127.0.0.1’

References

Configure Debugging with PHPStorm

Debugging is crucial part in being productive developer. Setting up debugging for PHPStorm IDE has been a painful process partly because the info on the subject matter online have been confusing to say at least. In this post, I hope to clarify some of the misconceptions and, perhaps, save some of your time.

It’s important to note, you will not be able to do remote debugging if your PHPStorm IDE is on machine with dynamic IP and is behind LAN that you don’t have control of configuring the router because Apache server Xdebug service needs to forward requests to the PHPStorm IDE

The post is divided into two main parts – Configure Server and Configure PHPStorm IDE both of which needed to make debugging work in PHPStorm

Tested: PHP5.4.10, Mac OSx 10.7, PHPStorm 5.0.4

How it works?

(in general)
When you start debugging at PHPStorm IDE, then your PHPStorm IDE makes request to the Apache Server Xdebug service and goes into wait mode. Next, you make request from your choice of browser to your site. Apache server receives this requests and from the session forwards it to the Xdebug service part of Apache suite. The Xdebug service forwards the request to PHPStorm IDE that steps through your break points. After done stepping thought the code, PHPStorm IDE sends response to Apache server Xdebug service which forwards it to Apache and Apache responds to your browser. A real good description on how it works is listed at http://xdebug.org/docs/remote

Part I – Configure Server

On server, we have to enable Xdebug service to be loaded with Apache and then configure Xdebug to be able communicate with PHPStorm IDE.

1) Install Xdebug

For Mac:

sudo port install php54-xdebug
port activate php54-xdebug

For Ubuntu:

sudo apt-get install php5-xdebug

For windows:
In the php.ini file add the following:

extension="/path/to/xdebug.so"
2. Configure Xdebug

To configure Xdebug add the following in php.ini file:

[Xdebug]
zend_extension="/usr/lib/php5/20090626/xdebug.so"
xdebug.remote_enable=1
xdebug.remote_port=9123
xdebug.remote_host=localhost//or IP of your remote server
//xdebug.remote_connect_back=1
xdebug.remote_log = /Users/mkapacs/xdebug/xdebug.log
xdebug.idekey="PHPSTORM"

The above configurations:

  • ‘zend_extension’ ensures to load xdebug module with apache
  • remote_port(i.e. ‘9123’) is the port used in communication between apache xDebug service and PHPStorm IDE.
  • remote_host(i.e. localhost) is the IP of machine with PHPStorm IDE.
  • ‘remote_connect_back’ if it set to 1, then you don’t need the remote_host because xDebugger determines PHPStorm machine’s location from the initial PHPStorm request header. This is preferred way when your PHPStorm IDE is on machine with dynamic IP, so you don’t have to reconfigure remote_host every time new IP is issued
  • idekey(i.e.’PHPSTORM’) can be any string. It is used for identifying unique session

Restart apache to load the new configurations.

3. Test Xdebug

In older versions and setups of Apache, we could test whatever xdebug was loaded by running:
apachectl -t -D DUMP_MODULES
If no xdebug configurations shows up in the list then your xdebug modules is not loaded

Today, i check by looking to see if there is logs written in locations as specified in xdebug.remote_log. If nothing is written than xdebug does not load…or a wrong assumption!!! The Xdebug Manual states:

xdebug.remote_enable
Type: boolean, Default value: 0
This switch controls whether Xdebug should try to contact a debug client which is listening on the host and port as set with the settings xdebug.remote_host and xdebug.remote_port. If a connection can not be established the script will just continue as if this setting was 0.

This makes me think that in past i didn’t see the logs written not because xdebug module was not loading but because Xdebug wasn’t able to establish connection with PHPStorm IDE, so it continued without xdebug and without writing logs. I don’t know, but when that happened I often forced xdebug module to load till the logs came up.

Force Xdebug to load with Apache

So, when there is no Xdebug logs written in location as specified at xdebug.remote_log, I was under impression that xdebug.so module is not loaded with Apache. In the following are steps to make to load xdebug module to see the Xdebug logs.
For Apache2.4 and Ubuntu14, it is as easy as installing package containing xdebug

sudo apt-get install libwxbase2.8-dbg

This will configure and load xdebug for Apache. Xdebug logs should be written

For older version of Apache, add the following line to apache configuration file httpd.conf:
LoadModule xdebug_module libexec/apache2/xdebug.so

!!!Make sure you copy the ‘xdebug.so’ into /path/to/apache2/modules directory

If you don’t know the path to xdebug.so, then To find location of your xdebug.so:

find / -name 'xdebug.co'

In our case it was in ‘/usr/lib/php/extensions’ dir

Part II – Configure PHPStorm IDE

Configure PHPStorm IDE involves configuring server info, so PHPStorm IDE knows where to make the initial request and then specify the port on which to wait from Server to forward user request. At last, set up the debugging.

1. Provide Server Info

a) Open PHPStorm settings(command + ‘,’) and navigate to PHP->Servers. Add server as following:


The drupal.local can be the IP address of the Apache Server. The port can be 80 as well so you don’t have to configure extra port(i.e. 9001) for Apache to listen

2. Provide Port

The port(i.e remote_port) you specified in the php.ini for communicating with PHPStorm IDE should go here. Navigate to PHP->Debug and then enter port under Xdebug section as following:

3. Set Up Debugging

At last, set up the debugging in the PHPStorm IDE by going to RUN->Edit Configurations and Add ‘PHP Remote Debug’ as following:

4. Test Drive

Lets test it by initiating the debugging in PHPStorm IDE:

a) Set break points

b) Start debugging as following:

To validate it started, look for message as follows:

c) Open browser

  • Turn on session via PHPStorm Marklets
  • –OR–

  • append ‘?XDEBUG_SESSION_START=PHPSTORM’ to your url

Both do the same thing add session id to the request
It should stop at your break point

Other

Enhance with xdebug.remote_autostart=1

If you include the following setting in php.ini:

[Xdebug]
...
xdebug.remote_autostart=1

Then XDebuger starts automatically without need for session id be sent from browser. As long there is PHPStorm Xdebuger listener waiting the apache server will forward all requests to it automatically. Turn off the PHPStorm Xdebuger listener and apache server will not forward request instead process them.

Start Debugger from Command Line

To start debugger from command line, everything is the same as above described except you have to give id to the command shell:

export XDEBUG_CONFIG="idekey=PHPSTORM"

Now, start the debugger at PHPSTORm and run script such as Drush command. Is should break at your breakpoints.

Troubleshooting

1. xdebug logs not showing up

This means xdebug module is not loading for apache(this is not always true, please see above). Besides php5-xdebug & it’s configurations in php.ini, the apache itself needs to load xdebug module. For ubuntu, you have to install a separate package – that includes xdebug module as following:

 sudo apt-get install libwxbase2.8-dbg
 service apache2 restart

Now, you see the logs of xdebug to help troubleshoot further

2. Address already in use: make_sock: could not bind to address [::]:80

This error happens when it is declared apache to listen on port 80 two times. Make sure you don’t have specified ‘Listen 80’ in two places httpd.conf and httpd-vhost.conf.

3. Scripts Run from command line always breaks at first line

Its the PHPStorm debugger configuration that you can change by unchecking the ‘Force break at [..]’ PHP->Debug PHPStorm configurations in xDebug section

Remote address found, connecting to [..]Time-out connecting to client. :-(…Could not connect to client

This happens when the Apache server Xdebug service is forwarding request to the PHPStorm IDE but it doesn’t have static IP or it’s behind firewall, NAT or some other configurations that prevents the communication between server to client. You can verify it by telneting from your Apache server machine to your PHPSTorm IDE machine:

>telnet
>O
(to) IP.COMES.HERE PORT

Solution is to open the port in the firewall or configure IP accordingly to make PHPStorm reachable from server. For Mac, make sure the ‘Remote Login’ is enabled at ‘Preferences/Sharing’

Remote file [..] is not mapped to any file path in project

You have to map the directory of your project locally to the one at the server. This can be done by going to File -> Settings->PHP->Server.

It doesn’t break at my breakpoints

One possible problem is that you haven’t mapped all of the files to the ones on the remote server. As result, Xdebug service doesn’t forward execution to your PHPSTorm IDE files. Go to ‘Settings’->’PHP’->’Server’ and configure dir mappings for your debug server

Useful Links