How To Run Multiple PHP Versions

The PHP programming language has gone long way since its beginnings. It made another big step by making it easy and convenient to run Apps in different versions of PHP at the same time. In this post, I go over the compiling of any PHP Version you wish, installation of the PHPFarm technology and the configuration of the Apache web server, all of which part of the task to run different PHP versions simultaneously on one machine with different apps

System: Ubuntu 14, Apache 2.4, LAMP

Installation of PHPFarm

Before compiling different versions of PHP, we install the PHPFarm. To do so

Step 1 – Install supporting necessary libraries:
udo apt-get install libxml2 libxml2-dev libssl-dev
sudo apt-get install libcurl4-openssl-dev pkg-config
sudo apt-get install libcurl4-gnutls-dev libjpeg-dev libpng12-dev libmysqlclient-dev
Step 2 – Install the PHPFarm

We grab the PHPFarm from PHPFarm Git repo

cd /home/tools
sudo git clone https://github.com/cweiske/phpfarm

This will clone PHPFarm at /home/tools/phpfarm directory

Compile PHP Versions

Once, we have the PHPFarm, lets compile some version of PHP.

Step 3 – Custom Configurations

First, we specify some custom configurations for new php version. Go to the ‘src’ directory of PHPFarm and create file of certain naming

cd /home/tools/phpfarm/src
sudo vim custom-options-5.3.29.sh

For some example of custom options, please see AmacGregor Git Repo
In our case, we use the following custom options by inserting them in the /home/tools/phpfarm/srccustom-options-5.3.29.sh:

#gcov='--enable-gcov'
configoptions="\
--enable-bcmath \
--enable-calendar \
--enable-exif \
--enable-ftp \
--enable-mbstring \
--enable-pcntl \
--enable-soap \
--enable-sockets \
--enable-sqlite-utf8 \
--enable-wddx \
--enable-zip \
--disable-debug \
--with-mysql \
--with-zlib \
--with-gettext \
--with-pdo-mysql \
--with-curl \
--with-openssl \
$gcov"
Step 4 – Compile PHP

Next, we compile new PHP version with our custom options:

cd /home/tools/phpfarm/src
sudo ./compile.sh 5.3.29

This will generate new version of PHP 5.3.29. Afterwards, test that it is working

cd /home/tools/phpfarm/inst/bin
./php-5.3.29 --version

Configure Virtual Host of Apache For Certain Version

Step 5 – Install supporting modules of PHPFarm for Apache

The FastCGI is technology to incorporate PHPFarm into Apache, so we are installing supporting libraries

sudo apt-get install libapache2-mod-fastcgi apache2-mpm-worker apache2-suexec
sudo a2enmod actions fastcgi suexec
sudo service apache2 restart

After restarting Apache service, the newly installed modules are included

Step 6 – Mapping CGI to PHPFarm

Next, we create web directory with configurations to map each PHP Version to the CGI service of Apache

sudo mkdir /var/www/cgi-bin
cd /var/www/cgi-bin
sudo vip php-cgi-5.3.29

Afterwards, we are adding mapping in form of Bash script named php-cgi-5.3.29 as follows

#!/bin/sh
PHPRC="/etc/php5/cgi/5.3.29/"
export PHPRC
PHP_FCGI_CHILDREN=3
export PHP_FCGI_CHILDREN
PHP_FCGI_MAX_REQUESTS=5000
export PHP_FCGI_MAX_REQUESTS
exec /home/tools/phpfarm/inst/bin/php-cgi-5.3.29

As you see the last line it maps to our PHP version. The variable “PHPRC” also specifies the location of php.ini file to use. By default, it uses the php.ini in the dir of /home/tools/phpfarm/inst/php-cgi-5.3.29/lib.php

Make our file of mapping is executable and chown rights to apache user („www-data“ by default):

sudo chmod +x /var/www/cgi-bin/php-cgi-5.3.29
sudo chown -R www-data:www-data /var/www/cgi-bin
Step 7 – Configure Virtual Host of Apache

At last, we configure particular application via Virtual Host Configurations at Apache to use the PHPFarm and our certain version of PHP by editing /etc/apache2/sites-enabled/some-domain.conf

<VirtualHost *:80>
	DocumentRoot /home/websites/min-view-drupal
	ServerName minnehaha.com
	ServerAlias www.minnehaha.com
	#LogLevel debug
	#FastCgiServer /home/websites/cgi-bin/php-cgi-5.3.29
    	FastCgiServer /home/websites/cgi-bin/php-cgi-5.4.27
	ScriptAlias /cgi-bin-php/ /home/websites/cgi-bin/
	<Directory "/home/websites/min-view-drupal">
		Options Indexes MultiViews FollowSymLinks
		AllowOverride All
		Require all granted
		AddHandler php-cgi .php
		#Action php-cgi /cgi-bin-php/php-cgi-5.3.29
		Action php-cgi /cgi-bin-php/php-cgi-5.4.27
		<FilesMatch "\.php$">
                	SetHandler php-cgi
            	</FilesMatch>
	</Directory>
	<Directory "/home/websites/cgi-bin">
   		Order allow,deny
   		Allow from all 
  		Require all granted
	</Directory>
 	Errorlog "${APACHE_LOG_DIR}/virtual-hosts/minnehaha/error.txt"
	CustomLog "${APACHE_LOG_DIR}/virtual-hosts/minnehaha/customLog.txt" common
</VirtualHost>

Here, all the highlighted lines involve configuring the PHPFArm. The line 7,8 includes the PHPFarm service. The lines 13-18, enables the certain version for our application. The lines 20-24 enables apache server to read our mapping configuration file which should be moved to general Apache configuration script since this directory may be shared between other Virtual hosts. In short if you have multiple Virtual Hosts then the following should be moved into general to declare only once:

...
FastCgiServer /home/websites/cgi-bin/php-cgi-5.4.27
ScriptAlias /cgi-bin-php/ /home/websites/cgi-bin/
...
<Directory "/home/websites/cgi-bin">
   		Order allow,deny
   		Allow from all 
  		Require all granted
	</Directory>
...

At last, restart Apache to load our newest changes:

sudoe services apache2 restart
Step 8 – Test it

To test it, include the phpinfo command into the index.php file of the root dir(i.e./home/websites/min-view-drupal) of the application

<?php phpinfo();?>

This should display php version along other setting once pointing the browser to the application of virtual host configured above(i.e. minnehaha.com)

Switching PHP Versions

After compiling another version of PHP as we did in Step 4, the switching between PHP Versions is as easy as configuring apache virtual host to read another mapping as done in Step 8(see commented out php version 5.4.27) to point to another php version done by creating new mapping as done in Step 7.

Troubleshooting

1. client denied by server configuration:

This error we got when the location(i.e. /home/websites/cgi-bin) of our mapping script of step 7 wasn’t configured to be loaded by Apache in virtual host missing the following:

<Directory "/home/websites/cgi-bin">
   		Order allow,deny
   		Allow from all 
  		Require all granted
	</Directory>

By including the above snippet and restarting Apache, the problem was solved

2. FastCgiServer: redefinition of a previously defined FastCGI server phpFarm

This error appeared when I declared the following in multiple Virtual host configurations:

FastCgiServer /home/websites/cgi-bin/php-cgi-5.4.27
ScriptAlias /cgi-bin-php/ /home/websites/cgi-bin/

Once, I moved it out into the general apache configuration while commenting out all other instances, it solved the problem

Sources

  • https://github.com/cweiske/phpfarm
  • http://www.jabommi.de/wiki/downgrade-php-5-5-to-5-3-ubuntu-14-with-multiple-php-versions/
  • https://wiki.apache.org/httpd/PHP-FPM
  • https://www.howtoforge.com/how-to-use-multiple-php-versions-php-fpm-and-fastcgi-with-ispconfig-3-ubuntu-12.04-lts
  • http://geekpad.ca/blog/post/compiling-php-and-php-fpm-part-2
  • http://www.sitepoint.com/run-multiple-versions-php-one-server/
  • https://gist.github.com/amacgregor/11041627#file-custom-options-sh

Sharing via Samba Instead VirtualBox Shares

The performance was so largely effected by using the VirtualBox shares on Apache server that I had to find a better solution to share directories between my VirtualBox Dev environment with Web server,database,etc and my machine Mac with the editor software. I turn to Samba Service as solution since it is one of the well known file sharing services available

Step 1 – Make Your Mac Sharing

By default, the file sharing is not enabled in the Mac, so the first step is to do just that

  1. On Mac, go to “System Preferences”->”Sharing” and check “File Sharing”
  2. And add the users and set permissions to “read & write” in the “Sharing” section
  3. On VBox Guest, add the ‘shares’ – directories that going to be shared in the “Sharing” section under Settings of your VirtualBox guest machine

Step 2 – Install Samba

While Mac comes with the Samba support, our guest Os is xUbuntu and it didn’t have Samba installed/enanbled which I did as following:

sudo apt-get install samba
sudo start smbd
sudo start nmbd

This installs and starts the Samba server on my guest Ubuntu

Mount Shares

Once the Samba installed and started, then we mount the share directory. The mounting was done from the Samba server( in our case guest OS) into the host – Mac OS as following:

To mount a share from guest to host:

sudo mount -t cifs //HOST-IP/SHARE-As-Declared-Step-1 /path/At/Guest/machine -o uid=1000,iocharset=utf8,rw,username="USER-HOST",password=USER-HOST-PASSWORD,nounix --verbose

See “man mount.cifs” for more info on options(i.e. “-o”)

To check what’s mounted so far on Ubuntu:

mount

This should display all mounted points including the shares

To mount automatically on reboot:

  1. Create Permanent IP for Host Machine

    Before configuring to auto mount the shares, it is important to have the permanent IP for the host. Otherwise, you will have to manually change the IP every time your Host WiFi IP changes. To have the permanent IP for the Host machine, you have to create “Host Only Network” as described in section 6.1 of post Virtualbox Gotchas

  2. Configure Auto Mount Shares

    To mount the shares at the start up, we edit /etc/rc.local at our Guest Ubuntu OS and added the following:

    mount.cifs //HOST-IP/SHARE-As-Declared-Step-1 /path/to/shared/dir -o uid=1000,iocharset=utf8,rw,username=HOST-USER,password=HOST-USER-PASSWORD,nounix
    

    So by adding the line in the rc.local file, it is executes the mount at the boot process. Do the same for each share you like to mount.

Sharing via AFP

Disclaimer: The AFP is very buggy and should be avoided.
Its possible to use AFP instead of SAMBA. I had to install afp modules on the guest Ubuntu:

sudo apt-get install afpfs-ng

Afterwards, we mounted the the directory as following:

sudo mount_afp afp://host_user:host_user_password@hostIp/share /srv/afp/share

Troubleshooting

Do Manual Mount

To better see the issue, mount the shared dir manually via “mount” command. It seems to display better error messages. Here is sample command taken out from rc.local with extra flags

sudo mount -t cifs //10.88.12.0/workspace /home/margots/workspace -o uid=1000,iocharset=utf8,rw,username=margots,password=PASSWORD,nounix
Check What is mounted?

First, look at what is mounted by issuing command:

mount

It will display a list of all the mounted directories

1. Check Logs

Log files location is declared smb.conf file in “/etc/samba/” dir. The default location “/var/log/samba”

2. See Samba Settings

There is nice way to see the Samba settings:

smbclient -L smb -N 
3. Check if Samba Running

Check if Samba is running:

ps -ef | grep smbd
4. Follow Kernel Logs Fix Mount Problem

Look at the logs to fix any mount problem:

tail -f /var/log/kern.log 

This will be output logs as they come in

5. mount_smbfs: mount error: /srv/samba/share: Unknown error: -1073741275

I haven’t found solution to this. It is happening when trying to mount share from host to the guest instead way around as described above. I guess my solution was just to mount the share from guest to host that accomplishes the task at hand.

6. Package smbfs is not available

According to this page https://wiki.ubuntu.com/MountWindowsSharesPermanently, it seems smbfs is deprecated and you should use cifs instead. try:

sudo apt-get install cifs-utils
7. mount error(22): Invalid argument.Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

This happened when trying to mount share into host mac

sudo mount -t cifs //192.168.1.4/srv /srv/samba/share -o uid=1000,iocharset=utf8,username="margots",password=38394815

To solve problem, we needed to disable the CIFS Unix Extensions for this mount that was done by adding option “nounix”

sudo mount -t cifs //192.168.1.4/srv /srv/samba/share -o uid=1000,iocharset=utf8,username=USERNAME,password=PASSWORD,nounix
8. Permission Denied For Apache When Writing On Samba Share

There was permission denied when Apache server tried to write in Samba Share. The solutions was to update apache user and group to one with Samba write permissions. To do so, I had to update Apache Environment variables – APACHE_RUN_USER, APACHE_RUN_GROUP in the /etc/apache2/envvars file as following:

export APACHE_RUN_USER=SAMBA_USER=samba_user
export APACHE_RUN_GROUP=share_dir_user_group

The Samba User with group of the share allowed Apache to make writes on Samba shares

9. mount error(13): Permission denied

I got this error when incorrectly enabled file sharing at host machine of Mac. You may need to have the 3 settings present in “System Preferences”->”Sharing”:

    Check “File Sharing”

  • Under “File Sharing”, click “Options” and add the user that will be used for remote sharing
  • Check “Remote Login”
  • Under “Remote Login”, add the user that will be used for remote sharing

This solved the error for me

10. /etc/rc.local Doesn’t Run at Startup

The /etc/rc.local contains our mount functions to run at startup, however. If it doesn’t run, try troubleshooting by running the following commands and looking for error messages for answers

/etc/rc2.d/S99rc.local start
/etc/init.d/rc.local start
/etc/rc.local

This is tracing through the sequence for scripts called before rc.local to find out where it fails

11. No Match Found in /etc/fstab

As I was tracing through the scripts to find out why the rc.local doesn’t seem to be executed at startup to mount my shared dir, i got the error “No Match xxx found in /etc/fstab” prusuant to 10 above. The solution was allow members of CDROM group to mount without prompting for root password by inserting the following line in /etc/sudoers:

%cdrom ALL = NOPASSWD:NOEXEC: /bin/mount, /bin/umount, /sbin/mount.cifs, /sbin/mount.nfs

Afterwards, i add “sudo mount” to mount command in rc.local script:

sudo mount.cifs ...
12. CIFS VFS: cifs_mount failed w/return code = -5 or mount error(5): Input/output error

These errors came up when the host, in my case Mac, didn’t have the shared directory enabled sharing according to Step 1 above

References

  • http://stackoverflow.com/questions/87442/virtual-network-interface-in-mac-os-x
  • http://www.dejadejoder.com/blog/posts/2010/07/21/mac-osx-10-6-snow-leopard-mount_smbfs-authentication-error-problem/
  • http://www.trickyways.com/2011/07/file-sharing-mac-os-x-lion-and-pc-windows-7/
  • http://www.samba.org/samba/docs/Samba24Hc13.pdf
  • http://www.clock.co.uk/blog/virtualbox-40-shared-folders-still-slow
  • https://forums.virtualbox.org/viewtopic.php?f=7&t=18044&start=0
  • https://forums.virtualbox.org/viewtopic.php?p=33519
  • https://help.ubuntu.com/10.04/serverguide/samba-fileserver.html
  • http://adrianmejia.com/blog/2011/07/12/how-to-set-up-samba-in-ubuntu-linux-and-access-it-in-mac-os-and-windows/
  • http://www.samba.org/samba/docs/man/manpages-3/mount.cifs.8.html

SendMail Email Server for Registration and Contact in Drupal

By default, Drupal uses php function mail() for sending email that instead uses the sendmail email server. So, in order for your email to work in Drupal you have to make sure the sendmail email server is installed and working on the server

Locate Sendmail

To see if sendmail is already installed and present:

which sendmail

This should return path to the sendmail executable. If it doesn’t then you have to install. If it does, then continue on the Verify Step

Install Sendmail

Here are steps to install and setup sendmail on linux machines:

  1. To install:
    apt-get install sendmail
    
  2. Ensure there is a route to localhost in the local route table
    vim /etc/hosts
    //And make sure the line looks like this:
    127.0.0.1 localhost localhost.localdomain yourhostnamehere
    
  3. To configure the sendmail:
    sendmailconfig
    
  4. Restart sendmail:
    services sendmail restart
    

Verify Sendmail

To verify, an email works with sendmail:

echo "my test email being send" | /usr/sbin/sendmail local@domain.com 

Here, the path to the sendmail executable is specified that you can get by running “which sendmail” command. This sends email to the local@domain.com by sendmail email server.

If you receive the email, then the sendmail email server is working. If you don’t receive the email then troubleshoot as described in next step

Troubleshoot Sendmail

To troubleshoot the sendmail, look at the logs located at /var/log/mail.log or /var/log/mail.err

Registration & Contact in Drupal

First, make sure the admin email is specified with valid email at Site Information(admin/config/system/site-information), otherwise, the email will not be send and you will receive error message such as ‘Connection timed out with example.com.” in mail.logs

To edit current settings on how the registration is handled, go to
admin/config/people/accounts

To configure contact form, go to admin/structure/contact

Troubleshooting

1. Unable to send e-mail drupal

This indicates that sendmail email server is not installed or configured correctly. You can install,configure as described in this post or turn it off for drupal sending email by adding the following line to the php.ini

sendmail_path = /bin/true
2. sendmail[6140]: My unqualified host name unknown; sleeping for retry …sm-msp-queue[6412]: unable to qualify my own domain name

Make sure that the domain name as specified in the error message is in the local route table(in file /etc/hosts) routing to 127.0.0.1.

Connection timed out with example.com.

Ensure the admin email address is configured with a valid email account in Site Information section admin/config/people/accounts

To enable sendmail to use STARTTLS
Resources
  • https://www.drupal.org/node/404190
  • http://holarails.wordpress.com/2013/11/17/configure-sendmail-in-ubuntu-12-04-and-make-it-fast/

Setup Dynamic Virtual Hosts To Automize Drupal With WorkFlow

A tool WorkFlow has a new functionality “construct-container” that sets up a new container, so you can spin(i.e. drush spin-it) new Drupal instances automatically. Besides creating database, directories in file system, the construct container task also sets up and configures virtual host for the new Drupal instance. In this post, we cover the approach that was taken to provide multiple virtual hosts solution for WorkFlow tool users.

The solution utilizes Apache module virtual_alias. To be able use the Wildcard domain names on the local machine, the DNS server – dnsmasq is used

Install virtual_alias Module

First, check to see if virtual_alias module is already enable and loaded with apache:

apachectl -M | grep "virtual_alias"

If this doesn’t return anything, then the virtual_alias module is not enabled. Lets do so:

sudo a2enmod virtual_alias
sude service apache2 restart

This should enable the virtual_alias module and load in apache. Go ahead and verify with “apachectl -M” to see if its loading

Set Up Universal Virtual Host Alias

In this example, we configure one universal virtual host for DEV Environment(for WorkFlow Tool there is additional for TEST environment) that is going to service all of the new Drupal instances. To do so, lets add new file – dev-virtual.conf in the /path/to/apache/sites-available(by default /etc/apache2/sites-available) as following:

<VirtualHost *:80>
  ServerAlias *.dev
  VirtualDocumentRoot /path/to/dev/%1
  ServerAlias localhost *.dev
  <Directory "/path/to/dev">
    Options +SymLinksIfOwnerMatch
    AllowOverride All
    Require all granted
  </Directory>
</VirtualHost>

Afterwards, lets enable the universal virtual host

sudo service a2ensite dev-virtual 
sudo service apache2 reload

Here, we specify root dir with ‘%1’ at the end to service each Drupal instance at its own directory. The ‘%1’ refers to the first part of domain name separated by dot. For example, metronic.dev will result the ‘%1’ refer to ‘metronic'(for more info see section ‘Directory Name Interpolation’).

As you see from the universal virtual host configurations above, the domain name we have set up to be *.dev, so next we configure all request *.dev point to the right machine.

Configure DNS or Set Up Own Local DNS

For public server, you will point all “*.dev” requests to the public server. In that case, all you need is to add the new DNS A entry in the DNS service as following:

*.dev   IN A    177.77.177.77

For local machines, we cannot use a wildcard in /etc/hosts, so we have to set up local DNS. Let’s use the light weight DNS solution – dnsmasq

sudo apt-get install dnsmasq

Once installed, we configure by editing /etc/dnsmasq.conf and adding line:

address=/dev/127.0.0.1

Here, we set *.dev requests point to 127.0.0.1 which is the local machine
Afterwards, restart the dnsmasq:

sudo service dnsmasq restart

Now, all request with the wild card (i.e. “*.dev”) will be pointed to local machine where the universal virtual host should service them from the directory specified.

Troubleshooting

1. client denied by server configuration: /path/to/dev/some

This error was caused by adding %1 at the end of path for the Directory configurations. Here is correct way:


VirtualDocumentRoot /path/to/dev/%1
...
//<Directory "/path/to/dev/%1">
<Directory "/path/to/dev">
...
                Order allow,deny
		Allow from all
...
</Directory>

By removing ‘%1’, the actual directory is found and no more errors. In short, when you have VirtualDocumentRoot, then the Directory root is the path to the virtual aliases instead the full path as it is with static virtual hosts configurations

References

  • http://brunodbo.ca/blog/2013/04/26/setting-up-wildcard-apache-virtual-host-wildcard-dns
  • http://float64.uk/blog/2014/08/22/dynamic-development-area-apache-php-fpm/
  • http://httpd.apache.org/docs/2.4/mod/mod_vhost_alias.html
  • https://httpd.apache.org/docs/2.4/rewrite/vhosts.html
  • https://httpd.apache.org/docs/2.4/vhosts/mass.html
  • https://code.google.com/p/mod-myvhost/wiki/Install
  • http://eosrei.net/articles/2012/08/create-dynamic-virtual-hosts-apache-http-vhostalias

VirtualBox Gotchas

This is ongoing post listing different challenges and how those can be overcome when utilizing VirtualBox for your development needs

1. VirtualBox host only supports 32bit guest OS

This was a problem on Windows 8 Intel processor machine that didn’t have vt-x enabled. The vt-x is feature of Intel processor that allows the virtualization of some kind. Once enabled, the VirtualBox host can load also 64bit Guest OS. To enable, boot Vindows 8 in BIOS and enable the vt-x feature

2. Freezes with “Building initial module for 3.x”

Make sure the guest OS you are installing is for the right processor.

3. Cannot Change Display Resolution

Install the VirtualBox additions at the Guest

sudo apt-get install virtualbox-guest-dkms

Afterwards, reboot and the display resolution works as you like

4. Cannot Copy & Paste from VirtualBox Host to Client and from Client to Host

Copy/paste is not enabled by default. You have to enable in host ‘Settings’ under ‘General’->’Advanced’ and check ‘Shared Clipboard’ and ‘Drag & Drop’. Note, that changes can only made when client OS is turned off.

5. Sharing folders Between VirtualBox Host and Guest

There are two ways to share directories between Host and Guest – via VirtualBox Shares or via independent Samba Server. As we learn later, the VirtualBox Shares solutions is very slow for Apache to service web applicaitons such as Drupal. So we strongly recommend Samba solution for sharing directories, however. Here are setup for both solutions:

  • Share Via Samba

    Please, see post “Sharing Vis Samba Instead VirtualBox Shares” for sharing directories via Samba

  • Share Via VirtualBox Shares

    There are two steps:

    1. In the host settings under ‘shared folder’ directory, add the shared directory and give a name ‘some-host-share-name’. Make sure you check “Permanent”, so its available on each reboot
    2. Next, physically mount the shared directory

      sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) some-host-share-name ~/guest-dir
      

    Make sure the ‘~/guest-dir’ exists on the guest OS. Otherwise, you will get error “/sbin/mount.vboxsf: mounting failed with the error: No such file or directory” or no any error and it will just not mount

To mount it automatically on reboot, add the entry in the /etc/rc.local as following:

  • For Samba solution:
    mount.cifs //HOST-IP/SHARE-NAME /path/to/shared/dir -o uid=1000,iocharset=utf8,rw,username=HOST-USER,password=HOST-USER-PASSWORD,nounix
    
  • For VirtualBox shares solution:
    mount.vboxsf -w some-host-share-name /home/<username>/guest-dir
    

This will mount the the shared drive on each reboot automatically.

5A. Share Between Guest OS of Windows XP

First ensure, the Guest addition is installed for the Guest Windows XP OS by grabbing appropriate ISO of your VirtualBox solution at the http://download.virtualbox.org/virtualbox/ dir.
Next, add the ISO to the Optical CD/DVD drive of the Guest OS.
At last, run the executable/installer in the ISO rood directory. This will install Guest additions necessary, so that at the next boot it creates the shared dir in the Guest Windows OS automatically in the “Network Drives”

6. Easily access your NAT configured virtual machine

You probably need host access servers on the guest OS, while your guest OS, probably, need access internet at the same time. Here is NAT + Host ONLY solution to address both needs configurable as following.

  1. Create new Network Go to VirtualBox VM->Preferences->Network->Host Only Networks and add new one with the following settings:
    IP:10.88.12.0
    Mask:255.0.0.0
    

    Keep DHCP server disable, so you can configure static IP much more easier than looking up the new IP every time it changes

  2. Add Network Interfaces Add the network interfaces by going to Guest Settings->Network->Adapter
    • Adapter 1: NAT
    • Adapter 2: Host Only Adapter with “name” of network you configured in step 1 above
  3. Configure Two Guest OS Interfaces. In your guest OS, edit /etc/network/interfaces for Deb or Ubuntu as following:
    # interfaces(5) file used by ifup(8) and ifdown(8)
    auto lo
    iface lo inet loopback
    
    #the primary network interface
    auto eth0
    iface eth0 inet dhcp
    
    #The secondary network interface
    auto eth1
    iface eth1 inet static
    address 10.88.12.4
    netmask 255.0.0.0
    

    This will configure two network interfaces one for NAT and another for Host Only network

Afterwards, reboot guest OS and you should access guest OS at 10.88.12.4 from host, access host at 10.88.12.0 from guest all which while your guest OS have internet connection via the NAT network

7. Slow Performance for Guest xUbuntu

The following settings improved Guest performance:

  • Check “Storage” -> “Use Host I/O Cache”
  • Check “Display” -> “Enable 3D Acceleration”
  • Install guest addtions, utilities and drivers:
    sudo apt-get install virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11
    
  • Add extra CPUs “System”->”Processor”
  • Check “Enable PAE/NX” under “System”->”Processor”
  • Add more RAM to Video Card “Display” -> “Video” -> “Video Ram”
  • Install VB extension pack on the host machine:
    1. Download the extension pack from VirtualBox Download page
    2. Run the following command:
      sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.3.18-96516.vbox-extpack 
      sudo VBoxManage list extpacks
      
  • look into logs to see if it shows any issues that may affect performance

Note: After analyzing performance for my Drupal site running on VirtualBox(see more details at post looking-at-drupal-performance-with-xprof-and-xhgui), it shows that the top 6 slowest functions are “file_scan_directory”. It made me conclude that reading/writing on disk is the slowest. I upgraded HD to SSD HD, but while there was performance improvement it still was noticeable. This applies to the shared directories between Guest and Host. If not shared then ‘file_scan_directory’ is not any more the bottleneck and Drupal site runs really fast.

UPDATE: The best improvement by far was moving away from sharing directories via VirtualBox Shares solution to the sharing via Samba Server as described in post Sharing Via Samba Instead VirtualBox Shares

8. Increase Size for VB Guest Hard Drive

After about 3 weeks running VB, it run out of space on Hard Drive that was set of fixed size. Here are steps to increase Hard Drive space

  1. [ONLY IF FIXED SIZE] Re-clone HD .VDI to be able use ModifyHD
    Resizing HD involves runing command ‘modifyhd’, however. This only works for HD that were not fixed, so before we are able to use “modifyhd”, we re-clone our HD .vdi file:

    sudo VBoxManage clonehd "/full/path/to/disk1.vdi" ClonedFixedHD.vdi --format VDI --variant Standard
    

    This will prepare our Hard Drive .vdi file for modifyhd command to use in next step

  2. Resize the HD Drive .vdi
    sudo VBoxManage modifyhd "/full/path/to/ClonedDevelLamp.vdi" --resize 20000
    

    This will increase the size of the hard drive to 20GB

  3. Update Partition Size for VB Guest
    Once the size is changed for hard drive, we need to update also the VB Guest partition that mounts the new sized Hard drive. It is accomplish by booting from GParted

    1. Download CD of GParted
    2. Mount to CD drive of the VB Guest
    3. Boot into your VB Guest
    4. Resize partition.Here is good article
  4. Remove Old Hard Drive & Attache New Resized HD
    Go to Setting->Storage and remove the old HD .vdi. Then add your new resized HD. Afterwards, reboot the machine

Check Logs

In the following are directories for logs per OS:

  • On Windows, this is %HOMEDRIVE%%HOMEPATH%\\.VirtualBox\\Machines\\\\Logs; typically some- thing like C:\\Documents and Settings\\Username\\.VirtualBox\\Machines\\\\Logs\\vbox.log
  • On Mac OS X, this is $HOME/Library/VirtualBox/Machines//Logs
  • On Unix-like systems (Linux, Solaris), this is $HOME/.VirtualBox/Machines//Logs

Troubleshooting

1. No extension pack by the name ‘Oracle VM VirtualBox Extension Pack’ was found

Go to VirtualBox Download page and download extension pack. Afterwards, install it as following:

sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.3.18-96516.vbox-extpack 

To verify:

sudo VBoxManage list extpacks

This should display descriptions of the extension pack or 0 for no extension pack present

2. VBoxManage: error: Cannot register the hard disk ‘/path/to/disk1.vdi’ {8e28be07-2543-4967-a83f-8a6a07d18c64} because a hard disk ‘path/to/disk1.vdi’ with UUID {..} already exists

We got this after cloning and then trying to resize hard drive. The solution was to reset UUID:

sudo VBoxManage internalcommands sethduuid "/full/path/to/disk1.vdi"

Make sure you provide full path to the HD file .vdi, otherwise, you will get another error at least on Mac OS.

3. Progress state: VBOX_E_NOT_SUPPORTED

This error came up when trying to resize the HD file .vdi, because the HD was configured fixed and command “modifyhd” doesn’t work with fixed HD .vdi. The solution was convert the HD .vdi into one that is not fixed size by re-cloning as following:

sudo VBoxManage clonehd "/full/path/to/disk1.vdi" ClonedNewVDIfile --format VDI --variant Standard

This will create new .vdi size ready to work with “modifyhd”

4. My .VDI is missing

I was looking at $HOME/VirtualBox VM/SOME_VM_BOX_DIR just to find there is no .vdi file, however, the particular Box was loading okey. I was unaware that the HD(i.e. .vdi file) can be outside the Box directory. See “Settings”->”Storage” to locate where is the Hard Drive .vdi file located

VBoxManage: error: Could not find file for the medium ‘/full/path/to/disk1.vdi’ (VERR_FILE_NOT_FOUND)/h6>
This error is because the path specified is not correct or it is not full path that is required for running VBoxManage commands

5. “the parameter is incorrect” Or “the network name cannot be found” OR “is not accessible”

This error happened when I was sharing files of mac host with Window XP as guest OS and I didn’t have the Guest Additions installed. To install, grab appropriate ISO of your VirtualBox solution at the http://download.virtualbox.org/virtualbox/ dir. Next, add the ISO to the Optical CD/DVD drive of the Guest OS. At last, run the executable/installer in the ISO rood directory. This will install Guest additions necessary, so that at the next boot it creates the shared dir in the Guest Windows OS automatically in the “Network Drives” dir

References

https://forums.virtualbox.org/viewtopic.php?f=24&t=50661
http://drwho.virtadpt.net/archive/2014/02/04/problems-cloning-virtualbox-disk-images

Resizing vmdk harddisks for VirtualBox

Setting Up and Managing Multiple SSH Accounts for Drush on Mac/Linux

It may be the case, you have multiple ssh connections some for github accounts and some for one or more virtual boxes for Drupal sites. In this post, we cover how to generate, configure and manage multiple SSH accounts for drush. In addition, we go over how to overcome the error ‘ssh-copy-id: command not found ‘ that happens for Mac machines when trying to setup ssh keys via drush command ‘pushkey’

Generating Public/Private SSH Key Pair

For each different ssh account, you would generate ssh public/private key pair as following:

ssh-keygen -t rsa -C "user@hostname.com"

By default, this will store the generated key pairs(two files one with .pub extensions) in the directory you run the command from. We moved it into $HOME/.ssh directory and we will be referring to this directory in this post

Manage SSH Keys

You can manage multiple SSH keys by creating host aliases for each ssh key pair. The host aliases are defined in the $HOME/.ssh/config file and all they do is reference the keys per host alias as following

Host virtual-box-main
        HostName hostdomain.com
        User user-name
        IdentityFile ~/.ssh/id_rsa_virtual_box

Host another-host-alias
        HostName some-remote-box.com
        User ssh-user
        IdentityFile ~/.ssh/private_key_gen_above

Here the host alias is ‘virtual-box-main’. The ‘hostname’ is actual domain name to the server you like to ssh. The ‘user’ is the actual ssh user account that will be used to ssh into the server. At last, the ssh key pair is referenced via ‘IdentityFile’ that was generated in the above step.

So, you can repeat the above steps to generate ssh key pairs for each ssh remote account. Then configure host alias by adding another entry as specified above for each different ssh login you need.

Setup Drush

Once you have the ssh host alias setup, then to configure drush via the alias(one location $HOME/.drush/aliases.drushrc.php) as following:

$aliases['ds.prod'] = array(
	'remote-host' => 'virtual-box-main',
	'remote-user' => 'root',
	'root' => '/home/websites/site',
	'uri' => 'http://site.com',
    );

Here the host alias configure in ssh is specified ‘remote-host’, so when drush makes the ssh connection it will look it up the alias to identify ssh key to use for login

If you try to connect with the current setup:

drush @ds.prod st

You will get following error:
Permission denied (publickey,password).

This is because the remote machine needs to have the ssh public key in order to be able authenticate. Lets push the key:

drush pushkey @ds.prod

This function copies the ssh key into the server for ssh authentication.

For Mac, this will result into the following error because Mac doesn’t have the function ‘ssh-copy-id’
ssh-copy-id: command not found
So, solution is to copy ssh key manually

Copy Public SSh Key Manually

To copy the public ssh key, run the following:

cat id_rsa_virtual_box.pub | ssh user@virtualbox.com 'cat >> .ssh/authorized_keys' 

This pipes the ssh key and then logs into the virtual server and pasts the key into authorized_keys file. You may be missing the .ssh folder on the remote server. In that case, you make it create one as following:

cat id_rsa_virtual_box.pub | ssh user@virtualbox.com 'umask 077; mkdir -p .ssh ; cat >> .ssh/authorized_keys'

Now, running:

drush @some.alias st

It should be displaying the status of the site on remote server

Troubleshooting

1. Test SSH Authentication

To test any of your SSH Aliases setups:

ssh virtual-box-main

Here, we test the alias ‘virtual-box-main’. This will prompt for password if logging in for the first time. All other times it logs you in without password
Note: for ssh services that has turned off interactive mode such as Git, you use -T flag:

ssh -T git-ssh-alias

This will confirm if your ssh is setup correctly or not

2. Permission denied (publickey,password).

You will get this message if you haven’t set up the ssh as described in this post

3. ‘pushkey’ could not be found drush

This is because the push key function is addition and needs to be downloaded as following:

drush dl drush_extras

This will download and install drush_extras module that includes “pushkey” command

4. “sudo: no tty present and no askpass program specified”

This is issue comes up when you open ssh without tty session. It can be solved in 2 ways depending on how much access you have on remote instance or whatever the remote instance has the capability of NOPASSWORD

  1. Turn on NOPASSWORD on remote instance as described here:
    http://askubuntu.com/questions/192050/how-to-run-sudo-command-with-no-password/443071#443071
  2. Configure Drush to pass in credentials at time of ssh-ing. Its being done by editing .drush/drushrc.php and adding following line:
    $options['ssh-options'] = '-o PasswordAuthentication=no -i $HOME/.ssh/id_rsa/lamp_vbox';
    

    Here the lamp_vbox is the private key generated by ssh-keygen

5. WARNING: UNPROTECTED PRIVATE KEY FILE!

The permissions need to be reset:

sudo chmod 600 /path/.ssh/id_rsa/lamp_vbox
sudo chmod 600 /path/.ssh/id_rsa/lamp_vbox.pub
sudo chmod 644 /path/.ssh/known_hosts
sudo chmod 755 /path/.ssh
References

http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201210.mbox
http://blogs.uoregon.edu/developments/2012/06/21/bash-function-ssh-copy-id-for-mac/
http://www.drush.org/sites/default/files/attachments/DGD7-Drush.pdf
http://stackoverflow.com/questions/3844393/what-to-do-about-pty-allocation-request-failed-on-channel-0

WYSIWYG with CKEditor in Drupal

It appears that CKEditor is the most advance and aesthetic WYSIWYG editor available for Drupal. As of February 2014, we run into issues installing the CKEditor as regular way WYSIWYG editors are installed in Drupal. In Part I, we summarize it. In the second part, we cover alternative integration of CKEditor as a stand alone editor

Install CKEditor Part of DRUPAL WYSIWYG

1. Enable WYSIWYG [wysiwyg] editors

To enable WYSIWYG editors in Drupal, install and enable wysiwyg module
With drush:

drush dl wysiwyg
drush en wysiwyg
2. Install Editor of your choice

Once module wysiwyg installed, then got to ‘admin/config/content/wysiwyg’ to see all the available wysiwyg editors with instructions how to install and enable any from the list.

We follow instruction and download the CKEditor library(in our case 4.3.2) from their site and extract it into the ‘site/all/libraries’ so that ‘ckeditor/ckeditor.js’ is as specified in the WYSIWYG configuration page.

After doing all this, the CKEDitor is not working and we have error:

The version of CKEditor could not be detected.

An alternative approach to install CKEditor as a separate module not part of WYSIWYG setup

Install CKEditor As Separate Module

Before continuing with this approach, make sure to disabale module – wysiwyg. The first step is as described above step 2 – Install Editor of Your Choice

A) Install the CKEditor Library

Download and past the CKEditor library into folder sites/all/libraries, so that ckeditor.js is in ‘sites/all/libraries/ckeditor/ckeditor.js’

B) Installing CKEditor Module

With Drush:

drush dl ckeditor
drush en ckeditor
C) Configure to use CKEditor for input fields

Go to admin/config/content/ckeditor/edit/Advanced or admin/config/content/ckeditor/edit/Full and ensure the checkbox ‘Text Formats’ for “Filtered HTML” or “Full HTML” is selected under ‘Basic Setup’ section.

For more custom configuration, go to ‘admin/config/content/ckeditor/’

D) Test Drive

Just open a node to ensure the editor is coming up for the input field with ‘full’ or ‘filter’ text formats

Other Stuff

Configure CKEditor Outside Drupal

If limited to what Drupal module – CKEditor configurations(admin/config/content/ckeditor) provides then you can configure CKEditor described in the CKEditor documentation native of CKEditor by updating the modules/ckeditor/ckeditor.config.js

For example:
To disable the advance auto filtering so that when CKEditor is enabled, it does not add and change extra html of the content:

CKEDITOR.editorConfig = function(config) {
    config.allowedContent = true;
...
}

This will disable the auto advance filtering. There other configurations to make like changing the color of the editor:

CKEDITOR.editorConfig = function(config) {
    config.uiColor = '#AADC6E';
...
}
Configure Allowed HTML Tags

To configure which html tags are allowed, consider to use wysiwyg_filter

sudo drush dl wysiwyg_filter
drush en wysiwyg_filter

Also uncheck ‘Limit allowed HTML tags’ in the Home » Administration » Configuration » Content authoring » Text formats

In addition, the CKEditor itself comes preconfigured to remove empty tags such span, em by default. Please see post Taking CKeditor Apart on how to avoid removing empty tags

Insert Photos in Content via IMCE Module

In order to insert photos in content via WYSIWYG, there is a Drupal Module called IMCE. To install and enable:

sudo drush dl imce
sudo drush en imce

This will install the necessary module to be able upload and insert photos via WYSIWYG editor in our case CKEditor

A) Make part of Image Dialog
The IMCE can be integrated in the CKEditor Image dialog. One of the benefits is that the functionality is add to current UI control(image button) so there is no need for additional buttons in the CKEditor Toolbar. Go to admin/config/content/ckeditor/edit/Full(or Advance) and select ‘IMCE’ for ‘File browser type (Image dialog window)’ in the section ‘FILE BROWSER SETTINGS’.

B)Make IMCE As Separate Control
The IMCE can be used as separate with its own button to upload files. To do that, check ‘Plugin for inserting files from IMCE without image dialog’ in the admin/config/content/ckeditor/edit/Full under ‘Plugins’ section. This will enable a separate IMCE button to be added to the CKEditor Toolbar.

Enhance Images to Resize, Align and Wrap text

With the newest version of CKEditor lib (4.4.0 and above) there is no more need for Drupal ckeditor_image module to have image enhancements such us resizing, aligning and wrapping text around. It comes out of box. You may want to combine the CKEditor Full source with plugin ‘Enhanced Image‘(image2) to have additional UI functionality for resizing by simply dragging a mouse.

Insert Photos in Content via Media & CKEditor_Media Module

Another module to insert images into content is Media. The benefit of using the Media module is that with ckeditor_media you are able to use the Media browser UI instead the CKEditor.
To install:

drush dl media
drush dl ckeditor_media
drush en media, ckeditor_media

Afterwards, configure the CKeditor toolbar to have the button for media browser by going to admin/config/content/ckeditor/edit/Full and drag/drop the media button.

Enabling CKEditor For Textarea

If you have custom form with textarea, then to enable the CKEditor, add the following:

          'some_textarea' => array(
            '#type' => 'text_format',
            '#format' => 'full_html',
            '#rows' => '5',
            '#prefix' => t('Content'),
            '#default_value' => variable_get('some_textarea', ''),
        ),

By adding ‘text_format’ and ‘full_html’ to the textarea element,it will trigger the CKEditor to load

Adding Theme Styles

http://docs.ckeditor.com/#!/guide/dev_styles

How to Externalized CKeditor settings?

Thomas asked the question how to externalized CKeditor settings(ckeditor/ckeditor.config.js)
Here is list of alternative potential solutions:

      1. It is possible to force ckeditor to read config file from theme path instead of module path. Its a setting under ckeditor / advanced – and then you can just copy the custom config file into the theme folder.
      2. [NEED TO BE TESTED] Use hook_js_alter to replace default ckeditor with your custom one. See post How To Upgrade JQuery UI In Drupal for the same approach just instead altering jQuery lib you would alter ckeditor.config.js location.
      3. [NEED TO BE TESTED] If overlays are not used for admin content then you can have custom Js floated with your custom settings set on the CKEDITOR variable. See post Taking CKEditor a Apart and look how CKEditor is started from console. This should give you idea on how to call CKEditor API on CKEDITOR global js variable with your custom settings
      Note:I don’t remember anymore, was it because overlays are in separate iframe or because CKEditor Drupal module removes CKEDITOR global var on load that makes it unavailable for calling CKEditor API directly. That being said, the second approach may not work

The first approach is more for UI development, the second is more for Drupal developers and third is more JS developers that like to have it done in js instead within Drupal framework

Troubleshooting

1. Many of the CKEditor Toolbar buttons doesn’t show

Make sure you have installed the FULL CKEditor library and not the other two. If a particular button don’t show up then the plugin or the code for that functionality is not present

2. The text doesn’t show up in CKEditor

This may be because the text color of theme is white and with CKEditor background color being white as well, the text appears missing. By changing “Editor CSS” in admin/config/content/ckeditor/edit/TEXT_FORMAT to “CKEditor Default” will use the CKEditor Theme with text font of color black

3. IMCE doesn’t work

The IMCE needs to be configured to work for all users. Go to admin/config/media/imce and create profile. Afterwards, assign this profile to user role

The Upload, Insert File, Resize,etc Buttons Doesn’t Show Up In IMCE Browser

Make sure the permissions for the public dir is set correctly

Useful links

  • CKeditor Source
  • https://drupal.org/project/ckeditor
  • http://docs.ckeditor.com/#!/guide/dev_configuration
  • http://docs.ckeditor.com/#!/api/CKEDITOR.config
  • http://docs.cksource.com/CKEditor_for_Drupal/Open_Source/Drupal_7/Tricks
  • http://drupal7tutorials.com/tutorial/3-getting-your-ckeditor-setup-drupal-7
  • http://docs.cksource.com/CKEditor_for_Drupal
  • https://drupal.org/node/1827434
  • http://ckeditor.com/forums/Support/How-to-build-and-implement-my-own-data-processor-and-disable-the-build-in-processor

Installing Apache on Mac/Linux

Mac comes with apache pre-installed, however. You may wish to install your own apache instance
http://margotskapacs.com/2013/09/installing-apache-on-mac/
Before start, make sure the default apache server is not running(see troubleshooting section below). To turn off, go to ‘System Preferences->Sharing’ and uncheck ‘Web Sharing’

Step 1. Installing Apache via macport

port selfupdate
sudo port install apache2

This installes Apache in /opt/local/apache2/ directory

Step 2. Configure Apache

Apache comes with sample configurations, lets copy it

sudo cp /opt/local/apache2/conf/httpd.conf.sample /opt/local/apache2/conf/httpd.conf

In the apache configuration file ../conf/httpd.conf change the following lines to specify root folder from where your websites is going to be serviced

...
<Directory "/opt/local/apache2/htdocs">
...
DocumentRoot "/opt/local/apache2/htdocs"

The path /opt[..]/htdocs’ can be anything you wish but in both places it have to point to the same folder

Next, add ‘index.php’ so that Apache grabs index.php file if present

<IfModule dir_module>
    DirectoryIndex index.html index.php
</IfModule>

Afterwards, configure Apache interpret all files ending with .php/.phps by adding these lines:

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

Finally, to enable user directories uncomment:

Include conf/extra/httpd-userdir.conf

Step 3. Activate PHP module

After installing and compiling PHP, there is PHP package that Apache needs to load, so here how to activate it

cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n "php5" libphp5.so
[activating module 'php5' in /opt/local/apache2/conf/httpd.conf] 

Step 4. Start Appache

To start and stop Apache:

sudo /opt/local/apache2/bin/apachectl start
sudo /opt/local/apache2/bin/apachectl stop

After starting, point your browser to ‘locahost’ and it should work. To test php, add the following file to the folder you specified ‘DocumentRoot’ in above:

<?php
phpinfo();
?>

Afterwards point to this file ‘localhost/fileName.php’ and it should display php configurations

Enable Virtual Hosting

To Enable virtual hosting, uncomment the following line in apache configuration file conf/httpd.conf

Include conf/extra/httpd-vhosts.conf

Afterwards, configure you virtual host at conf/extra/httpd-vhosts.conf. Here is an example to get you started:

NameVirtualHost *:80
...
<VirtualHost *:80>
    DocumentRoot "/path/to/your/site"
    ServerName domainName.com
    <Directory /path/to/your/site>
        Options FollowSymLinks
        AllowOverride All
        Allow from all
        Order Deny,Allow
    </Directory>
    ErrorLog "/private/var/log/apache2/domainName.com-error_log"
    CustomLog "/private/var/log/apache2/domainName.com-access_log" common
</VirtualHost>

Troubleshooting

1. To see Apache looaded configs:
httpd -V
//or for Debian servers
apache2ctl -V
2. To see if Apache deamon is running
ps -ef | grep httpd
3./.htaccess: Options not allowed here

This happen after specifying alias in the virtual host but never declaring directive:

     <Directory /path/to/the/alias>
        Options FollowSymLinks
        AllowOverride All
        Order Deny,Allow
        Allow from all
    </Directory>
    Alias /path /path/to/the/alias

So, by including the “<Directory..” gives the permissions for the .htaccess manage the ‘Options’ configuration solving the problem you see in the log

4. Virtual Hosting is Not Working

Make sure you have added "NameVirtualHost *:80" in one of the configurations that is being loaded such apache2.conf, some-v-host.conf,etc

5. client denied by server configuration (PHP FPM)

Make user to have Require all granted part of the Virtual Host configuration Directory section as following:

<Directory "/path/to/serving/dir">
		Options Indexes MultiViews FollowSymLinks
		AllowOverride All
		Order allow,deny
		Require all granted
	</Directory>
6. Forms Will not submit or “AH00128: File does not exist” in logs

The forms could be submitted and the apache logs were showing “AH00128: File does not exist[..]” for Apache 2.4. There issue is that mod_rewrite is not enabled by default. You can check if the mod_rewrite is enabled and loading by looking at phpinfo(). To enable mod_rewrite:

sudo a2enmod rewrite
sudo service apache2 restart

This will load mod_rewrite making the forms to Post for Drupal

caught SIGTERM, shutting down

This error was coming up for Apache 2.4 on Ubuntu after logging into Drupal. I am not sure which exactly fix it but did the following:

  • turn off limit for MaxKeepAliveRequests in the apache2.conf configuration file
  • Increase memory in php.ini by changing memory_limit
  • Increased max_input_time in php.ini
  • Increased max_execution_time in php.ini

In addition, it appears that apache restarts with the above error every time there is php syntax error.

Blank Screen

By default Apache will display blank screen when there is php error. To turn on errors to be displayed in browser instead of blank screen, edit php.ini as following:

display_errors=On
html_errors=On
error_reporting=E_ALL | E_STRICT 
display_startup_errors=On

This will enable error reporting in the browser. In case it doesn’t, then at your application level there must be ErrorHandler defined that hides all the errors. See more info here http://stackoverflow.com/questions/1475297/phps-white-screen-of-death

Troubleshooting Tip

When there is error, turn on max logging by changing ErrorLevel to trace8 in virtual host settings or Apache global settings:

...
 LogLevel trace8
 ErrorLog ${APACHE_LOG_DIR}/dev-virtual/error.log
..

Afterwards, tail the logs specified in ErrorLog from command line to follow real time Apache execution:

tail -f APACHE_LOG_DIR/dev-virtual/error.log

At last, make the requests and follow the logs in command line:

Useful Links

  • http://www.thegeekstuff.com/2008/08/9-tips-to-use-apachectl-and-httpd-like-a-power-user/
  • http://2tbsp.com/content/install_apache_2_and_php_5_macports

Interacting with Redis Cheat Sheet

Redis is elegant solution to provide fast storage while its supported by Node.js. However, with its different data structures such as List, Set and Hashes, the API of Redis takes time to memorize. Here is quick cheat sheet to help

Installation

Step 1 – Download & Compile

To install Redis:

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

Here, we installed Stable version of Redis, but if you need some older version then see Google Code Redis repo for version of your need

Step 2 – Test It

Try if your build works correctly by typing “make test”

Step 3 – Configure Redis

After the compilation the src directory inside the Redis distribution is populated with the different executables that are part of Redis:

  • redis-server is the Redis Server itself.
  • redis-sentinel is the Redis Sentinel executable (monitoring and failover).
  • redis-cli is the command line interface utility to talk with Redis.
  • redis-benchmark is used to check Redis performances.
  • redis-check-aof and redis-check-dump are useful in the rare event of corrupted data files.

It is a good idea to copy both the Redis server and the command line interface in proper places, either manually using the following commands:

  • sudo cp src/redis-server /usr/local/bin/
  • sudo cp src/redis-cli /usr/local/bin/

Or just using:

 cd src/
 make install.

NOTE: We assume that /usr/local/bin is in your PATH environment variable so that you can execute both the binaries without specifying the full path.

Starting Redis

To start redis with default configurations:

redis-server

Or if you like to start with custom settings then pass them in at the start up as following:

redis-server /etc/redis.conf

You should use the redis.conf file included in the root directory of the Redis source code distribution as a template to write your configuration file.

Testing Redis

Redis provides a command line utility “redis-cli” that can be used to send commands to Redis. To check if Redis is working properly is sending a PING command using redis-cli

redis-cli ping

This should respond with “PONG” if Redis working properly

To see all options with redis-cli utility such as running different port,host, etc:

redis-cli --help

To start Redis in interactive mode so you can type different commands and see their replies.

$ redis-cli                                                                
redis 127.0.0.1:6379> ping
PONG
redis 127.0.0.1:6379> set mykey somevalue
OK
redis 127.0.0.1:6379> get mykey
"somevalue"

If able set values, then Redis is working properly and your are ready to next step

Connecting And Other

Command Redis Node.js
CREATE CONNECTION
 redis-cli
db = redis.createClient();
CLOSE CONNECTION
 exit 
db.end
FLUSH DATABASE
 FLUSHALL 
db.flushdb(function (err, succes) {
...
}

Hashes

Purpose: good for saving objects that have properties
For example, we have object review ->[id:1,title:’some title’,…] that contains properties and values

Command Redis Node.js
SET
Single Attribute
 HSET myhash field1 "Hello"
db.hset('cache:review:1', 
   {id: '1', title: 'some title'}, 
   function(err, args){..})
GET
Singe Attributes
 HGET myhash field1
???
SET
Multiple Attributes
 HMSET myhash 
   field1 "Hello" 
   field2 "World"
db.hmset('review:2', 
   this, 
   function(err, args){..})
GET
Multiple Attributes
 HMGET myhash field1 field2 
db.hmget('cache:review:6', 
   'remote', 
   'local', 
   function(err, args){..})
LIST ALL
attribute = values
HGETALL myhash
db.hgetall('review:3',
    function(err,review){..}
LIST ALL
ONLY values
 HVALS myhash 
???
DELETE
 HDEL myhash field2 
db.del('review:2')

Sorted Set – non repeating collections of Strings

Purpose: Its good to keeping track of Objects Stored in Redis.
For example, we have set ‘Reviews’ -> [1,3,4,6] that contains all IDs of each review saved

Command Redis Node.js
ADD
ZADD myzset 1 "one"
db.zadd('myzset', 1, "one", function(err, args){
        if(err) {...});
LIST
 ZRANGE myzset 0 -1
db.zrange('reviews', 
  from, 
  to, 
  callBackFunc)
DELETE
 ZREM myzset "two"
db.zrem('ŗeviews', 
  this.id)

Lists

coming soon…

Security

Securing from CLI

Through command line you can setup security as follows:

redis-cli //to login redis cli
>CONFIG get requirepass //see if security & password setup already
1) "requirepass"
2) ""
>CONFIG set requirepass "MY_PASSWORD" //sets up password
OK
>AUTH "MY_PASSWORD" //this will test your new password
OK
>exit

This will setup your security with password

Securing from Configuration File

Instead of command line, the security can be configure via redis-config file as follows:

Either case the security is configured, once configured the apps need to authenticate. Here is sample from NodeJs app

     redis_client = redis.createClient();
     redis_client.port = port;
     redis_client.host = host;
     redis_client.auth('PASSWORD');

The highlighted line, authenticates the connection via the password provided in the code

Maintenance

To see Redis logs:

cat /var/log/redis_6379.log

To stop Redis server manually on Ubuntu:

/etc/init.d/redis-server restart

–OR–

redis-cli shutdown

–OR–

ps -ef | grep -i 'redis-server'
kill -9 PID owned by redis

To find version of Redis and other info:

redis-cli
>INFO

So, we login first Redis command line and then execute “INFO” to see info about current version of Redis

Useful Links

  • https://github.com/mranney/node_redis
  • http://redis.io/topics/data-types-intro
  • http://redis.io/topics/quickstart

Plugin Configurations in Grails

Perhaps, you have a plugin and you would like to configure it instead of hard coding the settings. Lets say, it is a url for REST service that your plugin consumes. In this post, we cover how to incorporate the plugin configurations into part of the default Grails configurations(conf/Config.groovy). In the second part, we cover how to separate the plugins configurations into a separate configurations file while still part of the application

Configurations From App conf/Config.groovy

First, add the configurations setting in the Grails app configuration file(conf/Config.groovy):

// This default value...
yourConfigurationName.url='http://service.com/restapi/'
...
environments {
  development {
    // configurations when in development environment
    yourConfigurationName.url='http://service.com/restapi/'
  }
}

The configuration name may be as you like. From your plugin, this configurations setting can be accessed as follows:

class NodeDriverProxyService {
    def grailsApplication
    def http

    public someMethod(){
        http = new HTTPBuilder(grailsApplication.config.yourConfigurationName.url)
...

As you see, first we inject grailsApplication and then access your setting via grailsApplicaiton.config

This way it is also possible to externalize plugin settings among the other grails configurations as described in post Externalizing Configurations per Grails Application

Configuration from plugins config file

Perhaps, you like to put plugin configurations into separate configuration file instead Grails default Config.groovy
First, create your configuration file and add the configurations. For example grails-app/conf/myPluginName.groovy:

// This default value...
yourConfigurationName.url=http://service.com/restapi/
...
environments {
  development {
    // configurations when in development environment
    yourConfigurationName.url=http://service.com/restapi/
  }
}

This is exact same as part of the Application configurations. Next, here is how to access from plugin:

GroovyClassLoader classLoader = new GroovyClassLoader(getClass().getClassLoader())
ConfigObject config
try {
   config = new ConfigSlurper().parse(classLoader.loadClass('myPluginName'))
}
catch (Exception e) {/*
    if exception, set default settings
*/}

//to access
config.yourConfigurationName.url

Here, we load the file and then use Groovy ConfigSlurper utility to parse and access plugin configurations

To summarize, we covered two ways for plugins configurations being incorporated in the application. One as part of the Grails configuration and another as a separate plugin configurations within the application.

Issues

1. grailsApplication is null
Make sure grailsApplication.config is used within a method. i.e:

class someClass{
def grailsApplication
def url = grailsApplication.config.pluginName.url//this is going to be null

public someMethod(){
   url = grailsApplication.config.pluginName.url//this works!!!
  }
}

Useful Links

  • http://groovy.codehaus.org/ConfigSlurper
  • http://stackoverflow.com/questions/843216/configuration-of-grails-plugin