Sharing via Samba Instead VirtualBox Shares

The performance was so largely effected by using the VirtualBox shares on Apache server that I had to find a better solution to share directories between my VirtualBox Dev environment with Web server,database,etc and my machine Mac with the editor software. I turn to Samba Service as solution since it is one of the well known file sharing services available

Step 1 – Make Your Mac Sharing

By default, the file sharing is not enabled in the Mac, so the first step is to do just that

  1. On Mac, go to “System Preferences”->”Sharing” and check “File Sharing”
  2. And add the users and set permissions to “read & write” in the “Sharing” section
  3. On VBox Guest, add the ‘shares’ – directories that going to be shared in the “Sharing” section under Settings of your VirtualBox guest machine

Step 2 – Install Samba

While Mac comes with the Samba support, our guest Os is xUbuntu and it didn’t have Samba installed/enanbled which I did as following:

sudo apt-get install samba
sudo start smbd
sudo start nmbd

This installs and starts the Samba server on my guest Ubuntu

Mount Shares

Once the Samba installed and started, then we mount the share directory. The mounting was done from the Samba server( in our case guest OS) into the host – Mac OS as following:

To mount a share from guest to host:

sudo mount -t cifs //HOST-IP/SHARE-As-Declared-Step-1 /path/At/Guest/machine -o uid=1000,iocharset=utf8,rw,username="USER-HOST",password=USER-HOST-PASSWORD,nounix --verbose

See “man mount.cifs” for more info on options(i.e. “-o”)

To check what’s mounted so far on Ubuntu:

mount

This should display all mounted points including the shares

To mount automatically on reboot:

  1. Create Permanent IP for Host Machine

    Before configuring to auto mount the shares, it is important to have the permanent IP for the host. Otherwise, you will have to manually change the IP every time your Host WiFi IP changes. To have the permanent IP for the Host machine, you have to create “Host Only Network” as described in section 6.1 of post Virtualbox Gotchas

  2. Configure Auto Mount Shares

    To mount the shares at the start up, we edit /etc/rc.local at our Guest Ubuntu OS and added the following:

    mount.cifs //HOST-IP/SHARE-As-Declared-Step-1 /path/to/shared/dir -o uid=1000,iocharset=utf8,rw,username=HOST-USER,password=HOST-USER-PASSWORD,nounix
    

    So by adding the line in the rc.local file, it is executes the mount at the boot process. Do the same for each share you like to mount.

Sharing via AFP

Disclaimer: The AFP is very buggy and should be avoided.
Its possible to use AFP instead of SAMBA. I had to install afp modules on the guest Ubuntu:

sudo apt-get install afpfs-ng

Afterwards, we mounted the the directory as following:

sudo mount_afp afp://host_user:host_user_password@hostIp/share /srv/afp/share

Troubleshooting

Do Manual Mount

To better see the issue, mount the shared dir manually via “mount” command. It seems to display better error messages. Here is sample command taken out from rc.local with extra flags

sudo mount -t cifs //10.88.12.0/workspace /home/margots/workspace -o uid=1000,iocharset=utf8,rw,username=margots,password=PASSWORD,nounix
Check What is mounted?

First, look at what is mounted by issuing command:

mount

It will display a list of all the mounted directories

1. Check Logs

Log files location is declared smb.conf file in “/etc/samba/” dir. The default location “/var/log/samba”

2. See Samba Settings

There is nice way to see the Samba settings:

smbclient -L smb -N 
3. Check if Samba Running

Check if Samba is running:

ps -ef | grep smbd
4. Follow Kernel Logs Fix Mount Problem

Look at the logs to fix any mount problem:

tail -f /var/log/kern.log 

This will be output logs as they come in

5. mount_smbfs: mount error: /srv/samba/share: Unknown error: -1073741275

I haven’t found solution to this. It is happening when trying to mount share from host to the guest instead way around as described above. I guess my solution was just to mount the share from guest to host that accomplishes the task at hand.

6. Package smbfs is not available

According to this page https://wiki.ubuntu.com/MountWindowsSharesPermanently, it seems smbfs is deprecated and you should use cifs instead. try:

sudo apt-get install cifs-utils
7. mount error(22): Invalid argument.Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

This happened when trying to mount share into host mac

sudo mount -t cifs //192.168.1.4/srv /srv/samba/share -o uid=1000,iocharset=utf8,username="margots",password=38394815

To solve problem, we needed to disable the CIFS Unix Extensions for this mount that was done by adding option “nounix”

sudo mount -t cifs //192.168.1.4/srv /srv/samba/share -o uid=1000,iocharset=utf8,username=USERNAME,password=PASSWORD,nounix
8. Permission Denied For Apache When Writing On Samba Share

There was permission denied when Apache server tried to write in Samba Share. The solutions was to update apache user and group to one with Samba write permissions. To do so, I had to update Apache Environment variables – APACHE_RUN_USER, APACHE_RUN_GROUP in the /etc/apache2/envvars file as following:

export APACHE_RUN_USER=SAMBA_USER=samba_user
export APACHE_RUN_GROUP=share_dir_user_group

The Samba User with group of the share allowed Apache to make writes on Samba shares

9. mount error(13): Permission denied

I got this error when incorrectly enabled file sharing at host machine of Mac. You may need to have the 3 settings present in “System Preferences”->”Sharing”:

    Check “File Sharing”

  • Under “File Sharing”, click “Options” and add the user that will be used for remote sharing
  • Check “Remote Login”
  • Under “Remote Login”, add the user that will be used for remote sharing

This solved the error for me

10. /etc/rc.local Doesn’t Run at Startup

The /etc/rc.local contains our mount functions to run at startup, however. If it doesn’t run, try troubleshooting by running the following commands and looking for error messages for answers

/etc/rc2.d/S99rc.local start
/etc/init.d/rc.local start
/etc/rc.local

This is tracing through the sequence for scripts called before rc.local to find out where it fails

11. No Match Found in /etc/fstab

As I was tracing through the scripts to find out why the rc.local doesn’t seem to be executed at startup to mount my shared dir, i got the error “No Match xxx found in /etc/fstab” prusuant to 10 above. The solution was allow members of CDROM group to mount without prompting for root password by inserting the following line in /etc/sudoers:

%cdrom ALL = NOPASSWD:NOEXEC: /bin/mount, /bin/umount, /sbin/mount.cifs, /sbin/mount.nfs

Afterwards, i add “sudo mount” to mount command in rc.local script:

sudo mount.cifs ...
12. CIFS VFS: cifs_mount failed w/return code = -5 or mount error(5): Input/output error

These errors came up when the host, in my case Mac, didn’t have the shared directory enabled sharing according to Step 1 above

References

  • http://stackoverflow.com/questions/87442/virtual-network-interface-in-mac-os-x
  • http://www.dejadejoder.com/blog/posts/2010/07/21/mac-osx-10-6-snow-leopard-mount_smbfs-authentication-error-problem/
  • http://www.trickyways.com/2011/07/file-sharing-mac-os-x-lion-and-pc-windows-7/
  • http://www.samba.org/samba/docs/Samba24Hc13.pdf
  • http://www.clock.co.uk/blog/virtualbox-40-shared-folders-still-slow
  • https://forums.virtualbox.org/viewtopic.php?f=7&t=18044&start=0
  • https://forums.virtualbox.org/viewtopic.php?p=33519
  • https://help.ubuntu.com/10.04/serverguide/samba-fileserver.html
  • http://adrianmejia.com/blog/2011/07/12/how-to-set-up-samba-in-ubuntu-linux-and-access-it-in-mac-os-and-windows/
  • http://www.samba.org/samba/docs/man/manpages-3/mount.cifs.8.html

VirtualBox Gotchas

This is ongoing post listing different challenges and how those can be overcome when utilizing VirtualBox for your development needs

1. VirtualBox host only supports 32bit guest OS

This was a problem on Windows 8 Intel processor machine that didn’t have vt-x enabled. The vt-x is feature of Intel processor that allows the virtualization of some kind. Once enabled, the VirtualBox host can load also 64bit Guest OS. To enable, boot Vindows 8 in BIOS and enable the vt-x feature

2. Freezes with “Building initial module for 3.x”

Make sure the guest OS you are installing is for the right processor.

3. Cannot Change Display Resolution

Install the VirtualBox additions at the Guest

sudo apt-get install virtualbox-guest-dkms

Afterwards, reboot and the display resolution works as you like

4. Cannot Copy & Paste from VirtualBox Host to Client and from Client to Host

Copy/paste is not enabled by default. You have to enable in host ‘Settings’ under ‘General’->’Advanced’ and check ‘Shared Clipboard’ and ‘Drag & Drop’. Note, that changes can only made when client OS is turned off.

5. Sharing folders Between VirtualBox Host and Guest

There are two ways to share directories between Host and Guest – via VirtualBox Shares or via independent Samba Server. As we learn later, the VirtualBox Shares solutions is very slow for Apache to service web applicaitons such as Drupal. So we strongly recommend Samba solution for sharing directories, however. Here are setup for both solutions:

  • Share Via Samba

    Please, see post “Sharing Vis Samba Instead VirtualBox Shares” for sharing directories via Samba

  • Share Via VirtualBox Shares

    There are two steps:

    1. In the host settings under ‘shared folder’ directory, add the shared directory and give a name ‘some-host-share-name’. Make sure you check “Permanent”, so its available on each reboot
    2. Next, physically mount the shared directory

      sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) some-host-share-name ~/guest-dir
      

    Make sure the ‘~/guest-dir’ exists on the guest OS. Otherwise, you will get error “/sbin/mount.vboxsf: mounting failed with the error: No such file or directory” or no any error and it will just not mount

To mount it automatically on reboot, add the entry in the /etc/rc.local as following:

  • For Samba solution:
    mount.cifs //HOST-IP/SHARE-NAME /path/to/shared/dir -o uid=1000,iocharset=utf8,rw,username=HOST-USER,password=HOST-USER-PASSWORD,nounix
    
  • For VirtualBox shares solution:
    mount.vboxsf -w some-host-share-name /home/<username>/guest-dir
    

This will mount the the shared drive on each reboot automatically.

5A. Share Between Guest OS of Windows XP

First ensure, the Guest addition is installed for the Guest Windows XP OS by grabbing appropriate ISO of your VirtualBox solution at the http://download.virtualbox.org/virtualbox/ dir.
Next, add the ISO to the Optical CD/DVD drive of the Guest OS.
At last, run the executable/installer in the ISO rood directory. This will install Guest additions necessary, so that at the next boot it creates the shared dir in the Guest Windows OS automatically in the “Network Drives”

6. Easily access your NAT configured virtual machine

You probably need host access servers on the guest OS, while your guest OS, probably, need access internet at the same time. Here is NAT + Host ONLY solution to address both needs configurable as following.

  1. Create new Network Go to VirtualBox VM->Preferences->Network->Host Only Networks and add new one with the following settings:
    IP:10.88.12.0
    Mask:255.0.0.0
    

    Keep DHCP server disable, so you can configure static IP much more easier than looking up the new IP every time it changes

  2. Add Network Interfaces Add the network interfaces by going to Guest Settings->Network->Adapter
    • Adapter 1: NAT
    • Adapter 2: Host Only Adapter with “name” of network you configured in step 1 above
  3. Configure Two Guest OS Interfaces. In your guest OS, edit /etc/network/interfaces for Deb or Ubuntu as following:
    # interfaces(5) file used by ifup(8) and ifdown(8)
    auto lo
    iface lo inet loopback
    
    #the primary network interface
    auto eth0
    iface eth0 inet dhcp
    
    #The secondary network interface
    auto eth1
    iface eth1 inet static
    address 10.88.12.4
    netmask 255.0.0.0
    

    This will configure two network interfaces one for NAT and another for Host Only network

Afterwards, reboot guest OS and you should access guest OS at 10.88.12.4 from host, access host at 10.88.12.0 from guest all which while your guest OS have internet connection via the NAT network

7. Slow Performance for Guest xUbuntu

The following settings improved Guest performance:

  • Check “Storage” -> “Use Host I/O Cache”
  • Check “Display” -> “Enable 3D Acceleration”
  • Install guest addtions, utilities and drivers:
    sudo apt-get install virtualbox-guest-dkms virtualbox-guest-utils virtualbox-guest-x11
    
  • Add extra CPUs “System”->”Processor”
  • Check “Enable PAE/NX” under “System”->”Processor”
  • Add more RAM to Video Card “Display” -> “Video” -> “Video Ram”
  • Install VB extension pack on the host machine:
    1. Download the extension pack from VirtualBox Download page
    2. Run the following command:
      sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.3.18-96516.vbox-extpack 
      sudo VBoxManage list extpacks
      
  • look into logs to see if it shows any issues that may affect performance

Note: After analyzing performance for my Drupal site running on VirtualBox(see more details at post looking-at-drupal-performance-with-xprof-and-xhgui), it shows that the top 6 slowest functions are “file_scan_directory”. It made me conclude that reading/writing on disk is the slowest. I upgraded HD to SSD HD, but while there was performance improvement it still was noticeable. This applies to the shared directories between Guest and Host. If not shared then ‘file_scan_directory’ is not any more the bottleneck and Drupal site runs really fast.

UPDATE: The best improvement by far was moving away from sharing directories via VirtualBox Shares solution to the sharing via Samba Server as described in post Sharing Via Samba Instead VirtualBox Shares

8. Increase Size for VB Guest Hard Drive

After about 3 weeks running VB, it run out of space on Hard Drive that was set of fixed size. Here are steps to increase Hard Drive space

  1. [ONLY IF FIXED SIZE] Re-clone HD .VDI to be able use ModifyHD
    Resizing HD involves runing command ‘modifyhd’, however. This only works for HD that were not fixed, so before we are able to use “modifyhd”, we re-clone our HD .vdi file:

    sudo VBoxManage clonehd "/full/path/to/disk1.vdi" ClonedFixedHD.vdi --format VDI --variant Standard
    

    This will prepare our Hard Drive .vdi file for modifyhd command to use in next step

  2. Resize the HD Drive .vdi
    sudo VBoxManage modifyhd "/full/path/to/ClonedDevelLamp.vdi" --resize 20000
    

    This will increase the size of the hard drive to 20GB

  3. Update Partition Size for VB Guest
    Once the size is changed for hard drive, we need to update also the VB Guest partition that mounts the new sized Hard drive. It is accomplish by booting from GParted

    1. Download CD of GParted
    2. Mount to CD drive of the VB Guest
    3. Boot into your VB Guest
    4. Resize partition.Here is good article
  4. Remove Old Hard Drive & Attache New Resized HD
    Go to Setting->Storage and remove the old HD .vdi. Then add your new resized HD. Afterwards, reboot the machine

Check Logs

In the following are directories for logs per OS:

  • On Windows, this is %HOMEDRIVE%%HOMEPATH%\\.VirtualBox\\Machines\\\\Logs; typically some- thing like C:\\Documents and Settings\\Username\\.VirtualBox\\Machines\\\\Logs\\vbox.log
  • On Mac OS X, this is $HOME/Library/VirtualBox/Machines//Logs
  • On Unix-like systems (Linux, Solaris), this is $HOME/.VirtualBox/Machines//Logs

Troubleshooting

1. No extension pack by the name ‘Oracle VM VirtualBox Extension Pack’ was found

Go to VirtualBox Download page and download extension pack. Afterwards, install it as following:

sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.3.18-96516.vbox-extpack 

To verify:

sudo VBoxManage list extpacks

This should display descriptions of the extension pack or 0 for no extension pack present

2. VBoxManage: error: Cannot register the hard disk ‘/path/to/disk1.vdi’ {8e28be07-2543-4967-a83f-8a6a07d18c64} because a hard disk ‘path/to/disk1.vdi’ with UUID {..} already exists

We got this after cloning and then trying to resize hard drive. The solution was to reset UUID:

sudo VBoxManage internalcommands sethduuid "/full/path/to/disk1.vdi"

Make sure you provide full path to the HD file .vdi, otherwise, you will get another error at least on Mac OS.

3. Progress state: VBOX_E_NOT_SUPPORTED

This error came up when trying to resize the HD file .vdi, because the HD was configured fixed and command “modifyhd” doesn’t work with fixed HD .vdi. The solution was convert the HD .vdi into one that is not fixed size by re-cloning as following:

sudo VBoxManage clonehd "/full/path/to/disk1.vdi" ClonedNewVDIfile --format VDI --variant Standard

This will create new .vdi size ready to work with “modifyhd”

4. My .VDI is missing

I was looking at $HOME/VirtualBox VM/SOME_VM_BOX_DIR just to find there is no .vdi file, however, the particular Box was loading okey. I was unaware that the HD(i.e. .vdi file) can be outside the Box directory. See “Settings”->”Storage” to locate where is the Hard Drive .vdi file located

VBoxManage: error: Could not find file for the medium ‘/full/path/to/disk1.vdi’ (VERR_FILE_NOT_FOUND)/h6>
This error is because the path specified is not correct or it is not full path that is required for running VBoxManage commands

5. “the parameter is incorrect” Or “the network name cannot be found” OR “is not accessible”

This error happened when I was sharing files of mac host with Window XP as guest OS and I didn’t have the Guest Additions installed. To install, grab appropriate ISO of your VirtualBox solution at the http://download.virtualbox.org/virtualbox/ dir. Next, add the ISO to the Optical CD/DVD drive of the Guest OS. At last, run the executable/installer in the ISO rood directory. This will install Guest additions necessary, so that at the next boot it creates the shared dir in the Guest Windows OS automatically in the “Network Drives” dir

References

https://forums.virtualbox.org/viewtopic.php?f=24&t=50661
http://drwho.virtadpt.net/archive/2014/02/04/problems-cloning-virtualbox-disk-images

Resizing vmdk harddisks for VirtualBox

After Vagrant Crash How To Recover Database

It becomes more common for businesses to use vagrant as single virtual machine solution in house that can be shared among developers that way ensuring every developer has the same version of services, frameworks and libraries running. It also saves developer time for upgrades and maintaining their development sandbox.

In short, vagrant manages the configurations defining what ports forwarded, memory size and others configuration including which virtual machine to load. The actual virtual machine is provided by Virtual Box technology and is available as separate entity.

Sometime it happens that vagrant crashes and hangs at the load process. For me, i got the following error message:

[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.

The logs are available in your Virtual Box folder or by default these images are in your home directory ‘VirtualBox VMs’ folder

Which Virtual Box image corresponds which Vagrant config?

In case you have more than one vagrant configurations or multiple virtual box images on your machine. To find out the particular Vagrant configuration pointing to which virtual image, go to the vagrant configuration folder and look at ‘.vagrant’ file. You will see something like:

{"active":{"default":"79a80fbd-2804-4dde-abd8-c23bb18a9b9a"}}% 

That is the UUID of the Virtual Box image to load from this vagrant configuration.
Next, how do you know what is the UUID for particular Virtual Box image. Go to the folder of your VirtualBox images and run:

VBoxManage list vms

This will display each VirtualBox name with its UUID

If your Vagrant crashes, perhaps, instead of spending time troubleshooting you would just load new instance. if so, you may still need the data from database of the old virtual image.

Step 1: Access VirtualBox image directly
With ‘VirtualBox’ software, you can run any Virtual box image you like. Once you start, the default login:

username:vagrant
password:vagrant

Step 2: Add Shared folder to access the mysql dump file
To add shared folder, go to ‘Settings’->Shared Folder and select folder(i.e.share) on your machine itself
Next mount that folder as following

sudo mount -t vboxsf share ~/host

Where the ‘share’ is the folder you specified in share folder settings above and the ‘~/host’ is the folder on the virtual image
Note: make sure your permissions set on virtual machine and your original machine accordingly
Step 3: Take mysql dump file
To take the back up, got the new mounted folder and run mysqldump command as follows:

mysqldump -u root -p nameOfDatabase>backupFile.sql

This will create the back up file and save into folder outside the virtual image that you have direct access. Afterwards, you can share this folder with another virtual image to import the database.

Useful links