Author Archives: Arun Natarajan

virt-manager without root

To be able to execute virt-manager without root privilege,

– create a new group

# group add libvirt

– Add the required users to this group by editing the /etc/groups file

– Edit the libvirtd configurations:

# vi /etc/libvirt/libvirtd.conf

– Add the following configurations.

unix_sock_group = "libvirt"
auth_unix_rw = "none"

– Restart libvirtd,

#service libvirtd restart

– Logout and try to access libvirtd

$ ssh -X <username>@<host> virt-manager

./arun

Monitor VMware ESXi hardware without root (Nagios)

Download and configure the plugin: http://exchange.nagios.org/directory/Plugins/Operating-Systems/*-Virtual-Environments/VMWare/check_esxi_hardware-2Epy/

– Create a new user in ESXi with no access privilege, you need to login to the ESXi directly to do that.

user

esxi_access

– Enable SSH, and add nagios user to root group:
# vi /etc/group
root:x:0:root,nagios

– Check from the command line, if it works
./check_esxi_hardware.py --host https://esxihost:5989 --user file:credentials.txt --pass file:credentials.txt
OK - Server: Cisco Systems Inc.....

– Configure the credentials files to use the nagios user credentials.

Setup GeoIP (PECL) for piwik geolocation and updating old visits

GeoIP is the recommended way to accurately determine the location of the visitor, by default geolocation settings may provide in accurate result.

To enable GeoIP(PECL) from redhat/centos machines:
# yum install php-pecl-geoip
#apachectl restart
# php -m | grep -i geo
geoip

From Piwik, Settings –> Geolocation –> GeoIP (PECL)

To reindex the old visits:
# cd misc/others
# php ./geoipUpdateRows.php
[note] Found working provider: geoip_pecl
90094 rows to process in piwik_log_visit and piwik_log_conversion....
.
.
.
100% done!

Send attachments from command line with mutt

To send e-mails from command line with attachments using mutt.

Set the from address with EMAIL=
-s – Subject
-a – attachment file
recipient name
-c – for CC
-b – for BCC
create a text file (eg: /tmp/testmessage) , with the body of the message.

EMAIL="foo@bar" mutt -s "Subject" -a test.doc foo1@bar -c foo2@bar < /tmp/testmessage

Could not connect to https://vcenter_address:7331/

This usually happens from the vSphere web client while opening a console session with virtual machine.

and the log (/var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log) shows something like:

[ERROR] Thread-42 System.err
INFO:oejsh.ContextHandler:started o.e.j.w.WebApp Context{/console,file:/tmp/jetty-0.0.0.0-7331-console.war-_console-any-/webapp/},/usr/lib/vmware-vsphere-client/server/work/tmp/console-distro/webapps/console.war

To fix this set the environment variable VMWARE_JAVA_HOME to proper path:

– SSH to vcenter
# vi /usr/lib/vmware-vsphere-client/server/wrapper/conf/wrapper.conf

– Under Environment variables add:
set.default.VMWARE_JAVA_HOME=/usr/java/jre-vmware

– Restart vsphere-client
# /etc/init.d/vsphere-client restart
Stopping VMware vSphere Web Client...
Stopped VMware vSphere Web Client.
Starting VMware vSphere Web Client...
Intializing registration provider...
Getting SSL certificates
Service with name was updated.
Return code is: Success
Waiting for VMware vSphere Web Client......
running: PID:

Reference: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2060604

Create bootable USB on OSX

– Identify the disk number for the USB disk inserted, usually you can find it from the “Name” and “Size” field.
Below eg: , We inserted a USB with 4.1 GB size (so as the identified: “disk2”.)
Open terminal and execute the following commands:

sh-3.2$ diskutil list
/dev/disk0
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:      GUID_partition_scheme                        *120.0 GB   disk0
1:                        EFI EFI                     209.7 MB   disk0s1
2:          Apple_CoreStorage                         119.0 GB   disk0s2
3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:                  Apple_HFS Macintosh HD           *118.7 GB   disk2
/dev/disk2
#:                       TYPE NAME                    SIZE       IDENTIFIER
0:                            UNTITLED        *4.1 GB     disk2

– Unmount the disk
sh-3.2$ diskutil unmountDisk /dev/disk3
Unmount of all volumes on disk3 was successful

– write the ISO file to USB using dd command
sh-3.2$ sudo dd if=Downloads/ubuntu-12.04.3-desktop-i386.iso of=/dev/disk2 bs=1m
Password:
707+0 records in
707+0 records out
741343232 bytes transferred in 139.059398 secs (5331126 bytes/sec)

– Unmount the disk
sh-3.2$ diskutil eject /dev/disk3
Disk /dev/disk3 ejected

vCenter Converter

Convert Linux Physical Server to VMware virtual machine

Download and install vCenter Converter on a windows machine.
http://www.vmware.com/products/converter
Unfortunately this tool does not have a Linux / MAC version.

vCenter Converter

vCenter Converter

In case you see an error: “Permission to perform this operation was denied”, right click and run the program as Administrator.

Permission to perform this operation was denied
Provide the source and destination information, the source is the physical server to be converted and the destination vCenter.
vCenter Converter

vCenter Converter

vCenter Converter

vCenter Converter

Follow the steps, to do the conversion , a temporary OS will be started on the destination, by default it try to get an IP address from the DHCP server so that it can connect to the source machine and fetch the files required. But in case if you don’t have DHCP server you might see error like: “Unable to obtain the IP address of the helper virtual machine” . Fix this issue by setting up a static IP to the helper virtual machine during the conversion setup. Basically the helper VM IP should be able to communicate with the source machine which needs to be migrated

vCenter Converter Static IP

vCenter Converter Static IP

Proceed with the conversion, the duration will be based on the size of the VM and the connectivity if it belongs to another site/LAN.

You may need to change the network configuration (eg: HWADDR) and the MAC address mapping (/etc/udev/rules.d) to get it connected.

ESXi host fails with a purple diagnostic screen PSOD

This happened while converting KVM VMs to VMware and power them on (method used:http://arunnsblog.com/2013/06/10/migrate-kvm-virtual-machines-to-vmware-esxi/) . It works for a while but then the ESXi crashes with PSOD.

Version : 5.1.0-799733

There were two sort of PSOD messages observed:
1) Crashed while the VM was running

 VMware NOT_IMPLEMENTED bora/vmkernel/sched/memsched.c:17724
 Code start: 0x41802b200000 VMK uptime: 10:19:25:27.335
 cpu4:8243)0x412200cdbaf0:[0x41802b27abff]PanicvPanicInt@vmkernel#nover+0x56 stack: 0x3000000008
 cpu4:8243)0x412200cdbbd0:[0x41802b27b4a7]Panic@vmkernel#nover+0xae stack: 0x100000000000000
 cpu4:8243)0x412200cdbc50:[0x41802b3d88eb]MemSched_WorldCleanup@vmkernel#nover+0x426 stack: 0x4100018a4fb0
 cpu4:8243)0x412200cdbef0:[0x41802b3033b8]WorldCleanup@vmkernel#nover+0x1cb stack: 0x4700cdbf40
 cpu4:8243)0x412200cdbf60:[0x41802b303829]WorldReap@vmkernel#nover+0x318 stack: 0x0
 cpu4:8243)0x412200cdbff0:[0x41802b2483c8]helpFunc@vmkernel#nover+0x517 stack: 0x0
 cpu4:8243)0x412200cdbff8:[0x0] stack: 0x0
 cpu4:8243)base fs=0x0 gs=0x418041000000 Kgs=0x0

VMWare_ESXi_PSOD

VMWare_ESXi_PSOD

2) Crashed during ESXi reboot.

#PF Exception 14 in world 8243:helper13-1 IP 0x41802b880a1e addr 0x410401503020

VMWare_ESXi_PSOD

VMWare_ESXi_PSOD

This seems to be a known issue in VMware ESXi 5.1 and is resolved in patch ESXi510-201212401-BG (Build 914609).
Ref: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2038767

To work around this issue, SSH to the ESXi host and increase the MinZeroCopyBufferLength to 512.

# esxcli system settings advanced set -o /BufferCache/MinZeroCopyBufferLength -i 512

To verify that the setting has been updated, run this command:

# esxcli system settings advanced list --option /BufferCache/MinZeroCopyBufferLength

Before and after change

Before and after change

ESXi host fails with a purple diagnostic screen PSOD

This happened while converting KVM VMs to VMware and power them on (method used: http://arunnsblog.com/2013/06/10/migrate-kvm-virtual-machines-to-vmware-esxi/) . It works for a while but then the ESXi crashes with PSOD.

Version : 5.1.0-799733

There were two sort of PSOD messages observed:
1) Crashed while the VM was running

 VMware NOT_IMPLEMENTED bora/vmkernel/sched/memsched.c:17724
 Code start: 0x41802b200000 VMK uptime: 10:19:25:27.335
 cpu4:8243)0x412200cdbaf0:[0x41802b27abff]PanicvPanicInt@vmkernel#nover+0x56 stack: 0x3000000008
 cpu4:8243)0x412200cdbbd0:[0x41802b27b4a7]Panic@vmkernel#nover+0xae stack: 0x100000000000000
 cpu4:8243)0x412200cdbc50:[0x41802b3d88eb]MemSched_WorldCleanup@vmkernel#nover+0x426 stack: 0x4100018a4fb0
 cpu4:8243)0x412200cdbef0:[0x41802b3033b8]WorldCleanup@vmkernel#nover+0x1cb stack: 0x4700cdbf40
 cpu4:8243)0x412200cdbf60:[0x41802b303829]WorldReap@vmkernel#nover+0x318 stack: 0x0
 cpu4:8243)0x412200cdbff0:[0x41802b2483c8]helpFunc@vmkernel#nover+0x517 stack: 0x0
 cpu4:8243)0x412200cdbff8:[0x0] stack: 0x0
 cpu4:8243)base fs=0x0 gs=0x418041000000 Kgs=0x0
VMWare_ESXi_PSOD

VMWare_ESXi_PSOD

2) Crashed during ESXi reboot.

#PF Exception 14 in world 8243:helper13-1 IP 0x41802b880a1e addr 0x410401503020
VMWare_ESXi_PSOD

VMWare_ESXi_PSOD

This seems to be a known issue in VMware ESXi 5.1 and is resolved in patch ESXi510-201212401-BG (Build 914609).
Ref: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2038767

To work around this issue, SSH to the ESXi host and increase the MinZeroCopyBufferLength to 512.

# esxcli system settings advanced set -o /BufferCache/MinZeroCopyBufferLength -i 512

To verify that the setting has been updated, run this command:

# esxcli system settings advanced list --option /BufferCache/MinZeroCopyBufferLength
Before and after change

Before and after change

 

 

Migrate KVM virtual machines to VMware ESXi

– Shutdown the KVM guest
– convert the QCOW2 or RAW format to VMDK format

# qemu-img convert image.img -O vmdk image.vmdk

– Upload this image to datastore

– Create a new virtual machine with this disk image

– There might be issues with network interface mapping, fix the network mapping at /etc/udev/rules.d/70-persistent-net.rules

Configure apache solr with tomcat

Install tomcat

$ sudo yum install tomcat6

Download and extract apache solr : http://lucene.apache.org/solr/

$ cp apache-solr-3.6.2.war /var/lib/tomcat6/webapps/solr.war
$ cp -r /data/apache-solr-3.6.2/example/solr /var/lib/tomcat6/

$ cat /etc/tomcat6/Catalina/localhost/solr.xml
$ chown -R tomcat: /var/lib/tomcat6/solr/
$ sudo service tomcat6 restart

Access url : http://<server_name/ip>:8080/solr/admin

Virus scanning for file uploads with clamav/php

Download and install the following packages, in case your repository has those package just use the management tool to install.

Ubuntu/Debian/Mint

# apt-get install clamav clamav-db clamd clamav-devel php-devel

Redhat

# yum install php-devel
# wget http://pkgs.repoforge.org/clamav/clamav-0.97.7-1.el5.rf.i386.rpm
# wget http://pkgs.repoforge.org/clamav/clamav-db-0.97.7-1.el5.rf.i386.rpm
# wget http://pkgs.repoforge.org/clamav/clamd-0.97.7-1.el5.rf.i386.rpm
# wget http://pkgs.repoforge.org/clamav/clamav-devel-0.97.7-1.el5.rf.i386.rpm
# rpm -Uvh clam*
# freshclam
# service clamd start

Configure php-clamav

Download php-clamav from from sf.net

# wget http://downloads.sourceforge.net/project/php-clamav/0.15/php-clamav_0.15.7.tar.gz
# tar xvzf php-clamav_0.15.7.tar.gz
# cd php-clamav-0.15.7/
# phpize
#./configure –with-clamav
# make
# cp modules/clamav.so /usr/lib/php/modules/

Add the modules to php.ini if required.

extension=clamav.so

Make sure the module is loaded

# php -i | grep -i clam
clamav

Incase you see the following error create a symlink to clamav path
LibClamAV Error: cl_load(): Can’t get status of /var/lib/clamav

# ln -s /var/clamav /var/lib/clamav

Test script
Get the testing virus file from http://www.eicar.org/86-0-Intended-use.html and save it on a file (eg: /tmp/virus.txt)

Create a php script:
cat > check_virus.php

<?php
$file = ‘/tmp/testing.txt’;
$retcode = cl_scanfile($file, $virusname);
if ($retcode == CL_VIRUS) {
echo .”Virus found name : “.$virusname;
} else {
echo .cl_pretcode($retcode);
}
?>

$ php check_virus.php
Virus found name : Eicar-Test-Signature

./arun

Create CSR and implement in apache

These steps are used to generate a CSR to get the SSL certificate signed with verisign. The filenames used are just examples.

$ /usr/bin/openssl genrsa -rand /dev/urandom -out <web_root>/domain_name.key 2048
$ /usr/bin/openssl req -new -key <web_root>/domain_name.key -out <web_root>/domain_name.csr

Country Name (2 letter code) [GB]:
State or Province Name (full name) [Berkshire]:
Locality Name (eg, city) [Newbury]:
Organization Name (eg, company) [My Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server’s hostname) []:<make sure it matches exactly with your domain>
Email Address []:

Create the certificate without passphrase if you dont want passphrase to be prompted for every webserver restart.

Upload the CSR to the Certificate Authority and get the signed certificate and save it as domain_name.crt. If it is from verisign get both intermediate certificates and add it to a file (eg: intermediate.ca.crt)

In virtual host configuration

SSLEngine on

SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:+EXP:!EXP-EDH-RSA-DES-CBC-SHA:!EXP-DES-CBC-SHA!EXP-RC2-CBC-MD5!EXP-RC4-MD5

SSLCertificateFile <web_root>/domain_name.crt

SSLCertificateKeyFile <web_root>/domain_name.key

SSLCACertificateFile <web_root>/intermediate.ca.crt

Restart webservice and verify the certificate, you may use the verisign cert checker (https://ssl-tools.verisign.com/#certChecker)

Install rich text editor in mediawiki

The WYSIWYG extension enables a more intuitive editing of pages on a MediaWiki-based site

Download the package suitable for your mediawiki version

http://www.mediawiki.org/wiki/Extension:WYSIWYG#Download

$ unzip WYSIWYG.zip

$ cp -prf WYSIWYG

$media_wiki/extensions/

$ vi LocalSettings.php added

require_once(“$IP/extensions/WYSIWYG/WYSIWYG.php”);

$wgGroupPermissions[‘*’][‘wysiwyg’]=true;

$wgGroupPermissions[‘registered_users’][‘wysiwyg’]=true;

define permissions as per your requirements.

Upgrade php to 5.3 – directadmin

# cd /usr/local/directadmin/custombuild
# ./build set php5_ver 5.3
# ./build update
# ./build php n
# /etc/init.d/httpd restart

This may break your softaculous, if it cannot load the correct ioncube.

Site error: the file /usr/local/directadmin/plugins/softaculous/images/inc.php requires the ionCube PHP Loader ioncube_loader_lin_5.3.so to be installed by the site administrator.

Edit your php.ini to fix it

# vi /usr/local/directadmin/plugins/softaculous/php.ini
zend_extension = "/usr/local/ioncube/ioncube_loader_lin_5.3.so" // replace it with correct path
# /etc/init.d/httpd restart

Install Raspbmc media center on RaspberryPi

RASPBMC

RASPBMC

Eventhoug I am happy with xbian , was just trying Raspbmc as well 🙂

Raspbmc  media center for the Raspberry Pi and is based on Raspbian and XBMC.

All credits go to Sam Nazarko, Thanks for excellent work.

Reference: http://www.raspbmc.com/,

You may use XPi Installer instead of the following method, http://www.xbmchub.com/blog/2012/11/02/installing-xbian-to-raspberry-pi-from-mac/

Install raspbmc on SDcard from MAC OSX / Linux / Windows ,

Download the raspbmc, change permission and execute. Make sure that you select the correct disk ( verify the info from disk utilities in MAC OS X)

$ curl -O http://svn.stmlabs.com/svn/raspbmc/testing/installers/python/install.py
$ chmod u+x install.py
$ sudo python install.py 
Raspbmc installer for Linux and OS X
http://raspbmc.com
----------------------------------------
Please ensure you've inserted your SD card, and press Enter to continue.
Enter the 'IDENTIFIER' of the device you would like imaged:
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *120.0 GB   disk0
   0:      GUID_partition_scheme                        *500.1 GB   disk1
   0:                  Apple_HFS Macintosh HD           *118.7 GB   disk2
   0:     Apple_partition_scheme                        *17.4 MB    disk3
   0:     FDisk_partition_scheme                        *15.9 GB    disk4
Enter your choice here (e.g. 'disk1', 'disk2'): disk4
It is your own responsibility to ensure there is no data loss! Please backup your system before imaging
You should also ensure you agree with the Raspbmc License Agreeement
Are you sure you want to install Raspbmc to '/dev/disk4' and accept the license agreement? [y/N] y
Downloading, please be patient...
Downloaded 16.35 of 16.35 MiB (100.00%)
Unmounting all partitions...
Unmount of all volumes on disk4 was successful
Please wait while Raspbmc is installed to your SD card...
This may take some time and no progress will be reported until it has finished.
0+1173 records in
0+1173 records out
76800000 bytes transferred in 14.809589 secs (5185829 bytes/sec)
Installation complete.
Would you like to setup your post-installation settings [ADVANCED]? [y/N]N
  • Once the installation is completed, eject it from the system.
  • Insert the sdcard to RaspberryPi.
  • Connect the RaspberryPi using Ethernet cable, continue with the post installation steps.

Create NAS/SAN storage with openfiler, work with VMware ESXi as shared storage

– Download the openfiler installation ISO , download link

I have downloaded the Installation ISO image (x86/64).

Basically we need to create open filer as a virtual machine with say 20 GB thin provisioning, and attach another disk/datastore to the virtual machine to configure it as SAN.

– Download and install  the Installation ISO image (x86/64) as a virtual machine, noting fancy here just do a normal installation.

– Once rebooted you get a web interface to login on port 446, https://< ip > :446/

– Login with username: openfiler, and password: password

Network access configuration:

set up network access configuration, enter the Network/Host , who are allowed to access.

System –> Network Access Configuration

Network Access Config

Network Access Config

Network Access Config

Network Access Config

Create a new physical volume

Volumes –> Block devices

Create Physical Volume

Create Physical Volume

Click on Edit devices on the hard disk where we going to create new physical volume, this is the extra hard drive we added to the virtual machine at the beginning. (/dev/sdb)

 

Create Physical Volume

Create Physical Volume

Select Physical volume as your partition type ( assuming that you are not using RAID), and set the mode to Primary, click create.

Create Physical Volume

Create Physical Volume

Create new Volume Group

Let us create a volume group for the physical volumes

Volumes –> Volume groups , enter a group name and select the physical drive and click Add volume

Create volume group

Create volume group

 

Create volume group

Create volume group

Create the Volume

Volume –> Add Volume

Enter Volume Name , Description, required space and select File System block.

Create Volume

Create Volume

Create Volume

Create Volume

Now Enable and Add  iSCSI Target

Services –> iSCSI Target Enable, Start

Start iSCSI target

Start iSCSI target

Volumes –> iSCSI Targets –> Target Configuration –> Add new iSCSI Target

Add new iSCSI target

Add new iSCSI target

Setup the LUN Mapping and allow access to iSCSI target

Setup LUN Mapping

Setup LUN Mapping

 

Allow access to iSCSI target

Allow access to iSCSI target

Now the open filer is ready to use.

In VMware ESXi

Create new VMKernel network adapter to use for iSCSI connection

VMKernel Adapter

VMKernel Adapter

VMKernel Adapter

VMKernel Adapter

VMKernel Adapter

VMKernel Adapter

Click on storage adapter –> Add new

Add storage adapter

Add storage adapter

Click on properties, add the VMkernel switch and the discover the iSCSI target.

Add VMKernel to iSCSI adapter

Add VMKernel to iSCSI adapter

Dynamic discovery

Dynamic discovery

Now your ESXi will show the openfiler as a datastore 🙂

Openfiler datastore

Openfiler datastore

More information about Openfiler:

http://www.openfiler.com/products