VMware esxi on a memory stick

http://www.vladan.fr/how-to-install-esxi-40-on-usb-memory-key/

Written in the past (some time ago…) , I published a how-to called “Howto install ESXi 3.5 update 3 on USB memory key“. The things has changed today a bit. The how-to is about the same but ESXi 4.0 and vSphere is new and it rocks…. We are in the ESX 4 Era… With ESX Server (vSphere) and more than 150 new features packed inside….
If you found this website useful and want to comeback for more, consider subscribing for FREE via RSS. If you don’t know what is RSS, don’t worry, I wrote a little guide what is RSS and how it saves your daily time.
VMware with vSphere are distancing even more the competition with the performances of vSphere greatly improved. Microsoft’s Hyper-V R2 is still in Beta….. Clearly, the time for an investment into a Virtual Infrastructure with VMware was never been more advantageous. Prices for the SMB market are starting from $166 per CPU. Each physical server may contain up to two phycial processors with up to 6 cores per processor.
Ok, here is the how-to:

01.) Get this 2 tools to acomplish the task.  IZArc, and also WinImage. (trial)
02.) Head to VMware Website and download the ESXi 4.0 vSphere – register for the free for the download. Here is the link.
03.) Save the file to the C drive for example.

04.) Open the ISO file with IZArc and extract the whole content to c:\Vmware-VMvisor-Installer-4.0.0-164009.x86_64 folder .

05.) Browse to “\image.tgz.temptar\usr\lib\vmware\installer\VMware-VMvisor-big-164009-x86_64.dd.bz2\”
Extract the “VMware-VMvisor-big-164009-x86_64.dd” file. You’ll have to extract the “image.tgz.temptar” file first…

 

06.) From just open Winimage, and goto file menu:  Disk > “Restore Virtual Harddisk Image on physical drive”

07.) Select your USB drive

08.) Select The file “VMware-VMvisor-big-164009-x86_64.dd” that you extracted earlier.

09.) Click Yes to write the image to the USB Memory Stick. (ooups… I’m running on French Windows here…… -:) ? It’s because I have 3 different language versions on my Desktop at home… Where I use VMware Workstation as a Virtual Lab. I hope that you don’t mind, the tuto is quite easy to follow…) . If you got any trouble, just comment below and I’ll try to help.. -:)

10.) And the result of your effort should look like this.

Read more: http://www.vladan.fr/how-to-install-esxi-40-on-usb-memory-key/#ixzz1ZURNuGZf

script to monitor a particular process & memory usage

#!/bin/bash



while [ 1 ]; do
   export phpcount=`ps -ef|grep php-cgi|wc| awk '{print $1}'`
   memtotal=`free | awk '/Mem/ { print }' | awk '{ print $2 }'`
   memused=`free | awk '/Mem/ { print }' | awk '{ print $3 }'`
   memfree=`free | awk '/Mem/ { print }' | awk '{ print $4 }'`
   echo "there are $phpcount php-cgi processes at `date` mem $memtotal used $memused free $memfree"
   #free
   sleep 2
done

virtualbox

Install virtualbox 4.0 in ubuntu 10.10
Edit /etc/apt/sources.list file

  deb http://download.virtualbox.org/virtualbox/debian maverick contrib
Add gpg key

  wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
Update source list

 sudo apt-get update

Install virtualbox 4.0

  sudo apt-get install virtualbox-4.0

may need:
sudo apt-get install dkms


from:
http://www.ubuntugeek.com/virtualbox-4-0-released-and-ubuntu-10-1010-049-10-installation-instructions-included.html

wine with office 2007 tips

apply "overrides" on this page:  http://appdb.winehq.org/objectManager.php?sClass=version&iId=4992

wine->wineconfig->libraries usp10 'native,builtin'

wine->wineconfig->libraries riched20 'Native (Windows)'  

wine->wineconfig->graphics 'emulate a virtual desktop'

bash notes

http://tldp.org/LDP/abs/html/fto.html

if [ -<opt> "$str" ]
then
     ...
else
     ...
fi

-e exists
-f is a file
-s is not 0 bytes
-d is directory
-b is block device
-c is char device
-p is pipe
-b is symlink
-L is symlink
-S is socket
-t  is terminal
-r has read perm
-w has write
-x has execute
-g has sgid flag
-u has suid flag
-k sticky bit
-O owner
-G gid same as yours
-N modifed since last read
f1 -nt f2 f1 newer than f2
f1 -ot f2 f2 older than f2
! nots above compares

ffmpeg conversion of 3gp to mp4 files

http://ffmpeg-users.933282.n4.nabble.com/3gp-to-mp4-td942632.html

ffmpeg -i test.3gp -sameq -ab 64k -ar 44100 test.mp4

-ab audio bitrage 
-ar audio resample


info from file ffmpeg -i video.avi
images to file ffmpeg -f image2 -i image%d.jpg video.mpg
file to images ffmpeg -i video.mpg image%d.jpg
sound to mp3   ffmpeg -i source_video.avi -vn -ar 44100 -ac 2 -ab 192 -f mp3 sound.mp3
 
* Source video : source_video.avi
* Audio bitrate : 192kb/s
* output format : mp3
* Generated sound : sound.mp3

avi to giff    ffmpeg -i source_video.avi animated_gif.gif

avi to ntsc    ffmpeg -i source_video.avi -target ntsc-svcd final_video.mpg 

cheatsheets:
http://www.webupd8.org/2009/08/ffmpeg-cheat-sheet-19-best-practices.html
http://rodrigopolo.com/ffmpeg/cheats.html

windows command line install:
http://www.arachneweb.co.uk/software/windows/avchdview/ffmpeg.html

Raid configuration example

http://ascend4.org/Installing_Raid_1_on_Existing_Ubuntu_Server

copied from above:

Useful web sites

* how to set up a raid disk - really good instructions * benchmarks comparing fstype options * how to set up fstab entries * what mount and umount are really all about

---

(NOTE: I have used these instructions for Ubuntu 7.04 and 8.04 also, and they work without any problems. AWW)
 
(NOTE April 27, 2009: I attempted to use these instructions when updating the operating system on my computer to Ubuntu 9.04 from 8.10. The raid 1 disk already existed, and the install process found it and set it up automatically. It called the raid disk md_d0 rather than md0, however. I had to mount it. I also looked into /etc/fstab and corrected the automount instruction that was there.)
 
The first web site above is really excellent. It points you to the tool mdadm, which is for setting up raid disks in linux. I basically followed the instructions on this web site. May God bless Mário, its creator. As you encounter each computer instruction in his guide, it should pay handsomely if you looked at the man pages for it. If the instruction was complex or a term confusing, I also did a search on the web for better explanations (giving rise to those above for fstype, fstab and mount)).

Background

I am a real novice at doing all this so I shall put in my thoughts as we go along. I purchased a PowerEdge 2950 rack mounted server computer from Dell in February, 2007. I bought it without any OS installed and without a hardware Raid controller. It shipped with a single 160 GB SATA disk but with slots for several more disks. I installed the Ubuntu 6.10 (Edgy) linux OS from a disk downloaded from the Ubuntu web site. We later ordered from Dell and received two additional 250 GB SATA disks, which I wished to install in a Raid 1 configuration AFTER having an "up and running" server. We could not obtain only the rails to install the disks into the front slots; we had to purchase the disks, too, from Dell. 


Before starting, and with the computer running, execute the following instruction:

cat /proc/diskstats 

and note the disks that are detected. I detected sda and its partitions only - i.e., my 160 GB disk. 


To install the disks, we halted the computer and installed them by simply slipping them into two slots on the front of the computer. We powered up the computer.

Creating the Raid 1 configuration

Go to the heading "The hard work" in the guiding web page. From a remote computer via ssh, it was my intention now to configure these disks to be in a Raid 1 configuration. The first instruction told me to check if the system identified the disks, which I checked by again running 

cat /proc/diskstats 

At the bottom of the output were the two new disks: sdb and sdc.
The next instruction tells us to format these two disks. If you are using disks that were previously in a Raid configuration, they are already formatted, and you can skip this step. Mine were new. I formatted the first one using 

sudo fdisk /dev/sdb 

The fdisk instruction expects you to "run it" interactively. Typing in m will give you the menu. As noted in the guiding web page, first type in n to establish a logic partition (I wrote down the number of cylinders as told to do). Then I typed in t and was presented with a list of options. I selected fd as this was to be part of a Raid 1 disk. I typed w as instructed to save this information.
I repeated the formatting for the second disk, sbc. 


The guiding web page now instructed me to use mdadm, which I did have to install 

sudo apt-get install mdadm 

I then read the man pages for this instruction carefully. Doing so one realizes this package is the secret behind setting up a Raid disk system. I wonder how anyone would do it without it. In the version of the man pages I read, the instruction has the form: 

mdadm [mode] <raiddevice> [options] <component-devices> 

The mode part is really meant to be the place to put the mode selection options. In the following instruction --create is such an option.
I then ran the instruction as advised 

sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 

and expected a miracle would occur, and it would work. It did not. The error message said something about md0 not existing. Of course it did not exist. I was "creating" it. Well, back to the web and a lot of not-too-helpful "help me" messages and their responses, until I came across a response that said "use the --auto option." I had read about that and had wondered about using it but lacked the courage. The response said that the --auto option is there to break the circle of md0 not being there but the mdadm instruction not functioning if it is not yet defined. They did not suggest which value to give that option so I ran the above instruction again with --auto=yes, as follows: 

sudo mdadm --create --verbose --auto=yes /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 

and now the miracle did occur. It worked. 


Our next instruction is to restart the computer and then to wait until the disks sync. Mário suggested getting a cup of coffee. Good idea, except you can do a lot more than that in the one to two hours you will be waiting. That they are syncing will not be obvious so you should monitor it, as noted by Mario, by issuing the command 

watch cat /proc/mdstat 

After it has synced, we are now told to format md0 with the instruction (if you are reinstalling disks that were in a previous Raid 1 configuration and you do not wish to destroy their content, skip this step): 

sudo mkfs -t reiserfs /dev/md0 

Okay, so what is the option -t reiserfs all about? On to the man pages for mkfs. Ah, reiserfs is an fstype. It tell us there are lots of different types of formating from which we can choose. Questions that occur: what are the options and which should I choose? Using google again on mkfs, I found Q&A pages noting that the types are indicated by choosing among installed programs of the form mkfs.fstype so I ran 

sudo locate mkfs\. 

to see what was in my system of that form. A list popped up that included ext2, ext3, and so forth but no reiserfs. Back to google. The best page I found to give me clues as to which to choose is in the useful web pages at the start of this page. First of all one wants a type that has journaling. Journaling says that if the system powers down suddenly and not nicely, one can power up and the system can directly recover the disk without me having to ask it to fix up all the inconsistencies it is finding. Yes, I want that. ext3 and reiserfs have journaling. I finally found a statement that said feiserfs is a good choice and a newer type than ext3. So, I chose reiserfs. Now does it exists?


I ran the instruction as advised

sudo mkfs -t reiserfs /dev/md0 

and sure enough reiserfs existed and ran successfully. Okay, I could have just done what I was advised, but I felt better about knowing why. 


The next instruction is to edit the /etc/fstab file with the apology that he was not going to tell us how to do that. Okay, his instructions have been great, and I can figure this out myself. Back again to google and up pops a really excellent website on fstab (see above). It directed me to a page on mount which I also read (I read it first, as advised). I found these two pages to be to the point and very easy to follow. All of a sudden I understood the mount instruction, and fstab no longer seemed a mystery. Basically, for our needs here you can mount a partition on a disk at a mount point that is any existing directory in your system (apparently even a directory that has things in it - not tested). That directory will be the root of the mounted partition, and, while mounted, the previous content of the directory is hiding from view. md0 is a partition. sdb1 is a partition on the hard disk sdb, and so forth. 


To check out my understanding, I decided to mount md0 manually at /back (I have my reasons for this name, but they are irrelevant here). /back has to exist so I first ran 

sudo mkdir /back 

and then I mounted md0 with the instruction 

sudo mount /dev/md0 /back 

Running the instruction 

df 

listed md0 as having the mount point /back, so everything seems to have worked. Armed with this general feeling of success, I added the following line to my /etc/fstab. You need to read about fstab in the above web page to see what auto, defaults and the two zeros are all about. The first two arguments are those for running the mount instruction. 

/dev/md0        /back           auto    defaults        0       0 

fstab is a file that tells the operating system which disks to mount automatically when restarting the system. The mount instruction it will run is 

sudo mount /dev/md0 /back 

So, if you are really curious that this restarting works and if you will not spoil it for anyone else who may be on your computer at this time, you could restart your computer and then run 

df 

again. md0 should be there at the mount point called /back.

Putting things on your Raid disk

Now it up to you to put things on md0 that you wish to be on your raid 1 disk. It seems most people put /home there (which makes complete sense). You can do this easily by tarring up the contents inside /home as the superuser (you need to be, and doing so guarantees file and directory ownership stays unchanged), umounting md0, remounting md0 to the mount point called /home, and untarring the contents of /home inside this directory. 

As we are running a server with MySQL running, it made sense to grab the files in /var/lib/mysql and put them there, too. But a word of warning. DO NOT put the whole directory /var onto the Raid disk. The sudo instruction that you need to move things around as the superuser first looks in /var for information, and it also writes back in there. You will almost certainly "mv" it to put it aside for a moment, and /var will no longer exist. sudo will no longer work. Ouch!! And you need to be the superuser to "mv" it back while attempting in full panic to rescue yourself from your not-so-clever misstep. Yes, I got bitten, and I had to take my install disk for ubuntu 6.10 to the machine and use the rescue facilities on it to put /var back in place. 

Note, I did not partition the raid 1 disk so I could have several mount points. As I have not played with this option (I did not want it), I offer no comments on how to do that.

duplicating devices via dd or via nc

using dd and gzip

Backup the drive

# dd if=/dev/hda conv=sync,noerror bs=64K | gzip -c  > /mnt/sda1/hda.img.gz

To restore your system

# gunzip -c /mnt/sda1/hda.img.gz | dd of=/dev/hda conv=sync,noerror bs=64K

view partitions in the image (once uncompressed)

# fdisk -l hda.img > /mnt/sda1/hda_fdisk.info 

netcat chatserver

create the chat server on this machine and set it to listen to 3333 TCP port:

$ nc -l 3333

On the other end, connect to the server with the following:

$ nc 192.168.0.1 3333

creating a partition image and sending it to a remote machine on-the-fly:
$ dd if=/dev/hdb5 | gzip -9 | nc -l 3333

On the remote machine, connect to the server and receive the partition image with the following command:

$ nc 192.168.0.1 3333 | pv -b > myhdb5partition.img.gz

This might not be as classy as the partition backups using partimage, but it is efficient.

http://www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/ 

audio examples, and named fifo examples here:
http://www.debian-administration.org/articles/145

file transfer:
tar cz /etc | ssh user@host -c ‘cd /bak ; cat > etc.tgz’

On the localhost, this will create an archive of /etc, which is piped into an ssh connection. On the remote host, the snippet enclosed in ” is executed: Change to the backup directory, and pipe the content of the connection into a file. If I wanted to extract the files again on the receiving side, I could do:

tar cz /etc | ssh user@host -c ‘cd /bak ; cat tar xz’

Note that the ‘-f -’ given in the article is redundant. If tar is not given a file, it uses STDIN/OUT. The advantage over recursive scp is that this method preserves device nodes and file permissions. The advantage over rsync is that it is using familiar tar semantics on both sides (of course, if you want diffing, you need rsync, which works just fine over SSH).

Of course, the other direction (pulling files from a host) is equally possible:

ssh user@host -c ‘tar cz /etc’ > /bak/etc.tgz

webmin notes

How do I change my Webmin password if I can't login?
/usr/local/webmin-1.150/changepass.pl /etc/webmin admin foo 

mounting images with kpartx or losetup + fdisk

http://madduck.net/blog/2006.10.20:loop-mounting-partitions-from-a-disk-image/

Loop-mounting partitions from a disk image

Update: it seems that kpartx pretty much does all of the below. Thanks to Faidon Liambotis for the pointer.


Every now and then, I have a disk image (as produced by cat, pv, or dd) and I need to access separate partitions. Unfortunately, the patch allowing partitions on loop devices to be accessed via their own device nodes does not appear to be in the latest (Debian) 2.6.18 kernels — the loop module does not have a max_partmodinfo. parameter, according to


So this time I sat down to come up with a recipe on how to access the partitions, and after some arithmetic and much swearing at disk manufacturers, and especially the designers of the msdos partition table type, I think I have found the solution, and the urge to document it for posterity.


It’s all about the -o parameter to losetup, which specifies how many bytes into the disk a given partition starts. Getting this number isn’t straight forward. Well, it is, if you know how, which is why I am writing this.


Let’s take a look at a partition table, with sectors as units:

$ /sbin/fdisk -lu disk.img
You must set cylinders.
You can do this from the extra functions menu.

Disk disk.img: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes

      Device Boot      Start         End      Blocks   Id  System
disk.imgp1   *          63       96389       48163+  83  Linux
disk.imgp2           96390     2056319      979965   82  Linux swap / Solaris
disk.imgp3         2056320    78140159    38041920    5  Extended
disk.imgp5         2056383     3052349      497983+  83  Linux
disk.imgp6         3052413    10859939     3903763+  83  Linux
disk.imgp7        10860003    68372639    28756318+  83  Linux
disk.imgp8        68372703    76180229     3903763+  83  Linux
disk.imgp9        76180293    78140159      979933+  83  Linux

The first few lines is fdisk complaining not being able to extract the number of cylinders, since it has to operate on a file which does not provide an ioctl interface.


The first important data are the units, which are stated to be 512 bytes per sector. We take note of this value as the factor for use in the next operation.
Let’s say we want to access the 7th partition, which is 10860003 sectors into the disk, according to the fdisk output. We know that each sector is 512 bytes, so:

10860003 * 512 = 5560321536

Passing this number to losetup produces the desired result:

# losetup /dev/loop0 disk.img -o $((10860003 * 512))
# file -s /dev/loop0
/dev/loop0: Linux rev 1.0 ext3 filesystem data
# mount /dev/loop0 /mnt
[...]
# umount /mnt
# losetup -d /dev/loop0

If the partition really holds a normal filesystem, you can also let mount set up the loop device, and manage it automatically:

# mount -o loop,offset=$((10860003 * 512)) disk.img /mnt
[...]
# umount /mnt

And since there’s aparently no means to automate the whole process for an entire disk, I hacked up plosetup. Enjoy:

# plosetup lapse.hda .
I: partition 1 of lapse.hda will become ./lapse.hda_p1 (/dev/loop0)...
I: plosetup: skipping partition 2 of type 82...
I: plosetup: skipping partition 3 of type 5...
I: partition 5 of lapse.hda will become ./lapse.hda_p5 (/dev/loop1)...
I: partition 6 of lapse.hda will become ./lapse.hda_p6 (/dev/loop2)...
I: partition 7 of lapse.hda will become ./lapse.hda_p7 (/dev/loop3)...
I: partition 8 of lapse.hda will become ./lapse.hda_p8 (/dev/loop4)...
I: partition 9 of lapse.hda will become ./lapse.hda_p9 (/dev/loop5)...
# ls -l
total 0
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p1 -> /dev/loop0
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p5 -> /dev/loop1
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p6 -> /dev/loop2
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p7 -> /dev/loop3
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p8 -> /dev/loop4
lrwxrwxrwx 1 root root 10 2006-10-20 13:25 lapse.hda_p9 -> /dev/loop5
# plosetup -c .
# ls -l
total 0

Kpartx:

Kpartx can be used to set up device mappings for the partitions of any 
partitioned block device. It is part of the Linux multipath-tools. With 

kpartx -l imagefile you get an overview of the partitions in the image 

file and with kpartx -a imagefile the partitions will accessible 

via /dev/mapper/loop0pX (X is the number of the partition). 

You can mount it now with mount /dev/mapper/loop0pX /mnt/ -o loop,ro. 

After unmounting you can disconnect the mapper devices with kpartx -d 

imagefile. 

doing control alt f on linux host with vmware

If you want to send a Ctrl-Alt-Fx command to the guest, you need to use a slightly different combination: Ctrl+Alt+Space

http://blog.gnu-designs.com/solved-how-to-use-ctrl-alt-f1-in-vmware-with-linux-guests/

Set Backspace’s Firefox behavior

set the backspace to be a back page (page up) or to match Windows / Mac behaviour (more useful) to go back a page in browser history:

default is now 2


Set "browser.backspace_action" to either 0 or 1
0 - Pressing backspace button will go back a page in the session history
1 - Pressing backspace button will scroll up a page in the current document and
(other) unmap function.  so new default value of 2 disables either of the above.

set it to 0 to match the WinMac action.

about:config browser.backspace_action;0

Kernel analysis tool

Kernelshark

http://rostedt.homelinux.com/kernelshark/
analyze scheduler
http://lwn.net/Articles/425583/
tutorial:
http://elinhttp://lwn.net/Articles/425583/ux.org/images/6/64/Elc2011_rostedt.pdf