git cherry-pick

This blog provides the most basic (and probably the most useful) usage of git cherry-pick:
https://ariejan.net/2010/06/10/cherry-picking-specific-commits-from-another-branch/

Details of git cherry-pick can be found here:
http://git-scm.com/docs/git-cherry-pick

Doxygen

To use Doxygen to document your source code on Ubuntu, first install doxygen and graphviz:

$ sudo apt-get install doxygen

$ sudo apt-get install graphviz

To generate a doxygen configuration file:

$ doxygen -g # this will generate a configuration file named Doxyfile

Then, edit Doxyfile. In my case, I only modified:

OUTPUT_DIRECTORY = doc

HAVE_DOT = YES

CALL_GRAPH = YES

CALLER_GRAPH = YES

Then run

$ doxygen Doxyfile

Doxygen will automatically find your source code if Doxyfile is placed in the same directory. The resulting documentations are in doc.

For more detail:

http://www.stack.nl/~dimitri/doxygen/index.html

Setting up the Data Serving benchmark (YCSB) from CloudSuite

(This post is mostly taken from http://parsa.epfl.ch/cloudsuite/dataserving.html)

Assuming you’re running on Ubuntu. We start with installing Oracle Java 6

$ sudo add-apt-repository ppa:webupd8team/java

$ sudo apt-get update

$ sudo apt-get install oracle-java6-installer

Alternatively, install Java 1.6.0_23 (1.6.023 were tested by the CloudSuite authors):
http://diegobenna.blogspot.com/2011/01/install-jdk-6-update-21-in-ubuntu-1010.html

Note that although later Java versions are also available, extra effort might be required to compile/run Cassandra and YCSB if not using Java 6.

Make sure to set your $JAVA_HOME. This can be done by

$ which java

In my case, the path is /usr/bin/java, so I did

$ export JAVA_HOME=/usr

Next, install Apache Ant

$ sudo apt-get install ant

Ant (another neat tool), is a tool for automating build, similar to Make (Ant:Java=Make:C/C++)

The next step is to download and install YCSB. The package can be downloaded from here:

http://parsa.epfl.ch/cloudsuite/software/dataserving.tar.gz

Although you can also clone YCSB using git, the one on github doesn’t seem to have the appropriate build.xml (build.xml is required for Ant). To make things easy, I use the tarball.

Follow the steps from here to build YCSB:

http://parsa.epfl.ch/cloudsuite/dataserving.html

While building, keep in mind that if you compile and run a program under different JDKs, an error will show up saying “Unsupported major.minor version”. So just remember to use a Java version consistently.

You’ll also need to download and install Cassandra. It can be found here:

http://archive.apache.org/dist/cassandra/0.7.3/apache-cassandra-0.7.3-bin.tar.gz

Apache Cassandra is basically a database management system. YCSB uses Cassandra to populate and operate on data systems.

Similarly, although there are also later versions of Cassandra, they are not always compatible with the Java/Ant/YCSB version you’re using. For instance, there’ll be a stack size problem when starting Cassandra if the Cassandra version and the Java version are not compatible. So again, to make things easy, I use the tarball instead of installing it via Ubuntu deb.

To install and run Cassandra, follow the steps in README inside the Cassandra directory. Alternatively, there are some instructions here:

http://parsa.epfl.ch/cloudsuite/dataserving.html

Now you should be ready to generate a dataset. To do so

$ cd <path to cassandra>

$ ./bin/cassandra –f # this will run Cassandra

Open another terminal

$ cd <path to cassandra>

$ ./bin/cassandra-cli -host localhost # this will invoke the cli utility, which is connected your local-running Cassandra

Upon entering the cassandra-cli prompt, use the commands provided in http://parsa.epfl.ch/cloudsuite/dataserving.html

Note that different versions of Cassandra use slightly different commands. In this example, we're using 0.73.

Then, populate the database

$ cd <path to ycsb>

$ ./run_load.command # modify this shell script and settings_load.dat accordingly

Finally, run the benchmark

$ ./run.command # modify this shell script and settings.dat accordingly

Please visit the CloudSuite and YCSB website for detailed information

http://parsa.epfl.ch/cloudsuite/cloudsuite.html

https://github.com/brianfrankcooper/YCSB/wiki

And also

http://www.webupd8.org/

http://ant.apache.org/

http://cassandra.apache.org/

Adding programs to disk images

(This post is mostly taken from http://marss86.org/~marss86/index.php/FAQ)

First create a loopback device and add partition mappings
$ sudo kpartx -a your.img

Then mount the loopback device
$ sudo mount -o loop /dev/mapper/loop0p1 mnt/

In most cases the loopback device is loop01, but it could be something else, such as loop1p1.
Also, mnt/ can be any directory. This is just an example.

After mounting the device, you can access the disk image via mnt/.

When finish with accessing the image, unmount it and delete the loopback device
$ sudo umount mnt/
$ sudo kpartx -d your.img

Resizing a disk image for MARSS (QEMU)

Assuming you are running on Ubuntu with “qemu”, “qemu-system”, and “gparted” installed. 
(If you haven’t: $ sudo apt-get install qemu qemu-system gparted)

Also assume you have already downloaded an image with an OS installed.
(Download disk image from MARSS: http://marss86.org/~marss86/index.php/Disk_Images)

To increase the size of a raw disk image, first create an additional image
$ qemu-img create –f raw additional.img 10G
additional.img is the file name you want to create, and 10G is the size for it.

After you get the additional image, append it to your existing image
$ cat additional.img >> exisiting.img

For example, if the original exisiting.img is 5G, then the new exisiting.img is 15G. You can check the size with $ ls -lh

To have the new existing.img properly partitioned (from now on I’ll just refer new exisiting.img as exisiting.img), we need to use gparted to partition it.

First associate exisiting.img to a loop device by
$ sudo losetup -f # this will return the path to a free loopback device, most likely /dev/loop0
$ sudo losetup /dev/loop0 existing.img # this will make /dev/loop0 a representation of existing.img
$ sudo partprobe /dev/loop0 # this will ask the kernel to load the partitions that are already on the image

Then open gparted to do the magic
$ sudo gparted /dev/loop0

A GUI window will pop out, and resizing the disk image should be intuitive. 
(Here is an example: http://softwarebakery.com/shrinking-images-on-linux)

After the image is successfully resized, close gparted, and unload the loop device
$ sudo losetup –d /dev/loop0

You can now check whether the image has the desired size
$ qemu-img info exisiting.img
Both the virtual and actual disk size should be 15G in this example.

You should also check if the disk image properly boots up
$ qemu-system-x86_64 existing.img

Once boot up, inside QEMU you should also see that the disk size is 15G in this example
$ df -h 

Now you have a resized image. You can convert it to qcow2 format if necessary.

Reference:
- http://michael.orlitzky.com/articles/resizing_a_kvm_or_qemu_disk_image.php
- http://softwarebakery.com/shrinking-images-on-linux