Time based tests with joda time

In unit tests it is helpful to set a fixed date and time. With joda time you can do this by simply calling

DateTimeUtils.setCurrentMillisFixed(new DateTime(2013, 3, 2, 0, 0).getMillis());

How to launch eclipse with a specific jdk on mac os

Edit /Eclipse.app/Contents/MacOS/eclipse.ini. Add the following before -vmargs:

-vm 
/path/to/java/home

The linebreak after -vm matters! To find out the path to java home run

/usr/libexec/java_home
or 
/usr/libexec/java_home -v 1.7

How to install the sun 6 / oracle 7 jdk on ubuntu

https://github.com/flexiondotorg/oab-java6 provides a script for building a java package for sun jdk 6 and oracle jdk 7 for ubuntu.

wget https://github.com/flexiondotorg/oab-java6/raw/0.3.0/oab-java.sh -O oab-java.sh
chmod +x oab-java.sh
sudo ./oab-java.sh

sudo apt-get install sun-java6-jdk

How to cache your git credentials

The following command tells git to cache your password for 5 minutes:

git config --global credential.helper 'cache --timeout=300'

See also http://git-scm.com/docs/git-credential-cache.


https://confluence.atlassian.com/display/STASH/Permanently+authenticating+with+Git+repositories

OS X

Follow these steps to use Git with credential caching on OS X:

1. Download the binary git-credential-osxkeychain.
2. Run the command below to ensure the binary is executable:
   

   chmod a+x git-credential-osxkeychain

3. Put it in the directory /usr/local/bin.
4. Run the command below:
   

   git config --global credential.helper osxkeychain

Use avro-tools to inspect avro files in hdfs

Since version 1.7.5 avro-tools include support for hdfs tools. To read an avro file in hdfs use the following:

hadoop jar avro-tools-1.7.5.jar  tojson hdfs://<hostname>:/path/to/file.avro | less

Working with data

A very important insight from http://radar.oreilly.com/2012/07/data-jujitsu.html I want to share with you:

 It’s impossible to overstress this: 80% of the work in any data project is in cleaning the data.

How to check the integrity of a bunch of gzip files

gunzip -t *.gz

will report broken gzip archives without unpacking them.