Tuesday, November 20, 2012

Why E-Commerce Applications Should Embrace Cloud?

Today is day before Thanksgiving and Express (a clothing brand) just sent an email to all its members about its exciting sale offer (50% of entire items). This is their once chance of the year to bring in shoppers to make good revenue. If only they could keep the site up..... If I was part of their engineering lead,  I would bring all engineers to the drawing table and rethink scalability and availability from ground up.

Saturday, July 21, 2012

Android: Set up automated backups

I am great fan of rsync backup Android app and use it a lot to sync files from/to Android with my home server. Combined with the Tasker app its a great way to automate the sync on a daily/weekly basis. This blog post lists the step by step instructions on how to setup rsync backup and Tasker to create automated backups.

Sunday, June 03, 2012

A smarter way to calculate distinct counts

I was recently writing a pig script to calculate distinct count over three fields on a big set of data and was getting an out of memory error on the reducer. The data types of these three fields are strings and The issue was the single reducer usage to calculate the distinct count. I couldn't figure out a way around the single reducer and instead used the below approach

Approach throwing the OOM error

A = LOAD '$inp' using PigStorage('\t');
uq_pid = DISTINCT H parallel 20;
guq_pid = GROUP uq_pid ALL;
itr_uq_pid = foreach guq_pid {
    generate COUNT_STAR(uq_pid);
store itr_uq_pid into '$otp/uq_metric';

Modified approach using a constant

A = LOAD '$inp' using PigStorage('\t');
pid = FOREACH A GENERATE $1,$3,$23,1;
uq_pid = DISTINCT pid parallel 20;
constant_uq_pid = FOREACH uq_pid GENERATE $3;
guq_pid = GROUP constant_uq_pid BY $0;
itr_uq_pid = foreach guq_pid {
    generate COUNT_STAR(constant_uq_pid);
store itr_uq_pid into '$otp/uq_metric';

Thursday, May 17, 2012

Get latest successful build number from Bamboo

This little script gets the latest successful build number.

for ln in `wget -O- --quiet --http-user="{user name}" --http-password="{user password}" 'http://{bamboo site url}/rest/api/latest/build/{plan key}?os_authType=basic' | tr '<' '\n'`
  build_number=`echo $ln | perl -n -e 'm/number="([\s\S]+?)" lifeCycleState="Finished" state="Successful"/ && {print "$1\n"}'`
  if [ $build_number ] 
echo "Latest successful build number $build_number"

Once you have the build number, you can use that to get latest code coverage, artifacts etc. Get the url for the intended resource and substitute with the $build_number variable above

Saturday, April 28, 2012

Ubuntu 11.10 amixer issue

I have previously used the amixer libraries in Ubuntu to programmatically control volume. I basically built a web UI for my media which has options to control volume. With the recent 11.10 (Oneiric) upgrade, the setup stopped working. There are known error cases where unmute does not work and the system does not support a global volume up and down functionality. Instead now you have to control both the Master and PCM to get the desired outcome. Below are the commands that I have ended up with. I have set hot keys that run each of these. The below is a generalized command that can be used with various options

ll={value};amixer set Master unmute;amixer set PCM unmute;amixer set PCM
$ll;amixer set Master 100;

value can be
mute - to mute the volume
numeric value - you can specify a number like 20 to set the volume to 20. On my box the max is set to 60