Exclude files from War packaging by Maven profile

To achieve this I have used the "packagingExcludes" property of Maven war plugin. For more info and examples of using this property see this page. To exclude files by Maven profile you need to first parameterize the value of "packagingExcludes" in war plugin declaration in pom file

     <plugin>  
          <artifactId>maven-war-plugin</artifactId>  
          <version>2.1.1</version>                      
          <configuration>  
                <packagingExcludes>${pkg.exclude}</packagingExcludes>                                     
          </configuration>  
     </plugin>  

In Maven profile you can define the value of this property by profile

            <profile>  
                  <id>prod</id>  
                  <properties>  
                       <pkg.exclude>WEB-INF/classes/com/internal/*.class,internal/*</pkg.exclude>                                                                                       
                  </properties>  
                  <activation>  
                       <property>  
                         <name>environment</name>  
                         <value>prod</value>  
                       </property>  
                  </activation>                      
            </profile>  

Mobile Browsers and Cookies

My attempt at succinctly describing first party and third party cookie support in various mobile browsers. For a good read on Cookies in general see this Wikipedia article

Browser	Default Cookie Behaviour	Default	Reference
Safari Mobile	Supports three cookie settings. "Never", "From Visited", "Always"	From Visited	http://support.apple.com/kb/HT1677
Chrome Mobile	Accept cookies: checked/unchecked. No way to enable frist part only.	Checked	Checked on my phone
Opera Mobile	Accept cookies: On /Off	On	Checked on my phone
Opera Mini on a Blackberry	Accept cookies: on/off.	On	http://www.usa.gov/optout-instructions.shtml
Android Browser	Accept cookies: Checked / Unchecked. No way to enable first part only.	Checked	http://www.usa.gov/optout-instructions.shtml
Firefox Android	Cookies: Options "Enabled", "Enabled, excluding 3rd party", "Disabled"	Enabled	Installed on my phone and checked
Internet Explorer Mobile	Allow cookies on my phone: Checked / Unchecked	Checked	http://www.windowsphone.com/en-us/how-to/wp7/web/changing-privacy-and-other-browser-settings
Blackberry Browser	Accept cookies: Checked / Unchecked	Checked	http://docs.blackberry.com/en/smartphone_users/deliverables/32004/Turn_off_cookies_in_the_browser_60_1072866_11.jsp

Elastic Beanstalk Tomcat Container Timezone

When deploying a Tomcat container webapp using AWS Elastic Beanstalk with default AMI, it defaults the EC2 instances to UTC/GMT timezone. To set a different timezone the hard way is to create your own custom AMI with the required timezone. If you are only running the Tomcat container on EC2 instance then an easier way to change timezone is to specify is as a JVM command line option. To change the configuration

1. Login to your AWS console
2. Go to "Elastic Beanstalk" service
3. Under your environment click on "Edit Configurations"
4. That should popup a screen and the last tab in the screen will say "Container"
5. Set the required timezone as shown in below screenshot. Java supports these timezones

DOJO Combobox On Enter Key Event

Below code can be used to setup a event to fire when a enter key is pressed inside combobox.

1:  require([  
2:  "dojo/ready", "dojo/store/JsonRest", "dijit/form/ComboBox", "dojox/storage", "dojo/request", "dojo/json"  
3:  ], function(ready, JsonRest, ComboBox, storage, request, json)  
4:    
5:  {   
6:      var comboBox = new ComboBox(  
7:      {  
8:         id: "IdCombo",  
9:         name: "NameCombo",  
10:         placeholder: "Enter text",     
11:      }, "placesomewhereid");  
12:    
13:      comboBox.startup();     
14:        
15:      require(["dojo/on", "dijit/focus", "dojo/keys", "dojo/domReady!"], function(on, focusUtil, keys)  
16:      {  
17:          on(document, "keyup", function(event)  
18:          {  
19:             if(event.keyCode == keys.ENTER && focusUtil.curNode.id == "IdCombo")  
20:             {  
21:                 //Write code here that needs to run when enter key is pressed  
22:             }        
23:          });  
24:      });  
25:  });  

Creating autocomplete text box in DOJO

Using DOJO combobox, you can create a autocomplete functionality. Below code creates a programmatic combobox and assigns a json store to combobox. A declarative input form element is used a placeholder on page.

1:  <script type="text/javascript">   
2:  require([  
3:  "dojo/ready", "dojo/store/JsonRest", "dijit/form/ComboBox"], function(ready, JsonRest, ComboBox)  
4:  {    
5:    var jsonStore = new JsonRest  
6:     ({  
7:        target: "URL for Ajax service"  
8:     });  
9:     var comboBox = new ComboBox(  
10:     {  
11:        id: "SomeUniqueID",  
12:        name: "SomeUniqueName",  
13:        value: "",  
14:        store: jsonStore,  
15:        pageSize: 3,  
16:        searchAttr: "key",  
17:        queryExpr: "${0}"  
18:     }, "StoreItemSelect");  
19:    comboBox.startup();   
20:  });   
21:  </script>  
22:  <body>  
23:     <input id="StoreItemSelect"/>  
24:  <body>  

The AJAX service should return items in below json format

 [{key: "Autocomplete value 1"},{key: "Autocomplete value 2"},{key: "Autocomplete value 3"}]  

Dojo TabContainer Tab select ContentPane reload

The usual way to implement a TabContainer is to define a ContentPane as tab childs. When a tab is selected and if you want the ContentPane to reload everytime the tab is selected then you need to specify the refreshOnShow property of ContentPane to true.

 var tc = new TabContainer(  
 {  
     style: "height: 100%; width: 100%;"  
 }, "tc1-prog");  
  var cp2 = new ContentPane(  
 {  
     title: "First Tab",  
     href: "page1.html",      
  });  
  tc.addChild(cp2);  
  var cp2 = new ContentPane({  
     title: "Second Tab",  
     href: "page2.html",   
     refreshOnShow: true     
  });  
  tc.addChild(cp2);    

In this case when "Second Tab" is selected, it refreshes every time. Refresh basically calls the dojo ready function on the page same as when loading the page using F5. "First tab" on the other hand will only load the first time its selected and will not refresh in later selects

Using dojo.place to add row to TableContainer

-- Assume you have a table container defined declaratively

                                                     

                                                       
 

To add a new row to this table container pro grammatically using dojo.place

dojo.place('

',"tb","first");

Why E-Commerce Applications Should Embrace Cloud?

Today is day before Thanksgiving and Express (a clothing brand) just sent an email to all its members about its exciting sale offer (50% of entire items). This is their once chance of the year to bring in shoppers to make good revenue. If only they could keep the site up..... If I was part of their engineering lead,  I would bring all engineers to the drawing table and rethink scalability and availability from ground up.

Android: Set up automated backups

I am great fan of rsync backup Android app and use it a lot to sync files from/to Android with my home server. Combined with the Tasker app its a great way to automate the sync on a daily/weekly basis. This blog post lists the step by step instructions on how to setup rsync backup and Tasker to create automated backups.

A smarter way to calculate distinct counts

I was recently writing a pig script to calculate distinct count over three fields on a big set of data and was getting an out of memory error on the reducer. The data types of these three fields are strings and The issue was the single reducer usage to calculate the distinct count. I couldn't figure out a way around the single reducer and instead used the below approach

Approach throwing the OOM error

A = LOAD '$inp' using PigStorage('\t');

H = FOREACH A GENERATE $1,$3,$23;

uq_pid = DISTINCT H parallel 20;

guq_pid = GROUP uq_pid ALL;

itr_uq_pid = foreach guq_pid {

    generate COUNT_STAR(uq_pid);

}

store itr_uq_pid into '$otp/uq_metric';

Modified approach using a constant

A = LOAD '$inp' using PigStorage('\t');

pid = FOREACH A GENERATE $1,$3,$23,1;

uq_pid = DISTINCT pid parallel 20;

constant_uq_pid = FOREACH uq_pid GENERATE $3;

guq_pid = GROUP constant_uq_pid BY $0;

itr_uq_pid = foreach guq_pid {

    generate COUNT_STAR(constant_uq_pid);

}

store itr_uq_pid into '$otp/uq_metric';

Get latest successful build number from Bamboo

This little script gets the latest successful build number.

SAVEIFS=$IFS

IFS=$'\n'

for ln in `wget -O- --quiet --http-user="{user name}" --http-password="{user password}" 'http://{bamboo site url}/rest/api/latest/build/{plan key}?os_authType=basic' | tr '<' '\n'`

do

  build_number=`echo $ln | perl -n -e 'm/number="([\s\S]+?)" lifeCycleState="Finished" state="Successful"/ && {print "$1\n"}'`

  if [ $build_number ] 

  then

     break

  fi

done

IFS=$SAVEIFS

echo "Latest successful build number $build_number"

Once you have the build number, you can use that to get latest code coverage, artifacts etc. Get the url for the intended resource and substitute with the $build_number variable above

Ubuntu 11.10 amixer issue

I have previously used the amixer libraries in Ubuntu to programmatically control volume. I basically built a web UI for my media which has options to control volume. With the recent 11.10 (Oneiric) upgrade, the setup stopped working. There are known error cases where unmute does not work and the system does not support a global volume up and down functionality. Instead now you have to control both the Master and PCM to get the desired outcome. Below are the commands that I have ended up with. I have set hot keys that run each of these. The below is a generalized command that can be used with various options

ll={value};amixer set Master unmute;amixer set PCM unmute;amixer set PCM

$ll;amixer set Master 100;

value can be

mute - to mute the volume

numeric value - you can specify a number like 20 to set the volume to 20. On my box the max is set to 60

JSPC Maven Plugin Package Name

Specifying package name when using the JSPC Maven plugin to pre-compile jsps

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>jspc-maven-plugin</artifactId>
    <configuration>
     <packageName>com.company.package.name</packageName>
    </configuration>
    <executions>
     <execution>
      <id>jspc</id>
      <goals>
       <goal>compile</goal>
      </goals>
     </execution>
    </executions>
   </plugin>

Amazon AWS Service Status

AWS currently does not provide an API to query service status. They only provide a health dashboard webpage. Although the page has RSS feed capability, I couldn't detect any specific pattern to watch for. This simple solution below works for now. We have created nagios monitoring out of these one liners

status0 = OK
status2 = Performance Issues
status3 = ERROR

The service and region name below is literally as is from the web page. 

wget -qO- "http://status.aws.amazon.com/" | grep -B 1 "Amazon Elastic Compute Cloud (N. California)" | head -n 1 | 
perl -n -e 'm/images\/([\s\S]+?)\.gif/ && {print "$1\n"}'

status0

wget -qO- "http://status.aws.amazon.com/" | grep -B 1 "Amazon Elastic Compute Cloud (N. Virginia)" | head -n 1 | 
perl -n -e 'm/images\/([\s\S]+?)\.gif/ && {print "$1\n"}'

status0

Rooted my Motorola Atrix

System version: 4.5.91
Android Version: 2.3.4

Instructions found below:

http://www.digitaltweaker.com/android/phones/2011/07/root-motorola-atrix-4-5-91-on-gingerbread-2-3-4/

If you get the "device not found" error when running "adb shell" then make sure you have sync drivers installed. When you connect Atrix to your computer the first time it installs a bunch of drivers and also gives an option to install "synch drivers". Make sure to install the "sync drivers".

Repository for JBoss RestEasy Releases

We have been using JBoss RestEasy for developing restful services. The framework is new and we have had to change the repo many times with various version releases and often times the repos fail with permission denied error or dependencies resolution error. A more stable repo for RestEasy is being maintained by the Atlassian folks. We use the below repo and its been working consistently and updates with new releases fairly quickly

https://maven.atlassian.com/content/repositories/jboss-releases

Check Logged Person In Campfire Room

We use campfire as a sole medium for team communication so its essential every team member is logged in so they are accessible. All team members use Mac and we use the Fluid app to run Campfire as a app. The app logs out automatically sometimes when in not use and we needed a way to get alerted when member was not logged into campfire.

Campfire has a pretty nifty API that allows to query many room attributes. Below script can be used to detect if a person is not logged in. Each of our team member has the below script cronned in their machines to run every 10mins or so to get alerted when they have been logged out. below are the attributes you need to substitute

{authcode here} - To get your auth code in Campfire go to "Edit my Campfire account" or "My Info" at the top screen in Campfire and there you will see your auth code. This is a secret code and should not be shared with others so be watchful and ensure the below script only has read/execute perms for the member user on the box.

"#{id}" - To get the room ID simply login to Campfire and click on the room. The browser url should show the room ID.

{member name} - Should be the same as the name specified when creating campfire account

#!/bin/bash
curl -s -u {authcode here}:X "https://{your specific domain}.campfirenow.com/room/#{id}.xml" > /tmp/campfire-room.xml
present=`grep "{member name}" /tmp/campfire-room.xml | wc -l`
if [ $present -eq 0 ]
  then
   `/usr/local/bin/growlnotify --name "Not Logged In Campfire" -s --message "You are not logged into campfire. 
This might reflect in your performance review next year so you better login now :)"`
fi

Transitioning Traditional Data Support Folks To New Breed ETL Systems

Like many other folks out there who are discovering the power of data compute grids, we recently transitioned a part of our traditional database based ETL system to Hadoop based processing system. Being in the digital advertising field, we get a whole lot of impression data both from our tracking systems and external. Our existing ETL system is mainly comprised of four components: cleansing, standardization, dimensioning, aggregation.

The cleansing and standardization components that involve a whole lot of text parsing and mapping take the major brunt as they deal with the raw volume of incoming data. As part of the transition we moved these two components to the new system. We have a dedicated product support team that handles most of the daily user data queries. Issues like missing/incorrect data, re-running jobs due to changed dimensions or external data outages, configuring and testing new data fields/sources, generating ad-hoc reports, configuring new clients etc. These folks have a thorough domain knowledge and are well versed with this data. They also fully understand the current ETL data flow and the various business rules thats gets applied as part of the data processing. Technology wise they are comfortable with databases, SQL, basic scripting, Excel and are usually enthusiastic about learning new technologies as need be.

To be able to perform the above mentioned issues, they essentially need a way to slice and dice raw/stage level data and with the data residing in HDFS this becomes an issue. We have been brainstorming ways on how to expose the new system to support folks and below are some options.

Apache Hive: Facebook was the first company to encounter this problem wherein they transitioned their analysts folks from a RDBMS based warehouse to a more scalable hadoop system. They developed Hive which is essentially a data warehouse system for Hadoop that facilitates easy data summarization, ad-hoc queries, and the analysis of large datasets stored in Hadoop compatible file systems. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. HiveQL is very similar to SQL although it does not support the full SQL-92 specification. As per my reading it more closely resembles MySql's SQL dialect and that makes sense because Facebook is a mysql shop and a similarity would make the transition easy for its folks. This option stands top in our list of possibilities given its similarity to SQL.

Apache Pig: Apache Pig is also a higher level abstraction for map/reduce. Pig uses Pig Latin language to express data flows. Although this is a powerful tool, it would require the support folks to learn a entirely new language.

Commercial tools like Datameer, IBM Big Sheets: It is a well know fact that Microsoft Excel is most versatile analytic tool. Analysts love the ease of use and the tools it provides to slice and dice/graph datasets. Imagine an Excel like tool with the power of Hadoop. Its essentially what these commercial tools are. We recently received a great demo from the Datameer folks and were impressed by its ease of use and especially its pluggable architecture. Easy and familiar spreadsheet-like interface for business users with complete set of data integration, transformation/analytic and visualization tools. It also has a neat scheduler for cron based job scheduling. This option is also a strong contender in our option set given its ease of use and spreadsheet like usability and feel.

We haven't decided on an option yet. The next couple of weeks will involve closely working with support folks to evaluate these options and ensuring a smooth transition.

Maven3 error "Could not find artifact"

We recently switched our artifactory repo to a new server and things were fine with build running fine. One of the team dev deployed a new artifact version to the repo and that caused the build to fail. We kept getting the below error on our continuous integration server while the build was working fine on all dev boxes

"Failed to execute goal on project... Could not resolve dependencies for project....SNAPSHOT: The following artifacts could not be resolved: {new artifact}. Failure to find ... in  http://localhost/artifactory/repo was cached in the local repository,  resolution will not be reattempted until the update interval of  artifactory has elapsed or updates are forced"

We debugged this issue for a while and made sure the artifact was deployed correctly to the repo and all that but the main issue was that in the error message the artifact location was wrong. It was looking in the localhost. We checked the POM and ensured the repositories link specified were correct. After quite a bit of head banging we leaned towards a issue with local cache of repo and decided to purge the local repository. We deleted the .m2/repository/artifact directory and that resolved the above error but gave the below error

"Failed to execute goal on project... Could not resolve dependencies for project....SNAPSHOT: The following artifacts could not be resolved: {new artifact} Could not find artifact in artifactory (http://localhost/artifactory/repo) "

We were still looking to find as to why it was not pointing to the correct repo location. The fact that it was working fine on all all places made us believe that there was a specific issue with the continuous integration box so we decided to specify a override on the local repo location so it downloads fresh a copy of all dependencies and thats when we saw a default mirror repo location specified in the local m2 settings.xml file. The file had the below properties set

<mirrors>
    <mirror>
        <id>artifactory</id>
        <mirrorOf>*</mirrorOf>
        <name>Artifactory</name>
        <url>http://localhost/artifactory/repo</url>
    </mirror>
</mirrors>

That was causing the issue. We modified this to point to the new repo location and boom the error was gone

Mac OSX Disable gconsync

I recently enabled synching with google contacts in ITunes while synching my IPhone. Ever since then I have been getting this nagging pop up where a program called "gconsync" keeps asking me password to access keychain. On my iphone I have now setup google account as exchange account and that automatically synchs my contacts with google so I no longer need to enable the google account synching with itunes but even after disabling that I still kept getting the gconsync popup for keychain access. Here's the way to disable this

Open the Address book app on Mac: /Applications/Address Book.app

Go to Preferences -> Account -> uncheck the "synchronize with Google" option

That should get rid of the popup.