Know Your Technology

Log4j multiple WAR files in single EAR configuration Problem: When you have multiple WAR files under the same EAR with multiple log4j config files in each of the war files, the logging API will pickup the first available config file and ignores the list. i.e, only a single log4j config file applies to entire EAR. Consider the below app app.ear |__admin.war    |__WEB-INF/classes/log4.xml |__web.war    |__WEB-INF/classes/log4j.xml When you deploy this app to the server all logs of two war files admin.war and web.war will go to admin.war' log. The log4j.xml in web.war has no effect here. There are some complex solutions for this like adding a logback, specifying package level log files in log4j configuration etc. Solution The easiest way is configuring the context param in web.xml In your web.xml add the following context param will send the log4j config locations specific to the contexts and fixes this issue. web.war/WEB-INF/classes/log4j.xml <conte

File Watch Service The NIO2 WatchService API provides the ability for java code to watch for file modifications in a directory in the filesystem. A WatchService object watches the folder it is registered with and signals the changes detected through event notifications.  Steps to create a WatchService Create a WatchService object (FileSystem.newWatchService()) Register the directories to be watched with the WatchService (WatchService.register()) Watch for file change events (poll()/take() methods) Retrieve the event keys and process them Rest the key and got to step 3 to watch again. Example code Below is a simple working code. import java.io.IOException; import java.nio.file.Path; import java.nio.file.Paths; import java.nio.file.StandardWatchEventKinds; import java.nio.file.WatchEvent; import java.nio.file.WatchKey; import java.nio.file.WatchService; import java.util.List; import java.util.concurrent.TimeUnit; public class FileWatcherTest { private static

Forging a stack trace in java The java.lang.Throwable.setStackTrace() method sets the stack trace elements that will be returned by getStackTrace() and printed by printStackTrace() and related methods. The signature of the method is below. public void setStackTrace(StackTraceElement[] stackTrace) This method can be used to modify the actual stack trace. When do you want to modify the original stack trace? There are certain scenarios when the application developer decides to append, prepend or curtail the original stack trace. Java RMI implementation constructs a stack trace by prepending the remote stack trace. Some libraries merge the stack traces from different sources which are all can be a reason for current stack trace. Sometimes you do not want the larger stack trace information to go into logs.   For example an application which handles validation errors by means of java exception chaining will log the validation error and print the stack trac

MapReduce Eclipse plugin not pointing to local file system rather hdfs. On your first run of WordCount from eclipse -> run on hadoop you may notice the java program uses local file system instead hdfs. Adding hdfs location in your Configuration object solves this issue. In case of WordCount.java add the below into your main method. public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); conf.addResource(new Path(HADOOP_INSTALL+"/conf/core-site.xml")); conf.addResource(new Path(HADOOP_INSTALL+"/conf/hdfs-site.xml")); }

Hadoop temp direcotry issue  While running hadoop in psudo-distributed mode  you may see the error 'directory /tmp/hadoop-{user}/dfs/name is inconsistent state'. This happens because each system shutdown cleans up unwanted temp folders and files from /tmp folder.So pointing your hadoop dfs location other than default /tmp folder will solve this issue. You can edit onf/hdfs-site.xml and set dfs location specific to your box <property>   <name>dfs.name.dir</name>   <value>/hadoopstorage/name/</value> </property> <property>   <name>dfs.data.dir</name>   <value>/hadoopstorage/data/</value> </property> This will remove namenode, datanode errors. But you may get errors related jobtracker and other process still the most other files pointing to the default /tmp directory. The default value for all of the hadoop related directories will be like below. dfs.name.dir -> ${hadoop.tmp.dir}

Java 7 New IO API - A program to Delete Directory including sub directories and files. /**  * Java program to delete folder.  */ package com.hadoop.test; import java.io.IOException; import java.nio.file.FileVisitResult; import java.nio.file.Files; import java.nio.file.Paths; import java.nio.file.SimpleFileVisitor; import java.nio.file.attribute.BasicFileAttributes; /**  * @author Bala  *  */ public class FileUtil { private static boolean deleteFolder(String location) throws IOException{ java.nio.file.Path path = Paths.get(location); Files.walkFileTree(path, new SimpleFileVisitor<java.nio.file.Path>(){ @Override public FileVisitResult postVisitDirectory(java.nio.file.Path dir, IOException exc) throws IOException { Files.delete(dir); return FileVisitResult.CONTINUE; } @Override public FileVisitResult visitFile(java.nio.file.Path file, BasicFileAttributes attrs) throws IOEx