Know Your Technology

MapReduce Eclipse plugin not pointing to local file system rather hdfs. On your first run of WordCount from eclipse -> run on hadoop you may notice the java program uses local file system instead hdfs. Adding hdfs location in your Configuration object solves this issue. In case of WordCount.java add the below into your main method. public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); conf.addResource(new Path(HADOOP_INSTALL+"/conf/core-site.xml")); conf.addResource(new Path(HADOOP_INSTALL+"/conf/hdfs-site.xml")); }

Hadoop temp direcotry issue  While running hadoop in psudo-distributed mode  you may see the error 'directory /tmp/hadoop-{user}/dfs/name is inconsistent state'. This happens because each system shutdown cleans up unwanted temp folders and files from /tmp folder.So pointing your hadoop dfs location other than default /tmp folder will solve this issue. You can edit onf/hdfs-site.xml and set dfs location specific to your box <property>   <name>dfs.name.dir</name>   <value>/hadoopstorage/name/</value> </property> <property>   <name>dfs.data.dir</name>   <value>/hadoopstorage/data/</value> </property> This will remove namenode, datanode errors. But you may get errors related jobtracker and other process still the most other files pointing to the default /tmp directory. The default value for all of the hadoop related directories will be like below. dfs.name.dir -...

Java 7 New IO API - A program to Delete Directory including sub directories and files. /**  * Java program to delete folder.  */ package com.hadoop.test; import java.io.IOException; import java.nio.file.FileVisitResult; import java.nio.file.Files; import java.nio.file.Paths; import java.nio.file.SimpleFileVisitor; import java.nio.file.attribute.BasicFileAttributes; /**  * @author Bala  *  */ public class FileUtil { private static boolean deleteFolder(String location) throws IOException{ java.nio.file.Path path = Paths.get(location); Files.walkFileTree(path, new SimpleFileVisitor<java.nio.file.Path>(){ @Override public FileVisitResult postVisitDirectory(java.nio.file.Path dir, IOException exc) throws IOException { Files.delete(dir); return FileVisitResult.CONTINUE; } @Override public FileVisitResult visitFile(java.nio.file.Path file, BasicFileAttri...

Ubuntu cursor jumping problem Issue : When you type in Ubuntu/Kubuntu you may get annoyed by cursor jumping here and there in the screen. This happens across all the applications. You might have tried adjusting key board settings, key stroke speed etc. But this is not related to your key board, but the touch pad! Cause: The cause of this issue is your touch pad. Yes, while typing parts of your hand/fingers often touch the keyboard. In windows this touch will not be deducted while typing. But for Ubuntu it is.  So when you type something and un-intentionally unknowingly touch your touch pad will take your cursor to the appropriate mouse position with respect to your touch forgetting you are typing something!!! Solution: Simple. Say your touch pad not to care the touches while I am eagerly typing something important.. How to say that? Open Synaptix Touch pad manager  Check Automatically switch of touch pad on keyboard activity Time to wait before switc...