Hadoop setJarByClass not working -


my wordcount example following structure:

public class wordcount extends configured implements tool {      public static class map extends             mapper<longwritable, text, text, intwritable> {}      public static class reduce extends             reducer<text, intwritable, text, intwritable> {}      public static void main(string[] args) throws exception {         basicconfigurator.configure();         logger.getrootlogger().setlevel(level.warn);         int res = toolrunner.run(new configuration(), new wordcount(), args);         system.exit(res);     }      @override     public int run(string[] args) throws exception {          configuration conf = new configuration();         filesystem fs = filesystem.get(conf);          if (fs.exists(new path(args[1]))) {             fs.delete(new path(args[1]), true);         }          job job = job.getinstance(conf, "wordcount");         long starttime = system.currenttimemillis();         job.setoutputkeyclass(text.class);         job.setoutputvalueclass(intwritable.class);          job.setmapperclass(map.class);         job.setreducerclass(reduce.class);          job.setinputformatclass(textinputformat.class);         job.setoutputformatclass(textoutputformat.class);          fileinputformat.addinputpath(job, new path(args[0]));         fileoutputformat.setoutputpath(job, new path(args[1]));         job.setjarbyclass(wordcount.class); //      job.setjar(wordcount.class.getsimplename());         job.waitforcompletion(true);         system.out.println("job finished in "                 + (system.currenttimemillis() - starttime) / 1000.0                 + " seconds");         return 0;     }  } 

the job.setjarbyclass() call not working, , "no job jar file set" message. also, job.getjar() after call shows "null" value. knows what's problem here?

i tried job.setjarbyclass(this.getclass()), job.setjar("wordcount") , job.setjar(wordcount.class.getsimplename()). first 1 has no effect, job.getjar() returns null, second , third both give me filenotfoundexception: file wordcount not exist. tried job.setjar("src/wordcount/wordcount.java") , job.setjar("bin/wordcount/wordcount.class"), both succeed within eclipse (without warning message), still fail filenotfoundexception when executed standalone jar file on command line. guess problem may relate class path setting if not unresolved dependencies.

think should add appropriate jar files.

in case must have jar org.apache.hadoop.mapreduce.job in project file.

i imported following classes , interfaces

import org.apache.hadoop.conf.configuration; import org.apache.hadoop.conf.configured; import org.apache.hadoop.fs.filesystem; import org.apache.hadoop.fs.path; import org.apache.hadoop.io.intwritable; import org.apache.hadoop.io.longwritable; import org.apache.hadoop.io.text; import org.apache.hadoop.mapreduce.job; import org.apache.hadoop.mapreduce.mapper; import org.apache.hadoop.mapreduce.mapper.context; import org.apache.hadoop.mapreduce.reducer; import org.apache.hadoop.mapreduce.lib.input.fileinputformat; import org.apache.hadoop.mapreduce.lib.input.textinputformat; import org.apache.hadoop.mapreduce.lib.output.fileoutputformat; import org.apache.hadoop.mapreduce.lib.output.textoutputformat; import org.apache.hadoop.util.tool; import org.apache.hadoop.util.toolrunner; import org.apache.log4j.basicconfigurator; import org.apache.log4j.level; import org.apache.log4j.logger; 

and project working fine. check after importing above mentioned classes. if problem, give me comment.


Comments

Popular posts from this blog

c++ - OpenMP unpredictable overhead -

ruby on rails - RuntimeError: Circular dependency detected while autoloading constant - ActiveAdmin.register Role -

javascript - Wordpress slider, not displayed 100% width -