How can I explain Hadoop not to split my file in some special MapReduce task? -

-

  1. given have file process hadoop , know size of file smaller block size of hdfs. guarantees file not splitted , dont need write inputsplit because default 1 not split it?

  2. given file saved sequencefileoutputformat (or other output format) bigger block size, consists of 1 key-value pair. implies file block's stored on same node (except replicated copies) , mapreduce task not waste time fetch them? means dont need write own inputsplit because key not splitted (key size smaller block size , there 1 key)?


Comments

Post a Comment

Popular posts from this blog

c++ - OpenMP unpredictable overhead -

-
i have simple openmp code runs 4 empty threads , reports time: #include "omp.h" #include <sys/time.h> #include <cstdio> double start_time; void process_thread() { } int main(){ struct timeval tv; gettimeofday(&tv, null); start_time = 1000*((double)tv.tv_sec + 1e-6*(double)tv.tv_usec); printf("beginning time: %.3fms ",start_time-start_time); #pragma omp parallel schedule(static,1) for(int i=0;i<4;i++) process_thread(); gettimeofday(&tv, null); double cur_time = 1000*((double)tv.tv_sec + 1e-6*(double)tv.tv_usec); printf("finishing time: %.3fms\n\n",cur_time-start_time); } without openmp running time small , consistent: beginning time: 0.000ms finishing time: 0.050ms beginning time: 0.000ms finishing time: 0.050ms beginning time: 0.000ms finishing time: 0.049ms beginning time: 0.000ms finishing time: 0.049ms beginning time: 0.000ms finishing time: 0.050ms beginnin...
Read more

ruby on rails - RuntimeError: Circular dependency detected while autoloading constant - ActiveAdmin.register Role -

-
i using activeadmin , have file doing this: activeadmin.register role something--- end however, server gives me error: runtimeerror: circular dependency detected while autoloading constant role /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:461:in `load_missing_constant' /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:184:in `const_missing' ~/desktop/boxfox/app/admin/role.rb:1:in `<top (required)>' /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:424:in `load' /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:424:in `block in load_file' /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:616:in `new_constants_in' /library/ruby/gems/2.0.0/gems/activesupport-4.0.10/lib/active_support/dependencies.rb:423:in `load_file' /library/ruby/gems/2.0.0/gems/activesuppor...
Read more

javascript - Wordpress slider, not displayed 100% width -

-
so im making first wordpress theme scratch going fine got problem. want slider on homepage 1 thing wrong. im setting width of image 100% !important doesnt go 100% width. makes image wider not 100%. example here: http://bit.ly/11ouebg see image doesnt contain whole width. can see background-color. really weird, im using meteor slides. i hope can me out :d
Read more