java - How to loop in oozie using sub-workflow? -


it has been suggested 1 use oozie-subworkflow execute action multiple times. can't figure out how it.

let's need invoke map-reduce action n times. let's encapsulate map-reduce action in sub-workflow. can manually invoke sub-workflow n times having n actions in oozie-workflow (as outlined in approach). n needs dynamic. so:

  • how decrease loop-counter ensure don't execute sub-workflow more n times.
  • can execute n sub-workflows in parallel.

i have thought of 2 approaches:

approach 1: thinking of having java action decreases counter, capture arguments , use way keep track of loop-counter. if loop-counter not 0 invoke subworkflow. however, how call java action multiple-times...a catch-22?

approach 2: approach ditch sub-workflow idea , encapsulate map-reduce (mapred-workflow.xml) job in normal workflow, implement java action executes oozie-workflow (mapred-workflow.xml) n times. in parallel, wait jobs finish return main workflow. main concern approach lack of reliability. happen if java action dies half-way through? example schedule n/2 oozie jobs die (for reason).

i think can use subworkflow option. below rough idea can develop on this.

....

 .... <action name="somejava">                 <java>                      <capture-output />                 </java>                 <ok to="countercheck" />                 <error to="fail" />     </action>   <decision name="countercheck">          <switch>             <case to="continuefurther">               ${wf:something == $countervariable}             </case>             <default to="anotheraction" />          </switch> </decision>  <action name="continuefurther">         <sub-workflow>                 <app-path>${repeatworkflow}</app-path>                 <propagate-configuration/>         </sub-workflow>         <ok to="somejava"/>         <error to="fail"/> </action> ..... ..... 

Comments

Popular posts from this blog

ruby on rails - RuntimeError: Circular dependency detected while autoloading constant - ActiveAdmin.register Role -

c++ - OpenMP unpredictable overhead -

javascript - Wordpress slider, not displayed 100% width -