java - How to loop in oozie using sub-workflow? -
it has been suggested 1 use oozie-subworkflow execute action multiple times. can't figure out how it.
let's need invoke map-reduce action n times. let's encapsulate map-reduce action in sub-workflow. can manually invoke sub-workflow n times having n actions in oozie-workflow (as outlined in approach). n needs dynamic. so:
- how decrease loop-counter ensure don't execute sub-workflow more n times.
- can execute n sub-workflows in parallel.
i have thought of 2 approaches:
approach 1: thinking of having java action decreases counter, capture arguments , use way keep track of loop-counter. if loop-counter not 0 invoke subworkflow. however, how call java action multiple-times...a catch-22?
approach 2: approach ditch sub-workflow idea , encapsulate map-reduce (mapred-workflow.xml) job in normal workflow, implement java action executes oozie-workflow (mapred-workflow.xml) n times. in parallel, wait jobs finish return main workflow. main concern approach lack of reliability. happen if java action dies half-way through? example schedule n/2 oozie jobs die (for reason).
i think can use subworkflow option. below rough idea can develop on this.
....
.... <action name="somejava"> <java> <capture-output /> </java> <ok to="countercheck" /> <error to="fail" /> </action> <decision name="countercheck"> <switch> <case to="continuefurther"> ${wf:something == $countervariable} </case> <default to="anotheraction" /> </switch> </decision> <action name="continuefurther"> <sub-workflow> <app-path>${repeatworkflow}</app-path> <propagate-configuration/> </sub-workflow> <ok to="somejava"/> <error to="fail"/> </action> ..... .....
Comments
Post a Comment