atlas_maneframe
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
atlas_maneframe [2016/03/31 14:47] – mcf | atlas_maneframe [2016/10/28 08:18] – [SLURM and Resource Management] sekula | ||
---|---|---|---|
Line 31: | Line 31: | ||
**Acknowledgement Policy** | **Acknowledgement Policy** | ||
- | During the period July - December | + | During the period July 2015 - December |
We do not ask for acknowledgement in public papers, conference notes, or presentations; | We do not ask for acknowledgement in public papers, conference notes, or presentations; | ||
Line 204: | Line 204: | ||
maneframe-run --inDS / | maneframe-run --inDS / | ||
+ | </ | ||
+ | |||
+ | So an example bash submission script might look like: | ||
+ | < | ||
+ | #!/bin/bash | ||
+ | |||
+ | INPUT=ThisIsYourInputFileDirectoryPATH | ||
+ | OUTPUT=/ | ||
+ | EXECUTABLE=" | ||
+ | EMAIL=" | ||
+ | MAXFILES=10 | ||
+ | MAXTIME=00: | ||
+ | |||
+ | ~sekula/ | ||
</ | </ | ||
Line 346: | Line 360: | ||
} // main | } // main | ||
</ | </ | ||
+ | |||
+ | === Combining Output Files === | ||
+ | When maneframe-run submits your jobs to ManeFrame it will do so in batches. Each of these will run on ManeFrame and each batch will return output to the OUTPUT directory you passed to maneframe-run. You will now want to combine these outputs to be able to look at the combined results in a single file. The following script will use ROOT's hadd utility to produce a single root file (location and name given by the user) as output: | ||
+ | < | ||
+ | #!/bin/bash | ||
+ | |||
+ | SOURCE=() | ||
+ | TARGET=~This/ | ||
+ | i=0 | ||
+ | |||
+ | # find all the .root files of interest | ||
+ | while IFS= read -r file; do | ||
+ | SOURCE[i]=$file | ||
+ | ((i++)) | ||
+ | done < <(find / | ||
+ | |||
+ | hadd $TARGET ${SOURCE[*]} | ||
+ | </ | ||
+ | |||
+ | === Using the EventLoop (EL) SlurmDriver === | ||
+ | |||
+ | As of about November of 2016, AnalysisBase releases (after 2.4.21) contained a new driver in the EventLoop package, SlurmDriver. Like the other drivers for running code locally or on other batch management systems, the EL:: | ||
+ | |||
+ | The present EL:: | ||
+ | |||
+ | < | ||
+ | EL:: | ||
+ | driver.SetJobName(" | ||
+ | driver.SetAccount(" | ||
+ | driver.SetPartition(" | ||
+ | driver.SetRunTime(" | ||
+ | driver.SetMemory(" | ||
+ | driver.SetConstrain("" | ||
+ | </ | ||
+ | |||
+ | When you create an EL:: | ||
+ | |||
+ | * The Job Name (appears in squeue and labels your individual jobs) | ||
+ | * Account - this is always " | ||
+ | * Partition - the name of the partition where jobs will run, e.g. serial or development, | ||
+ | * RunTime - the maximum running time of the job. Here, the example code uses 30 minutes in HH:MM:SS format | ||
+ | * Memory - the memory limit for your job | ||
+ | * Constrain - this will constrain your job to specific resources, if there are multiple resource options in the partition. | ||
===== Known Issues ===== | ===== Known Issues ===== | ||
atlas_maneframe.txt · Last modified: 2018/10/28 10:28 by sekula