atlas_maneframe
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
atlas_maneframe [2015/07/05 16:27] – [SLURM and Resource Management] sekula | atlas_maneframe [2018/10/28 10:28] (current) – sekula | ||
---|---|---|---|
Line 20: | Line 20: | ||
For now (July - December, 2015), SMU is hosting a few "ATLAS power users" to conduct their work on ManeFrame and provide quick access to resources for fast turnaround of time-sensitive results; the work these power users are conducting is synergistic with the technical, upgrade, and physics interests of the SMU ATLAS Group. We are exploring other models by which we might share our Tier 3 resources with ATLAS collaborators, | For now (July - December, 2015), SMU is hosting a few "ATLAS power users" to conduct their work on ManeFrame and provide quick access to resources for fast turnaround of time-sensitive results; the work these power users are conducting is synergistic with the technical, upgrade, and physics interests of the SMU ATLAS Group. We are exploring other models by which we might share our Tier 3 resources with ATLAS collaborators, | ||
+ | |||
+ | **Mailing List** | ||
+ | |||
+ | External ManeFrame users should request to be added to the mailing list '' | ||
===== Acknowledgement by External Users ===== | ===== Acknowledgement by External Users ===== | ||
Line 27: | Line 31: | ||
**Acknowledgement Policy** | **Acknowledgement Policy** | ||
- | During the period July - December | + | During the period July 2015 - December |
We do not ask for acknowledgement in public papers, conference notes, or presentations; | We do not ask for acknowledgement in public papers, conference notes, or presentations; | ||
Line 186: | Line 190: | ||
< | < | ||
- | maneframe-run --inDS <input directory> | + | maneframe-run --inDS <input directory> |
| | ||
Line 202: | Line 206: | ||
</ | </ | ||
- | === Example Code: Loading and Processing a File Listing | + | So an example bash submission script might look like: |
+ | < | ||
+ | # | ||
+ | |||
+ | INPUT=ThisIsYourInputFileDirectoryPATH | ||
+ | OUTPUT=/ | ||
+ | EXECUTABLE=" | ||
+ | EMAIL=" | ||
+ | MAXFILES=10 | ||
+ | MAXTIME=00: | ||
+ | |||
+ | ~sekula/ | ||
+ | </ | ||
+ | |||
+ | === Example Code: Loading and Processing a list of ROOT files === | ||
- | '' | + | '' |
< | < | ||
+ | #include < | ||
+ | // This is an expansion of the example code from the xAOD Tutorial Page for ATLAS: | ||
+ | // | ||
// use SampleHandler to scan all of the subdirectories of a directory for particular MC single file: | // use SampleHandler to scan all of the subdirectories of a directory for particular MC single file: | ||
std::string pathstring = " | std::string pathstring = " | ||
if (argc > 2) { | if (argc > 2) { | ||
pathstring = static_cast< | pathstring = static_cast< | ||
- | std::cout << pathstring << std::endl; | ||
} | } | ||
const char* inputFilePath = gSystem-> | const char* inputFilePath = gSystem-> | ||
Line 218: | Line 238: | ||
// This is an input.txt file - it contains a CSV list of ROOT files. | // This is an input.txt file - it contains a CSV list of ROOT files. | ||
// Parse it and add the root files to the job | // Parse it and add the root files to the job | ||
- | SH:: | + | SH:: |
| | ||
std:: | std:: | ||
Line 227: | Line 247: | ||
std::string line; | std::string line; | ||
getline( fs, line ); | getline( fs, line ); | ||
- | |||
- | std::cout << line << std::endl; | ||
TObjArray* rootfilelist = TString(line).Tokenize("," | TObjArray* rootfilelist = TString(line).Tokenize("," | ||
Line 246: | Line 264: | ||
</ | </ | ||
- | ===== Known Issues ===== | + | so that (keeping with the tutorial' |
+ | < | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include " | ||
+ | #include < | ||
+ | #include < | ||
+ | #include < | ||
+ | #include < | ||
- | ==== Clock Skew Warnings During Builds ==== | + | #include " |
- | There is a known effect that, while doing builds, you get a lot of "clock skew" | + | int main (int argc, char *argv[]) { |
+ | // Take the submit directory from the input if provided: | ||
+ | std::string submitDir = "submitDir"; | ||
+ | if (argc > 1) submitDir = argv[1]; | ||
- | ===== Questions or Problems? ===== | + | // Set up the job for xAOD access: |
+ | xAOD:: | ||
- | Please email both of the following people with questions, comments, suggestions, | + | // Construct |
+ | SH:: | ||
- | | + | |
- | | + | |
- | Or just fill out the form below, which will result in us receiving an email. | + | if (argc > 2) { |
+ | pathstring = static_cast< | ||
+ | } | ||
+ | const char *inputFilePath = gSystem-> | ||
- | < | + | if (TString(inputFilePath).Contains(" |
- | Action mail ssekula@smu.edu ahkumar@smu.edu | + | // This is an input.txt file - it contains |
- | Thanks | + | // Parse it and add the root files to the job |
+ | SH:: | ||
- | textarea "Your Name" x2 | + | std:: |
- | select " | + | |
- | textarea "Your Comments" | + | |
- | email "Your E-Mail Address" | + | if (fs.good() && fs.is_open()) { |
+ | while (!fs.eof()) { | ||
+ | std::string line; | ||
+ | getline(fs, line); | ||
- | Fieldset | + | TObjArray *rootfilelist = TString(line).Tokenize(","); |
- | submit "Submit" | + | |
- | </form> | + | for (Int_t i = 0; i < rootfilelist-> |
+ | TString sampleName = ((TObjString*)rootfilelist-> | ||
+ | sample-> | ||
+ | } | ||
+ | } | ||
+ | } | ||
+ | |||
+ | sh.add(sample); | ||
+ | } | ||
+ | else { | ||
+ | SH:: | ||
+ | sh, inputFilePath); | ||
+ | } | ||
+ | |||
+ | // Set the name of the input TTree. It's always " | ||
+ | sh.setMetaString(" | ||
+ | |||
+ | // Print what we found: | ||
+ | sh.print(); | ||
+ | |||
+ | // Create an EventLoop job: | ||
+ | EL::Job job; | ||
+ | job.sampleHandler(sh); | ||
+ | job.options()-> | ||
+ | |||
+ | // Create an output and an assocaited ntuple | ||
+ | EL:: | ||
+ | job.outputAdd(output); | ||
+ | EL:: | ||
+ | job.algsAdd(ntuple); | ||
+ | |||
+ | // Add our analysis to the job: | ||
+ | MyxAODAnalysis* alg = new MyxAODAnalysis(); | ||
+ | job.algsAdd(alg); | ||
+ | |||
+ | alg-> | ||
+ | |||
+ | // Run the job using the local/ | ||
+ | EL:: | ||
+ | driver.submit(job, submitDir); // This will be obsolete when a SLURM driver is available | ||
+ | |||
+ | return 0; | ||
+ | } // main | ||
+ | </ | ||
+ | |||
+ | === Combining Output Files === | ||
+ | When maneframe-run submits your jobs to ManeFrame it will do so in batches. Each of these will run on ManeFrame and each batch will return output to the OUTPUT directory you passed to maneframe-run. You will now want to combine these outputs to be able to look at the combined results in a single file. The following script will use ROOT's hadd utility to produce a single root file (location and name given by the user) as output: | ||
+ | < | ||
+ | # | ||
+ | |||
+ | SOURCE=() | ||
+ | TARGET=~This/ | ||
+ | i=0 | ||
+ | |||
+ | # find all the .root files of interest | ||
+ | while IFS= read -r file; do | ||
+ | SOURCE[i]=$file | ||
+ | ((i++)) | ||
+ | done < <(find / | ||
+ | |||
+ | hadd $TARGET ${SOURCE[*]} | ||
+ | </ | ||
+ | |||
+ | === Using the EventLoop (EL) SlurmDriver === | ||
+ | |||
+ | As of about November of 2016, AnalysisBase releases (after 2.4.21) contained a new driver in the EventLoop package, SlurmDriver. Like the other drivers for running code locally or on other batch management systems, the EL:: | ||
+ | |||
+ | The present EL:: | ||
+ | |||
+ | < | ||
+ | EL:: | ||
+ | driver.SetJobName("MyAnalysis"); | ||
+ | driver.SetAccount(" | ||
+ | driver.SetPartition(" | ||
+ | driver.SetRunTime(" | ||
+ | driver.SetMemory(" | ||
+ | driver.SetConstrain("" | ||
+ | </code> | ||
+ | |||
+ | When you create an EL:: | ||
+ | |||
+ | * The Job Name (appears in squeue and labels your individual jobs) | ||
+ | * Account - this is always " | ||
+ | * Partition - the name of the partition where jobs will run, e.g. serial or development, | ||
+ | * RunTime - the maximum running time of the job. Here, the example code uses 30 minutes in HH:MM:SS format | ||
+ | * Memory - the memory limit for your job | ||
+ | * Constrain - this will constrain your job to specific resources, if there are multiple resource options in the partition. | ||
+ | ===== Known Issues ===== | ||
+ | |||
+ | ==== Clock Skew Warnings During Builds ==== | ||
+ | |||
+ | There is a known effect that, while doing builds, you get a lot of "clock skew" warnings. This is under investigation. It appears to be harmless | ||
+ | |||
+ | |||
+ | ===== Questions or Problems? ===== | ||
- | We welcome suggestions and feedback on how to make the ManeFrame/ATLAS experience better. We are grateful for your interest in using our Tier 3 resources to advance ATLAS research and development activities. | + | Please contact |
atlas_maneframe.txt · Last modified: 2018/10/28 10:28 by sekula