Getting Started

OSCER account

Our group computes on OSCER, the OU Supercomputing Center for Education & Research. Steps to get an account:

1. Complete the New Account Request Form.
2. If approved, you will receive an email when your account is created.
3. Log in with your initial password and change it IMMEDIATELY if prompted (or manually if not prompted).

The OSCER documentation is the reference for anything not covered on this page. For account problems, quota issues, or anything else cluster-related, contact support@oscer.ou.edu (see the OSCER support page).

Logging in

Once your account is active, connect with:

ssh -XY [username]@sooner.oscer.ou.edu

To save typing, add an entry to ~/.ssh/config on your laptop:

Host sooner
    HostName sooner.oscer.ou.edu
    User [username]
    ForwardX11 yes
    ForwardX11Trusted yes

after which ssh sooner is all you need.

Requesting a node on Sooner

The group has a dedicated Slurm partition, currently named ouheptmp (this will change to ouhep in the future). Never run real work on the login nodes — they are shared by everyone and only meant for editing files and submitting jobs. Always run your code on an interactive node. To request one:

srun -p sooner_test --container=el9hw --pty $SHELL

This drops you into a shell on a compute node inside an EL9 container, which is what ATLAS software expects.

GPU access

For GPU work (e.g. machine learning training), request a GPU node with:

srun -p ouheptmp --container=el9hw --gres=gpu:1 --ntasks=1 --cpus-per-task=16 --mem=40G --pty $SHELL

Check what you were allocated with nvidia-smi once the shell opens.

Submitting batch jobs

For anything longer than a quick test, submit a batch job instead of holding an interactive shell. Write a submission script like:

#!/bin/bash
#SBATCH --partition=ouheptmp
#SBATCH --container=el9hw
#SBATCH --job-name=myjob
#SBATCH --output=logs/%x-%j.out
#SBATCH --time=12:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --mem=8G

setupATLAS
asetup Athena,main,latest
# your commands here

and submit it with:

sbatch myjob.sh

Useful commands for managing your jobs:

squeue -u $USER        # list your queued and running jobs
scancel [jobid]        # kill a job
sacct -j [jobid]       # accounting info for a finished job

%x and %j in the output path expand to the job name and job ID. Make sure the logs/ directory exists before submitting. Request only the resources you need — smaller jobs schedule faster.

Storage: OURDisk

The group's OURDisk partition lives at:

/ourdisk/hpc/ouhep/

We have a shared quota of 93 TB. Please be mindful not to fill it up — clean up datasets you no longer need, and check with the group before copying in anything large.

Python environments with mamba

Manage Python with mamba, a fast drop-in replacement for conda. On OSCER, mamba is already available as a module — do not install your own. Load it with:

module load Mamba

(The first time, run mamba init bash and log out and back in so environment activation works.) Then create one environment per project — never install packages into the base environment:

mamba create -n myanalysis python=3.12
mamba activate myanalysis

Environments keep each project's dependencies isolated and reproducible. Note that environments can get large — if your home area fills up, contact OSCER support.

Machine learning on OSCER

For machine learning work, create a dedicated environment and install PyTorch and Lightning into it (the PyTorch wheels bundle CUDA, so the same install works on CPU and GPU nodes):

mamba create -n ml python=3.12
mamba activate ml
pip install torch lightning

Then request a GPU node (see the OSCER tab) and check that the GPU is visible:

python -c "import torch; print(torch.cuda.is_available())"

Do the installation itself from an interactive node rather than a login node — the downloads are large. Add whatever else your project needs (numpy, matplotlib, jupyterlab, …) to the same environment.

HEP data analysis: the Scikit-HEP stack

For ntuple-level analysis we recommend the Scikit-HEP ecosystem: modern, columnar Python analysis that plays well with the broader scientific Python world (NumPy, matplotlib, machine learning frameworks) — no compiled ROOT macros required. The core pieces:

• uproot — reads and writes ROOT files directly into arrays, with no ROOT installation needed
• awkward — NumPy-style operations on jagged arrays (variable-length collections like jets per event)
• hist — histograms with a clean API for filling, slicing, and projecting
• mplhep — matplotlib styles and helpers for publication-quality HEP plots (including the ATLAS style)
• vector — Lorentz vectors and kinematics on arrays

Install them into your environment from conda-forge:

mamba install uproot awkward hist mplhep vector jupyterlab

A minimal analysis looks like:

import uproot
import awkward as ak
import hist
import mplhep as hep
import matplotlib.pyplot as plt

tree = uproot.open("ntuple.root")["tree"]
jet_pt = tree["jet_pt"].array() / 1000.0  # MeV to GeV

h = hist.Hist.new.Reg(50, 0, 500, label="Jet $p_{T}$ [GeV]").Double()
h.fill(ak.flatten(jet_pt))

hep.style.use(hep.style.ATLAS)
h.plot()
plt.savefig("jet_pt.pdf")

The HSF Scikit-HEP training module is a good hands-on introduction to the whole stack.