Skip to content

Targeting a Casper Node Architecture

Casper is a heterogeneous system that has several node architectures. The chart below provides architecture specifications and example PBS select statements to target a specific node type. Targeting a specific node may impact scheduler time, particularly for architectures with low node counts. The PBS select statements are basic examples to request the node and you will likely want to modify the select parameters to better suit your job workflow needs.

Be mindful that these examples only show a way to guarantee the scheduler gives you that node type. There are instances where modifying the example CPU count may overlap with other node types. A specific example would be selecting #PBS -l select=1:ncpus=4 which could place you on any of the High-Throughput Computing node types.

Two parameters that are helpful for targeting node types are cpu_type and gpu_type. The table below provides example on how to use them and if you need to use them for requesting a node type.

The examples below also do not include options for MPI ranks or OpenMP threads but should be added if you want parallelism for multiple CPUs in your job. Memory requests are also omitted in the examples unless it is a requirement to be placed on large memory nodes. You can reference the job script examples to help build your submission script or contact the NCAR Research Computing help desk if you have any questions about targeting a node type with PBS select statements that are optimal for your workflow.

Node Type CPU Cores Core Speed Node Memory GPU GPU Memory Node Count PBS Select Statements
Data & Visualization Intel Skylake 36 2.3GHz 384 GB GP100 16 GB 9 -l select=1:ncpus=1:ngpus=1 -l gpu_type=gp100
Intel Skylake 36 2.3GHz 384 GB A100 40 GB 3 -l select=1:ncpus=1:ngpus=1:cpu_type=skylake -l gpu_type=a100
AMD EPYC 9474F 48 3.6GHz 733 GB L40 48 GB 6 -l select=1:ncpus=1:ngpus=1 -l gpu_type=l40
ML & GPGPU Cascade Lake 36 2.6GHz 768 GB V100 (x4) 32 GB 4 -l select=1:ncpus=1:ngpus=4 -l gpu_type=v100
Intel Cascade Lake 36 2.6GHz 1152 GB V100 (x8) 32 GB 6 -l select=1:ncpus=8:mpiprocs=8:ngpus=8 -l gpu_type=v100
AMD EPYC Milan 128 2.45GHz 1024 GB A100 (x4) 80 GB 8 -l select=1:ncpus=4:mpiprocs=4:ngpus=4:cpu_type=milan -l gpu_type=a100
Intel Xeon Gold 6430 64 2.10Ghz 985 GB H100 (x4) 80 GB 2 -l select=1:ncpus=4:mpiprocs=4:ngpus=4 -l gpu_type=h100
High-Throughput Computing Cascade Lake 36 2.6GHz 384 GB 62 -l select=1:ncpus=36:cpu_type=cascadelake
Cascade Lake 36 2.3GHz 1500 GB 2 -l select=1:ncpus=36:cpu_type=cascadelake:mem=800GB
AMD Epyc 9554P 62 3.1GHz 768 GB 64 -l select=1:ncpus=62:cpu_type=genoa
AMD Epyc 9554P 64 3.1GHz 1536 GB 6 -l select=1:ncpus=64:cpu_type=genoa:mem=800GB