Differences

This shows you the differences between two versions of the page.

--- techstaff:slurm [2018/05/04 12:47] – [Partitions / Queues] kauffman
+++ techstaff:slurm [2019/10/08 17:24] – kauffman
@@ Line 1: / Line 1: @@
 ===== Notice =====
+**2019-10-08**: New computer nodes added under the partiton ''%%genfast%%''
 **2017-08-31**: Configuration change to allow allocation on CPUs and RAM. Please read the 'Default Quota' section under https://howto.cs.uchicago.edu/techstaff:slurm#usage
@@ Line 112: / Line 113: @@
 | ^ SLURM ^ Example ^
 ^ Submit a batch serial job | sbatch | sbatch runscript.sh |
-^ Run a script interatively | srun | srun --pty -p interact -t 10 --mem 1000 \\ /bin/bash \\ /bin/hostname |
+^ Run a script interactively | srun | srun --pty -p interact -t 10 --mem 1000 \\ /bin/bash \\ /bin/hostname |
 ^ Kill a job | scancel | scancel 4585 |
 ^ View status of queues | squeue | squeue -u cnetid |
@@ Line 320: / Line 321: @@
+==== Checking how many Generic RESources are being consumed ====
-===== Paths =====
+Simple use the ''%%-O%%'' option for ''%%squeue%%'' and you can see how many generic resources any particular job is consuming.
-You will need to add the following to your ''%%$PATH%%'' and ''%%$LD_LIBRARY_PATH%%''.
+<code>
+$ squeue -O username,nodelist,gres
+USER                NODELIST            GRES
+someusername        gpu3                gpu:1
+otherusername       gpu3                gpu:3
+...
+</code>
+===== Environment Variables =====
+==== CUDA_HOME, LD_LIBRARY_PATH ====
+Please make sure you specify $CUDA_HOME and if you want to take advantage of CUDNN libraries you will need to append /usr/local/cuda-x.x/lib64 to the $LD_LIBRARY_PATH environment variable.
+  cuda_version=9.2
+  export CUDA_HOME=/usr/local/cuda-${cuda_version}
+  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_HOME/lib64
+Currently we support the same versions of CUDA that the latest version of CUDNN supports. This is not written in stone and we can accommodate most other versions if required; just let techstaff know what your needs are.
+==== PATH ====
+You may also need to add the following to your ''%%$PATH%%''
   export PATH=$PATH:/usr/local/cuda/bin
-  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH=/usr/local/cuda/lib
+==== CUDA_VISIBLE_DEVICES ====
+Do not set this variable. It will be set for you by SLURM.
+The variable name is actually misleading; since it does NOT mean the amount of devices, but rather the physical device number assigned by the kernel (e.g. /dev/nvidia2).
+For example: If you requested multiple gpu's from SLURM (--gres=gpu:2), the CUDA_VISIBLE_DEVICES variable should contain two numbers(0-3 in this case) separated by a comma (e.g. 1,3).
@@ Line 383: / Line 413: @@
 STDERR should be blank.
 ====== More ======
-If you feel this documentation is lacking in some way please let techstaff know. Email [[techstaff@cs.uchicago.edu]], call (773-702-1031), or stop by our office (Ryerson 154).
+If you feel this documentation is lacking in some way please let techstaff know. Email [[techstaff@cs.uchicago.edu]], call (773-702-1031), or stop by our office (Crerar 357).