parallelly::availableCores() returns the number of cores available for multicore processing. R itself provides parallel::detectCores() for this. There is also the mc.cores option (default to environment variable MC_CORES) set when parallel is loaded. Beyond this, various systems/setups set specific environment variables to reflect the number of available/allocated cores. For instance, resource manager PBS, sets environment variable PBS_NUM_PPN on the compute node specifying the number of allotted cores.

Currently, availableCores() defaults to return the first valid value of (in order):

1. PBS_NUM_PPN
2. mc.cores (and MC_CORES)
3. parallel::detectCores()

I would like to add support for more resource/workload managers and other distributed processing environments. For instance,

• PBS (Portable Batch System)
  • PBS_NUM_NODES,
  • PBS_NUM_PPN,
  • PBS_NP (see comment below)
• PBSPro
  • NCPUS
• Slurm (Simple Linux Utility for Resource Management)
  • Manual: https://slurm.schedmd.com/
  • SLURM_CPUS_PER_TASK, e.g. --cpus-per-task=3 (or short -N 3)
  • SLURM_CPUS_ON_NODE, e.g. --ntasks=3 (or short -n 3): only truthworty when --nodes=1
  • Support for multi-node job requests, e.g. --nodes=2 --ntasks=3. Identify the number of cores to run on the current node.
• Oracle Grid Engine (aka Sun Grid Engine, CODINE, GRD). Environment variable: NSLOTS (?), cf. http://cc.in2p3.fr/docenligne/969
• LSF/OpenLava (Platform Load Sharing Facility).
  • LSB_DJOB_NUMPROC - "The number of processors (slots) allocated to the job." (https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_config_ref/lsf_envars_ref.html), cf. PR availableCores: add LSF/OpenLava future#360
  • LSB_MAX_NUM_PROCESSORS(?) - "The maximum number of processors requested when the job is submitted. [...]
    Set during job execution based on bsub options. For example, for a job submitted with -n 2,4, the maximum number of processors requested is 4." (https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_config_ref/lsf_envars_ref.html). This setting is recommended by https://grid.rcs.hbs.org/parallel-r
  • ...?
• Spark
• OAR
• HTCondor
  • Handled in Issue availableCores(): Add support for HTCondor #50
• Moab
• PJM (https://staff.cs.manchester.ac.uk/~fumie/internal/Job_Operation_Software_en.pdf)
  • PJM_VNODE_CORE
  • PJM_PROC_BY_NODE (not always)

What else am I missing?

• BBS_HOME - (env var) if set (to anything) BiocParallel will "throttle number of cores used on Bioconductor build systems [...] to 4".
• snowFT's ./.clustersize file that specifies number of workers?

I appreciate any feedback on what environment variables or commands that are available to a compute node to query the number of allotted cores, iff at all. Please try to provide links to documentations if you can.

References

• http://slurm.schedmd.com/rosetta.pdf