Schumer lab: Lab group on Sherlock

From OpenWetWare
Jump to navigationJump to search

Please remember to clean up files you will not use or keep most of your temporary files on /scratch! It is really easy to forget about this and use up all of our space so make a habit of routinely removing temporary files.

Lab organization on Sherlock

Overall organization

Files and folders containing active scripts or small data files for analyses can be kept in:


which is the equivalent path to:


This directory does not have much space (1 TB shared) so should not be used for big files of any kind. A better place for that is:


which is the equivalent path to:


$GROUP_SCRATCH files are automatically deleted after being inactive for 6 months so make sure to backup on $GROUP_HOME or $OAK if you need files or scripts long-term

Long term data storage and large files should kept in our lab directory:


which is the equivalent path to:


Where to find specific files or programs

Genome assemblies, annotation files, recombination maps and other shared resources can be found here:


Shared scripts and the lab git repository are here:


Commonly used programs in the lab that are not available via module loading can be found here:


To make these programs globally available, you can export the path to this folder:

export PATH="/home/groups/schumer/shared_bin:$PATH"

To avoid doing this every time you login to Sherlock you can edit you .bashrc file, which can be found here:


and add the export command below the line that says:

# User specific aliases and functions

export PATH="/home/groups/schumer/shared_bin:$PATH"

In addition to the above command you might also want to automatically load certain modules, e.g.:

module load perl

module load R

Getting started

Make your own person folder inside our lab group folder here:


Storing data and data backup

Our lab raw data repository and repository for large files is on Oak, and can be accessed in three ways:

1) cd /oak/stanford/groups/schumer/data

2) cd $OAK/data

3) I've put a link to this folder in our lab Sherlock directory: cd $GROUP_HOME/data

  • NOTE: Oak is not regularly backed up and cannot be seen as a permanent data repository. Other options include external hard drives and NCBI SRA. It is possible to set the release data on the SRA so that release can occur after publication if necessary, this can be edited any time.
  • For directions on how to deposit on the SRA, see this file in dropbox: depositing_data_to_NCBIs_SRA.txt

Interactive jobs

Before running any programs from the login node on Sherlock, remember to request an interactive job:

you can use our dedicated nodes:

srun --pty -p schumer -t 0-2 bash

or request general nodes:

srun --pty -t 0-2 bash

You can also submit jobs, this is covered elsewhere Submitting slurm jobs

Common programs/dependencies and how to load them

Many programs on Sherlock are available as modules. To see the modules that are available, type:

module avail

Lots of biology specific modules have to be loaded after loading the general biology module:

module load biology

Any module can be loaded by typing:

module load $MODULE_NAME

after which the commands will be available globally


You will probably want to load R interactively on Sherlock at some point, to do quick analyses of your data or run analysis/generate figures on a dataset too large to load quickly on your Desktop.

To access R, first load the module:

module load R

To run R from the command line simply type R then enter.

To quit, type: q()


Perl is globally available on Sherlock, but to install packages you need to load the perl module to use cpan:

module load perl

Many scripts we have in the lab depend on certain perl modules, they are listed below:




You can install perl modules by running the following from your terminal:


for example:

cpan Math::Random