Schumer lab: Lab group on Sherlock

From OpenWetWare
Jump to navigationJump to search

Please remember to clean up files you will not use or keep most of your temporary files on /scratch! It is really easy to forget about this and use up all of our space so make a habit of routinely removing temporary files.

Lab organization on Sherlock

Overall organization

Files and folders containing active analyses can be kept in:

$GROUP_HOME

which is the equivalent path to:

/home/groups/schumer

Data storage and large files should kept in our lab directory:

$OAK

which is the equivalent path to:

/oak/stanford/groups/schumer

Where to find specific files or programs

Genome assemblies, annotation files and other shared resources can be found here:

/home/groups/schumer/shared_bin/shared_resources


Shared scripts and the lab git repository are here:

/home/groups/schumer/shared_bin/Lab_shared_scripts


Commonly used programs in the lab that are not available via module loading can be found here:

/home/groups/schumer/shared_bin


To make these programs globally available, you can export the path to this folder:

export PATH="/home/groups/schumer/shared_bin:$PATH"


Commonly used shared resources such as genome assemblies, annotation files, etc, can be found here:

/home/groups/schumer/shared_bin/shared_resources

Getting started

Make your own person folder inside our lab group folder here:

/home/groups/schumer/lab_member_folders

Storing data and data backup

Our lab raw data repository and repository for large files is on Oak, and can be accessed in three ways:

1) cd /oak/stanford/groups/schumer/data

2) cd $OAK/data

3) I've put a link to this folder in our lab Sherlock directory: cd $GROUP_HOME/data

  • NOTE: Oak is not regularly backed up and cannot be seen as a permanent data repository. Other options include external hard drives and NCBI SRA. It is possible to set the release data on the SRA so that release can occur after publication if necessary, this can be edited any time.
  • For directions on how to deposit on the SRA, see this file in dropbox: depositing_data_to_NCBIs_SRA.txt

Interactive jobs

Before running any programs from the login node on Sherlock, remember to request an interactive job:

srun --pty -t 0-2 bash

You can also submit jobs, this is covered elsewhere Submitting slurm jobs

Common programs/dependencies and how to load them

Many programs on Sherlock are available as modules. To see the modules that are available, type:

module avail

Lots of biology specific modules have to be loaded after loading the general biology module:

module load biology

Any module can be loaded by typing:

module load $MODULE_NAME

after which the commands will be available globally

R

You will probably want to load R interactively on Sherlock at some point, to do quick analyses of your data or run analysis/generate figures on a dataset too large to load quickly on your Desktop.

To access R, first load the module:

module load R

To run R from the command line simply type R then enter.

To quit, type: q()