User:Timothee Flutre/Notebook/Postdoc/2013/12/27: Difference between revisions
From OpenWetWare
(→About organizing a project: add lots of details) |
(→About organizing computer-based research: add Rlibs and PYTHONPATH) |
||
Line 9: | Line 9: | ||
* '''Motivation''': when starting a new project, it is very handy to ''quickly'' and ''easily'' set up a ''portable'' structure allowing the project to be ''backed-up'' on other machines, ''shared'' with collaborators and the work to be ''reproduced/replicated'' by colleagues. | * '''Motivation''': when starting a new project, it is very handy to ''quickly'' and ''easily'' set up a ''portable'' structure allowing the project to be ''backed-up'' on other machines, ''shared'' with collaborators and the work to be ''reproduced/replicated'' by colleagues. | ||
* '''OS choice''': concerning computers, one usually has a preferred [https://en.wikipedia.org/wiki/Operating_system operating system]. Yet, in scientific projects where computing is an important aspect of research, the most frequent is [https://en.wikipedia.org/wiki/Linux GNU/Linux]. Thus, even if it's always good to know how to find our way on other operating systems, such as [https://en.wikipedia.org/wiki/Microsoft_Windows Microsoft Windows] and [https://en.wikipedia.org/wiki/OS_X Apple Mac OS X], I will focus in the following on GNU/Linux. | * '''OS choice''': concerning computers, one usually has a preferred [https://en.wikipedia.org/wiki/Operating_system operating system]. Yet, in scientific projects where computing is an important aspect of research, the most frequent is [https://en.wikipedia.org/wiki/Linux GNU/Linux]. Thus, even if it's always good to know how to find our way on other operating systems, such as [https://en.wikipedia.org/wiki/Microsoft_Windows Microsoft Windows] and [https://en.wikipedia.org/wiki/OS_X Apple Mac OS X], I will focus in the following on GNU/Linux. | ||
* '''Home''': | * '''Home''': | ||
** I create a set of directories, via <code>mkdir -p bin include lib share src src_ext texmf tmp work</code>: | ** I create a set of directories, via <code>mkdir -p bin include lib share src src_ext src_ext/Rlibs texmf tmp work</code>: | ||
*** <code>bin</code>: contains executables; | *** <code>bin</code>: contains executables; | ||
*** <code>include</code>: contains C/C++ header files; | *** <code>include</code>: contains C/C++ header files; | ||
Line 20: | Line 18: | ||
*** <code>src</code>: contains source code from my own packages; | *** <code>src</code>: contains source code from my own packages; | ||
*** <code>src_ext</code>: contains source code from external packages; | *** <code>src_ext</code>: contains source code from external packages; | ||
**** <code>src_ext/Rlibs</code>: for external R packages; | |||
*** <code>texmf</code>: contains LaTeX packages; | *** <code>texmf</code>: contains LaTeX packages; | ||
*** <code>tmp</code>: contains temporary tasks; | *** <code>tmp</code>: contains temporary tasks; | ||
Line 38: | Line 37: | ||
LDFLAGS="$LDFLAGS -L$HOME/lib" | LDFLAGS="$LDFLAGS -L$HOME/lib" | ||
export LDFLAGS | export LDFLAGS | ||
export R_LIBS_USER=$HOME/src_ext/Rlibs | |||
export PYTHONPATH=$HOME/lib/python/:$PYTHONPATH | |||
</nowiki> | </nowiki> | ||
* '''External packages''': for each external package in <code>src_ext</code>, I create a directory | * '''External packages''': for each external package in <code>src_ext</code>, I create a directory with its usual name, say emacs, in which I create a file <code>install.bash</code> with the necessary commands to compile and install the package, typically: | ||
<nowiki> | <nowiki> | ||
Line 82: | Line 83: | ||
date | date | ||
</nowiki> | </nowiki> | ||
* '''My history''': in 2006, during an internship in a bioinformatics lab, I discovered GNU/Linux. More specifically, I worked on a [https://en.wikipedia.org/wiki/Fedora_%28operating_system%29 Fedora] distribution and was able to install it on my laptop. From 2007 to 2010, during my PhD, I switched to [https://en.wikipedia.org/wiki/Debian Debian] and then [https://en.wikipedia.org/wiki/Ubuntu_%28operating_system%29 Ubuntu] for my laptop, and I used several computer clusters running with [https://en.wikipedia.org/wiki/Solaris_%28operating_system%29 Solaris] and [https://en.wikipedia.org/wiki/CentOS CentOS]. | |||
** This looks like an anthology of weird names but, fundamentally, all these distributions are more or less similar to each other and can be described as [https://en.wikipedia.org/wiki/Unix-like Unix-like systems]. Please note, however, that all these are not equivalent in terms of protecting your ''freedom''. Michael Kerrisk presents this quite well ([http://man7.org/conf/udes2012/Linux_and_Free_Software.pdf pdf]). It is indeed important to know about the difference between [https://en.wikipedia.org/wiki/GNU/Linux_naming_controversy GNU and Linux] and, for those who read the [http://www.amazon.com/dp/1451648537/ biography of Steve Jobs], I highly recommend reading the [https://en.wikipedia.org/wiki/Free_as_in_Freedom:_Richard_Stallman%27s_Crusade_for_Free_Software biography of Richard Stallman] (founder of GNU). | |||
<!-- ##### DO NOT edit below this line unless you know what you are doing. ##### --> | <!-- ##### DO NOT edit below this line unless you know what you are doing. ##### --> |
Revision as of 02:40, 10 February 2014
Project name | <html><img src="/images/9/94/Report.png" border="0" /></html> Main project page <html><img src="/images/c/c3/Resultset_previous.png" border="0" /></html>Previous entry<html> </html> |
About organizing computer-based research
# Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$HOME/bin:$PATH export PATH CFLAGS="$CFLAGS -I$HOME/include" export CFLAGS LDFLAGS="$LDFLAGS -L$HOME/lib" export LDFLAGS export R_LIBS_USER=$HOME/src_ext/Rlibs export PYTHONPATH=$HOME/lib/python/:$PYTHONPATH
#!/usr/bin/env bash wget http://gnu.mirrors.hoobly.com/gnu/emacs/emacs-24.3.tar.gz tar xzvf emacs-24.3.tar.gz cd emacs-24.3 ./configure --prefix=$HOME --with-x-toolkit=no --with-xpm=no --with-jpeg=no --with-gif=no --with-tiff=no make make install
#!/usr/bin/env bash # backup.bash <path_to_backup> >& backup.log & date RSYNC_OPT="--compress --recursive --times --perms --links --exclude="*~" --delete --delete-excluded --progress" rsync $RSYNC_OPT ~/remote1/work/project1 $1 rsync $RSYNC_OPT ~/remote1/work/project2 $1 date
|