User:Timothee Flutre/Notebook/Postdoc/2012/05/16

From OpenWetWare

(Difference between revisions)
Jump to: navigation, search
(Typical templates for Python scripts, C++ programs and others: use getopt in bash instead of getopts)
(About programming: add list free c++ courses)
(24 intermediate revisions not shown.)
Line 6: Line 6:
| colspan="2"|
| colspan="2"|
<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->
<!-- ##### DO NOT edit above this line unless you know what you are doing. ##### -->
-
==Typical templates for Python scripts, C++ programs and others==
+
==About programming==
-
It is always rewarding on the long term to start any piece of computer software with a minimum amount of generic code (verbose, command-line options, help message, license, etc). But it's a pain to write all this every time, right? So below are my typical templates for any Python script, C++ program, Beamer presentation, Bash script, etc.
+
* '''Resources''': most of the time, it's not necessary to buy a book, search online instead!
 +
** [http://software-carpentry.org/ Software Carpentry] to learn about tests, versioning, Makefile, regular expressions, etc
 +
** [http://www.tldp.org/LDP/abs/html/ Advanced Bash-Scripting Guide] by Mendel Cooper
 +
** my own page on [http://openwetware.org/wiki/User:Timothee_Flutre/Notebook/Postdoc/2011/11/07 R]
 +
** [http://www.cplusplus.com/doc/tutorial/ C++ tutorial], a must-read (and a [http://www.mycplus.com/featured-articles/best-free-programming-courses-online/ list] of the best free C/C++ resources online)
 +
** [http://scipy-lectures.github.io/ Python tutorial] for scientists
 +
** [http://resrc.io/list/10/list-of-free-programming-books/ list] of free programming books
-
* '''Python''': it is assumed that the code below is copied into a file named "MyClass.py".
+
* '''Templates''': it is always rewarding on the long term to start any piece of computer software with a minimum amount of generic code (command-line options, help message, license, usage of gzipped files, running time, etc). But it's a pain to write all this every time, right? And often we know how to do something in one language but not in another. So below are my typical templates for any C++/Python/R/Bash program, as well as Beamer presentation.
 +
** '''C++''': download the file [http://github.com/timflutre/quantgen/blob/master/myprogram.cpp myprogram.cpp], as well as [http://github.com/timflutre/quantgen/blob/master/utils_io.cpp utils_io.cpp] along with its header [http://github.com/timflutre/quantgen/blob/master/utils_io.hpp utils_io.hpp].
 +
** '''Python''': download the file [http://github.com/timflutre/quantgen/blob/master/myprogram.py myprogram.py]
 +
** '''R''': download the file [http://github.com/timflutre/quantgen/blob/master/myprogram.R myprogram.R]
 +
** '''Bash''': download the file [http://github.com/timflutre/quantgen/blob/master/myprogram.bash myprogram.bash]
-
<nowiki>
+
* '''Language-independent user documentation''': I'm a firm believer that it is necessary to add some user documentation, even  minimal, to any program. An easy way to do this is to simply generate such documentation from the "help" message, as long as it is "properly" formatted (see [http://www.gnu.org/s/help2man/ help2man]). The following commands work for any programming language:
-
#!/usr/bin/env python
+
-
 
+
-
# Author: Timothee Flutre
+
-
# License: GPL-3
+
-
# Aim: does this and that
+
-
# help2man -o MyClass.man ./MyClass.py
+
-
# groff -mandoc MyClass.man > MyClass.ps
+
-
 
+
-
import sys
+
-
import os
+
-
import getopt
+
-
import time
+
-
import datetime
+
-
import math
+
-
 
+
-
 
+
-
class MyClass(object):
+
-
   
+
-
    def __init__(self):
+
-
        self.verbose = 1
+
-
        self.input = ""
+
-
       
+
-
       
+
-
    def help(self):
+
-
        msg = "`%s' does this and that.\n" % os.path.basename(sys.argv[0])
+
-
        msg += "\n"
+
-
        msg += "Usage: %s [OPTIONS] ...\n" % os.path.basename(sys.argv[0])
+
-
        msg += "\n"
+
-
        msg += "Options:\n"
+
-
        msg += " -h, --help\tdisplay the help and exit\n"
+
-
        msg += " -V, --version\toutput version information and exit\n"
+
-
        msg += " -v, --verbose\tverbosity level (0/default=1/2/3)\n"
+
-
        msg += " -i\tinput"
+
-
        msg += "\n"
+
-
        msg += "Examples:\n"
+
-
        print msg; sys.stdout.flush()
+
-
       
+
-
       
+
-
    def version(self):
+
-
        msg = "%s 0.1\n" % os.path.basename(sys.argv[0])
+
-
        msg += "\n"
+
-
        msg += "License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>\n"
+
-
        msg += "This is free software; see the source for copying conditions.  There is NO\n"
+
-
        msg += "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n"
+
-
        print msg; sys.stdout.flush()
+
-
       
+
-
       
+
-
    def setAttributesFromCmdLine(self):
+
-
        try:
+
-
            opts, args = getopt.getopt( sys.argv[1:], "hVv:i:",
+
-
                                        ["help", "version", "verbose="])
+
-
        except getopt.GetoptError, err:
+
-
            sys.stderr.write("%s\n" % str(err))
+
-
            self.help()
+
-
            sys.exit(2)
+
-
        for o, a in opts:
+
-
            if o == "-h" or o == "--help":
+
-
                self.help()
+
-
                sys.exit(0)
+
-
            elif o == "-V" or o == "--version":
+
-
                self.version()
+
-
                sys.exit(0)
+
-
            elif o == "-v" or o == "--verbose":
+
-
                self.verbose = int(a)
+
-
            elif o == "-i":
+
-
                self.input = a
+
-
            else:
+
-
                assert False, "unhandled option"
+
-
               
+
-
               
+
-
    def checkAttributes(self):
+
-
        if self.input == "":
+
-
            msg = "ERROR: missing compulsory option -i"
+
-
            sys.stderr.write("%s\n\n" % msg)
+
-
            self.help()
+
-
            sys.exit(1)
+
-
        if not os.path.exists(self.input):
+
-
            msg = "ERROR: can't find '%s'" % self.input
+
-
            sys.stderr.write("%s\n\n" % msg)
+
-
            self.help()
+
-
            sys.exit(1)
+
-
           
+
-
           
+
-
    def run(self):
+
-
        self.checkAttributes()
+
-
       
+
-
        if self.verbose > 0:
+
-
            msg = "START %s" % time.strftime("%Y-%m-%d %H:%M:%S")
+
-
            startTime = time.time()
+
-
            print msg; sys.stdout.flush()
+
-
           
+
-
        # ... specific code ...
+
-
       
+
-
        if self.verbose > 0:
+
-
            msg = "END %s" % time.strftime("%Y-%m-%d %H:%M:%S")
+
-
            endTime = time.time()
+
-
            runLength = datetime.timedelta(seconds=
+
-
                                          math.floor(endTime - startTime))
+
-
            msg += " (%s)" % str(runLength)
+
-
            print msg; sys.stdout.flush()
+
-
           
+
-
           
+
-
if __name__ == "__main__":
+
-
    i = MyClass()
+
-
    i.setAttributesFromCmdLine()
+
-
    i.run()
+
-
</nowiki>
+
-
 
+
-
* '''C++''': it is assumed that the code below is copied into a file named "myprogram.cpp" and that the file "[https://github.com/timflutre/quantgen/blob/master/utils.cpp utils.cpp]" is present in the same directory, along with the corresponding header <nowiki>utils.h</nowiki>.
+
  <nowiki>
  <nowiki>
-
/** \file myprogram.cpp
+
help2man -o myprogram.man ./myprogram
-
*
+
man ./myprogram.man
-
*  `myprogram' does this and that.
+
groff -mandoc myprogram.man > myprogram.ps
-
*  Copyright (C) 2011,2012 Timothee Flutre
+
ps2pdf myprogram.ps myprogram.pdf
-
*
+
-
*  This program is free software: you can redistribute it and/or modify
+
-
*  it under the terms of the GNU General Public License as published by
+
-
*  the Free Software Foundation, either version 3 of the License, or
+
-
*  (at your option) any later version.
+
-
*
+
-
*  This program is distributed in the hope that it will be useful,
+
-
*  but WITHOUT ANY WARRANTY; without even the implied warranty of
+
-
*  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+
-
*  GNU General Public License for more details.
+
-
*
+
-
*  You should have received a copy of the GNU General Public License
+
-
*  along with this program. If not, see <http://www.gnu.org/licenses/>.
+
-
*
+
-
*  g++ -Wall -g utils.cpp myprogram.cpp -lgsl -lgslcblas -lz -o myprogram
+
-
*  help2man -o myprogram.man ./myprogram
+
-
groff -mandoc myprogram.man > myprogram.ps
+
-
*/
+
-
 
+
-
#include <cmath>
+
-
#include <ctime>
+
-
#include <getopt.h>
+
-
 
+
-
#include <iostream>
+
-
#include <string>
+
-
using namespace std;
+
-
 
+
-
#include "utils.h"
+
-
 
+
-
/** \brief Display the help on stdout.
+
-
*/
+
-
void help (char ** argv)
+
-
{
+
-
  cout << "`" << argv[0] << "'"
+
-
      << " does this and that." << endl
+
-
      << endl
+
-
      << "Usage: " << argv[0] << " [OPTIONS] ..." << endl
+
-
      << endl
+
-
      << "Options:" << endl
+
-
      << "  -h, --help\tdisplay the help and exit" << endl
+
-
      << "  -V, --version\toutput version information and exit" << endl
+
-
      << "  -v, --verbose\tverbosity level (0/default=1/2/3)" << endl
+
-
      << "  -i, --in\tinput" << endl
+
-
      << endl
+
-
      << "Examples:" << endl
+
-
      << "  " << argv[0] << " -i <input>" << endl
+
-
      << endl
+
-
      << "Remarks:" << endl
+
-
      << "  This is my typical template file for C++." << endl
+
-
    ;
+
-
}
+
-
 
+
-
/** \brief Display version and license information on stdout.
+
-
*/
+
-
void version (char ** argv)
+
-
{
+
-
  cout << argv[0] << " " << __DATE__ << " " << __TIME__ << endl
+
-
      << endl
+
-
      << "Copyright (C) 2011,2012 T. Flutre." << endl
+
-
      << "License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>" << endl
+
-
      << "This is free software; see the source for copying conditions.  There is NO" << endl
+
-
      << "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." << endl
+
-
      << endl
+
-
      << "Written by T. Flutre." << endl;
+
-
}
+
-
 
+
-
/** \brief Parse the command-line arguments and check the values of the
+
-
*  compulsory ones.
+
-
*/
+
-
void
+
-
parseArgs (
+
-
  int argc,
+
-
  char ** argv,
+
-
  string & input,
+
-
  int & verbose)
+
-
{
+
-
  int c = 0;
+
-
  while (true)
+
-
  {
+
-
    static struct option long_options[] =
+
-
      {
+
-
        {"help", no_argument, 0, 'h'},
+
-
        {"version", no_argument, 0, 'V'},
+
-
        {"verbose", required_argument, 0, 'v'},
+
-
        {"input", required_argument, 0, 'i'},
+
-
        {0, 0, 0, 0}
+
-
      };
+
-
    int option_index = 0;
+
-
    c = getopt_long (argc, argv, "hVv:i:",
+
-
                    long_options, &option_index);
+
-
    if (c == -1)
+
-
      break;
+
-
    switch (c)
+
-
    {
+
-
    case 0:
+
-
      if (long_options[option_index].flag != 0)
+
-
        break;
+
-
    case 'h':
+
-
      help (argv);
+
-
      exit (0);
+
-
    case 'V':
+
-
      version (argv);
+
-
      exit (0);
+
-
    case 'v':
+
-
      verbose = atoi(optarg);
+
-
      break;
+
-
    case 'i':
+
-
      input = optarg;
+
-
      break;
+
-
    case '?':
+
-
      printf ("\n"); help (argv);
+
-
      abort ();
+
-
    default:
+
-
      printf ("\n"); help (argv);
+
-
      abort ();
+
-
    }
+
-
  }
+
-
  if (input.empty())
+
-
  {
+
-
    printCmdLine (argc, argv);
+
-
    fprintf (stderr, "ERROR: missing compulsory option -i\n\n");
+
-
    help (argv);
+
-
    exit (1);
+
-
  }
+
-
  if (! doesFileExist (input))
+
-
  {
+
-
    printCmdLine (argc, argv);
+
-
    fprintf (stderr, "ERROR: can't find '%s'\n\n", input.c_str());
+
-
    help (argv);
+
-
    exit (1);
+
-
  }
+
-
}
+
-
 
+
-
int main (int argc, char ** argv)
+
-
{
+
-
  string input;
+
-
  int verbose = 1;
+
-
 
+
-
  parseArgs (argc, argv, input, verbose);
+
-
 
+
-
  time_t startRawTime, endRawTime;
+
-
  if (verbose > 0)
+
-
  {
+
-
    time (&startRawTime);
+
-
    cout << "START " << argv[0] << " (" << time2string (startRawTime) << ")"
+
-
        << endl
+
-
        << "compiled -> " << __DATE__ << " " << __TIME__
+
-
        << endl << flush;
+
-
    printCmdLine (argc, argv);
+
-
  }
+
-
 
+
-
  // ... specific code ...
+
-
 
+
-
  if (verbose > 0)
+
-
  {
+
-
    time (&endRawTime);
+
-
    cout << "END " << argv[0] << " (" << time2string (endRawTime) << ")"
+
-
        << endl
+
-
        << "elapsed -> " << elapsedTime(startRawTime, endRawTime)
+
-
        << endl
+
-
        << "max.mem -> " << getMaxMemUsedByProcess () << " kB"
+
-
        << endl;
+
-
  }
+
-
 
+
-
  return EXIT_SUCCESS;
+
-
}
+
</nowiki>
</nowiki>
-
* '''Bash''': sometimes it's easier to write a script in bash rather than in python. This is especially the case when one wants to use pipes, eg. <nowiki>zcat data.txt.gz | awk '{print $1}'</nowiki>, or when one wants to use a [http://en.wikipedia.org/wiki/Glob_%28programming%29 glob] to pass multiple files to the script, eg. <nowiki>./myscript.bash -i "input*.txt"</nowiki>.
+
* '''Benchmarking''':
-
 
+
** via the command-line: [http://man7.org/linux/man-pages/man1/time.1.html time] (see also [https://github.com/jhclark/memusg memusg])
-
<nowiki>
+
** in R: [http://cran.r-project.org/web/packages/rbenchmark/index.html rbenchmark]
-
#!/usr/bin/env bash
+
-
 
+
-
# Author: Timothee Flutre
+
-
# License: GPL-3
+
-
# Aim: does this and that
+
-
 
+
-
function help () {
+
-
    msg="\`$0' does this and that.\n"
+
-
    msg+="\n"
+
-
    msg+="Usage: $0 [OPTIONS] ...\n"
+
-
    msg+="\n"
+
-
    msg+="Options:\n"
+
-
    msg+="  -h, --help\t\tdisplay the help and exit\n"
+
-
    msg+="  -V, --version\t\toutput version information and exit\n"
+
-
    msg+="  -v, --verbose\t\tverbosity level (0/default=1/2/3)\n"
+
-
    msg+="  -i, --in\t\tinput\n"
+
-
    msg+="\n"
+
-
    msg+="Examples:\n"
+
-
    msg+="  $0 -i <input>\n"
+
-
    echo -e $msg
+
-
}
+
-
 
+
-
function version () {
+
-
    msg="$0 0.1\n"
+
-
    msg+="\n"
+
-
    msg+="Copyright (C) 2012 T. Flutre.\n"
+
-
    msg+="License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>\n"
+
-
    msg+="This is free software; see the source for copying conditions.  There is NO\n"
+
-
    msg+="warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n"
+
-
    msg+="\n"
+
-
    msg+="Written by T. Flutre.\n"
+
-
    echo -e $msg
+
-
}
+
-
 
+
-
# source http://www.linuxjournal.com/content/use-date-command-measure-elapsed-time
+
-
function timer () {
+
-
    if [[ $# -eq 0 ]]; then
+
-
        echo $(date '+%s')
+
-
    else
+
-
        local  stime=$1
+
-
        etime=$(date '+%s')
+
-
        if [[ -z "$stime" ]]; then stime=$etime; fi
+
-
        dt=$((etime - stime))
+
-
        ds=$((dt % 60))
+
-
        dm=$(((dt / 60) % 60))
+
-
        dh=$((dt / 3600))
+
-
        printf '%d:%02d:%02d' $dh $dm $ds
+
-
    fi
+
-
}
+
-
 
+
-
function parseArgs () {
+
-
    TEMP=`getopt -o hVv:p:f:c:g:o:i: -l help,version,verbose: \
+
-
        -n "$0" -- "$@"`
+
-
    if [ $? != 0 ] ; then echo "ERROR: getopt failed" >&2 ; exit 1 ; fi
+
-
    eval set -- "$TEMP"
+
-
    while true; do
+
-
        case "$1" in
+
-
            -h|--help) help; exit 0; shift;;
+
-
            -V|--version) version; exit 0; shift;;
+
-
            -v|--verbose) verbose=$2; shift;;
+
-
            -i|--in) input=$2; shift 2;;
+
-
            --) shift; break;;
+
-
            *) echo "ERROR: options parsing failed"; exit 1;;
+
-
        esac
+
-
    done
+
-
    if [ "x${input}" == "x" ]; then
+
-
        echo "ERROR: missing compulsory option -i"; echo; help; exit 1
+
-
    fi
+
-
    if [ ! -f ${input} ]; then
+
-
        echo "ERROR: can't find '${input}'"; exit 1
+
-
    fi
+
-
}
+
-
 
+
-
verbose=1
+
-
input=""
+
-
parseArgs $@
+
-
 
+
-
if [ $verbose -gt "0" ]; then
+
-
    printf "START %s %s\n" $(date +"%Y-%m-%d") $(date +"%H:%M:%S")
+
-
    startTime=$(timer)
+
-
fi
+
-
 
+
-
# ... specific code ...
+
-
sleep 1
+
-
 
+
-
if [ $verbose -gt "0" ]; then
+
-
    printf "END %s %s" $(date +"%Y-%m-%d") $(date +"%H:%M:%S")
+
-
    printf " (%s)\n" $(timer startTime)
+
-
fi
+
-
</nowiki>
+
-
* '''Latex-Beamer''': it is assumed that the code below is copied into a file named "mypresentation.tex" and that several packages are already installed. The current directory is also assumed to contain a sub-directory named "figures" in which are saved all picture files included in the presentation.
+
* '''Latex-Beamer''': it is assumed that the code below is copied into a file named "mypresentation.tex" and that several packages are already installed. The current directory is also assumed to contain a sub-directory named "figures" in which are saved all picture files included in the presentation. In the future, I should have a longer look at Karl Broman's [http://kbroman.wordpress.com/2013/10/07/better-looking-latexbeamer-slides/ post], and try [http://sourceforge.net/projects/qpdfpresenter/ QPdfPresenter].
  <nowiki>
  <nowiki>
-
% Author: Timothée Flutre
+
% Copyright (C) 2012 Timothee Flutre.
\documentclass{beamer}
\documentclass{beamer}
Line 479: Line 124:
end{document}
end{document}
</nowiki>
</nowiki>
-
 
<!-- ##### DO NOT edit below this line unless you know what you are doing. ##### -->
<!-- ##### DO NOT edit below this line unless you know what you are doing. ##### -->

Revision as of 10:34, 1 January 2014

Project name Main project page
Previous entry      Next entry

About programming

  • Templates: it is always rewarding on the long term to start any piece of computer software with a minimum amount of generic code (command-line options, help message, license, usage of gzipped files, running time, etc). But it's a pain to write all this every time, right? And often we know how to do something in one language but not in another. So below are my typical templates for any C++/Python/R/Bash program, as well as Beamer presentation.
  • Language-independent user documentation: I'm a firm believer that it is necessary to add some user documentation, even minimal, to any program. An easy way to do this is to simply generate such documentation from the "help" message, as long as it is "properly" formatted (see help2man). The following commands work for any programming language:
help2man -o myprogram.man ./myprogram
man ./myprogram.man
groff -mandoc myprogram.man > myprogram.ps
ps2pdf myprogram.ps myprogram.pdf

  • Latex-Beamer: it is assumed that the code below is copied into a file named "mypresentation.tex" and that several packages are already installed. The current directory is also assumed to contain a sub-directory named "figures" in which are saved all picture files included in the presentation. In the future, I should have a longer look at Karl Broman's post, and try QPdfPresenter.
% Copyright (C) 2012 Timothee Flutre.

\documentclass{beamer}
\usepackage{amsmath}
\usepackage{bm} % to have mathematical symbols in bold
\usepackage{hyperref}
\hypersetup{colorlinks, linkcolor=black, urlcolor=gray}
\usepackage{multirow}
\usepackage{tikz}
\usepackage[francais]{babel}
\usepackage[utf8]{inputenc}

\graphicspath{{./figures/}}

%-----------------------------------------------------------------------------

\setbeamertemplate{caption}[numbered]
\setbeamerfont{caption}{size=\scriptsize}
\setbeamertemplate{navigation symbols}{}
\setbeamercolor{alerted text}{fg=purple}

\setbeamertemplate{footline}
{
  \leavevmode
  \hbox{
    \hspace*{-0.06cm}
    \begin{beamercolorbox}[wd=.2\paperwidth,ht=2.25ex,dp=1ex,center]{author in head/foot}
      \usebeamerfont{author in head/foot}\insertshortauthor \hspace*{1em} \insertshortinstitute
    \end{beamercolorbox}
    \begin{beamercolorbox}[wd=.50\paperwidth,ht=2.25ex,dp=1ex,center]{section in head/foot}
      \usebeamerfont{section in head/foot}\insertshorttitle
    \end{beamercolorbox}
    \begin{beamercolorbox}[wd=.27\paperwidth,ht=2.25ex,dp=1ex,right]{section in head/foot}%
      \usebeamerfont{section in head/foot}\insertshortdate{}\hspace*{2em}
      \insertframenumber{} / \inserttotalframenumber\hspace*{2ex}
    \end{beamercolorbox}
  }
  \vskip0pt
}

\AtBeginSection[]
{
  \begin{frame}
    \frametitle{Outline}
    \addtocounter{framenumber}{-1}
    \tableofcontents[currentsection]
  \end{frame}
}

%-----------------------------------------------------------------------------

\title[Short title]{Long title}
\author[T. Flutre]{Timoth\'{e}e Flutre}
\institute[Short affiliation]{Long affiliation}
\date{\today}

begin{document}

\begin{frame}
\titlepage
\end{frame}                                                                                                                       

\begin{frame}
\frametitle{Outline}
\tableofcontents
\end{frame}

\section{First section}
\begin{frame}
\frametitle{I.1.}
\begin{itemize}
\item 
\end{itemize}
\end{frame}

\section{Second section}

\begin{frame}
\frametitle{II.1.}
\begin{center}
%\includegraphics[width=0.95\textwidth,height=0.90\textheight,keepaspectratio=true]{myplot}%
\end{center}
\end{frame}

end{document}


Personal tools