Resources: HOWTO collection

I find myself constantly looking up common scripts, workarounds and howtos on the internet for usual tasks in a research lab -- moving large files around from server to server, setting up SSH keys for BitBucket, etc. This is a dump of all those resources I picked up, starting out as a greenhorn at the UMass Vision lab.

BitBucket and Git

SSH for BitBucket setup:
RSA key gen
X11 forwarding
On some of the newer vision servers, after a git clone, we would get X-11 error messages. They turn out to be mostly warning messages and can be usually ignored. Some details on that.

Git branch/pull/add/etc. [under construction]
Git is really essential, and has something of a steep learning curve for first-timers. As usual, I studiously avoided it in the beginning but the initial time+effort investment always pays off in long-term productivity. I really intend to expand this portion to give short one-line answers and commands for specific tasks (instead of long-winded explanations some Git howtos throw up).

  • A quick introduction to setting up MatConvNet: slides
  • Installation instructions: MatConvNet site
  • Make sure the versions of CUDA and Matlab are matching before MEXing: MatConvNet issue 33
  • Example compile command with the usual paths to Matlab and CUDA on the vision servers:
    vl_compilenn('enableGpu', true, 'enableImreadJpeg', true, 'imreadJpegFlags', ...
      {'-I/usr/local/libjpeg/include/','-L/usr/local/libjpeg/lib','-ljpeg'}, ...
      'cudaRoot', '/usr/local/cuda-6.5', 'matlabRoot', '/exp/comm/matlab');
  • The path to CUDA must be exported before running Matlab and MatConvNet:
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64

Shell Commands
A handy list of common shell commands.
  • wget very large files:
    wget --continue --progress=dot:mega --tries=0
  • Rsync remote files to current location:
    rsync -azP source destination

    With a single status bar, instead of scrolling down:
    rsync -az --info=progress2 source destination
    More on Rsync

  • Remove all files in folder:
    rm -rf directoryname
  • Tarball
     tar -cvzf filename.tar.gz folder/ 

    tar -xzvf filename.tar.gz 
  • Printing from the command line:
    Sometimes accessing colour printers from the local machine has some complications (Ubuntu older versions etc.). In such cases, one can use the printers setup on the newer vision servers for printing colour versions of papers/pdfs.
    lpr -P cscolor3 -o sides=two-sided-long-edge filename.pdf

    LPSTAT for displaying printer info
    CUPS options

Linux Screen setup

Screen is a utiity that allows you to log off from a remote session and then resume it from another location or the same location at a later time. The programs continue running. Naturally, it is very handy when setting large tasks to run on the servers. The first link provides a really nice overview of the setup.


Single LaTeX make command to generate PDF, keeping cross-references and bibtex correctly:
latexmk -pdf mydocument.tex
Compressing PDF filesize
Bit twiddling hacks

Awesome collection of bit-level hacks for all ages by Sean Eron Anderson :)
Bit Twiddling Hacks webpage