Resources: HOWTO collection

About

I find myself constantly looking up common scripts, workarounds and howtos on the internet for usual tasks in a research lab -- moving large files around from server to server, setting up SSH keys for BitBucket, etc. This is a dump of all those resources I picked up, starting out as a greenhorn at the UMass Vision lab.

BitBucket and Git
MatConvNet
Shell commands
GNU Screen setup
LaTeX utils
Bit Twiddling Hacks

BitBucket and Git

SSH for BitBucket setup:

Setup walkthrough and documentation

RSA key gen

Generate SSH key at command line:
```
ssh-keygen -t rsa
```
Add an SSH key to an account [BitBucket permission denied due to no public key]: Walkthrough from StackOverflow answer

X11 forwarding
On some of the newer vision servers, after a git clone, we would get X-11 error messages. They turn out to be mostly warning messages and can be usually ignored. Some details on that.

Git branch/pull/add/etc. [under construction]
Git is really essential, and has something of a steep learning curve for first-timers. As usual, I studiously avoided it in the beginning but the initial time+effort investment always pays off in long-term productivity. I really intend to expand this portion to give short one-line answers and commands for specific tasks (instead of long-winded explanations some Git howtos throw up).

Adding and removing Git branches: Git branches
Adding files to a Git repo: Git Adding files
Adding only untracked files to the repo: Add untracked files

MatConvNet

A quick introduction to setting up MatConvNet: slides
Installation instructions: MatConvNet site
Make sure the versions of CUDA and Matlab are matching before MEXing: MatConvNet issue 33

Example compile command with the usual paths to Matlab and CUDA on the vision servers:

vl_compilenn('enableGpu', true, 'enableImreadJpeg', true, 'imreadJpegFlags', ...
  {'-I/usr/local/libjpeg/include/','-L/usr/local/libjpeg/lib','-ljpeg'}, ...
  'cudaRoot', '/usr/local/cuda-6.5', 'matlabRoot', '/exp/comm/matlab');

The path to CUDA must be exported before running Matlab and MatConvNet:
```
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64
```

Shell Commands

A handy list of common shell commands.

wget very large files:

wget --continue --progress=dot:mega --tries=0

Rsync remote files to current location:
```
rsync -azP source destination
```
With a single status bar, instead of scrolling down:
```
rsync -az --info=progress2 source destination
```
More on Rsync

Remove all files in folder:
```
rm -rf directoryname
```

Tarball
Compress:

 tar -cvzf filename.tar.gz folder/

Extract:

tar -xzvf filename.tar.gz

Printing from the command line:
Sometimes accessing colour printers from the local machine has some complications (Ubuntu older versions etc.). In such cases, one can use the printers setup on the newer vision servers for printing colour versions of papers/pdfs.
```
lpr -P cscolor3 -o sides=two-sided-long-edge filename.pdf
```
LPSTAT for displaying printer info
CUPS options

Linux Screen setup

Screen is a utiity that allows you to log off from a remote session and then resume it from another location or the same location at a later time. The programs continue running. Naturally, it is very handy when setting large tasks to run on the servers. The first link provides a really nice overview of the setup.

Screen setup
Screen window by default does not have mouse scrolling enabled. The keyboard shortcuts are frankly rather hairy. Add mouse scroling support in .screenrc
Kill a specific screen session

LaTeX

Single LaTeX make command to generate PDF, keeping cross-references and bibtex correctly:

latexmk -pdf mydocument.tex

Compressing PDF filesize

Bit twiddling hacks: Awesome collection of bit-level hacks for all ages by Sean Eron Anderson :)
Bit Twiddling Hacks webpage