Compiling and submitting Matlab jobs on swarm

The following summary was composed by Yariv Z Levy and includes information gathered from discussions with Bruno Castro da Silva, Valerie Caro, Tyler Trafford and Laura Sevilla, as well as from previously written material by Manjunath Narayana and Benjamin Mears.

MATLAB Compiler uses the MATLAB Compiler Runtime (MCR), a standalone set of shared libraries that enables the execution of MATLAB files on computers without an installed version of MATLAB. This allows users to run matlab jobs on the swarm cluster without using the department Matlab licenses. 

  If you are running on swarm and using the matlab version in

/share/apps/matlab you can use the MCR installed in
/share/apps/matlab/MATLAB_Compiler_Runtime/v714
or you can build your own copy:

STEP 1: installing the MCR (MATLAB Compiler Runtime) into your account

  1. log into your swarm account
  2. type "qlogin" and hit enter
  3. type "/share/apps/matlab/bin/matlab -nojvm" and hit enter
  4. now that you are in MATLAB, type "mcrinstaller" and hit enter
    MATLAB will give you the path to the file "MCRInstaller.bin", e.g. :
    /share/apps/matlab/toolbox/compiler/deploy/glnxa64/MCRInstaller.bin
    (For Swarm, you need the GLNXA64 version.)
  5. exit from Matlab by typing "exit" and hitting enter
  6. exit from your qlogin session by typing "exit" and hitting enter
  7. copy the GLNXA64 version of "MCRInstaller.bin" into your account (or call it using the full path) and run it:
    ./MCRInstaller.bin

More info here:
http://www.mathworks.com/access/helpdesk/help/toolbox/compiler/f12-999353.html

STEP 2: compile your matlab code as a standalone application

Note - check http://www.mathworks.com/help/toolbox/compiler/br2cqa0-20.html for a list of matlab functions you cannot compile with MATLAB Compiler.

  • clean up your code - removing all graphical output (e.g., plot
commands, etc.) and function calls not supported by the compiler
In the example here below, the lines 11, 12, and 13 should be removed.
                1       function testSwarm(b,c)
                2       current_path = pwd;
                3       if(~isdeployed)
                4               addpath(current_path)
                5       end
                6       j= 50000;
                7       for i=1:j
                8               a1(1,i) = i * (b(1,1) + c(1,1));
                9               a2(1,i) = rand;
                10      end
                11      plot(a1, 'r')
                12      hold on
                13      plot(a2, 'b')
                14      save('testSwarm_data');

NOTE 1 : Failing to include lines 2 to 5 may cause errors while running
the standalone application.

NOTE 2 : It is a good idea to save your output to the Lustre filesystem,
which path is "/lustre/work1/yourAdvisorUsername/yourUsername/"
because you have more disk quota there.
  • log into your swarm account and make sure to have a local copy of your matlab file (.m)
  • type "qlogin" and hit enter
  • type "/share/apps/matlab/bin/matlab -nojvm" and hit enter
  • compile your matlab file (let's call it "example.m"), with the command
    "mcc -mv example.m -d ./ -o test".
    More info about mcc here: http://www.mathworks.com/help/toolbox/compiler/mcc.html
    If the compilation is successful, two files will be created: "test" and
    "run_test.sh" (in fact, also the files "readme.txt" and
    "mccExcludedFiles.log" are created).
  • exit from Matlab by typing "exit" and hitting enter
  • exit from your qlogin session by typing "exit" and hitting enter

STEP 3: launch your matlab standalone application on swarm

  • log into your swarm account
  • assuming that "test" and "run_test.sh" are in your home folder ("~"),
    and assuming that you want to use b=10 and c=5 as inputs to your
    program ("example.m"), type
    qsub -b y -cwd -e logE.txt -o logO.txt ./run_test.sh ./MCRfiles/v714 10 5
    and hit enter.
    or
    qsub -b y -cwd -e logE.txt -o logO.txt ./run_test.sh /share/apps/matlab/MATLAB_Compiler_Runtime/v714 10 5

    NOTE: There are different queues for short/long jobs. Depending on your
    task, you may need to use a different queue.
    More info here:
    http://people.cs.umass.edu/~swarm/index.php?n=Main.PolicyDOC
    and here: http://people.cs.umass.edu/~swarm/index.php?n=Main.UserDoc
  • wait until the run is done (check using "qstat -u yourUsername") and enjoy.