Collecting Data on Amazon Mechanical Turk (AMT)

Subhransu Maji



This page contains details of several interfaces I wrote for collecting image annotations using Amazon Mechanical Turk. These jobs can be launched using the 'external hit' specification of the AMT command line interface. You can donwload the command line interaface for AMT for the unix platform here. Here is the list of interfaces written mostly in Java/Javascript using the canvas tag and relies on url-encoding to pass parameters such as image names, etc., to the GUI. To view the code for the interfaces, you can view-source in any modern browser.

  • Image Segmentation | AMT script
    • Draw the outer boundary of the object to segment the object from the background. Read the README in the AMT script to launch the jobs.
    • The name of the image is encoded in the url parameters. In the above example, the url parameters are "?category-image=person,2011_000745_10.jpg", which means that the image location is - base_url + "/images/person/2011_000745_10.jpg". The base_url parameter is specified in the source code of the interface.
  • Attributes of people
    • Label various attributes like hairtype, clothing type, gender, etc of people.
  • Tagging keypoints for PASCAL VOC categories
    • Tag each image with a list of keypoints. See source code to configure the names of keypoints and category types.
  • Head and torso pose tagger
    • Estimate the 3d pose of the head and torso by adjusting two guage figures

Here is a short report describing the details of the interface and other design choices.

Here is a presentation I gave at RAD Lab, Berkeley and Intel Research, Berkeley on the same.

Here is a social experiment I did about measuring the influence of external sources of information on how people rate abstract art on AMT. This was a course project in the Social Choice and Networks, Fall'10 course taught by Elchanan Mossel.

The data collected is being used in several projects. See the relevant publications to access the data. In addition please cite the following if you use the data and/or the inferface for your work:

@techreport{Maji:EECS-2011-79,
    Author = {Maji, Subhransu},
    Title = {Large Scale Image Annotations on Amazon Mechanical Turk},
    Institution = {EECS Department, University of California, Berkeley},
    Year = {2011},
    Month = {Jul},
    URL = {http://www.eecs.berkeley.edu/Pubs/TechRpts/2011/EECS-2011-79.html},
    Number = {UCB/EECS-2011-79},
}