Programming Assignment 07: Image Mosaic
Estimated reading time: 15 minutes
Estimated time to complete: 60–150 minutes (plus debugging time)
Prerequisites: Assignment 05, Lab 05
Starter code: image-mosaic-student.zip (see note!)
Collaboration: not permitted
Note (2016-10-28)
The starter code was updated around 4pm today to include the final test for Mosaic. If you want to use this test, you can extract the zipfile and move two files from its contents into your local workspace. MosaicTest.java needs to go into test/mosaic (the same directory as TileFactoryTest.java), and mosaic.png needs to go into test/resources (the same directory as nature.jpg). Finally, note that MosaicTest.java requires that you use an unmodified copy of MosaicDriver.java.
I’m still waiting on Gradescope for (I hope) one last problem to be resolved before I can open up the autograder. But the tests on the autograder and the JUnit tests you have are the same, so you are able to work on the assignment. I’ll send a message using Piazza once the autograder tests are up, and if necessary, extend the due date so that everyone has a full 24 hours to submit.
Overview
Image mosaics (or photographic mosaics) are pictures that have been divided into tiles, and had each tiles replaced by a smaller picture of approximately the same color as in the tile in the original picture.
In this assignment, you’ll implement a simplified image mosaic generator. We’ve provided scaffolding code, but you’ll need to develop and debug a system for loading images as tiles, and for selecting and compositing image tiles based upon a target image.
We’ve provided a small set of unit tests to help with automated testing, as well as an example “driver” class with a main method. The Gradescope autograder includes a few more tests, but they exist primarily to verify you’re not gaming the autograder. If your code can pass the tests we’ve provided, it is likely correct.
Note that if you run into trouble with the Eclipse debugger mysteriously quitting during unit tests, it’s due to the timeout rule that we use to catch infinite loops:
@Rule
public Timeout globalTimeout = Timeout.seconds(10); // 10 seconds
Comment out the above two lines in all test files, and the debugger will no longer exit (and test cases will now get stuck in infinite loops).
Goals
- Translate written descriptions of behavior into code.
- Practice representing state in a class.
- Practice interacting with the
MapandListabstractions. - Practice using external JARs.
- Test code using unit tests.
Downloading and importing the starter code
As in previous assignments, download and save (but do not decompress) the provided archive file containing the starter code. Then import it into Eclipse in the same way; you should end up with a image-mosaic-student project in the “Project Explorer”.
Once again, we’re using the Processing library for graphics manipulation.
Top-down: Code walkthrough
Start by taking a look at the MosaicDriver. The MosaicDriver draws an image mosaic based upon a target image and several tile images, and it depends upon each other part of the other two parts of the program (the Mosaic and the TileFactory). Let’s walk through it to give you a sense of what each component should be doing.
Note that this section describes the functionality of MosaicDriver.setup.
Palette selection
There are many ways you can create an image mosaic, but they all revolve around dividing a target image into tiles, then replacing those tiles with tiles from a source library based upon some attribute. Our code will compare colors to find matching tiles.
Our code uses a single value to represent the color; the MosaicDriver has a very simple palette of just three colors, of red, green, and blue. The three parameters to PApplet.color are each on the 0..255 scale, and represent how much red, green, and blue each color is composed of. (We could have a bigger palette, or use a more advanced way to find matching tiles, but that’s not the focus of this assignment.)
Building the TileFactory
Next we instantiate the TileFactory. The TileFactory needs to know about the PApplet instance (in order to call various useful instance methods), it needs to know the palette, and it needs to know the tile width and height.
Once it’s instantiated, it is populated with tiles. Tiles are images that are loaded from disk, resized to the tile size, and then placed into lists, organized by hue.
Loading and mosaic-ing the target image
The MosaicDriver then loads the target image from disk, and builds a Mosaic using the current PApplet instance, the target image, and the TileFactory. The Mosaic instance is responsible for breaking the target image into tiles, and replacing each tile with a tile from the TileFactory. Once it’s done so (in the buildMosaic method), that image is drawn to the screen.
Bottom-up: What to do
If you run the MosaicDriver before making any changes, you’ll see it just draws the input image, not a mosaic. That’s because Mosaic.buildImage is not complete; right now it just returns the target image.
It’s probably easiest to start “bottom-up” on this assignment. That is, rather than working on the Mosaic code first, start with the TileFactory. The rest of this writeup walks you through what you’ll need to do.
TileFactory, the palette, and tiles
First you’ll want to adjust how TileFactory stores the palette. You’re going to associate each hue (which is a property of a color, see: https://en.wikipedia.org/wiki/Hue)in the palette with one or more tile images. Right now, there’s an int[] array declared and initialized to hold the palette of hues, but it’s not a great choice.
You should change the TileFactory to instead use a Map to hold the mappings between hues and lists of images. As you’ve seen in the MosaicDriver, images will be represented as instances of the class PImage, so declare your map and initialize its key-value pairs appropriately. You’ll need to keep the use of pApplet.hue() to convert the input color (an integer) into an hue (also an integer).
Now look at addImage; you’ll see it already computes the average hue of an image for you, but it doesn’t do anything with that information. You’ll want to determine which hue in the palette is closest to the average hue of the image and insert the image into the corresponding list in your map.
How are you going to find the closest hue? Take a look at both of closestHue and hueDistance. The former will rely upon the latter (since “closest” relies upon a notion of distance), so let’s tackle it first. In Processing (the image library we’re using) a hue is a by default value between 0 and 255, but kind of like the numbers on a clock (1–12), hues “wrap around.” Your first instinct may be to use something like Math.abs(hue1 - hue2), which is close, not not quite right. Use the clock analogy to figure out the right mathematical expression. For example, in a 12-hour clock, the difference between 11 o’clock and 3 o’clock is four hours, not eight (abs(11 - 3)). Once you figure this out for a 12-hour clock, then you can use the almost the same expression for a 256-value hue.
closestHue is a straightforward application of hueDistance. Compare the input hue against each of the hues in the palette, and return the best match. Use closestHue to complete addImage by adding the image to the list associated with the hue closest to the image’s average hue.
Finally, you’ll need to write getTile. getTile should look up the list of images associated with the hue in the palette most closely matching the input hue. It should return the “next” image in that list. The idea here is that each time that getTile returns a tile, it should return a different one from the one before. In particular, it will return the next (to the right) element of the list. You could keep a counter, but I suggest you rotate the list left one space (perhaps using Collections.rotate with the appropriate argument) instead.
Mosaic and building the image
The buildMosaic method is where the magic happens, and probably where you’re going to have to spend the most time.
Note that the method starts by calling loadPixels on the image. Before you start working on an image’s pixels, you need to call this method once. And when you’re done manipulating an image, you need to call updatePixels.
The int[] pixels array in a PImage represent each pixel in the image as a RGB color value, stored as an int. What you need to know here is that each pixel has an X and Y coordinate; the upper-left of the image is (0, 0); moving right one pixel increases the X coordinate, and moving down increases the Y coordinate.
First, figure out how to divide the image into tiles. You know the tile width and height (from the TileFactory), so it should be straightforward to determine how many tiles wide and high the image is, based upon the image‘s’ width and height. Note that if the target image is not exactly a multiple of the tile width (or height), the mosiac image should be truncated so that it is an even multiple. That is, you can ignore the “edges” of the target with the largest X and Y coordinates if they’re not a full tile wide.
Then create a new PImage to hold the mosaic – don’t modify the original image in place! To create the mosaic image, you should not use PImage‘s constructor. Instead, use the createImage method of the pApplet. The first two parameters should be the width and height of the mosiac, and the third, the color format, should be PImage.RGB. (How would you know this? By reading the documentation for Processing. But I’m trying to spare you some of that work in this assignment.)
Then, you’ll need to consider each tile of the target image. You’ll probably end up writing a pair of nested for loops to do this, but whatever approach you take is fine. Use the get method on the target image to retrieve a new PImage, where get takes four parameters: the x and y coordinate of the upper-left pixel of the tile, then width and height, which are exactly the tile width and height. Don’t forget to call loadPixels on this PImage of the tile, then you can access its .pixels (an int[]).
How do you find the upper-left coordinate of a tile? The tile in the upper left is at (0,0). The tile to its right is at (tile width, 0); the tile to the right of that is at (2 * tile width, 0). The tile below the upper-left tile is at (0, tile_height); the tile to its right is at (tile width, tile height); and so on.
Now that you can select a tile-sized image from the original image, you need to get a tile from the TileFactory that matches it. Remember, we’re matching by hue, so use ImageUtils.averageHue to find the average hue of your tile from the source image, then use getTile from your TileFactory to get the next mosaic tile of this hue from its palette. Finally, copy this mosaic tile into the mosaic image by using the copy method of the mosaic image. copy takes seven (!!) parameters. The first should be the mosaic tile image; the next four are the source coordinates (and should be 0, 0, tile width, tile height), and the next four are the location in the mosaic to copy into (and should be x coordinate, y coordinate, tile width, tile height).
To pass the final test in MosaicTest, you’ll need to select tiles in a particular order. In particular, select columns (left-to-right), then within each column, top-to-bottom. This will happen naturally if you write a pair of nested for loops, where the outer loop iterates over the number of tiles across in the mosaic, and the inner loop over the number of tiles up-and-down.
When you’re all done, don’t forget to call updatePixels on your mosaic PImage before returning it.
Et voila
Running the MosaicDriver once you’ve finished should turn this:
into this:
Oh dear. Well, perhaps the world will have to wait for the unveiling of our next masterpiece.
Submitting the assignment
When you have completed the changes to your code, you should export an archive file containing the src/ directory from your Java project. To do this, follow the same steps as from Assignment 01 to produce a .zip file, and upload it to Gradescope.
Remember, you can resubmit the assignment as many times as you want, until the deadline. If it turns out you missed something and your code doesn’t pass 100% of the tests, you can keep working until it does.
Improving the mosaic
There are many, many ways that Mosaic and TileFactory could be improved. First, a (much) larger selection of tiles and palette hues will allow closer matching. Next, taking into account other aspects of color (saturation and brightness) for better matches will make further improve the output. A more advanced tile matcher will also match areas of color within tiles, and possible rotate or flip tiles to make the best possible matches. You might also “cheat” by modifying each tile as its placed, adjusting its hue, saturation, or brightness (“tinting,” etc.) to better match the source image at that location.
All of this is well beyond the scope of this assignment, but might be interesting to do in your spare time if you are so inclined.