“By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.” ― Eliezer Yudkowsky

We’ve reached the end of the semester.

This semester, I tried to give you a guided tour of topics in Artificial Intelligence. I hope that the material we covered has weighted your (biological) neural networks appropriately, and helps you in the future when the need or interest arises.

Please get in touch if you’re around in the future. I’d love to hear what you’re up to.

Cheers,
Marc

(90,100]: 2  
(80,90]: 10
(70,80]: 13
(60,70]:  8
(50,60]:  4
(0,50]:   5

The test cases referenced in the autograder’s output for are available for download here: assignment10-test-cases.tar.gz

A sample solution in Java is available here: HMMInference.tar.gz. The implementation of the filtering algorithm is straightforward and two-part as described in class. The implementation of the Viterbi algorithm is an ugly port of my first draft in Python to Java. I wish I had time to make it beautiful for you all, but like you, I am pressed for time at the end of the semester.

As usual, if you think something in the grading was incorrect (and not in your favor), please email or come to see me.

(90,100]: 8  
(80,90]: 10
(70,80]:  9
(60,70]:  5
(50,60]:  6
(0,50]:   4

I will have regular office hours Monday; I will email and post on the blog if there is a location change, and if I plan a separate review session.

A sample solution in Java is available here: KMeans.tar.gz. It implements the algorithm for k-means clustering with the constraints given in the assignment. When computing Euclidean distance between points, it computes the sum of the squares of the dimension-wise differences, but doesn’t take the square root of that sum. This is fine, since the square root is a monotonically increasing function of its input.

Some notes on the grading from Patrick:

Some people misunderstood the algorithm.

• For example, one person updated each cluster center by assigning it the value of the data point closest to the mean of the cluster rather than the mean itself.
• Another person did something with random numbers that I didn’t understand.
• Another did not iterate, outputting the first cluster assignment.

I also noticed a pair of mistakes crop up more than once.

• One mistake was not handling greater than 1 dimension correctly (by ignoring the additional values, or doing something else that was unclear to me). In this case, the two 1d tests passed, but the other tests had errors.
• One more subtle mistake was using Manhattan distance (the sum of the absolute value of the dimension-wise differences) rather than the Euclidean distance as specified. This error resulted in only a small number of mis-labelings in the 2d case, but many in the 10d case.

As usual, if you think something in the grading was incorrect (and not in your favor), please email or come to see Patrick or me.

A sample solution in Java is available here: DecisionTree.tar.gz. It mostly follows the algorithm in Russell and Norvig. It omits the first termination condition (“if examples is empty”) since that case will not arise in a tree composed of only boolean attributes (though it didn’t hurt to leave it in). You can run the program with only training data specified on the command line to see a crude printout of the tree it learns.

Some notes on the grading:

• Unlike the naive Bayes classifier, there is ambiguity in the tree your program could correctly learn. If there is a tie in information gain, it’s unspecified which attribute the program should select to split on.
  • The small test cases are all intended to be unambiguous in terms of output, so your program’s output was evaluated strictly. That is, they had to match the expected output exactly. If you believe I am mistaken here, please send me an email outlining your reasoning. If I’m wrong I’ll be more than happy to give you (and everyone else) their points back.
  • The large test cases are somewhat ambiguous, depending upon the exact tree your program constructed, so I evaluated liberally. This means that I checked that the labels were right (I did not examine the probabilities), and that I allowed for a small fraction of misclassification relative to my sample implementation (up to 15% different and you still received full credit).
• A common mistake was to stop splitting early when all attributes had an information gain of zero. Some functions require splitting on several attributes. The XOR small test caught this error.

As usual, if you think something in the grading was incorrect (and not in your favor), please email or come to see Patrick or me.

At 1200 on Wednesday 03 December in room 343 203 of the Computer Science building, I will hold a question-and-answer-style review session for Exam 3. This room is not my office – it’s a conference room down the hall.

It may be helpful to look over the review sheet before this session.

As is the case for the previous assignment, a couple of extensions are still in effect, so for reasons of fairness I have not yet posted the test cases and the sample solution. I will post them as soon as all deadlines are expired. I’ll have more to say about the grading then.

A couple of extensions are still in effect, so for reasons of fairness I have not yet posted the test cases and the sample solution. I will post them as soon as all deadlines are expired. I’ll have more to say about the grading then.

The test cases referenced in the autograder’s output are available for download here: assignment07-test-cases.tar.gz

A Java sample solution is available here: NaiveBayesClassifier.tar.gz. The useAlternativeMethod flag indicates which way the classifier will handle ? values (see below).

Some notes on the tests and grading:

• Correctness was defined as outputting both the correct class and the correct probability of that class. With a few minor exceptions detailed here and that we accounted for when grading, there was no ambiguity in the correct output. Just outputting the right class label was not sufficient for credit, nor was getting anything but close (±0.01) to the correct probability.

• There were two sets of tests. The small tests were on synthetic data and intended to evoke specific types of errors. For one of them, I used a student’s test that was posted to Moodle; many people used it to validate their submission so I thought you should get credit for it. The large tests were on subsets of the full vote data.

• No test explicitly pushed at the boundaries of underflow. (Do not take this to mean that the tests in Assignment 10 won’t!) The log transform can lose a small bit of precision, but not enough to matter to the tests we did (within ±0.01 of the correct value). It’s not actually necessary to use it unless underflow occurs—look at estimateProbability() in the solution for an example.

• We accounted for floating-point roundoff errors around 0.5, for example, democrat,0.5000000000000004 was acceptable when republican,0.5 was the correct answer.

• As noted in the assignment question and answer section, we accepted two distinct ways of handling unknown values. If your program’s output didn’t match the way that I specified, the autograder re-checked it against the way that Patrick specified. These are the .alternative solutions.

As usual, if you think something is amiss, please email or come see me or Patrick.

To complete it, you can re-do a previous assignment or choose a new assignment.

The test cases referenced in the autograder’s output are available for download here: assignment06-test-cases.tar.gz

Two notes:

• Because the simulator(s) you wrote were stochastic, the autograder accepted as correct any output within 15% of the correct value. That is, for each possible setting of query values, if |prob_expected - prob_observed| / prob_expected < 0.15, then your simulator passed the given test case. There was a chance some values would be wrong just due to randomness. Patrick watched for this case and adjusted grades upward if he suspected it was occurring. If you think he overlooked such a case, send us both email and we’ll investigate. Generally, if this occurred at all, it occurred on test case 5-j-m.query.
• According to Patrick, “Many students parsed and/or created json with their own code rather than a library, which caused many mistakes.”

As usual, if you think something is amiss, please email or come see me or Patrick.

I had the brilliant idea (ha-ha) that I’d write this one up in a largely imperative style, passing and mutating arrays and array indices, in an attempt to get it to run quickly. It does run fairly fast (the Gibbs sampler gets around 2M samples / second on my desktop computer). Part of the cost of using this style is that it took a little longer than I expected to complete; it also is less readable than I’d like.

A note: Each term exam is weighted equally. This exam is not worth twice as much as Exam 1!

This is a fairly direct translation that takes the same approach as the Python previously posted. The entire dataset is loaded, then filtered by the condition in the query. Then, each line of the output is generated by creating another subset of the data, corresponding to exactly one possible setting of the query variables.

I suggest you eyeball the Cartesian product code in Query.java. The implementation I wrote is recursive. Recursion is a convenient way to think about things, but can cause problems in some languages (like Java) if too many recursive calls are nested. With prior knowledge of the data set we were working with, I knew it wouldn’t be a problem here.

You should understand how this method works, and perhaps take the time to translate it to an iterative form if you’re rusty on the equivalence between recursive and iterative algorithms. If you don’t know how to do that, then dust off your 220 notes, or come see me and we can talk about it.

The solution clocks in at about 150 source lines of code, which is within the typical range of expansion I expect when going from Python to Java (1.5x – 3x). The extra length in Java comes from a few things. For example, Python has less boilerplate and supports very useful syntax sugar to reduce code length (such as list comprehensions).

Python also has a more comprehensive standard library. The input code in Python is a line or two, where Java took about a dozen; similarly, generating the Cartesian product of variable settings is a library call in Python but needs to be written in Java. You can add third-party JARs to Java to help cut down on code length (e.g., OpenCSV to read car.data and Guava for many things, including Cartesian product). I chose not to do that so as to keep this solution self-contained, but you might consider it in future assignments. A drawback to this approach is that you have to understand the third-party API well enough to be sure you’re using it correctly.