Paul Utgoff - In Memoriam (1951 - 2008)

Contact Information

Paul Utgoff directs the Machine Learning Laboratory in the College of Information & Computer Sciences at the University of Massachusetts at Amherst.

Research Focus

My main research interests are rooted in the longstanding fundamental problem of how intelligent systems can acquire features, terms, and other representational structures that are prerequisite to further learning. Simple learning can be accomplished by application of a statistical pattern-finding algorithms, but how are learned patterns employed in the grander design of sustained learning over the lifetime of an intelligent entity? In terms of Clark & Thornton's compelling distinction between Type-1 and Type-2 learning, I am completely intrigued with the issues involved in understanding and modeling Type-2 learning, which involves building representational mappings when no statistical relationship exists between the independent and dependent variables. Quartz has also described well this dichotomy between setting up representations (Type-2) and then making use of those representations for the better understood processes of identifying simple input/output mappings (Type-1). What mechanisms can account for sustained Type-2 and Type-1 learning? For me, this is the most exciting and most central problem in attaining new levels of mechanical intelligence.

I have been working on algorithms that incidentally solve Type-1 problems when present and possible, each one producing a building block that joins a many-layered deeply-nested organization of such elements. Earlier learning lays the groundwork for later learning. I am guided by the principle that all learning is simple if the prerequisites are in place. I am quite fascinated by textbooks, as each presents a large body of coherent knowledge carefully organized in terms of building blocks and their dependencies.

Present Students

I am ably assisted by my graduates students:

• David Stracuzzi (Ph.D. expected Spring 2005)
• Laylaa Ali (M.S. expected Summer 2004)
• Stephen Murtagh (Ph.D. student)
• Gary Holness (Ph.D. student)

Classification of Phytoplankton Images

UMass researchers from the Computer Vision Laboaratory and the Machine Learning Laboratory are collaborating with marine scientists from Bigelow Laboratory to build image classification systems that are capable of discriminating a large variety of phytoplankton. The ability to automate analyses of samples in various settings will enable studies of such ocean life to be conducted at much lower cost, and in a much more timely manner. We are investigating classification algorithms, ensemble classifiers, and feature sets that enable separation of useful classes to apply to challenging real-world problems.

Music Perception

I am collaborating on automated music perception with Prof. Chris Raphael of our Mathematics and Statistics Department. Ideally, we would like to be able to take as input an audio signal of a music performance, and produce as output a music score for the selection that was performed. There are many subproblems, including determining the fundamental pitch events, pulse, meter, key signature, intentional variations of pitch and note duration, articulation identification, voice leading and chord identification. We approach many of these problems starting with a midi performance instead of an audio signal, to factor out the problem of correctly identifying pitch events. If these kinds of problems interest you, and you have studied music performance, come get involved.

Decision Tree Induction

I have developed and I maintain the ITI (Incremental Tree Inducer) and DMTI (Direct Metric Tree Induction) decision tree induction systems. I welcome questions, suggestions, and hearing of useful applications.

Ph.D. Graduates

• Jeffery Clouse, Research Scientist, Lucent Technologies
• Carla Brodley, Associate Professor, Purdue University
• Tom Fawcett, Research Scientist, Hewlett-Packard Laboratories
• Jamie Callan, Associate Professor, Carnegie-Mellon University
• Sharad Saxena, Senior Member Technical Staff, Texas Instruments.

Publications

• Stracuzzi, D.J., & Utgoff, P.E. (accepted). Randomized variable elimination. Journal of Machine Learning Research.
• Precup, D., & Utgoff, P.E. (2004). Classification using Phi-machines and constructive function approximation. Machine Learning, 55, 31-52.
• Stracuzzi, D.J., & Utgoff, P.E. (2002). Randomized variable elimination. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 594-601).
• Utgoff, P.E., & Stracuzzi, D.J. (2002a). Many-layered learning. (.ps) (.pdf) Neural Computation, 14, 2497-2539.
• Utgoff, P.E., & Stracuzzi, D.J. (2002b). Many-layered learning. Proceedings of the Second International Conference on Development and Learning (pp. 141-146).
• Utgoff, P.E., & Cochran, R.P. (2001). A least-certainty heuristic for selective search. (.ps) (.pdf) Proceedings of the Second International Conference on Computers and Games (pp. 1-18). Springer Verlag.
• Utgoff, P.E. (2001). Feature construction for game playing (pp. 131-152). (.ps) (.pdf) In Fuerenkranz & Kubat (Eds.), Machines that learn to play games. Nova Science Publishers.
• Utgoff, P.E., & Stracuzzi, D.J. (1999). Approximation via value unification. (.ps) (.pdf) Proceedings of the Sixteenth International Conference on Machine Learning (pp. 425-432). Ljubljana: Morgan Kaufmann.
• Piater, J.H., Riseman, E.M., & Utgoff, P.E. (1999). Interactively training pixel classifiers. International Journal of Pattern Recognition and Artificial Intelligence, 13, 171-193.
• Utgoff, P.E. (1998). Decision trees (pp. 222-224). (.ps) (.pdf) In Wilson & Keil (Eds.), The MIT encyclopedia of cognitive sciences. Bradford.
• Precup, D., & Utgoff, P.E. (1998). Classification using Phi-machines and constructive function approximation. Proceedings of the Fifteenth International Conference on Machine Learning (pp. 439-444).
• Utgoff, P.E., & Precup, D. (1998b). Constructive function approximation (pp. 219-235). (.ps) (.pdf) In Motoda & Liu (Eds.), Feature extraction, construction, and selection: A data-mining perspective. Kluwer.
• Piater, J., Riseman, E., & Utgoff, P.E. (1998). Interactively training pixel classifiers. Eleventh International FLAIRS Conference (FLAIRS-98) (pp. 57-61).
• Schmill, M.D., Rosenstein, M.T., Cohen, P.R., & Utgoff, P.E. (1998). Learning what is relevant to the effects of actions for a mobile robot. Proceedings of the Second International Conference on Autonomous Agents (pp. 247-253).
• Moss, J.E.B., Utgoff, P.E., Cavazos, J., Precup, D., Stefanovic, D., Brodley, C., & Scheeff, D. (1998). Learning to schedule straight-line code. Advances in Neural Information Processing Systems (pp. 929-935). San Mateo, CA: Morgan Kaufmann.
• Utgoff, P.E., Berkman, N.C., & Clouse, J.A. (1997). Decision tree induction based on efficient tree restructuring. (.ps) (.pdf) Machine Learning, 29, 5-44.
• Brodley, C.E., & Utgoff, P.E. (1995). Multivariate decision trees. Machine Learning, 19, 45-77.
• Draper, B.A., Brodley, C.E., & Utgoff, P.E. (1994). Goal-directed classification using linear machine decision trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 888-893.
• Utgoff, P.E. (1994). An improved algorithm for incremental induction of decision trees. Proceedings of the Eleventh International Conference on Machine Learning (pp. 318-325). Morgan-Kaufmann.
• Brodley, C.E., & Utgoff, P.E. (1994). Dynamic recursive model class selection for classifier construction. In Cheeseman & Olford (Eds.), Selecting models from data: AI and statistics IV.
• Brodley, C.E., & Utgoff, P.E. (1992). Dynamic recursive model class selection for classifier construction. Proceedings of the Fourth International Workshop on Artificial Intelligence and Statistics.
• Clouse, J.A., & Utgoff, P.E. (1992). A teaching method for reinforcement learning. Proceedings of the Ninth International Conference on Machine Learning (pp. 92-101). Aberdeen: Morgan Kaufman.
• Fawcett, T.E., & Utgoff, P.E. (1992). Automatic feature generation for problem solving systems. Proceedings of the Ninth International Conference on Machine Learning (pp. 144-153). Aberdeen: Morgan Kaufman.
• Utgoff, P.E., & Clouse, J.A. (1991). Two kinds of training information for evaluation function learning. (.ps) (.pdf) Proceedings of the Ninth National Conference on Artificial Intelligence (pp. 596-600). Anaheim, CA: AAAI Press/The MIT Press.
• Callan, J.P., & Utgoff, P.E. (1991). Constructive induction on domain information. Proceedings of the Ninth National Conference on Artificial Intelligence (pp. 614-619). Anaheim, CA: AAAI Press/The MIT Press.
• Fawcett, T.E., & Utgoff, P.E. (1991). A hybrid method for feature generation. Proceedings of the Eighth International Workshop on Machine Learning (pp. 137-141). Evanston, IL: Morgan Kaufman.
• Callan, J.P., & Utgoff, P.E. (1991). A transformational approach to constructive induction. Proceedings of the Eighth International Workshop on Machine Learning (pp. 122-126). Evanston, IL: Morgan Kaufman.
• Yee, R.C., Saxena, S., Utgoff, P.E., & Barto, A.G. (1990). Explaining temporal-differences to create useful concepts for evaluating states. Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 882-888). Cambridge, MA: AAAI Press/The MIT Press.
• Utgoff, P.E., & Brodley, C.E. (1990). An incremental method for finding multivariate splits for decision trees. (.ps) (.pdf) Proceedings of the Seventh International Conference on Machine Learning (pp. 58-65). Austin, TX: Morgan Kaufmann.
• Utgoff, P.E. (1989a). Perceptron trees: A case study in hybrid concept representations. Connection Science, 1, 377-391.
• Utgoff, P.E. (1989b). Incremental induction of decision trees. (.ps) (.pdf) Machine Learning, 4, 161-186.
• Utgoff, P.E. (1989c). Improved training via incremental learning. Proceedings of the Sixth International Workshop on Machine Learning (pp. 362-365). Ithaca, NY: Morgan Kaufmann.
• Utgoff, P.E. (1988a). Perceptron trees: A case study in hybrid concept representations. Proceedings of the Seventh National Conference on Artificial Intelligence (pp. 601-606). St.Paul, MN: Morgan Kaufmann.
• Utgoff, P.E. (1988b). ID5: An incremental ID3. Proceedings of the Fifth International Conference on Machine Learning (pp. 107-120). Ann Arbor, MI: Morgan Kaufmann.
• Utgoff, P.E., & Heitman, P.S. (1988c). Learning and generalizing move selection preferences. Proceedings of the AAAI Symposium on Game Playing (pp. 36-40). Palo Alto, CA.
• Connell, M.E., & Utgoff, P.E. (1987). Learning to control a dynamic physical system. Computational Intelligence, 3, 330-337.
• Utgoff, P.E., & Saxena, S. (1987). Learning a preference predicate. Proceedings of the Fourth International Workshop on Machine Learning (pp. 115-121). Irvine, CA: Morgan Kaufmann.
• Connell, M.E., & Utgoff, P.E. (1987). Learning to control a dynamic physical system. Proceedings of the Sixth National Conference on Artificial Intelligence (pp. 456-460). Seattle, WA: Morgan Kaufmann.
• Utgoff, P.E. (1986a). Machine learning of inductive bias. Hingham, MA: Kluwer.
• Utgoff, P.E. (1986b). Shift of bias for inductive concept learning (pp. 107-148). In Michalski, Carbonell & Mitchell (Eds.), Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
• Utgoff, P.E. (1984). Shift of bias for inductive concept learning. Doctoral dissertation, Department of Computer Science, Rutgers University, New Brunswick, NJ.
• Utgoff, P.E. (1983a). Adjusting bias in concept learning. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (pp. 447-449). Karlsruhe, FRG: Morgan Kaufmann.
• Utgoff, P.E. (1983b). Adjusting bias in concept learning. Proceedings of the Second International Workshop on Machine Learning (pp. 105-109). Monticello, IL.
• Mitchell, T.M., Utgoff, P.E., & Banerji, R.B. (1983). Learning by experimentation: Acquiring and refining problem-solving heuristics (pp. 163-190). In Michalski, Carbonell & Mitchell (Eds.), Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
• Utgoff, P.E., & Mitchell, T.M. (1982). Acquisition of appropriate bias for inductive concept learning. Proceedings of the Second National Conference on Artificial Intelligence (pp. 414-417). Pittsburgh, PA: Morgan Kaufmann.
• Mitchell, T.M., Utgoff, P.E., Nudel, B., & Banerji, R.B. (1981). Learning problem-solving heuristics through practice. Proceedings of the Seventh International Joint Conference on Artificial Intelligence (pp. 127-134). Vancouver, B.C.: Morgan Kaufmann.