CS 692P: Hot Topics in Software Engineering Research

Fall 2024

News:

Description:

CS 692P is a seminar-style course that introduces students to the cutting edge literature in software engineering.


Logistics:

Room:CS Building, room 140
Seminar:Thursday 2:30–3:30PM
Instructors:

Juan Zhai juanzhai@umass.edu
Yuriy Brun

Grading:

Each student will be responsible for being part of the presentation group for eight papers during the semester. Grading is based on active participation during discussions and effectiveness of the presentations made by the students.

Class Format:

Each week will have an assigned paper covering research software engineering research advances. Each student will be responsible for reading the paper, every week. In class, we will discuss the paper we all read that week. There will be a team of five students presenting the paper each week, with each student in charge of a specific role. The roles are paper author, academic researcher, academic historian, industry practitioner, and scientific peer reviewer. These roles are inspired by Colin Raffel's paper-reading seminars.

Paper author needs to study the paper and prepare a presentation of about 7 minutes to deliver in class, focusing on highlighting the contributions and main results on the work.

Academic researcher needs to use the paper to propose 2–3 creative follow up research ideas inspired by this paper. The proposed research should rely on technology or findings made by the paper discussed. This proposed work should not be a copy of future work mentioned in the paper, but rather be new ideas. The academic researcher should prepare a set of slides, one per new research idea proposed and be prepared to talk for about 7 minutes about these ideas.

Academic historian is in charge of describing the timeline of work that came before and after the paper under discussion. The historial should produce 3–4 slides that detail the context of prior work that this paper is based on and subsequent work that is based on this paper. (You can use Google scholar to find subsequent papers that cite this paper.) The presentation should briefly describe at least one older paper that the discussed paper is based on, and at least one newer paper that cites the current paper (if none exist, discuss another older paper). The presentation should take about 5 minutes.

Industry practitioner works for a company or organization that may choose to implement the technology described in the paper as part of their application or product. Prepare a convincing pitch (2 slides) for why you should be paid to implement the method in the paper, and discuss at least one possible benefit and one potential disadvantage for the application.

Scientific peer reviewer reviews this paper as if it is under submission to a top-tier conference. The reviewer writes a review following a typical format, including a summary paragraph of the paper, a list of strengths of the work, a list of weaknesses of the work, and supporting evidence for each strength and weakness. The reviewer will summarize their review in class, arguing for each of the strengths and weaknesses. This presentation should take about 7 minutes. You can see instructions for writing a review below.


Schedule:

(subject to change; check regularly)

week date paper student presenter
Paper Author Academic Researcher Academic Historian Industry Practitioner Scientific Peer Reviewer
Week 1
Sep 5 Organizational meeting Juan and Yuriy
Week 2
Sep 12 ITER: Iterative Neural Repair for Multi-Location Patches (ICSE 2024) Kyla Levin Nisarg Parikh Harold Thidemann Yuriy Brun
Week 3
Sep 19 RUNNER: Responsible UNfair NEuron Repair for Enhancing Deep Neural Network Fairness (ICSE 2024) Yesugen Baatartogtokh Gehao Zhang Anders Freeman Nisarg Parikh Kyla Levin
Week 4
Sep 26 A Deep Dive into Large Language Models for Automated Bug Localization and Repair (FSE 2024) Anders Freeman Harold Thidemann Yesugen Baatartogtokh Gehao Zhang Nisarg Parikh
Week 5
Oct 3 NO CLASS (Attend DLS on Oct 2 instead)
Week 6
Oct 10 RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair (FSE 2023) Kyla Levin Zhanna Kaufman Gehao Zhang Harold Thidemann Anders Freeman
Week 7
Oct 17 Automated Repair of Programs from Large Language Models (ICSE 2023) Gehao Zhang Nisarg Parikh Anders Freeman Ruchi Gupta Kyla Levin
Week 8
Oct 24 PyTy: Repairing Static Type Errors in Python (ICSE 2024) Ruchi Gupta Anders Freeman Harold Thidemann Zhanna Kaufman Gehao Zhang
Week 9
Oct 31 Impact of Code Language Models on Automated Program Repair (ICSE 2023) Harold Thidemann Yesugen Baatartogtokh Nisarg Parikh Anders Freeman Ruchi Gupta
Week 10
Nov 7 KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair (ICSE 2023) Gehao Zhang Nisarg Parikh Yesugen Baatartogtokh Zhanna Kaufman Harold Thidemann
Week 11
Nov 14 Constraint Based Program Repair for Persistent Memory Bugs (ICSE 2024) Ruchi Gupta Harold Thidemann Zhanna Kaufman Kyla Levin Gehao Zhang
Week 12
Nov 21 Template-Guided Program Repair in the Era of Large Language Models (ICSE 2025) Nisarg Parikh Zhanna Kaufman Ruchi Gupta Yesugen Baatartogtokh
Week 13
Nov 28 NO CLASS (Thanksgiving)
Week 14
Dec 5 Code Comment Inconsistency Detection and Rectification Using a Large Language Model (ICSE 2025) Zhanna Kaufman Ruchi Gupta Kyla Levin Yesugen Baatartogtokh

Paper Review:

A paper review consists of three elements:

Paper Summary: The summary should be a paragraph (about 10 sentences) that overviews the main points of the paper. The summary should explain at a high level: what problem the paper focuses on; why the problem is important; why existing solutions are insufficient; the main insights and contributions of the proposed algorithm(s), method, approach, or system, and how the paper argues that the proposal is successful (its evaluation). The summary needs to show an understanding of the paper's main focus and correctly identify the main ideas that the paper has to offer.

Paper Strengths: Identify at least two strengths of the paper with respect to its contributions. These should be about the scientific contributions and not about writing style (e.g., "very well-written paper" is not an acceptable strength). A strength is something that can be used to argue that this paper should be accepted for publication. The strengths should identify important insights (e.g., "the paper solves X through an innovative combination of techniques A and B; this is surprising, because technique B was designed for a completely different problem"). Each strength should be supported by evidence from the paper and be concrete (general comments, such as "strong evaluation" are too vague to be used to argue for acceptance.

Paper Weaknesses: Identify at least two weaknesses or extensions of the paper. Again, focus on concrete examples, such as insufficient evidence the evaluation provides for a claim, unstated or unrealistic assumptions, or a result that is not likely to generalize beyond the presented evaluation. These should not be superficial comments, such as "the evaluation can be augmented." Instead, identify exactly what you find is missing and why you think it is important. If you cannot find weaknesses, then list meaningful extensions to the work presented in the paper. What would be the next step to take in terms of altering the problem focus, removing assumptions, augmenting the methodology in some way, etc? The strengths and weaknesses need to be specific to the work being evaluated. If you find that the stated strength of weakness can be easily applied to virtually any paper (e.g., "the writing is missing details about the implementation"), then it is not a good strengths and weaknesses to use in your review.


Nondiscrimination policy:

This class includes and welcomes all students regardless of age, background, citizenship, disability, sex, education, ethnicity, family status, gender, gender identity, geographical origin, language, military experience, political views, race, religion, sexual orientation, socioeconomic status, and work experience. Our discussions and learning will benefit from these and other diverse points of view. Any kind of language or action displaying bias against or discriminating against members of any group, or making members of any group uncomfortable are against the mission of this course and will not be tolerated. The instructor welcomes discussion of this policy, and encourages anyone experiencing concerns to speak with him.


Academic integrity:

Students are allowed to work together on all aspects of this class. Students are encouraged to use the Internet, literature, and other publicly-available resources, but must disclose whenever such materials were used, e.g., in the preparation of the in-class presentation.

The use of AI-based and other generative technologies (such as, but not limited to ChatGPT) is not allowed in this class.

The students should familiarize themselves with the UMass Academic Honesty Policy and Guidelines for Classroom Civility and Respect. These policies and guidelines apply to this class.

Students who violate University standards of academic integrity are subject to disciplinary sanctions, including failure in the course and suspension from the university. Since dishonesty in any form harms the individual, other students, and the university, policies on academic integrity have been and will be strictly enforced.