CS 621: Advanced Software Engineering: Analysis and Evaluation

Spring 2023

News | Description | Logistics | Grading | Schedule | Nondiscrimination | Academic integrity | Reading | Prerequisites | Acknowledgements

News:

Description:

Software engineering goes well beyond software development. It involves understanding customer needs, planning the development process, teamwork, maintenance, and analysis and evaluation tasks, such as testing, verification, and validation. In CS 621, we will explore the state-of-the-art research in techniques that make analysis and evaluation of software easier. Concurrently, student groups will engage in research to push forward that state of the art.

This term will cover the following topics: static, dynamic, and speculative analyses; model inference, model checking, and formal verification; bug localization; mutation and regression testing, and symbolic execution.

Becoming familiar, and a part of cutting-edge research will constitute a major part of this class. There will be three research aspects of the class:

  1. The lectures and assignments will draw on the latest research in software engineering
  2. The students will, either individually or in pairs, engage in reading and presenting to the class existing research from recent premier conferences and journals. The students will summarize the work in writing and in a class presentation, and will lead a class discussion.
  3. The students, either individually or in groups of up to four, will engage in a term-long project to advance the state of the art of one of the above topics. The students will identify a shortcoming of today's techniques, review the relevant literature, develop a novel technique that addresses the shortcoming, and evaluate the technique against the state of the art. Finally, the goal is for students to write a research paper describing their work worthy of submission to a research conference.

The students will present an existing research paper once during the semester, and will complete the research project.


Logistics:

Room:140 computer science building
Lecture:Tuesday and Thursday 11:30AM–12:45AM
Instructor:



Yuriy Brun
office: 302 computer science building
office hours: TBD
email:
TA:



Qizheng Yang
office: LGRT T220
office hours: Monday 2:30–3:30PM and Thursday 4–5PM
email: qizhengyang@umass.edu
TA:



Rachana Ponagandla
office: LGRT T220
office hours: Monday 9–10AM
email: rponagandla@umass.edu

All assignment submissions are through Moodle.

Late policy: Assignment due dates and times are listed on the schedule. All deadlines are sharp and the submission site will be closed at the specified time. No extensions will be granted after the assignment is due. Early requests for extensions will be considered only in extenuating circumstances. No more than one extension will be granted per student.


Grading:

Students are responsible for submitting all homework and project assignments. A student who fails to submit at least one of the homework or project assignments, or does not participate in the midterm, will receive the grade F for the entire class.

621
 Assignment Grade
Midterm 25%
Homework 32%
Free points!15%
Participation 3%
Project 25%

The project's 25% are further broken down:

literature review 7%
project plan and presentation 7%
final report and presentation 11%
total: 25%

Schedule:

(subject to change; check regularly)

Due: Tu May 9, 2023, 9:00AM EDT
week date day topic reading homework
project
Week 1
Feb 7 Tu Course introduction
Feb 9 Th Static and dynamic analyses overview
Week 2
Feb 14 Tu Dynamic analysis Dynamically discovering likely program invariants to support program evolution & Purify
Homework 1
Due: Th March 2, 2023, 9:00AM EST
Feb 16 Th Guest lecture: Yi Ding: A Holistic View on Machine Learning for Systems NURD: Negative-Unlabeled Learning for Online Datacenter Straggler Prediction
Week 3
Feb 21 Tu Guest lecture: Jingbo Wang: Trustworthy Software Enabled by Program Analysis and Synthesis Data-Driven Synthesis of Provably Sound Side Channel Analyses
Feb 23 Th Snow Day! Models, tests, bugs, and symbols overview: Watch a video of part of the lecture
Idea Proposal assignment
due: M Mar 20, 2023, 9:00AM EDT
Week 4
Feb 28 Tu Snow Say! No Class.
Mar 2 Th Guest lecture: Mariam Guizani:
From Barriers to Bridges: Designing Processes and Tools for Inclusive Open-Source Communities
No extra class held on Thursday.
Attracting and Retaining OSS Contributors with a Maintainer Dashboard
Week 5
Mar 7 Tu Guest lecture: Jenna DiVincenzo:
Gradual Verification: Assuring Programs Incrementally
Homework 2
Due: Tu March 28, 2023, 9:00AM EST
Mar 9 Th Guest lecture: Juan Zhai (12–1PM, room CS151): Software Quality Assessment via Specification Synthesis
Week 8
Spring break: no class
Week 6
Mar 21 Tu Automated test generation Korat: Automated testing based on Java predicates
Mar 23 Th Project idea presentations
Literature review
Due: Tu Apr 11, 2023, 9:00AM EDT
Week 7
Mar 28 Tu Reproducing field failures Chronicler: Lightweight Recording to Reproduce Field Failures and BugRedux: Reproducing Field Failures for In-house Debugging
Mar 30 Th Testing
Homework 3
Due: Tu Apr 18, 2023, 9:00AM EDT
Week 9
Apr 4 Tu Quality of automated repair and Search Repair Is the Cure Worse than the Disease? Overfitting in Automated Program Repair
Apr 6 Th Guest lecture: Shiqing Ma: Exploring the DNN Memorization Effect in Transparent Computing Complex Backdoor Detection by Symmetric Feature Differencing
Week 10
Apr 11 Tu Software Fairness Fairness Testing: Testing Software for Discrimination
Apr 13 Th Midterm review and Speculative analysis Early Detection of Collaboration Conflicts and Risks
Project plan
Due: Th April 20, 2023, 9:00AM EDT
Week 11
Apr 18 Tu No class (Patriots' Day Break)
Apr 20 Th Designs to Support Better Visual Data Communication Corrigendum: The Science of Visual Data Communication: What Works
Homework 4
Due: Tu May 9, 2023, 9:00AM EDT
Week 12
Apr 25 Tu Project plan presentations
Final project report
Due: Wed May 17, 2023, 11:55PM EDT
Apr 27 Th Midterm (in class)
Week 13
May 2 Tu Groupthink exercise part 1
May 4 Th Groupthink exercise part 2
Week 14
May 9 Tu How to make team-based engineering successful
May 11 Th Final project presentations
Week 15
May 16 Tu No Class. Work on final projects.

Nondiscrimination policy:

Software engineering is at its nature a collaborative activity and it benefits greatly from diversity. This class includes and welcomes all students regardless of age, background, citizenship, disability, sex, education, ethnicity, family status, gender, gender identity, geographical origin, language, military experience, political views, race, religion, sexual orientation, socioeconomic status, and work experience. Our discussions and learning will benefit from these and other diverse points of view. Any kind of language or action displaying bias against or discriminating against members of any group, or making members of any group uncomfortable are against the mission of this course and will not be tolerated. The instructor welcomes discussion of this policy, and encourages anyone experiencing concerns to speak with him.


Academic integrity:

Students are allowed to work together on all aspects of this class except the midterm. However, for the homework assignments, each student must submit his or her own write up, clearly stating the collaborators. Your submission must be your own. When in doubt, contact the instructors about whether a potential action would be considered plagiarism. If you discuss material with anyone besides the class staff, acknowledge your collaborators in your write-up. If you obtain a key insight with help (e.g., through library work or a friend), acknowledge your source and write up the summary on your own. It is the student's responsibility to remove any possibility of someone else's work from being misconstrued as the student's. Never misrepresent someone else's work as your own. It must be absolutely clear what material is your original work. Plagiarism and other anti-intellectual behavior will be dealt with severely. Note that facilitation of plagiarism (giving your work to someone else) is also considered to be plagiarism, and will carry the same repercussions.

The UMass Academic Honesty Policy prohibits the use of ChatGPT and similar AI text generators. Representing material generated by such tools as your own will be considered plagiarism.

Students are encouraged to use the Internet, literature, and other publicly-available resources, except the homework solutions and test (including quizzes, midterms, finals, and other exams) solutions, from past terms' versions of this course and other academic courses, whether at UMass and at other institutions. To reiterate, the students are not allowed to view and use past homework and test solutions, unless explicitly distributed by the COMPSCI 621 staff as study material.

Whenever students use Internet, literature, and other publicly-available resources, they must clearly reference the materials in their write ups, attributing proper credit. This cannot be emphasized enough: attribute proper credit to your sources. Failure to do so will result in a zero grade for the assignment and possibly a failing grade for the class, at the instructor's discretion. Copying directly from resources is not permitted, unless the copying is clearly identified as a quote from a source. Most use of references should be written in the words of the student, placing the related work in proper context and describing the relevant comparison.

The students should familiarize themselves with the UMass Academic Honesty Policy and Guidelines for Classroom Civility and Respect. These policies and guidelines apply to this class.

Students who violate University standards of academic integrity are subject to disciplinary sanctions, including failure in the course and suspension from the university. Since dishonesty in any form harms the individual, other students, and the university, policies on academic integrity have been and will be strictly enforced.


Reading:

There is no required textbook for the course. Reading assignments will come from publicly available research papers. Students who wish to read established textbooks beyond the assigned reading should consider:


Prerequisites:

Students should have taken an introductory course in software engineering or have the equivalent background. Students are expected to be familiar with an object oriented programming language, such as Java or C++. Some programming and the ability to download and use off-the-shelf tools are expected.

Acknowledgements:

Various materials used in this course have greatly benefited from materials developed by Alex Aiken, Lori Clarke, Carlo Curino, Sebastian Elbaum, Michael Ernst, David Notkin, Nenad Medvidovic, Alex Orso, Lee Osterweil, Willem Visser. Thank you.