When designing a software system, architects make a series of design decisions that directly impact the system's quality. The number of available design alternatives grows rapidly with system size, creating an enormous space of intertwined design concerns that renders manual exploration impractical. We present eQual, a model-driven technique for simulation-based assessment of architectural designs. While it is not possible to guarantee optimal decisions so early in the design process, eQual improves decision quality. eQual is effective in practice because it (1) limits the amount of information the architects have to provide and (2) adapts optimization algorithms to effectively explore massive spaces of design alternatives. We empirically demonstrate that eQual yields designs whose quality is comparable to a set of systems' known optimal designs. A user study shows that, compared to the state-of-the-art, engineers using eQual produce statistically significantly higher-quality designs with a large effect size, are statistically significantly more confident in their designs, and find eQual easier to use.
@inproceedings{Shahbazian20fse,
author = {Arman Shahbazian and Suhrid Karthik and Yuriy Brun and Nenad Medvidovic},
title = {\href{http://people.cs.umass.edu/brun/pubs/pubs/Shahbazian20fse.pdf}{eQual: Informing Early Design Decisions}},
booktitle = {Proceedings of the 27th ACM Joint European Software
Engineering Conference and Symposium on the Foundations of Software
Engineering (ESEC/FSE)},
venue = {ESEC/FSE},
month = {November},
year = {2020},
date = {11--13},
pages = {1039--1051},
address = {Sacramento, CA, USA},
doi = {10.1145/3368089.3409749},
accept = {$\frac{101}{360} \approx 28\%$},
note = {ACM artifact badges granted:
\href{https://www.acm.org/publications/policies/artifact-review-and-badging-current}{\raisebox{-.75ex}{\includegraphics[height=2.5ex]{ACMArtifactAvailable}}~Artifact Available}.
\href{https://doi.org/10.1145/3368089.3409749}{DOI: 10.1145/3368089.3409749}},
fundedBy = {NSF CCF-1453474, NSF CCF-1618231, NSF CCF-1717963, NSF CNS1823354,
U.S. Office of Naval Research N00014-17-1-2896},
abstract = {<p>When designing a software system, architects make a series of design
decisions that directly impact the system's quality. The number of available
design alternatives grows rapidly with system size, creating an enormous
space of intertwined design concerns that renders manual exploration
impractical. We present eQual, a model-driven technique for simulation-based
assessment of architectural designs. While it is not possible to guarantee
optimal decisions so early in the design process, eQual improves decision
quality. eQual is effective in practice because it (1) limits the amount of
information the architects have to provide and (2) adapts optimization
algorithms to effectively explore massive spaces of design alternatives. We
empirically demonstrate that eQual yields designs whose quality is comparable
to a set of systems' known optimal designs. A user study shows that, compared
to the state-of-the-art, engineers using eQual produce statistically
significantly higher-quality designs with a large effect size, are
statistically significantly more confident in their designs, and find eQual
easier to use.</p>},
}