Improving impact of self-adaptation and self-management research through evaluation methodology
by Yuriy Brun
Abstract:

Today, self-adaptation and self-management approaches to software engineering are viewed as specialized techniques and reach a somewhat limited community. In this paper, I overview the current state and expectation of self-adaptation and self-management impact in industry and in premier publication venues and identify what we, as a community, may do to improve such impact.

In particular, I find that common evaluation methodologies make it relatively simple for self-adaptation and self-management research to be compared to other such research, but not to more-traditional software engineering research. I argue that extending the evaluation to include comparisons to traditional software engineering techniques may improve a reader's ability to judge the contribution of the research and increase its impact. Finally, I propose a set of evaluation guidelines that may ease the promotion of self-adaptation and self-management as mainstream software engineering techniques.

Citation:
Yuriy Brun, Improving impact of self-adaptation and self-management research through evaluation methodology, in Proceedings of Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2010, pp. 1–9.
Bibtex:
@inproceedings{Brun10seams,
  author = {Yuriy Brun},
  title =
  {\href{http://people.cs.umass.edu/brun/pubs/pubs/Brun10seams.pdf}{Improving
  impact of self-adaptation and self-management research through evaluation
  methodology}},
  booktitle = {Proceedings of Software Engineering for Adaptive and
  Self-Managing Systems (SEAMS)},
  venue = {SEAMS},
  month = {May},
  date = {3--4},
  year = {2010},
  address = {Cape Town, South Africa},
  pages = {1--9},
  accept = {$\frac{13}{32} \approx 41\%$},
  doi = {10.1145/1808984.1808985},

  note = {\href{http://dx.doi.org/10.1145/1808984.1808985}{DOI:
  10.1145/1808984.1808985}},

  abstract = {<p>Today, self-adaptation and self-management approaches to
  software engineering are viewed as specialized techniques and reach a
  somewhat limited community. In this paper, I overview the current state and
  expectation of self-adaptation and self-management impact in industry and in
  premier publication venues and identify what we, as a community, may do to
  improve such impact.</p>

  <p>In particular, I find that common evaluation methodologies make it
  relatively simple for self-adaptation and self-management research to be
  compared to other such research, but not to more-traditional software
  engineering research. I argue that extending the evaluation to include
  comparisons to traditional software engineering techniques may improve a
  reader's ability to judge the contribution of the research and increase its
  impact. Finally, I propose a set of evaluation guidelines that may ease the
  promotion of self-adaptation and self-management as mainstream software
  engineering techniques.</p>},
	
  fundedBy = {NSF CNS-0937060 to the CRA for the CIFellows Project},
}