herraiz.org | Blog

Main | Blog | Research papers | PhD thesis | GnuPG (PGP)

In recent times, the replicability of Software Engineering empirical studies has become a main concern in the research community. One way to achieve replicability is by reusing datasets, so everybody base their results on the same data. However, if these datasets contain any kind of problem, they could cause more harm than benefits.

In the case of software defects, there are datasets that are known to contain bias, mainly when referencing a fix to a particular bug report.

We have studied a different kind of bias: popularity bias. A software project with less bugs is of higher quality. However, in open source software development, more bugs may mean more quality. Why? Because more found bugs imply more people looking for those bugs. This is, if you have no bugs it is because nobody is using your software and reporting them. If you have more bugs, it is because your software is popular; should your software be less popular, the number of bugs would be lower. We have studied this effect in the case of Debian, using the Ultimate Debian Database, and we indeed find that only very popular Debian packages will present a very high number of bugs, and that non-popular packages get very few bug reports.

If you want to know more, read our WCRE 2011 paper, entitled "Impact of Installation Counts on Perceived Quality: A Case Study on Debian". A tag cloud of the contents of the paper:

To cite this paper, there is a BibTeX file available, or you can copy from below

@InProceedings{debian_wcre2011,
  author = 	 {Israel Herraiz and Emad Shihab and Thanh H.D. Nguyen and Ahmed E. Hassan},
  title = 	 {Impact of Installation Counts on Perceived Quality: A Case Study on {D}ebian},
  booktitle = {Proceedings of the 18th Working Conference on Reverse Engineering},
  year = 	 {2011},
  publisher = {IEEE Computer Society},
}