The assessment of development practices in Free Libre Open Source Software (FLOSS) projects can contribute to the improvement of the development process by identifying poor practices and providing a list of necessary practices. Available assessment methods (e.g., Capability Maturity Model Integration (CMMI)) do not address sufficiently FLOSS-specific aspects (e.g., geographically distributed development, importance of the contributions, reputation of the project, etc.). We present a FLOSS-focused, CMMI-like assessment/improvement model: the QualiPSo Open Source Maturity Model (OMM). OMM focuses on the development process. This makes it different from existing assessment models that are focused on the assessment of the product. We have assessed six FLOSS projects using OMM. Three projects were started and led by a software company, and three are developed by three different FLOSS communities. We identified poorly addressed development activities as the number of commit/bug reports, the external contributions, and the risk management. The results showed that FLOSS projects led by companies adopt standard project management approaches as product planning, design definition, and testing, that are less often addressed by community led FLOSS projects. The OMM is valuable for both the FLOSS community, by identifying critical development activities necessary to be improved, and for potential users that can better decide which product to adopt. 1. Introduction Free/Libre Open Source Software (FLOSS) development approaches differ from the traditional software development approaches [1] such as the waterfall or the spiral. The FLOSS approaches have specific characteristics as the geographical distribution of the development team [1]. The developers usually do not know personally each other, there are no budget constraints, and so forth. However, some traditional software development issues as [2]: faults insertion, continuous change of requirements, and growing complexity, are present also in agile and FLOSS projects with additional critical aspects that have to be addressed. Some of these, are for example, issues related to a strongly distributed development process, and absence of formal responsibility of developers for meeting deadlines. The software development process is increasingly being defined and standardized [3]. Assessment models have been defined for evaluating the quality of the software development process. Only by assessing it, it is possible to identify poorly implemented practices, identify missing practices, and improve the development process.
References
[1]
E. S. Raymond, The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary, O'Reilly & Associates, 2001.
[2]
T. Dyb？ and T. Dings？yr, “Empirical studies of agile software development: a systematic review,” Information and Software Technology, vol. 50, no. 9-10, pp. 833–859, 2008.
[3]
A. Fuggetta, “Software process: a roadmap,” in Proceedings of the Conference on the Future of Software Engineering (ICSE '00), pp. 25–34, ACM, Limerick, Ireland, June 2000.
[4]
Process Maturity Profile of the Software Community 1999 Year End Update, Software Engineering Institute, 2000.
[5]
E. Petrinja, R. Nambakam, and A. Sillitti, “Introducing the opensource maturity model,” in Proceedings of the ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development (FLOSS '09) collocated with 31st International Conference on Software Engineering, pp. 37–41, Vancouver, Canada, May 2009.
[6]
E. Petrinja, A. Sillitti, and G. Succi, “Comparing OpenBRR, QSOS, and OMM assessment models,” in Proceedings of the 6th International Conference on Open Source Systems (OSS '10), pp. 224–238, Notre Dame, Ind, USA, May 2010.
[7]
Qualipso Consortium: QualiPSo—Quality Platform for Open Source Software, http://www.qualipso.org/index.php.
[8]
E. Petrinja, A. Sillitti, and G. Succi, “Overview on trust in large FLOSS communities,” in Proceedings of the 4th International Conference on Open Source Systems (OSS '08), pp. 47–56, Milan, Italy, 2008.
[9]
U. Raja and M. J. Tretter, “Defining and evaluating a measure of open source project survivability,” IEEE Transactions on Software Engineering, vol. 38, no. 1, pp. 163–174, 2012.
[10]
M. Khalifa and J. M. Verner, “Drivers for software development method usage,” IEEE Transactions on Engineering Management, vol. 47, no. 3, pp. 360–369, 2000.
[11]
S. Matook and M. Indulska, “Improving the quality of process reference models: a quality function deployment-based approach,” Decision Support Systems, vol. 47, no. 1, pp. 60–71, 2009.
[12]
T. L. Roberts, M. L. Gibson, K. T. Fields, and R. Kelly Rainer, “Factors that impact implementing a system development methodology,” IEEE Transactions on Software Engineering, vol. 24, no. 8, pp. 640–649, 1998.
[13]
C. G. Von Wangenheim, J. C. R. Hauck, A. Zoucas, C. F. Salviano, F. McCaffery, and F. Shull, “Creating software process capability/maturity models,” IEEE Software, vol. 27, no. 4, pp. 92–94, 2010.
[14]
S. C. Misra, V. Kumar, and U. Kumar, “Identifying some important success factors in adopting agile software development practices,” Journal of Systems and Software, vol. 82, no. 11, pp. 1869–1890, 2009.
[15]
E. Yourdon, “Where’s the basis for year 2000 optimism?” Computerworld, vol. 32, no. 7, p. 68, 1998.
[16]
M. Agrawal and K. Chari, “Software effort, quality, and cycle time: a study of CMM level 5 projects,” IEEE Transactions on Software Engineering, vol. 33, no. 3, pp. 145–156, 2007.
[17]
M. Staples, M. Niazi, R. Jeffery, A. Abrahams, P. Byatt, and R. Murphy, “An exploratory study of why organizations do not adopt CMMI,” Journal of Systems and Software, vol. 80, no. 6, pp. 883–895, 2007.
[18]
F. Guerrero and Y. Eterovic, “Adopting the SW-CMM in a small IT organization,” IEEE Software, vol. 21, no. 4, pp. 29–35, 2004.
[19]
A. Qumer and B. Henderson-Sellers, “A framework to support the evaluation, adoption and improvement of agile methods in practice,” Journal of Systems and Software, vol. 81, no. 11, pp. 1899–1919, 2008.
[20]
M. C. Paulk, “Extreme programming from a CMM perspective,” IEEE Software, vol. 18, no. 6, pp. 19–26, 2001.
[21]
C. G. von Wangeheim, A. Anacleto, and C. F. Salviano, “Helping small companies assess software processes,” IEEE Software, vol. 23, no. 1, pp. 91–98, 2006.
[22]
T. Dyb？, “Factors of software process improvement success in small and large organizations: An empirical study in the scandinavian context,” in Proceedings of the 9th European Software Engineering Conference Held Jointly with 11th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 148–157, ACM Press, September 2003.
[23]
G. B. Dietrich, D. B. Walz, and J. L. Wynekoop, “The failure of SDT diffusion: a case for mass customization,” IEEE Transactions on Engineering Management, vol. 44, no. 4, pp. 390–398, 1997.
[24]
Navica Inc., The Open Source Maturity Model is a vital tool for planning open source success, http://www.oss-watch.ac.uk/resources/osmm.xml#body.1_div.2.
[25]
Atos Origin, Method for Qualification and Selection of Open Source Software (QSOS), 2009, http://www.qsos.org/.
[26]
A. Wasserman, M. Pal, and C. Chan, Business Readiness Rating Project, BRR Whitepaper 2005 RFC1, http://www.openbrr.org/.
[27]
D. Taibi, L. Lavazza, and S. Morasca, OpenBQR: A Framework for the Assessment of OSS, Open Source Software 2007, Limerick, Ireland, 2007.
[28]
D. Izquierdo-Cortazar, G. Robles, J. M. González-Barahona, and J.-C. Deprez, “Assessing FLOSS communities: an experience report from the QualOSS project,” Open Source Ecosystems: Diverse Communities Interacting, vol. 299, p. 364, 2009.
[29]
I. Samoladas, G. Gousios, D. Spinellis, and I. Stamelos, “The SQO-OSS quality model: Measurement based open source software evaluation,” IFIP International Federation for Information Processing, vol. 275, pp. 237–248, 2008.
[30]
J.-C. Deprez and S. Alexandre, Comparing Assessment Methodologies for Free/Open Source Software: OpenBRR and QSOS, Book chapter in Lecture Notes in Computer Science, Springer, Berlin, Germany, 2008.
[31]
V. R. Basili, “Software modelling and measurement: the Goal/Question/Metric paradigm,” Computer Science Technical Report Series CS-TR-2956 (UMIACS-TR-92-96), University of Maryland, College Park, Md, USA, 1992.
[32]
S. Morasca, “On the use of weighted sums in the definition of measures,” in Proceedings of the 2010 ICSE Workshop on Emerging Trends in Software Metrics (WETSoM '10), pp. 8–15, ACM Press, May 2010.
