MeiSE (Measurement in Software Engineering) comprehends a series of initiatives we have been performed for more than ten years aiming to support measurement in software and IT organizations. The starting point was the doctorate thesis entitled A Strategy for Software Measurement and Software Measurement Repository Evaluation for Statistical Process Control in High Maturity Organizations, by Monalessa P. Barcellos. Since then, we have proposed ontologies, strategies, methods and tools to support software measurement. Some of them are cited on this page.
Measurement is a key process for software process and product improvement. It provides useful information for organizations to analyze their performance, guiding both decision-making and daily activities. Successful organizations use measurement as part of their day-to-day activities. Depending on its purpose and maturity level, software organizations perform measurement in different ways. For example, organizations at initial maturity levels of models such as CMMI (Capability Maturity Model Integration) and MR-MPS-SW (Brazilian Reference Model for Software Process Improvement) use measurement with focus on supporting management information needs. On the other hand, organizations at high maturity levels, such as CMMI levels 4 and 5 and MR-MPS-SW levels B and A, perform measurement also for the purpose of statistical process control (SPC), to understand processes behavior and support process improvement efforts.
The Reference Ontology for Software Measurement (RSMO)
RSMO is a well-founded reference ontology that provides a conceptualization about software measurement and statistical process control. Its first version was proposed in 2009, in Monalessa P. Barcellos’ doctorate thesis. RSMO evolved over the years and currently it is a networked ontology of SEON (Software Engineering Ontology Network).
- Barcellos, M. P.; Falbo, R.A.; Rocha, A. R.. A Strategy for preparing Software Organizations for Statistical Process Control. Journal of the Brazilian Computer Society, vol. 19, p. 1–31, 2013.
- Barcellos, M. P.; Falbo, R.A.; Rocha, A. R.. Using a Reference Domain Ontology for Developing a Software Measurement Strategy for High Maturity Organizations. In: Proceedings of the IEEE 16th International Enterprise Distributed Object Computing Conference Workshops, 2012.
- Barcellos, M. P.; Falbo, R.A.; Rocha, A. R.. Establishing a Well-Founded Conceptualization about Software Measurement in High Maturity Levels. In: Proceedings of the Seventh International Conference on the Quality of Information and Communications Technology (QUATIC), p. 467–472, 2010.
- Barcellos, M. P.; Falbo, R.A.; Rocha, A. R.. A Well-Founded Software Process Behavior Ontology to Support Business Goals Monitoring in High Maturity Software Organizations. In: Proceedings of the 14th IEEE International Enterprise Distributed Object Computing Conference Workshops, p. 253–262, 2010.
- Barcellos, M. P.; Falbo, R.A.; Dalmoro, R.. A Well-Founded Software Measurement Ontology. In Frontiers in Artificial Intelligence and Applications, p. 213–226, 2010.
RSMO represents software measurement structural knowledge. To address behavioral aspects of the software measurement process, we developed a Software Measurement Task Ontology (STMO), which was introduced in this paper:
Ontology-based Approach for Software Measurement Systems Integration (OBA-MSI)
Typically, organizations use different tools to support different processes. For example, schedule and budget tools are used to support project management, modeling tools are used to support requirements engineering, and development environments and version control systems are used to support coding and source code management. Although these tools are not usually conceived to support software measurement, many times they store useful data related to the supported processes (e.g., number of defects, time spent on activities, number of lines of code, test failure rate, etc.). To properly support software measurement, these tools must be integrated. However, this is not an easy task. The heterogeneity between systems to be integrated is the major difficulty. OBA-MSI is an approach that uses software measurement ontologies (RSMO and TSMO previously cited) as a basis to integrate tools and support software measurement. OBA-MSI was presented in this paper:
Software Measurement aiming at Statistical Process Control
We have proposed a strategy for preparing software organizations for statistical process control. The strategy is presented in:
In this context, we defined a set of recommendations for software measurement and an instrument to evaluate measurement repositories.
The set of recommendations provides recommendations to help organizations to perform measurement aiming at SPC. A summary of the recommendations is available here (in Portuguese). An overview of the set of recommendations is presented in:
The instrument to evaluate measurement repositories provides checklists and a process to evaluate the suitability of measurement repositories for SPC. The instrument specification is available here (in Portuguese). The following paper addresses the use of the instrument to evaluate the measurement repository of an organization:
MePPLa (Measurement Planning Pattern Language)
The growing interest of organizations in improving their software processes has led them to aim at achieving high maturity, where statistical process control (SPC) is required. One of the challenges to perform SPC is selecting measures suitable for it. Measures used in SPC can be found in the literature and can be reused by organizations, but the information is dispersed and non-structured, not favoring reuse.
From measures used in SPC initiatives, it is possible to identify patterns of measures used to monitor certain measurement goals and to perform statistical control of certain processes. Patterns can be organized into pattern languages, which represent patterns and their relationships and also define a process that guides pattern selection and use.
MePPLa is a pattern language composed of a set of interrelated patterns that when used in a combined way help in the elaboration of measurement plans for SPC. The patterns were identified from the literature and from a survey with practitioners.
In MePPLa each pattern is related to a process and follows the Goal-Question-Metric (GQM) format. Thus, a MePPLa pattern includes a measurement goal, questions that indicate information needs that must be met so that it is possible to monitor the measurement goal and measures that meet the information needs. By using MePPLa, organizations select patterns related to processes they want to submit to SPC and the patterns are added to the measurement plan. Currently, MePPLa includes patterns related to Project Management, Coding, and Testing processes. Being a pattern language, MePPLa is constantly evolving, with new patterns and processes being gradually added to it. MePPLa was introduced in this paper:
MePPLa specification is available here. The use of MePPLa is supported by MePPLa Tool, in which the user can select patterns and include them in a document for the elaboration of the measurement plan.
Measurement in IT Services
The literature on IT Service states that IT services should support critical business processes and should be measured in order to provide useful information for decision-making. In a partnership with prof. Gleison Santos (UNIRIO) and prof. Tayana Conte (UFAM), we have investigated aspects related to IT services measurement. In the context of Bianca Trinckenreich’s master dissertation (UNIRIO), we defined SINIS, a method to identify Goals, Strategies and Indicators for IT Services, which extends GQM+Strategies. It supports IT service departments in identifying IT service goals, strategies and indicators to provide information for decision-making at different organizational levels and in alignment with business goals. SINIS supports defining strategies to achieve IT service goals and identifying indicators to evaluate the strategies and goals achievement. It is based on process improvement approaches (mainly GQM+Strategies) and approaches related to IT service management (mainly COBIT Goals Cascade).
- Trinkenreich, B.; Santos, G.; Barcellos, M. P.; Conte, T.. Combining GQM+Strategies and OKR: Preliminary Results from a Participative Case Study in Industry. In: Proceedings of the 20th International Conference on Product-Focused Software Process Improvement (PROFES), 2019.
- Trinkenreich, B.; Santos, G.; Barcellos, M. P.. SINIS: A GQM+Strategies-based approach for identifying Goals, Strategies and Indicators for IT Services. Information and Software Technology, vol. 100, p. 147–164, 2018.
- Trinkenreich, B.; Conte, T.; Barcellos, M. P.; Santos, G.; Defining, Measuring and Monitoring IT Service Goals and Strategies: Preliminar Practical Results and Pitfalls from a Qualitative Study with IT Service Managers. In: Proceedings of the 12th International Symposium on
Empirical Software Engineering and Measurement (ESEM), 2018.
- Trinkenreich, B.; Santos, G.; Barcellos, M. P.; Conte, T.. Eliciting Strategies for the GQM+Strategies Approach in IT Service Measurement Initiatives. In: Proceedings of the 11st ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2017.
Other works related to Software Measurement
- Brito, D.F.; Barcellos, M. P.; Santos, G.. Investigating Measures for applying Statistical Process Control in Software Organizations. Journal of Software Engineering Research and Development, vol. 6, p. 1–31, 2018.
- Ferreira, E.; Trinkenreich, B.; Santos, G.; Barcellos, M. P.. Using SINIS and GQM+Strategies to Align Organizational Goals and Service Level Agreement Indicators. In: Proceedings of the 17th Brazilian Symposium on Software Quality, 2018.
- Bonelli, S.; Santos, G.; Barcellos, M. P.. SANTOS, and M. P.. ASM.br: A Template for Specifying Indicators. In: Proceedings of the 21st Evaluation and Assessment in Software Engineering Conference (EASE), 2017.
- Trinkenreich, B.; Santos, G.; Barcellos, M. P.. Metrics to Support IT Service Maturity Models: A Systematic Mapping. In: Proceedings of the 17th International Conference on Enterprise Information Systems (ICEIS), 2015.
- Fonseca, V.S.; Barcellos, M. P.; Falbo, R.A.. Tools Integration for Supporting Software Measurement: A Systematic Literature Review. iSys – Revista Brasileira de Sistemas de Informação, vol. 8, p. 80–108, 2015.
- Fonseca, V.S.; Barcellos, M. P.; Falbo, R.A.. Integration of Software Measurement Supporting Tools: A Mapping Study. In: Proceedings of the 27th International Conference on Software Engineering and Knowledge Engineering (SEKE), 2015.
- Maretto, C.X.; Barcellos, M.P.. A Levels-based Approach for Defining Software Measurement Architectures.CLEI Electronic Journal, vol. 17, 2014.
- Maretto, C.X.; Barcellos, M.P.. Software Measurement Architectures: A Mapping Study. In: Proceedings of the XVI Iberoamerican Conference on Software Engineering (CIbSE), p. 20–33, 2013.
General information about software measurement and statistical process control, including knowledge obtained from the set of recommendations and evaluation instrument previously mentioned can be found in this book (in Portuguese):