• Home
• About
• Services
• PDPI^SM
• Models
• Results
• Reference
• Contact

• PDPI^SM
• Motive
• Model
• Method
• Managing Change
• Measures

PDPI^SM | Measures

The final step in process improvement (and the first step in the next improvement cycle) is to determine the impact on the organization of the changes that have been implemented. This implies some set of measures which can be compared against a baseline in order to determine quantitatively how successful the process improvement program has been. To be effective, software measurement should be integrated with an overall strategy for software process improvement.

There are many approaches to defining measures and setting up a measurement program. MDM bases its work on the Practical Software Measurement project and the Goal-Question-Measures approach.

The PSM project was developed to transition measurement into day-to-day practice. Measurement professionals from a wide variety of organizations participated in the PSM project, including representatives from government, industry, and overseas. PSM is based on actual measurement experience on DoD, government and industry programs, and treats measurement as a flexible process, not a pre-defined list of graphs or reports. The guidebook, “Practical Software and Systems Measurement,” is available from their web-site, www.psmsc.com.

PSM is built around nine measurement principles:

1. Objectives and issues are used to drive the measurement requirements. Project objectives are goals and requirements: cost, schedule, quality, functionality, and technical performance. Issues are areas of concern that present obstacles: problems, risks and lack of information.
2. Define and collect measures based on the technical and management processes. Measures should be collected as a natural by-products of the work performed. Consider the processes of other team members and subcontractors as well as your own project processes.
3. Collect and analyze data at a level of detail sufficient to identify and isolate problems. Periodically collect, process and analyze measurement data. Specific data depends on project objectives and issues and the kinds of questions you need to answer.
4. Implement an independent analysis capability. Measurement data should be assessed by an independent group. This ensures objectivity and accurate, unbiased Appraisal of project status.
5. Use a systematic analysis process to trace the measures to the decisions. The meaning of the numbers must be understood. There should be a clear flow from the data through the analysis to the conclusions. The analysis process should provide repeatable results.
6. Interpret the measurement results in the context of other project information. No measurement result is good or bad by itself. A variance between planned and actual only indicates a possible problem, not the cause.
7. Integrate measurement into the project management process. Measurement provides insight into the current phase. It also can project consequences of current actions on later phases.
8. Use the measurement process as a basis for objective communications. Involve the entire project in developing the measurement process. All parties should use same data and have a common understanding of the data definitions and commitment to the value of the measurement program.
9. Focus initially on project-level analysis. Project success means meeting specific project objectives. Implement a consistent measurement process on all projects. Organization-level data can be derived from well-defined project measures.

GQM Approach

The GQM Approach is very useful in determining what measures to collect. (See the first principle of PSM above.) In goal-driven measurement, the primary question is not "What metrics should I use?” but "What do I want to know or learn?" Because the answers depend on your goals, no fixed set of measures is universally appropriate. The goal-driven measurement process is based on three precepts:

1. Measurement goals are derived from business goals. The goal-driven process begins with identifying business goals and breaking them down into manageable sub-goals.
2. Evolving mental models provide context and focus. The primary mechanisms for translating goals into issues, questions, and measures are the mental models that you have for the processes you use. These mental models gain substance and evolve as you begin to make them explicit. They are the engines that generate the insights that guide you to useful measures and actions.
3. GQM translates informal goals into executable measurement structures. The process ends with a plan for implementing well-defined measures and indicators that support the goals. Along the way, it maintains traceability back to the goals, so that those who collect and process measurement data do not lose sight of the objectives.

GQM involves steps:

• Step 1: Identify your business goals.
• Step 2: Identify what you want to know or learn.
• Step 3: Identify your sub-goals.
• Step 4: Identify the entities and attributes.
• Step 5: Formalize your measurement goals.
• Step 6: Identify your measurement questions & indicators.
• Step 7: Identify the data elements.
• Step 8: Define and document measures and indicators.
• Step 9: Identify the actions needed to implement your measures.
• Step 10: Prepare a plan for implementing the measures.

GQM Guidelines

In using the GQM approach to develop a set of metrics, use the following guidelines:

1. State a goal. Use brainstorming technique or start from established goals.
2. Do we understand the goal? If not, break it down to a lower level. Ask questions that will translate the goal into measurable attributes. We understand it when the focus or subject of the goal can be described in a single unit of measure (number)
3. Once we understand the goal (or its attributes), identify the measures (or metrics) that characterize it. Use brainstorming to identify candidate metrics/measures. Select the primary metrics from the list of measures generated. Is it at the right level of detail? Is it a sub-element of another metric? Is it another (redundant) point of view? Is it derivable from directly measurable quantities? If so, have we identified those measures? Is it cost effective to collect?
4. Decide what question(s) management should ask to determine if we are meeting the goal. This is Q in GQM that is our end product. Use brainstorming. Derive from it the desired metrics/measures.