What Are Software Engineering Metrics?

Metrics are units of measurement. The term "metrics" is also frequently used to mean a set of specific measurements taken on a particular item or process. Software engineering metrics are units of measurement that are used to characterize:

software engineering products, e.g., designs, source code, and test cases,

software engineering processes, e.g., the activities of analysis, designing, and coding, and

software engineering people, e.g., the efficiency of an individual tester, or the productivity of an individual designer.

If used properly, software engineering metrics can allow us to:

quantitatively define success and failure, and/or the degree of success or failure, for a product, a process, or a person,

identify and quantify improvement, lack of improvement, or degradation in our products, processes, and people,

make meaningful and useful managerial and technical decisions,

identify trends, and

make quantified and meaningful estimates.

Over the years, I have noticed some common trends among software engineering metrics. Here are some observations:

A single software engineering metric in isolation is seldom useful. However, for a particular process, product, or person, 3 to 5 well-chosen metrics seems to be a practical upper limit, i.e., additional metrics (above 5) do not usually provide a significant return on investment.

Although multiple metrics must be gathered, the most useful set of metrics for a given person, process, or product may not be known ahead of time. This implies that, when we first begin to study some aspect of software engineering, or a specific software project, we will probably have to use a large (e.g., 20 to 30, or more) number of different metrics. Later, analysis should point out the most useful metrics.

Metrics are almost always interrelated. Specifically, attempts to influence one metric usually have an impact on other metrics for the same person, process, or product.

To be useful, metrics must be gathered systematically and regularly -- preferably in an automated manner.

Metrics must be correlated with reality. This correlation must take place before meaningful decisions, based on the metrics, can be made.

Faulty analysis (statistical or otherwise) of metrics can render metrics useless, or even harmful.

To make meaningful metrics-based comparisons, both the similarities and dissimilarities of the people, processes, or products being compared must be known.

Those gathering metrics must be aware of the items that may influence the metrics they are gathering. For example, there are the "terrible H's," i.e., the Heisenberg effect and the Hawthorne effect.

Metrics can be harmful. More properly, metrics can be misused.

Object-oriented software engineering metrics are units of measurement that are used to characterize:

object-oriented software engineering products, e.g., designs, source code, and test cases,

object-oriented software engineering processes, e.g., the activities of analysis, designing, and coding, and

object-oriented software engineering people, e.g., the efficiency of an individual tester, or the productivity of an individual designer.