Practical Framework for Measuring Application Performance

This article introduces a performance measurement framework for applications. The framework provides a simple and effective approach to measuring the performance of any application, identifying areas of improvement, and taking corrective action to optimize performance. By following this framework, businesses can ensure that their applications meet performance goals, enhance user experience, and achieve optimal efficiency.

© Copyright Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Adapted
from Field Guide to Consulting and Organizational Development

Suggested Pre-Reading

Overview
of Performance Management Process for any Application

Sections of This Topic Include

Where do I focus my performance
efforts in the organization?
How do I identify which organizational
results to measure?
How do I know what measures to
make to evaluate results?
What about measures after I’ve
made efforts to improve performance?

(Author’s note: The publication “Performance Improvement
Theory and Practice” (in Advances in Developing Human
Resources, Number 1, 1999, by Richard A. Swanson) served as
a very valuable resource while developing the contents of this
library page. Professor Swanson’s document suggests approaches
that answer some basic questions when undertaking efforts to improve
performance. I only hope my overview does justice to his fine
presentation of ideas.)

Where Do I Focus My Measurements in the Organization?

Look for Domains (or Areas of Focus) in the Organization

Swanson suggests four performance domains in organizations,
including 1) mission, 2) process, 3) critical performance subsystem
and 4) individual. These domains suggest areas in which to focus
improvement efforts. More explanation follows.

Mission Performance Domain

The mission is the ultimate purpose of the organization. The
mission of the organization is identified or updated usually during
strategic planning. Typically, reaching the mission requires providing
certain outputs or services to external customers. Identify results
by establishing units of performance with these products or services.
Describe these results in terms of quantity, quality, time and
cost. For example, results might be increasing marketshare by
25% over the next fiscal year, increasing profits by 25% over
the next fiscal year or ensuring that at least 90% of unwed mothers
under 18 years of age in the Hopeful Neighborhood can read and
write.

Process Performance Domain

Swanson cites a process as “a series of phases designed
to produce a product or service.” Quality and reengineering
efforts focus on the process performance domain. Swanson explains
that processes typically cut across various subsystems. There
are numerous processes in an organization, too many to mention
here. Start by thinking about where problems have appeared in
the organization, or where improvement will be needed to meet
goals identified during strategic planning. To get in the mindset
for identifying processes, think about, e.g., processes of market
research to identify customer needs, product design, product development,
budget development, customer service, financial planning and management,
program development, etc.

Critical Performance Subsystems Performance Domain

This domain defines internal performance subsystems that always
directly connect to the internal environment, and frequently with
the external environment (the mission domain always interacts
with the external environment). These subsystems differ from processes
in that processes cut across multiple performance subsystems.
Examples include:
1. programs (implementing new policies and procedures to ensure
a safe workplace; or, for a nonprofit, ongoing delivery of services
to a community)
2. products or services to internal or external customers
3. projects (automating the billing process, moving to a new building,
etc.)
4. teams or groups organized to accomplish a result or an internal
or external customer

Individual Performance Domain

Swanson asserts that performance professions have generally
focused too much on this domain. Individual performance management
typically focuses on the individual working to achieve results
and goals with some performance standard. These results and goals
are recorded and referenced during a performance appraisal process.
Ongoing training and development is provided as needed. Ideally,
the supervisor and employee exchange ongoing feedback during the
appraisal period to enhance the individual’s performance.

How Do I Identify Which Results To Measure?

Look for System Outputs as Results to Measure

An organization is comprised of very complex and dynamic processes,
coordinated by various structures and other controls. In addition,
the organization is continually exchanging various resources and
information with its external environment. Consequently, organizational
performance management can be very complex. There are numerous
results, measurements and standards to consider among the numerous
levels and related areas (or domains) in the organization.

When thinking about results, measurements and standards in
the organization, it helps a great deal to think of the organization
as a system. This system has various aspects, including inputs,
processes, outputs and outcomes. Ongoing feedback occurs among
these aspects of the system. The overall system has various subsystems,
e.g., financial management processes, departments, teams, employees,
etc. Ongoing feedback occurs among aspects of these systems, as
well as with the overall organization and its external environment.

As noted above, when looking for what results to measure, consider
outputs from the system. Measure results in terms of units of
performance, considering timeliness, cost, quantity or quality.

How Do I Know What to Measure to Evaluate Results?

Look for Outcome Measures and Driver Measures

Swanson describes outcomes as measures of effectiveness or
efficiency relative to the core outputs of a system or subsystem.
Outcomes are generic across organizations. Drivers “measure
elements of performance that are expected to sustain or increase
system, subsystem, process or individual ability and capacity
to be more effective or efficient in the future” (p. 33).
Swanson adds that drivers are leading indicators of future outcomes,
and tend to be unique to the organization.

For example, a) management development and change, and b) employee’s
noticing the change and increasing productivity may be drivers
to the outcome of c) increased sales.

Must View Outcomes and Drivers Together

Drivers and outcomes should be considered together. Otherwise,
one gets a flawed picture of the performance system.

For example, focusing on drivers alone might lead one to believe
that the driver (e.g., learning) always produces the outcome (e.g.,
units of sales). Focusing on outcomes alone might lead one to
take short-term actions (e.g., massive layoffs) which adversely
effect long-term outcomes (e.g., sales).

Classification of a Measure (Outcome and Driver) Depends on
the Context

A measure can be either an outcome or a driver, depending on
the situation. For example, a specified increase in customer satisfaction
might be a driver for the organizational outcome of a specified
increase in sales. However, the specified increase in customer
satisfaction might also be an outcome measure for the Customer
Service Department.

Consider Outcomes and Drivers Relative to the Performance
Domains

For example, the performance driver measures of knowledge and
expertise in the individual performance domain may produce the
performance outcome measure of productivity for that domain. The
drivers of innovation and leadership might produce the outcome
of team effectiveness for the critical subsystem domain.

What About Measures After I’ve Made Efforts to Improve Performance?

Driver Results Assessment Matrix

There are a wide variety of results to measure and numerous
types of drivers that can produce these results. It may seem much
too complex to know what to measure and when. Swanson disagrees.

He explains that assessing outcomes from modifying drivers
to affect performance improvements (that is, from efforts such
as training people, instilling quality management, etc., made
ultimately to improve performance) differ in only two basic ways:
1) predictability of performance results and 2) timing of performance
results.

Four Types of Assessments of Outcomes

Swanson designed a two-row, four-cell matrix to depict four
types of drivers, or types of efforts to improve performance.

Basic

Outcomes are known in advance and expected to occur soon. An
example is training. Standard assessment techniques work well
here.

Type 1

Improvement efforts are rather straightforward, but take a
longer time to show results. An example is management development.
In this case, it’s useful to develop a “pattern of events”
(e.g., managers learn and change, employees notice and become
more motivated, and ultimately sales increase). Each step of the
pattern can be measured. Phases before the end result are drivers
to measure. The end result is an outcome to measure.

Type 2

Here, performance results may appear anywhere and take longer
to appear. An example is developing a learning organization. Open-ended
assessment strategies should be used. Emerging outcomes should
be measured, since known outcomes do not exist. Conduct ongoing
assessment over time and compare results.

Type 3

Here, results are still less predictable, but they occur more
quickly. An example is instilling a quality program. The same
assessment approach used in Type 2 is useful here. However, measurements
can be conducted sooner and usually only once.

Return to Performance
Measures.