The Mihalik Blog

Quality Improvement 101: Benchmarking

I want to talk about a pet peeve of mine—the misuse of the words benchmark and benchmarking. 

 

It seems that folks often use the term benchmark and goal interchangeably.  They’re very different. 

 

A benchmark is the best of the best of actual performance.  It is a level of measured performance that some organization has actually achieved.  A benchmark cannot be “set” the way a goal can.  You get to choose your goal.  A benchmark is something that “is.”  In this context, one might define benchmarking as the process of measuring an organization’s performance on a key customer requirement against the best in the industry, or the best in any industry. 

 

The last little twist (“the best in any industry”) is often a characteristic of benchmarking efforts.  You may need to go out of your industry to find a benchmark.  For example, it is unlikely that the benchmark for customer service would be found in a health care organization.  You might want to consider Disney or Nieman Marcus in that regard. 

 

Usually, I consider benchmarking to relate to the process of measuring an organization’s performance against the best known performance.  Another definition of benchmarking, though, is the search for industry best practices that will lead to exceptional performance through the implementation of these practices.  This approach to benchmarking focuses not on the level of performance itself, but what the actual practices are that lead to exceptional performance. 

 

Notice that this second definition of benchmarking does not actually include consideration of the measurement of performance.  It speaks more to evaluating the processes of another organization and adopting those that are considered industry best.  To satisfy NCQA requirements for either a benchmark or a goal, you need a number so this approach to benchmarking alone will not be adequate without the adoption of a quantitative goal.

 

Another way to define benchmarking is the practice of being humble enough to admit that someone else is better at something than you are and being wise enough to try to surpass them at it! 

 

We’re always here to help you achieve your performance objectives. 

Quality Improvement 101: Selection Grid

A Selection Grid is another way to establish priorities among an array of options.  It is a more structured process than either multi-voting  or nominal group technique.  It is used when there are clear criteria for making a decision.

 

One of the major conceptual differences between using a selection grid and using multi-voting or nominal group technique is the emphasis on conscious decision-making.  Explicitly articulated criteria are not used in multi-voting or nominal group technique. 

 

There are many selection grid methodologies.  Some can get quite complex but have the advantage of being able to establish priorities among a large number of options.  In health care, I generally use a consensus-based method, which can be time-consuming and difficult to use for more than about 8-10 options. 

 

In order to use a selection grid, there need to be decision-making criteria.  If you do not already have criteria (some organizations already have criteria to be used when making certain types of decisions even if the criteria are not used in the context of a selection grid) you may want to consider multi-voting or nominal group technique unless you have a good reason for wanting the decision-making criteria to be explicitly stated.

 

As with the other techniques, pull together a representative group of participants.  Clearly state the question and identify the options from which the selection is to be made. 

 

Construct a grid that lists the decision-making criteria down the left side and the options to be decided on across the top. 

 

Using a scale of 0 to 9, rate each option on each criterion by group consensus.  Alternative methods allow each individual to do his/her own rating and to average the results to arrive at the rating for each cell in the table.  It is also possible to weight the criteria so that more important criteria carry more decision-making weight. 

 

When all the ratings are done, add up the score for each option and put the scores in a row across the bottom of the table.  Discuss the ranking.  Does it make sense?  Remember that small differences in total score may not be significant but large differences almost certainly are. 

Quality Improvement 101: Nominal Group Technique

Like, multi-voting, Nominal Group Technique is a method of paring down a broad list of ideas and establishing priorities. I use nominal group technique more than multi-voting. I suggest you try both a few times and decide which you prefer.

Gather together a representative cross-section of individuals, write the ideas on flip chart pages, and letter each idea just as you would for multi-voting. Most likely. the ideas will come from one or more brainstormingsessions.

Now, ask the participants to write down the letters of the five ideas that they think are most important on an index card or piece of paper. By now, if you’ve read the last few entries in this series, you know that this process should happen in silence.

The facilitator then asks each participant to identify the most important idea from his/her five ideas and to put the number “5” next to it. Allowing time for each step so that no one is rushed, the facilitator asks the participants to put the number “1” next to the least important of the remaining four items; then a “4” next to the most important of the remaining three items; then a “2” next to the least important of the remaining two items, then a “3” next to the lone remaining item.

The cards are passed into the scribe who, usually with an assistant, transcribes the votes to the flip chart adjacent to the appropriate idea. Before discussing the ranking, the group should decide if the results seem to have adequately pared down the list to a group of high-priority items. If not, the process can be repeated after eliminating all items that got no votes and probably those that got only one vote.

A big difference between the analysis of the results of nominal group technique and the results of multi-voting is that the group considers the total point count for each item (adding up the values of each individual vote) as well as the number of votes that each item got.

Consider the difference between an item that got five votes of “1” and an item that got one vote of “5.” In the former instance, five different people put the item in their top five but in the latter, only one person felt the idea made the cut into the top five. Don’t dismiss the single vote of “5” though. Consider, for example, whether the individual who cast the vote has a unique perspective on the process that prompted her/him to rate that idea as the most important.

It can be particularly difficult to facilitate sessions like this since the process is unfamiliar to most individuals. There will be a tendency to want to switch into “committee mode” and begin discussing ideas. I suggest you rigidly adhere to the rules to get a sense of what the technique can accomplish.