Resources > APICS Magazine > November/December 2005 > FOR RENT

Metrics at Work
A new approach to assessing supply chain performance

By KENT LINFORD

The tension builds as the senior vice president of operations walks in the room. Service rates haven't met expectations and he wants to know why. He asks the all-too-familiar questions. Why are fill rates so low? Why were so many orders missed? Where is the process breakdown? In response, the planning group blames the production group, claiming the plan isn't being executed. The production group blames the planning group, saying they could execute if the planning was better. Let the finger pointing begin.

Does this situation sound familiar? As with any process, when assessing the supply chain, metrics have to be in place. Otherwise, politicking and posturing ensue as employees try to protect their turf. All this is pointless, and it doesn't make things better. In fact, blaming makes things worse. It creates an adversarial relationship between groups that should be working together.

Furthermore, if responsibility for problems is not identified via metrics, then nothing will be improved. Nobody is concerned about the process or methodology because the other group is at fault.

Insufficient measurement
The traditional measurement of forecast error often is used as a gauge to assess whether a service miss is a planning or execution problem. The idea is that, if sales are above forecast, then planning is to blame for a poor forecast and the resulting poor service levels. This type of analysis is missing a key component of planning—safety stock. Planners use safety stock to cover variability during replenishment lead time. Hence, the greater the variability of demand or supply, the more safety stock that is carried in inventory.

Why is forecast error insufficient when measuring planning performance? Because an item may be highly variable and show a large forecast error, but may have a safety stock sufficiently large to absorb the forecast error, thus protecting the item from stockout.

When using only forecast error, an item that had sufficient planning (both forecast and safety stock) could be erroneously attributed to poor planning, when, in fact, the item had an execution issue. With this incorrect attribution, the wrong link in the supply chain is targeted as the culprit, and worse, the link in the supply chain that needs attention—execution—is completely ignored. Thus, both forecast and safety stock need to be considered when measuring and analyzing supply chain performance.

A new approach
I'd like to propose several new metrics and a new process to assess supply chain performance. The new metrics are planning error, planning service level, and execution service level. The new approach is called supply chain operations performance analysis (SCOPA). Developed amidst service challenges with one of our customers here at Moen Incorporated, SCOPA is a better and more robust way to assess planning and execution than forecast error analysis and the blame game. It provides more opportunities to truly improve the supply chain.

This new approach is based on a belief that problems need to be addressed from two perspectives—alleviating the current situation and reducing and eliminating future problems. Here's an analogy: While we try to build better guardrails, we still must rescue the stockkeeping units (SKUs) that have gone over the edge. And while we try to prevent service wrecks through planning and forecasting, we still must have a responsive ambulance.

SCOPA begins with the attribution analysis report, which is generated using the new metrics of planning error, planning service level, and execution service level. These metrics help identify whether planning or execution is responsible for each item missed (or if both groups are partially responsible). They also are used to measure the service level for each group. See Figure 1 for an example of an attribution analysis report using Microsoft Excel.

The attribution analysis report is fashioned after the traditional forecast error report. The given information is as follows: part number, offset forecast, quantity ordered/sold, quantity shipped, and offset safety stock. The calculated information incorporates the quantity missed, forecast error, safety stock protection, planning error, missed quantity attributed to planning, missed quantity attributed to execution, planning service level, execution service level, and supply chain service level. (For more information about how to calculate the figures associated with the attribution analysis, please see "SCOPA in Short," in the sidebar.)

Safety stock protection is calculated by dividing the offset safety stock by replenishment lead-time days multiplied by the number of business days being analyzed (the number of days that orders were taken). The idea of safety stock protection is that it's the quantity that could be sold over and above the forecast during a given period without incurring a stockout. Planning error equals forecast error minus safety stock protection. If this metric is a positive number, then planning was not sufficient to prevent stockout. If this metric is a negative number, then planning was sufficient.

If an item's planning error is positive, then that item's service miss, if any, is attributed to planning up to the planning error quantity. If the actual quantity missed for an item is greater than the planning error, then the quantity missed minus the planning error is attributed to execution. (Note: If line fill or dollar fill is used, then calculate an attribution ratio using planning error and quantity missed and allocate line fill or dollar fill miss based on each item's attribution ratio.)

With the completed attribution analysis report, each group can analyze its misses. Master data, such as product line, life cycle, ABC class, and more, can be used to see patterns in the attributed misses. For example, a disproportionate percentage of A items may be causing service problems.

In overall terms, the planning service level answers the following questions: Did the planning groups give the execution groups a chance at hitting the supply chain's service level targets? Was the planning service level so low that, in effect, the execution group was doomed from the beginning? Or was the planning service level fine but execution blew it? Each organization must establish baselines and targets for its planning and execution groups.

The value of these new service level metrics is in the ability to track planning and execution performance just as the overall supply chain performance is traditionally tracked.

Corrective action processes
The second part of the SCOPA process is to take corrective action on the problems identified in the attribution analysis. For the success of the supply chain, problems need to be addressed from the prevention (guardrail) and treatment (ambulance) perspectives.

Businesses spend a lot of time taking care of immediate problems, but it's important to consistently work on process improvement. Because the two processes are different—one being tactical, the other strategic—each process needs separate meetings. The approach to each meeting also must be different.

The guardrail process improves planning and execution, and it is strategic in nature. The first meeting should be held in order to review the attribution analysis report. There should be no finger pointing. The report clearly shows the group responsible for each item missed. This meeting should give each group the opportunity to analyze why items were missed. After this meeting, groups should be given time to process data and look for opportunities for improvement.

The second meeting of the guardrail initiative is the process development meeting, which gives each group the opportunity to report its findings. These findings include root cause and proposed process changes. It's important that both planning and execution groups meet together so the process flow between the two groups can be analyzed and discussed. Depending on the current service situation, this meeting could be held as often as weekly, but monthly probably will suffice. Keeping track of ideas and process improvement initiatives is important in order to allow for follow-up. Discussion during the process development meeting need not be limited to items or issues listed on the attribution analysis report. The report may serve as a launching pad for other continuous improvement ideas. Get the two groups talking and get the creative juices flowing.

Next, the ambulance process prioritizes and expedites current problem items. This process is tactical in nature. It takes care of urgent supply chain issues and includes the use of shortage reports, service miss reports, expedite reports, and related items.

A prioritizing and expediting meeting is held so that planning and execution can coordinate these urgent issues. This meeting is not about assigning blame; it's about what needs to happen to alleviate emergencies and recover items to acceptable inventory position.

Again, a weekly meeting probably will suffice. Two metrics I've found useful in identifying items to discuss during prioritizing and expediting meetings are weeks of supply and percent to safety stock. At Moen, we established thresholds where items become either alert or critical. One key is to keep the SKU set manageable. Some groupings that may be helpful are A items, SKUs sold to major customers, and items planned by a particular group or individual.

SCOPA at the office
At Moen, we've seen SCOPA improve safety stock methodology. Through the attribution analysis, we found a weakness in our methodology for SKUs that recently transitioned from Moen's new product planning group to the core planning group. Without seeing the SKUs' service misses grouped together on the attribution analysis report, we probably would have missed the opportunity.

SCOPA also has allowed Moen to see its forecasting and fulfillment problems sooner and, therefore, handle them in a proactive rather than reactive manner. Sometimes somebody will say, "I'm doing everything I can; nothing more can be done." However, when you focus on a particular problem, something can be done. We established triggers that indicate when an order needs to be expedited and who to contact to make things happen swiftly. Plus, we improved communication up and down the supply chain.

Finally, we discovered execution problems were being masked by a positive forecast error. For a particular month, the attribution analysis revealed that a significant number of SKUs with execution misses were faucets with a common product characteristic. In discussing this set of SKUs, we determined that either our replenishment lead times were wrong or we needed to improve the component supply chain. (One countermeasure is to increase component safety stock.) Spurred by SCOPA information, two people were able to investigate further and suggest process improvements.

In all of these cases, if we had used traditional forecast error analysis, we would have falsely targeted finished goods planning and possibly raised safety stocks on finished goods. This would have been an inefficient place to increase inventory.

Using SCOPA to deal with one particular customer that was experiencing service problems, Moen extended its cross-functional team to include plant representatives, and top-level management communicated to the whole supply chain the importance of servicing this customer. These process changes helped Moen raise this customer's on-time quantity fill rate to almost 100 percent within five months. SCOPA's success at Moen is reinforced because all of the company's business planning groups now use the tool.

SCOPA only takes consideration of forecast and safety stock, ownership and accountability, and prevention and treatment. If you already perform forecast error analyses, SCOPA will not be hard to do. The resulting improvements to the supply chain are worth it.

Kent Linford is the senior planning analyst at Moen Incorporated, and a member of the Cleveland, Ohio, chapter of APICS. His operations management experience includes demand planning, master scheduling, production planning, inventory control, and order fulfillment. Linford may be contacted at klinford@moen.com.