Member Needs Help - Performance Metrics Virtual Brainstorm

Well guys,

Firstly may I say thanks for all the great responses. My thought processes are racing once more after having been stuck looking at the problem.

The main things that I take away from this brainstorming are:

Involvement: I can sit here and define indicators to the nth degree but without the involvement of the team in defining these there will be no buy in and therefore limited success.

Feedback: I think it can be very easy to underestimate the power of the feedback process, and as problems hit it can be very easy to focus on the negative behaviours with the result that positive feedback suffers.

Community: What a great sense of community this site is developing. I know where to come when I am stuck, and will gladly offer my two pennies worth when I think it will help.

A BIG THANKYOU to everyone out there!

Alastair

Data collected in small batches can be misleading due to daily fluctuations. Data collected over longer periods may be more effective.

Time may be a reasonable indicator. Thinking along the lines of: How much time is Bob spending on line one? Line two? How does that compare to Joe?

Consider the source of your gut feeling. Gut feelings generally have an underlying reason, this may give you another starting point.

Factors I've used in performance management (as they may apply here):

Time - average handle time, delay times

Counts - how many items within an hour, how many items within a day, how many items within a week, how many items within a month, how many of which type of items

Quality - how many damaged parcels in a set time, how many dropped items in a set time, how well organized are the parcels if it affects delivery times, how well stacked are the items to avoid falling/dropping damage

Cost - if two people are paid the same amount but work different lines and each line has a different difficulty level, what is the approximate cost per package per line, is cost of service affected by someone specializing for line one vs. line two

Rotating shifts for line work (Person A works Line 1 on day 1, Person B works Line 2 on day 1, they swap the next day)

A few others for the list:
* Safety stats - often a good indicator of efficiency (usually the poorer performers in safety are the poorer performers from a productivity point of view. This is a difficult one to pin people on (works better for teams than individuals), but something that adds to your data. (Please don't just use negative indicators like LTIs but also look to positive/proactive ones - eg hazards identified and addressed. Not sure how many hazards are likely to exist in a sorting environment, but the point can also be applied to the identification of productivity improvements/initiatives which could be rewarded... so...
* Number of improvements/initiative identified to help productivity (who helps follow through on implementing the improvements - not just the ideas)
* Training records - which of the individuals are actively seeking to learn more about their roles. Who has completed what? Feedback from trainers.
* Feedback from van drivers. People contributing to efficient rounds
* Feedback from new people - ask them who are the guys that are really helping them out (help to identify the ones you want to keep).

Most of the above tend to identify those who are performing well. Perhaps the focus should then be on coaching/feedback for those whose names haven't come up on how to be more effective.

Mark and Alastair,

Good luck cracking this puzzle.

A couple thoughts.

Most contributors have already covered the "x items in y minutes" type of volume based metrics. I think those metrics are important.

I think quality metrics are good too. Fewer defects, fewer dropped items, fewer damaged items, fewer mis-routed items. Maybe there's something in this area you can measure for your operation.

Collaboration should be encouraged as well, I think. I'm not sure of all the details of your operation, but when there are opportunities for mentoring, coaching, etc., I think the performance metrics should measure this area.

Another comment -- my kids report cards come back from school initially with very rough metrics (above average, average, below average) (think kindergarten). Slowly the grading scale moves to percentages and 10ths of percentages (think high school and college boards). I suggest that all metrics do not need an IT department and do not need to be measured to the nth degree. Start with basic "above average", "average" and "below average" in the key areas you're measuring. Over time hone the measurements to a more granular level.

Again, best of luck with setting up your metrics.

Regards,
Steve

You could meet with your team and get them to give you the answer(s).

Destructive Brainstorm (it's fun!) - Ask

"What could we do to screw up our performance?"

When you have your list of ideas, ask your team to look for the opposites or counter ideas that will deliver excellent performance. Task members of your team to implement their ideas and, hopefully, some ideas (screw up's and linked excellents) will be bahaviours that you can look for when observing. As the team have been on this journey WITH you, one would expect a better chance of buy in from them.

When I read:

"I have gut feelings about which staff work harder than others, but there is no way of gathering data on what they are actually doing."

Therein lies an opportunity, which I will come back to.

Do you actually NEED time consuming manually collected data that could be open to errors and/or exaggeration by your cleaver staff? Don't get me wrong, performance data is great, but consider the cost to collect it and it's eventual quality.

Anyway, back to the opportunity........

This was also said:

"I started using the feedback model about 4 months ago and it deals with behaviours I don’t like but doesn’t help with identifying and managing performance."

That, for me, is a great step forward, but do more - it CAN identify performance issues!

* Observe your staff.
* Feedback, feedback, feedback, feedback - correct/improve/encourage/acknowledge those behavours that impact performance.
* Define best practice (if not already done).
* Set clear expectations and test everyones understanding of them (if not already done).
* Reward/Incentivise GREAT performance.

Thought: You could video tape your staff and then during your one2one with them, play the video back and ask them to self critique.

Eric-

GREAT POST.

Thank you for taking the time to help a member.

Mark

I have been developing a matrix to measure performance in my area of responsibility for the past year or so. It is a complicated, first in industry system that uses data from satellite technology that was pre-existing. I have worked long and hard on this project, and it is finally coming to fruition. We are currently gleaning Q1 data. The December data was exciting for everyone.

Here is what I have learned so far throughout the process.

1. My subjective opinions about performance were correct. Trust yours.

2. Measuring the performance absolutely had more to do with improving the same performance than anything else. Pneuhardt alluded to this earlier.

3. My people are much more creative than I ever imagined. They have already begun developing their own systems to "fool" the measurement process, thus I will have to change it again eventually.

The process has enabled me to share data with others who have a stake in the process to see where their performance affects the group as a whole, but has ruffled some feathers across the board. One of the negative side effects is that a lot of people have offered to 'help' with the project, now that it looks to be successful. This sounds good at first glance, but keep in mind, there are always those who want to take credit...Not help.

My suggestion about this process is this:

Trust your instincts, but be gentle. Someone always comes in last place in any race, when you measure people they know this and you have to learn to regulate this kind of pressure to help avoid apathy or cynicism.

Keep trying to measure, but be prepared to change that system. You won't be able to develop a perfect system, especially if you have talented people. Even the people who you may view as less resourceful, will surprise you when it comes to the "will of autonomy."

And finally, in any kind of environment, there are subjective measures that you have to use. Attitude, creativity, potential, willingness, etc. to name a few. Be prepared to factor these in whatever system you decide to use.

Eric

In each 'team' operation there is usually a 'pivotal' position that acts as a bottleneck when slow person occupies it - or speeds things up when someone good is there. Identify that position and swap people around, observing the results and interviewing people afterwards.

Peer reviews

Primary Sortation: Number of parcles sorted per person, per hour, per day, per shift, per week and/or per month. Deviations from the normal.
Secondary sortation: Vehicle Data. eg number of parcles in van, capacity %, number of trips, distance, stops, time

Hmm. Potential team metrics:

- % of parcels processed within X amount of time
Could then break down into % of parcels processed by phase 1, phase 2, etc. If IT system does not yet support this, then get your guys to keep tally score pads in the interim.

- % of parcels mis-sorted/directed (i.e. go to the wrong van/location and have to be re-processed thus costing the company money)

Ok, outside the box now...

Give each one a pedometer to see who puts on the most miles (or least while still getting the job done)

go down to FedEx or UPS and see what they do.

Tim-

Brillliant! A Guinness to go with it.

Mark

oops, forgot one thing...

Peanut Butter

Start with group metrics, like how long each phase takes (understanding that they overlap in their timelines). As a third metric, track how long the entire process takes for each load.

Determine a baseline and come up with some sort of reward or incentive for improvement.

Then I'd do two things:
1) Talk to the team about ideas for making further improvements (beyond just each person moving a bit faster)
2) Observe the team closely to see if any ideas for individual metrics jump out. The observable difference may get more pronounced when some people are shooting for a goal and others continue to just try not to get fired.

As a new manager for a company without established procedures, internal and external metrics and without a sophisticated IT department (i know, why did i go there?) i used the following technique to begin assessing what to measure and how:

A) through interviews with key personel and managers i value mapped each action that took place within my department

B) I am always interviewing my manager and his manager and his manager to determine their objectives, values etc.

C) I created a hierarchy of needs: company vision, operational objectives, departmental objectives and individual objectives. i attributed as much information i had accumulated for the first 2. that allowed me to visualize and align myself with upper management to develop objectives for my department and my team members (and myself, of course)....

D) after identifying what is of most value to my manager and ownership as the expert in my field i mapped out a short term plan on how to achieve their objectives. by doing so i identified key areas that need to be measured and analyzed. in my case, as a purchasing manager on time delivery, quality and cost control are basic requirements. by knowing my internal and external customer expectations i know what to measure.

e) another manager and myself created the IT tools to measure.

i hope i helped a little bit. good luck.

The ideas of having people measure themselves are good ones, but use caution on this one. Heisenberg was right (and not just at the sub-atomic level) when he said that the act of measuring something influences the measurement. In fact, the more precisely you measure something, the greater the effect. (At the quantum level, measuring a thing destroys it. That's not what happens here, but it's not a bad thing to keep in mind so that you dont get too crazy with metrics.)

If people see you watching and counting parcels, they may work artifically hard as long as you are there so that they look good. If they do the measuring themselves, their taking the time to collect slips or record extra data will slow them down.

It's your choice as to which impact you would rather have: make people work harder or slow them down.

* Create a schedule for observing staff based on randomization (say over several weeks)
* Establish what you're looking for in the observation (e.g. total parcels sorted)
* With all due apologies for possibly adding a question, finish this sentence: "The easiest way to tell if this job is being done well is ..."

Lynne and BFlynn-

Shame on you! Questions!

Mark

My first thought - what really drives the business? What makes your parcel business more profitable? I could be wrong, but it strikes me that the main drivers are speed with accuracy. Be fast enough to meet the demand and you're profitable.

Hook's idea is good - I'll expand it and say "ask your people". Its pretty simple, and I suspect pretty reliable. Take what they say with a grain of salt, but you'll see a trend. The person on the bottom with either be the worst performer or someone who doesn't fit in with the team.

Its possible that your team is too independent for this to work, in which case my only idea is that you have to get a report from IT. Base the report on what drives profitablity in your company (which could be different than your division).

1. Forget "timely" and get what data when you can from your IT support group.
2. organize your group into teams and have the team members stay together
3. get a little team competition going
4. The last guys chosen for the team might not be your best workers
5. let the teams keep their own scores

As a young student I worked for a pizza specialized restaurant with home delivery (like Domino's Pizza, for example).

In a heavy hour of work, when the deliver boy arrived to the restaurant to pick the next pizza to be delivered, lazy workers used to pick the pizzas to be delivered as nearest to the restaurant as possible, so they could procrastinate but having delivered the same number of pizzas as the rest of the team. That represented a problem, as those orders weren't necessarily the first ones in the cue, so customers who lived farther had to wait more, not only for being farther but also because of lazy deliver boys.

However, each order included the bill with the address on it. So the Manager decided that each order should include two copies of the bill, one to be delivered to the customer and another to be kept by the deliver boy, which by the end of the day should give all the tickets to the Manager. The Manager then analyzed the locations of the deliveries of ech worker and identified whether there was a geographic pattern or not.

Perhaps you could develop a system by which each worker should keep a ticket with the data of each parcel they process and then give them to you.

My experience in a service business (in my case, either customer service call centers or insurance claim audit centers) is that the best metrics focus not just on overall performance, but on performance under the worst conditions. Since you have few metrics in place, focus first on seeing how people handle the worst situations, in your case heavy volumes. That will be the most revealing set of data.

To second what Lynne says, get out on the floor and see what's up. Take a watch and a notepad. Pick a person. Count parcels handled in 10 minutes. Do it again with someone else. Repeat on the next heavy traffic day with two other people, and maybe the next day with yet two more people. I recommend picking someone you think is a high performer, then someone you think is a low performer. If there isn't much of a real difference, re-examine your assumptions. That might lead you to another idea on how to measure performance.

Lastly, try asking your people how they feel they should be measured. It's amazing what they can come up with some times. When you do, make sure you ask because you are interested in recognizing and rewarding superiour performance and not as a way of identifying and punishing poor performance. They will know it's about finding the slackers. It's up to you to prove to them that the same system will reward the stars.

Two thoughts outside of an IT solution:
1) You don't mention that you get out on the floor to observe what's actually taking place during your primary and secondary sorts. You may consider spending a little time each day out on the floor to see if your assumptions are justified.

2) Are there opportunities to change the process so that there is more clarity when the folks doing the primary sorts move to the secondary sort?

Barcodes
RFID