Is your performance stifling performance measurement?
Lee R. Feldman, ICMA-CM
City of Palm Bay, Florida
Editor's Note: What follows is the "counterpoint" to the article on performance measurement on page 26.
About two years ago I had a visit from some software developers that desired the City as a Beta Site for their new data management software package. Their pitch was that they could link into all of the City's real-time databases and could provide management with a digital dashboard that would indicate when performance was on the mark or running above or below specific performance benchmarks. I envisioned myself in data heaven; able to sit behind a desk all day and constantly monitoring everyone's performance. Just think what I could do with the Blackberry!
While I never had a second conversation with those software developers, I became convinced that our organization was very good at collecting data. We had all sorts of systems that were designed to gather data. We knew how many drainage complaints we were getting and how many potholes were being filled and how many man-hours were being spent on preventive maintenance of vehicles. We could even tell you the exact latitude and longitude of stop signs. What we were not very good at was using the data collected for any meaningful purpose, such as enhancing performance.
Collecting data for the sake of collecting data is a lot like playing with a gorilla. It may seem fun at first, but you really can't stop when you want to. Data collection must exist for valid purposes: (1) compliance with regulatory schemes (such as NPDES program); (2) to tell a story; and (3) to enhance the operation of the department by reducing a performance deficit. Since we generally do not have any control over what we gather for compliance purposes, I will focus my discussion on using data for storytelling and performance enhancement.
During budget time, when I meet with departments, I am often barraged with everyone's priorities. Programs, which if not funded will surely result in the immediate demise of the city and possibly the end of mankind as we know it today, are often promulgated by well-meaning individuals who truly believe in the value of the project they are pitching. Here is a hypothetical interaction at a typical budget meeting with a Housing Director:
City Manager: At some point, the Federal Government will no longer fund your First Time Homeowners Down Payment Assistance Program and you will be looking for General Fund money.
Housing Director: Yes, that is correct. The program has been very successful in creating new ownership opportunities. Our clients become stakeholders in the community and that is good for the city.
City Manager: Do these stakeholders have a lower incidence of code enforcement violations than others in their neighborhood?
Housing Director: I don't know. I would imagine so.
City Manager: Are your clients less likely to be delinquent on their water and sewer bills? Do we have more calls for police service at this address?
Housing Director: I don't know.
City Manager: Then how can you tell us that these stakeholders are good citizens? How can you convince me that this program accomplishes anything?
Of course that type of discussion would never happen in the public works field. The point of this illustration, however, is that while I am sure that the Housing Director could give me information on the number of clients in the program, the average amount disbursed and even the average number of days to process the application, none of that data tells the story of the program. This is all data that is collected and stored and from time to time even looked at, but it is not data that tells the story of the mission of the department. Letting the City Manager know that in the last three years, the Down Payment Assistance Program assisted 62 clients, of which only three have had code enforcement complaints, only one of which was delinquent of their water bill and none of which have had any reason for the Police Department to visit with them is a compelling story that helps convince the City Manager that the program should continue to be funded once the federal dollars dry up.
As public works officials, you have a story to tell to the administration of the city, to its elected officials and to the public. We are not interested in the fact that you can pave one mile of street in record time or that you have cleaned more culverts than this time last year. We are interested in knowing that the claims for front-end alignment due to road conditions have decreased because of your road paving program and that claims for losses as a result of neighborhood flooding are lower now that you can clean culverts faster. Select data to measure that will help you tell your story and secure future funding for the activities you believe are essential for the public.
In our city, we have developed a performance measurement system which is targeted on improving performance in areas where performance deficits(1) exist. This strategy, which we call PalmStat, serves to allow departments to determine performance deficits (sometimes these are determined for them by the City Manager, Elected Officials or the public), identify strategies to reduce or eliminate the performance deficit and the data measures (performance measurements) to measure the success of the strategies. During the Action Phase, the data is examined to determine if the strategies employed have been effective. If not, the strategies will be refined until success is achieved.
Let's take one of the examples found in Sue Hann's article (p. 26)—the fleet manager and preventive maintenance. In this example, the fleet manager has determined that he or she has a performance deficit of too many vehicles having preventive maintenance performed beyond the mileage criteria established by policy. In this case, the data utilized for analysis is simple—how many vehicles are in compliance. The fleet manager establishes a target of 95% compliance. Using the strategy discussed in the article, the fleet manager quickly comes into compliance and reduces the deficit, but has in turn created additional deficits for the fleet manager and the police chief. So, the fleet manager might want to further analyze the data. Are the non-compliant vehicles all assigned to one shift? Are these pool vehicles? What is the anticipated downtime for the type of vehicles not in compliance? How will the Police Department provide service if the vehicles are scheduled for preventive maintenance?
Potential strategies the fleet manager may explore utilizing are altering the mechanics' hours to accommodate the schedule of the vehicle operator, ferrying pool cars over, or scheduling preventive maintenance on police vehicles immediately post-shift (I could have inserted a donut joke here). The data is then gathered and examined to determine if the target has been achieved.
What is important here is that there is not a lot of data that needs to be collected to handle the performance deficit. The data gathered and examined must be consistent with identifying if the strategies employed are effective. Use the data to determine success (or failure). The police chief is not concerned that proper preventive maintenance will add .26 years to the life of the vehicle, but he is concerned about how much downtime the officers have if they are waiting for you to change their oil.
Look closely at the data you are collecting every day. If it is not required for regulatory compliance, does not help you tell your story or reduce your performance deficits, ask yourself why you are collecting it. If you can't justify the collection of the data, save your staff's time and concentrate on something else. When the vendor comes knocking on your door selling you the software that will enhance your data collection ability, remember that someone needs to enter that data and someone else (probably you) will need to read the reports generated.
Performance measurement will only be detrimental to the organization if it becomes the sole activity. When properly employed it will be your friend.
Lee R. Feldman can be reached at email@example.com.
(1) For a discussion on the definition of "performance deficits" please review Bob Behn's Public Management Report (September 2006), entitled "On Why Public Managers Need to Focus on Their Performance Deficit," http://www.ksg.harvard.edu/TheBehnReport/September2006.pdf.