Corruption Matters - November 2016 | Issue 48
Have you got the right KPIs?
We all know what they are. But when it comes to writing key performance indicators (KPIs), how many of us know where to start? The right KPIs in contracts can mean the difference between a job done well and a job done to tick a box.
KPIs are often used to establish the core deliverables under a contract and to measure performance. Designing effective KPIs for outcomes-based contracts can be challenging because of the subjective judgments sometimes made.
Any uncertainty around what a contractor is required to deliver, or problems with measuring delivery, because of poorly drafted KPIs can provide real opportunities for under-performance and corrupt conduct to flourish. For example, a KPI that requires a contractor to keep a bathroom clean will only be effective if what is meant by clean is defined as “no mould on tiles”.
KPIs that are unclear or unrealistic may also place a heavy burden on a contractor that may become too much to bear. If contractors experience financial stress as a result of unachievable targets, for example, they may become motivated to recover costs through other means that may involve colluding with public officials for corrupt purposes. An agency may then become susceptible to practices such as inflated variations claims or poor performance driven by a desire to cut costs. Conversely, insufficiently rigorous targets can result in an agency paying too much and getting poor service because it never realised that the outcomes specified by the agency were suboptimal, even though the contractor may have known this all along.
Understanding an agency’s specific operational processes and required standards can help in the writing of effective KPIs. It is equally important to consult operational staff to better understand these processes and gain an understanding of what drives optimum performance. If an agency lacks sufficient data about its operations, this knowledge and the performance measures can take time to develop. If an agency is uncertain about its KPIs or lacks experience in the design of KPIs, another option is to ensure that a contract is written in such a way as to allow for the flexibility to change performance measures over a period of time.
Writing effective KPIs also requires an understanding of the impact that performance measures can have on contractor behaviour. The relationship between measuring and reporting on certain outcomes provides incentives that influence contractor behaviour. The old adage “what gets measured gets done” holds true in this context. The temptation is for contractors to perform well with regard to what is measured and neglect other areas of delivery.
In Operation Monto, for example, the Commission found that the board and senior staff of the agency at the centre of the investigation had focused on safety and the on-time running of trains with little regard to maintenance costs and preventing corruption. As contractor performance and costs attracted minimal attention, opportunities for corruption had been created.
In its publication, Developing and Managing Contracts, the Australian National Audit Office cites a case whereby a KPI measureable was the time taken for a contractor to answer a telephone call. The contractor met the target by picking up the telephone and hanging up again. Of course, the agency’s real intention was to understand the amount of time it took the contractor to adequately answer a client’s enquiry but the KPI motivated the contractor to address the measurement itself and bypass the true requirement.