SLAs & KPIs: Managing Vendors

Managers who will be interacting with the vendor should understand these points as they will be an important component of the delivery interactions.

In the Fundaments Article, we referenced examples of a performance measure where the vendor missed the SLA for Severity 2 Incident Resolution. We will utilize that example to illustrate what would happen in the service management group to determine the appropriate disposition of that result.

Exemptions:  one of the core principles of an SLA framework is that the vendor can only be held accountable for something that they control. In the event that an outside party caused the vendor to miss the performance target, then that task would be excluded from the calculation.  The exclusion requires the client to agree to the exclusion.  If the revised calculation results in a measurement that is equal to or higher than the minimum performance target, then the penalties do not apply.

Law of Small Numbers:  there is a principle in the SLA framework that a single miss should not trigger a financial penalty.  In our Severity 2 Resolution example above, you will see that the vendor missed 1 out of 10 incidents causing a 90% performance rating where the target was 95%.  In order for a single miss to not trigger this  performance measure, there needs to be at least 20 Severity 2 incidents in the month, i.e. 19 successful events against a total of 20 = 95% performance.  The typical approach to remedy this situation would be to carry over the results to the following month and aggregate the volume of the two months to reach the required volume of incidents for evaluation.

Exception:   some measures that are so important that a single miss can cause a financial penalty, for example application availability. 

Earn Back:   in some agreements, the vendor has the right to earn back penalties that are incurred. The ability to earn back is based on the achievement or overachievement of performance targets in subsequent months. It is worth noting that Earn Back provisions are no longer common in the competitive outsourcing marketplace.

Dead Band:  when a vendor presents a solution it is based on a series of assumptions, one of the most critical being volume of work.  As an example, for application support engagements the number of incidents per month is an important factor in staffing and has a direct impact on the ability of the vendor to meet the agreed upon service levels.  Dead Bands are constructed to indicate when variability is deemed high enough to impact the vendor’s ability to perform, thereby giving them relief from service level penalties.

In the example shown, the expected volume of incidents is 1,500 per month. The parties have agreed to establish the dead bands at +/- 500 incidents per month.  In the month of January and February, the incident volumes were within the expected range. 

Sample Dead Band:

 In March there was an unexpected surge of incidents that resulted in volumes above the upper range. Provided that these incidents were not the result of something the vendor did, there would be relief granted for SLA misses that were “volume related” during that month.  SLAs such as Incident Remediation for Severity 3-4 might be excused because Severity 3-4 incidents is the vast majority of the incidents. Remediation for Severity 1-2 incident would not be excused because the volume is typically less than 5% of the total incidents. 

June, July and August all are above the upper range. When 3 or more months are outside the range (above or below) a formal meeting is called to review the expected volume. Adjustments can be made to either the service level performance requirements (no cost impact) or the necessary staffing to align with the new expected volume  (potential cost impact +/-).    

Application Tiers:   It is not unusual for large scale organizations to split their application portfolios into different groups (ex: Gold, Silver, Bronze) based on their criticality. In this case, each application tier would have different service levels associated with them. As an example, Gold applications would have higher requirements for system availability than the Silver applications, such as Gold 99.99% availability and Silver 99.9% availability.