Governing Vendor's Work

I had come across a situation wherein I was asked by a person that his organization do not actually build the product, or execute projects end to end, instead it is done by vendors. The vendors come with their project managers and all other development, QA resources, and they follow usually their standards; they end up following their way in managing project. In nutshell, their (client) only job is to supervise vendor's work. The questions is how to ensure that project is managed in a best possible way under this scenario.

The way I see it, they have already passed the state of "Vendor Assessment" for contractual award, and it had more to do with their "performance assessment"; let's concentrate on the "performance" evaluation part alone.

It is true that you will only be able to assess the performance if you are clear on goal(s). If you know what you need to achieve, and the purpose of hiring contractors instead of building that internally, you will also know the attributes to assess their performance.

In addition, making (or seeing) vendor as partners is good; however, relying on them completely may not be in your best of interests. Please note that by choosing the alternative of getting it done through vendors, you have only passed on the responsibility; the accountability still lies with you.

What it essentially means, you need to identify the attributes from your list of objectives and goals, which are critical for your success, and need to measure / assess the vendor's performance against those attributes. In "service" parlance those are termed as "Service Level Agreements (SLA)". That is, you need to identify the service levels that you require / expect from Delivery, Quality, Time, Cost, Cycle Time, Productivity, System Availability, Performance, Disaster Recovery, Business Continuity etc.

That is, all attributes which affects your business and which you would like to control for favorable business results.

Now, what attributes will constitute SLA, will depend on nature of assignment / relationship with vendor, i.e. whether its Development, Engineering, Maintenance, Application Outsourcing, Business Process Outsourcing, Voice related services outsourcing, Infrastructure Outsourcing etc. Given the context, you would have SLA tuned to that situation.

Please note that outsourcing does not mean you have to follow what vendor says; instead outsourcing warrants more focused (and may be stringent) governance, i.e. governance on all aspects (Finance, Delivery, Quality, Resource Skills, Productivity, etc.).

If a client decides to outsource, it means there would be situations where it would be saving costs; however, at the same time there would be situations where it would be incurring costs. Moreover, there would be some one time and some recurring costs and savings; for example, it would be incurring following onetime costs;
a. Legal Fee
b. Transition Costs
c. Severance Package (if ends the contract prematurely)

On the other hand in a long-term event, it would be having costs / savings and accrual on a recurring basis;
a. Budget Savings
b. Facilities and other savings
c. Training Costs and Staff Turnover
d. Staff Augmentation
e. Inefficient Business Operations – temporary and short-period; after that it should improve. That is cost initially and savings later.
f. Governance cost, etc.

To summarize, identify your objectives and goals, identify measurements for those objectives / goals, set minimum acceptable levels against those measurements (your SLA), decide on incentives for improved results, and make vendor work towards your objectives and goals. Further, if you really want to analyze the perfromance, then you should also consider of comparing the cost of peforming a work internally vis-a-vis getting it done through vendors.

Sample Metrics

Process

1) Variances - Schedule, Effort, Cost
2) SPAN - The elapsed duration since original baseline
3) In process metrics - EV metrics
4) Productivity
5) Cost of Quality (CoQ)
6) Efficiency - Review / Test
7) Effectiveness - Review / Test / Internal / External

Product

1) Defect Density
2) Defect Slip-Age
3) Field Errors

Metrics for Self (Customer)

1) Requirements Stability Index