How can I quote for this project? Do I know enough to provide a definitive estimate? Or do I need to keep it high level until we know more? Which algorithm should I use to get the answer? Whose input do I need? And how much input is enough? And what to do if things go wrong?
If you’ve been in or around the IT industry for a while, you’ve almost certainly found yourself in this situation when looking to win a new project from a client.
At ITMAGINATION I’m asked to produce cost estimates or provide input for cost estimates for our clients quite regularly. It’s not always easy. Sometimes, the extent to which a project is defined doesn’t go beyond a few sentences. Not ideal, but we all know what it’s like to hear “just give me an idea of what it’ll cost”. That’s right, sometimes we have to work with less than what we feel is enough information, or we risk a customer walking away and finding somebody who will give them ‘that number’; that “idea of what it’ll cost”. So I often ask myself:
How can I scope and quote this project in a way that a) is as accurate and professional as possible, and b) minimizes the risk for us as the vendor (ITMAGINATION) and my project team?
The 3 (or 4) methods we consider when providing a cost estimate for an IT project
There are at least three viable methods that can be used to scope out a project and then provide a time and cost estimate. These three methods provide us with enough of a basis to back up our estimates in an informed and industry-recognized manner. The three well-known methods I’m referring to are:
- Estimation by analogy
- Expert judgment
There’s also a fourth way, which is a little more creative and perhaps nonstandard (although, arguably, it could be considered an extension of the Expert Judgement approach). I call this t. In this approach, we work on the basis that, if we put X number of people on the project and give then 6 months, we absolutely know that the project will get done. It might not take all of that time and perhaps not everybody will be needed 100% of the time, but – if a number and time estimate is needed – then this is a safe and solid estimation.
Estimation by analogy
The ‘estimation by analogy’ method works on the basis that we’ll be doing something that we’ve done before, know well, and should have sufficient historical information or data to guide us with estimating this new scope and generating a cost estimate. Although this approach is theoretically sound, experience tells me that most IT professionals rarely refer back to documented ‘lessons learned’. Projects are completed, but then the experiences and lessons learned are left to gather dust, rarely referred back to for the future. Of course, as professionals, experiences contribute to our expertise and this is reflected in all our work, but this is rather informal. At the same time, the effectiveness of the estimation-by-analogy approach to project scoping and estimation is limited unless we formally review and discuss data and information from past experiences.
Using expert judgment to scope and estimate a project
The method closest to my heart is the one that relies primarily on the input of experts (i.e. people that have worked on projects of similar scale and complexity). In my work at ITMAGINATION, I typically invite at least three experts to take part in the process. Each of them will lean on their own experience and expertise to provide an independent estimate based on our understanding of a project’s requirements. Providing that there are no extreme outliers in the respective estimates (which would need to be discussed), we can take the average estimates from across our panel of expert inputs and treat this as our safe estimate.
This method has a fair bit in common with the single-minded ‘expert’ assumption method. The main difference is that one method invites several people to provide their input and thus allows for consensus to be reached, while the other typically comes from one or two strong-minded experts (you could say it’s a little bit dictatorial, even). Arguably, a consensus-led approach and one that has the ‘buy in’ or at least has considered input from several people, is the safer approach and is the most likely to be accurate (and thus successful). Furthermore, the single-minded ‘expert’ assumption often comes with some caveats and disclaimers, such as a need to focus 100% on a single topic for the duration of the project (which isn’t always realistic) or a lack of awareness or anticipation of potentially disruptive external factors. It is bold, and it can be risky.
In my opinion, the ‘expert input’ model is among the most accurate methods for project scoping and estimation … on the condition that you pick the right experts, and that they are experts with the appropriate knowledge and expertise within a given technology and/or business domain.
Algorithm-based approach to project scoping and estimation
I must confess that the algorithmic approach to project scoping and estimation is the one that I’ve used least in my 10+ years in IT. One example of the type of algorithmic model that is used to quote projects is COCOMO, which stands for COst COnstruction Model. COCOMO helps predict the effort and schedule for the development of a software project based on historical data and inputs relating to the size of the software and a number of cost drivers that affect productivity. There are various different versions of COCOMO that I’m familiar with, ranging from basic to advanced.
In the version I’m most familiar with, which is the ‘Intermediate’ version, the first action to take is to estimate the number of lines of code or function points. Thereafter, there are four categories of cost drivers that are considered. These are: software/product attributes, hardware/platform attributes, personnel attributes and project attributes.
Here’s a link for you to see how COCOMO and other such algorithmic models work.
Models like COCOMO are useful in that they are transparent, factual, easy to interpret and they’re recognized in the industry. They also force you to consider all relevant factors before arriving at a number, and they help you to understand the impact of these factors on a project scope and plan. On the downside, how can we properly estimate the number of lines of code at the outset? And is the number of lines of code the appropriate way of understanding scope and measuring effort? What’s more, COCOMO doesn’t factor in the importance of factors such as the quality of project management, the nature of the relationship between vendor and customer, turnover in personnel, and other occurrences and variables that affect projects. Importantly, COCOMO also traditionally works on the basis that Design has taken place and that there will be few, if any, changes to a project. Given the prevalence of agile and the need for products to evolve in line with constantly changing business and technical requirements, one could argue that the use of models like COCOMO is rarely feasible.
My preferred method for project scoping and estimation? Count on experts
When considering which approach is right for you, you need to consider your own environment. Do you have experts with the right level of experience and knowledge, whose judgement you can trust? Do you have project managers and platform experts around you that can help facilitate a smooth product journey? Do you have relationships with your clients and stakeholders that feel more like partnerships than traditional vendor-client relationships? If you don’t, then an algorithmic approach might be the safest way for you to proceed and the easiest approach to defend.
Ultimately, though, my take on this is that algorithmic approaches to project scoping and quoting still require expert knowledge and input in order for useful inputs to be entered into the algorithm. This being the case, I personally prefer to take an end-to-end approach, which means learning about and defining the requirements, evaluating the circumstances (history, availability of expertise, project management presence, relationship with the client) and then feeding all of this information into an informed statement of work and estimate. At ITMAGINATION, I benefit from having healthy, collaborative relationships with my clients and their stakeholders – both business and technical. I can also be confident in the project management and the level of technical knowledge and experience around me. All of this gives me and my team the confidence to build scope of work statements and estimates that we can confidently present, discuss and probe further with clients and stakeholders.
Sometimes shit happens. What do we do then?
Having a proper and comprehensive understanding of a project’s requirements is great and it gives you a high chance of success. But it’s not everything. We must be prepared for those inevitable moments when – for one reason or another – shit happens and a project deviates from its originally desired path. For this reason, it’s always wise to factor in a buffer or margin for error. This isn’t about a vendor increasing its margin – applied properly, it’s a safety measure that protects both the client and vendor. A safety buffer should allow for a variety of different events that could negatively influence the progression of a project. It might be a case of force majeure or it might be due to human factors such as changes in personnel, staff turnover, change in leadership on client side. Or it might simply be that something you all thought was rather small has become quite big and will now require more time from everybody. Either way, the presence and amount of the buffer should be clearly and transparently communicated by the project manager. Buffers are important, and how they are communicated and managed says much about the relationship between client and vendor. At ITMAGINATION, we pride ourselves on the transparency and openness with which we handle this with our clients. A buffer naturally leads to an increase in the total possible cost, but most people recognize that factoring in a buffer is part of good project management and that it’s better to see such numbers ahead of time, rather than be shocked by them later.
Now we know how to estimate, what’s next? How do we make the difference?
During my experience at ITMAGINATION, I’ve learned that the most important element of building a project scope or work document and estimation is understanding the client’s expectations, especially their functional requirements. This is much easier said than done. After all, we all understand things differently and our understandings are shaped by a variety of factors, such our own experiences (both as professionals and as users) and positive or negative sentiments towards specific solutions.
I’ll always remember the way one of my first managers explained this to me. His name was Piotr (Peter in English) and he told me that quoting for an IT project is like buying a car. The most expensive car will get you from A to B. The cheapest car will get you from A to B. Ostensibly, they both do the same thing. But, if that’s the case, why would anybody spend more to buy the more expensive car? It’s because getting us from A to B is only one part of our expectation (albeit probably the most important one) – most of us also care about how the car looks, how it responds when we put our foot down, and how comfortable our kids are in the back.
This analogy with cars is useful and brings us on to another point. When most of us buy cars, we typically decide that we need a car, then we establish some basic criteria (e.g. size and shape) and decide how much money we can spend (our budget). Once we’ve decided what our budget is, we then go out and try and get the best deal (i.e. the best car, with the most features) for our money. I often think that it would be useful for clients to tell us what their budget is to deliver a project that fulfills X, Y and Z requirements. Equipped with this knowledge, all vendors that have been invited to make a proposal then have the opportunity to show the client just how much they can deliver for that budget. Are they delivering a base-level product that serves the basic purpose, or are they promising to deliver a high-specification product that the client will be so enamored with that they would consider keeping it for longer than they originally planned? Will it come with a user experience (UX) that is functional but uninspiring, and thus is likely to be received meekly by its users? Or will it come with a fully thought-through UX that compels users to do more in the tool and maximize the benefits for everybody? And does your vendor build solutions that simply tick boxes or do they care about things like automating back-end functions that nobody really sees, but – in the long-term – might free up half of a person’s time (and thus generates a whole load of associated cost savings)? These are the areas that the better vendors will differentiate themselves and ultimately deliver longer-lasting value to their clients’ business. Unfortunately, such scenarios are rare in the industry today, which is why project estimates – and the outputs – vary so wildly when IT projects are put out to tender. This is a shame, as it often means that the best mid- or long-term solution is overlooked in favor of the cheapest (in the short term, at least) solution.
How do you want to get from point A to point B?
In my opinion, IT services are very much a case of you get for what you pay for. Sometimes you pay less money at the start and end up paying more later due to mismanaged projects and poor vendor performance (either financially or through stress, time, effort or lost business). Sometimes you pay a little bit more at the start, and you get what you wanted, and you get to derive much more value later. In both scenarios, you might get from point A to point B, but the quality and comfort of your ride will differ, as will the time it takes you to get there and also your desire to continue the journey on to points, C, D and beyond.
The question is: How would you and your business like to get where you want to go?
The answer is never that simple, which is why us IT professionals all love to say:
Well, it all depends on …
Author: Jakub Miąsik
Learn it. Know it. Done.