An implementation specialist who works with hundreds of Salesforce shops every year recently came to us because he was interested in source control and continuous integration for Salesforce. But he said we first needed to calculate in dollars and cents the cost of not investing in these kinds of tools for Salesforce shops. So we chatted with him about some basic numbers and put together this simple example.
For this ROI exercise, we are going to use a typical but small enterprise Salesforce implementation. This is not meant to be a comprehensive look at what the costs for every type of team will be, but it should provide guideposts that you can use to evaluate the costs for your own team.
Our hypothetical team has the follow roles:
- 1 team lead
- 2 admins in charge for production releases
- 2 development admins in charge of sandbox releases
- A team of 10 Salesforce Developers developers
We chose this team size after speaking with several consultants about a typical team size. With this team size you can easily halve, double or multiply the numbers based on your own team size.
Cost 1: Code Clobbering
Code clobbering occurs when one developer overwrites another’s work. This happens all too often in Salesforce. It can happen during a sandbox refresh or a routine deployment. Salesforce’s development model makes this easy because source control is not built into the process. When you deploy in Salesforce you immediately overwrite the code that was in the environment previously. There is no concept of making an incremental change.
If you ask your development team, they will know this problem first hand though hopefully on a relatively small scale. Unfortunately, we have talked to Salesforce teams that have seen serious challenges with code clobbering.
One such shop who shall remain nameless had a team of 20 Salesforce developers. They had no release management process in place. One day a sandbox that contained 3 months of work for all 20 of their developers was refreshed and overwritten. Given the average Salesforce developer salary of $87,000 before including benefits, that means that they lost $435,000 of developer time during a 1 minute incident. They had no recourse and no way to recover any of that work. Sadly, the VP of Engineering was fired when the board learned of this setback.
2012 Salesforce MVP Matt Botos tells of some similarly frightening tales on his blog titled $36,000 Reasons Why I Backup Salesforce.
Let’s imagine a similar incident for our hypothetical team of 10 developers and 4 admins, this would be $281,500 if the same thing happened.
3 months of lost work: (10 developers * $7,250/mo * 3 months) + (4 admins * $5,333 * 3 months) = $281,500
But let’s say that even it happens just once a year and you only lose two weeks of work because you are deploying in two week sprints. That is still $37,000 per year.
2 weeks of lost work: (10 developers * $7,250/mo * 2 weeks) + (4 admins * $5,333 * 2 weeks = $37,500
Code Clobbering Cost: $37,500-$281,500
Cost 2: Deploying Bugs Without Rollback
Your Salesforce developers are there to support valuable parts of your business infrastructure. In some instances they may be supporting a revenue generating sales team, in others they may be a critical part of your ERP or inventory management system. Regardless, the work they do is essential to running your business and this is why companies trust Salesforce and invest millions of dollars a year in Salesforce licenses.
Let’s say a bug is introduced into your Salesforce instance. You have no release management process or source control in place, so you have no record of the state of your application before the bug was introduced. We have talked to dozens of consultants who have seen this happen with their clients. We have heard horror stories of all-hands-on-deck, war-room situations that could last at least 2 days to get everything rolled back and returned to normal.
That means 2 days of your business infrastructure down. Our team of 10 developers might be supporting a sales organization of 500. So there are 500 sales people who cannot do their jobs for 2 days. There are many costs associated with this.
First, there is the wasted salary on 500 sales people who cannot do any productive work. Assuming conservatively that your sales team make $50,000 on average that’s $189k in forgone salary for each such bug. Not to mention forgone sales. Sales that could have been made in that time but won’t be. What if it happens at the end of a quarter? Or during the holidays? For a $40M/year business this would be well over $200,000 in lost revenue.
Cost of Lost Productivity: 500 salespeople * $50,000/salesperson/year ($189/day) * 2 days lost = $189,394
Cost of Lost Sales: $40M annual sales is $109,589/day * 2 days lost = $219,178
Total Deploying Bugs Cost: $189,394-$408,572
Cost 3: Manual Effort Wasted
The third and final major cost is most annoying because it’s the most guaranteed. Bugs and code clobbering are horrific events that are catastrophic when they happen. But hopefully major cases of them at the very worst occur only once a year. But there is a more insidious cost that is guaranteed to happen week in week out for you team: slow, manual deployments.
The way Salesforce deployments work today, are slow and manual. You are required to log into a web interface and write down everything that you have changed. You have to remember not only what has changed but also the dependencies that are affected by your change. Not only is this error prone which feeds into the second cost described above, it’s wasteful and inefficient.
Typically, a team might spend a full day or two just preparing a release. This requires lots of trial and error to set dependencies and make sure that they are working. If this process were to be automated it could take 20 minutes.
If your team is spending a day with every release doing manual deployments, this cost adds up. Using our average developer salary of $87,000 let’s assume you need four developers to support deployments for a team of this size. And let’s assume releasing every two weeks. That’s 24 releases per year. This wasted day every two weeks for four people is worth $23,000 annually.
And that doesn’t account for the benefits that come from releasing continuously on a daily basis. If you release more frequently you:
Reduce the risk of catastrophic code clobbering events like the 3 month event described above.
Reduce risk of bugs getting into production because you can deploy to test environments first. If deployments take only 20 minutes it’s easier to deploy to testing first. But if deployments are taking a full day, then it’s unlikely that you will have the patience to deploy to testing first. Because then it costs you another day to do a production deployment.
Further, most teams that work on Salesforce at this scale use some kind of professional services or outside consulting. For every $1 spent on Salesforce licensing, $3 is spent on professional services alone. This is a $14B market according to IDC. A huge portion of your professional services fees are going to manual deployments. Imagine if your professional services provider could do a deployment in 20 minutes instead of a day. How much more would you save?
Manual effort wasted cost: 4 release management admins * 1 day lost/release * 24 annual releases = $23,000+
Total Cost of Not Having Release Management in Place
For a team of the size we described, the cost of not having a proper release management and source control solution in place ranges from $250,000 to $713,000 annually. This can be avoided by taking a relatively simple step of implementing source control and a proper software development lifecycle management process. There is a reason that every other software development platform invests in tools like Git and source control. It is just too costly not to.
Of course the final cost is morale. Teams that have proper source control and release management in place have happier developers who are able to do better work. And this is a gift that has no price tag.