10 Lessons From Analyzing 15 Billion Metadata Items

The power of Salesforce lies in the ability to customize an org, much of which can be done quickly, by an admin or consultant using declarative (point-and-click) configuration. What this boils down to is this: adding and modifying metadata is easy to do, but get it wrong and face serious consequences. Deleting metadata is an even scarier thought.

The only way to increase both the speed that changes can be made, and to mitigate risk, is to have a well understood implementation lifecycle that manages changes to the org metadata rigorously. Lose control of the metadata and you lose control of the org. This is what we advocate!

So, let’s look at how well other Salesforce customers are doing in taking care of their org.  Here are the 10 things that the Elements Product Management team have discovered and are food for thought.

1. The scale of Salesforce Metadata in Orgs is staggering

Salesforce is a strategic, enterprise platform.  It is reinforced by these numbers. These are the highest numbers of metadata items we have seen in orgs: 250,000 reports, 54,146 email templates, 2,000 custom objects, 20,000 custom fields, 250 record types on an object, 49,000 dashboards, 60 managed packages, 18 million cases, 1.4x storage limit, 114 million task records!

We haven’t analyzed Org62 (Salesforce’s own instance) but we’ve heard that they can beat these numbers!

2. Technical debt is killing agility

COVID has accelerated digital transformation. So organizations need agility which means faster development cycles. But we are seeing staggering levels of technical debt that is stopping organizations migrating from Classic to Lightning. It is killing agility as a recent Forrester article suggested.

We are hearing “throw all that investment away and start again”. This is a huge risk and cost. Surely the answer should be to continuously work at reducing technical debt – every day, every sprint, every release. That is where better org analysis is helping.

3. Hitting the limiters for custom fields

Top 5 standard objects (Lead, Account, Contact, Opportunity and Case) often hit custom field limits (500 or 800 depending on the licensing tier). This is because teams fear that deleting a field will break the org. The common practice is to “hide” or “deactivate” a field – but never actually delete it. We’ve seen examples of orgs with extra custom objects – Opportunity1 & Opportunity2 – for the extra fields alongside the Opportunity standard object. This is wrong, wrong, wrong.

4. Mastering the Metadata API is challenging

We’ve been working with the Metadata API for 4 years and it is a love-hate relationship. It is powerful and we need it, but it can be really quirky. There is a balance of the amount of data you request so that you minimize the API calls, but at the same time not having it time out.

We took 12 months to completely rewrite our sync architecture to make sure we can scale for any size org. Part of that is 1,000s of lines of code that we have added to make sure that we are getting all the data, catching the timeouts, and not blowing up customer’s API limits.

5. Dependency API is no longer supported by Salesforce

The Dependency API launched in beta but the development stopped 3 years ago and the team are now warning developed to understand the implications and risks. But the power of dependency trees is you can show not just where a metadata item is used, but the knock on impact.

Rather than really on the Dependency API, we’ve developed thousands of lines of custom code to be able to build a more complete and accurate where-used and dependency picture.

6. Org Impact analysis takes processing power

Big orgs take a long time to process, so a really well thought through architecture and efficient code is critical. And don’t underestimate the processing power required to create the org impact analysis. Pulling all the metadata into a spreadsheet is a waste of effort because the key is understanding the interdependencies and importance of the metadata items.

And every day, the org has changed so it needs to be reanalyzed because the analysis is only valuable if it is up to date – for every org – Prod and Sandboxes. We make over 500,000 API calls to sync and analyze customer orgs. And we do this on AWS so we don’t trash the org’s API limits.

7. Managed Packages are difficult to remove safely

The difficulty and risk of removing managed packages may seem like great news for ISVs. But our org analysis is making it safer for customers to remove them and potentially save on unnecessary license fees. Apart from the costs, there are risks with managed packages in orgs. They install fields on standard objects and add them to page layouts. And a managed package can be upgraded and suddenly new metadata items appear unannounced.

Be really ruthless about installing managed packages and remove them after the trial ends if they are not going to be used. There are orgs with the Trailhead managed packages installed into production – and the objects being used!!

8. Record Types are like glitter

Record types are really powerful. They are like glitter. They are so powerful and shiny, but they are almost impossible to get rid of. And you instantly regret having so many.

Why is so hard to get rid of them? Like a field, record types can be referenced by so many metadata types. If you remove them without checking everywhere they are used then you risk breaking the org. So you don’t or can’t delete them. Without our record type analysis you are stuck with unused record types cluttering up the org. Just like the glitter you find, weeks after the party happened, that the hoover hasn’t managed to catch.

9. No one wants to document

The impact on agility, risk and technical debt is due to the lack of documentation. Even at the most basic level (e.g. completing description fields) it is embarrassingly bad. Even standard object metadata and Salesforce managed packages have very few description fields filled out. Documentation of “why” you made a change is critical institutional knowledge that will make future impact assessment easier. It is ‘paying it forward’.

10. Shorter release cycles leads to a greater ROI

Fact. Faster releases drive huge business agility and also engage end users who feel their requests are being addressed. ROI and confidence skyrocket as the business get incremental changes more quickly. All of this drives up the ROI of Salesforce.

But this doesn’t happen because release cycles are often over a month. Why? Every change goes through a rigorous development cycle because there is very poor impact analysis, no documentation, and little thought given to streamlining the development process.