Reducing Data Analytics Costs In 2023 – Doing More With Less

If you haven’t started looking for ways to improve your data analytics budget for 2023, then you’re probably already behind. The truth is that between all of the various economic indicators and investor letters, everyone is looking to improve audit all parts of their business.

Especially where there has likely been bloat. One of those likely areas has been the data and analytics investments. It makes sense, over the last few years money has been cheap and the focus on data science and machine learning and moving to the cloud seemed to have no limit in spend at many companies.

This has lead to a lot of budgets expanding overnight on Snowflake, Databricks, Looker, Fivetran and other solutions. We know, we have helped several companies cut their data analytics spend often by as much as 50% by auditing and improving their data infrastructure.

So whether you have already started or need to start let’s go over some

Simplify Data Infra

There are plenty of fancy diagrams out there that had every solution category under the sun. I know, I have made at least two of them. But the truth is I also have helped clients use far simper set-ups. It’s nice to diagram out all the fanciest tools until its time to look at your bottomline.

So before getting to drawn into overly complex data infrastructure, take a moment to ask, what are you really trying to do? Are you just trying to answer a few basic questions, and is that all your stakeholders need to know for the next few years or are you trying to build a complex data stack that can help support machine learning models and AI.

Be truthful. As companies are trying to reduce data budgets, it might be time to put some initiatives on hold. If you can easily create reports off of a replicant database, then why build a data warehouse and hire an employee to support it?

If you can use a cheaper data visualization option compared to Looker or Tableau and have it answer the same questions and provide the same functionality, then why not? Now there is a lot of nuance in all of this as TCO is still a reality.

But view this as an opportunity to clean house. There are likely some contracts and vendors that are barely used. It’s a great time to close them out and migrate your team away from them.

Do You Need A Data Infrastructure Audit? Contact Our Team Of Data Infrastructure And Machine Learning Experts Today For A Free 30 Minute Consultation

Pull In Only What You Need

One of the ongoing jokes in the data world is the fact that most companies are like hoarders with data. It’s said that between 60 percent and 73 percent of data that is stored is unused after all. They just store all of it because they never know what they might need.

But this gets expensive. Especially if you’re using solutions that process and charge you via the amount of rows you process. Again, while money was cheap, this behavior was “fine” ( not really), but now you need to be more scrupulous. This was a standard practice in the data world a decade or two ago when everything was on local servers. Back then, you couldn’t get more space on your server in seconds. It’d likely take a few months to get a new server. Leading to data management teams caring about every bit and byte. Now we can focus more on gigabytes and terabytes. But it still matters.

If you’re don’t need to pull in data. Don’t.

Be Mindful Of Your Data Workflows

Snowflake has allowed many companies access to a data platform that usually would require a six to seven-figure contract for as low as five figures. However, this cost-for-consumption model can quickly start to bite users as they start building more and more complex workflows.

Especially if your tables are configured incorrectly or you’re constantly running dashboards live that could rely on pre-aggregated data. This is not unique to Snowflake. Most cloud data warehouses can have sky-rocketing costs when they are poorly set up.

In the era of the DBA, we cared about the number of bytes in a column. Now we need to care about the number of gigabytes, terabytes, and petabytes being processed. They are directly leading to inflated cloud costs.

In the end, I have personally(as well as talked with others who also have) helped cut multiple companies’ data warehouse costs by improving their underlying data structure.

To be clear, it’s not just me. Multiple consultants and newly minted tech leads have all informed me of similar situations(some of which have led to promotions). Often simply by improving some minuscule details in a company’s overall data workflow.

Understand TCO

One concept that often gets passed over is TOC or total cost of ownership. When times are good, it can feel like it doesn’t matter. If your team wants to use a tool that requires more fine tuning and expertise then so be it. But now, if your team is picking solutions that require more fine tuning because it’s the new hot thing to learn. You should question whether this is the right choice. If you can find a solution that costs 10% more but perhaps requires one less person to manage it. That might be the right tool.

As long as the math adds up.

Audit Data Infra

One of the easiest places to start your data analytics cost cutting is auditing your data infrastructure. Yes we already referenced simplify your data infrastructure, but that all starts with knowing what you have.

Do you have dashboards that aren’t utilized? Delete them.

Do you have servers or storage that is under utilized, change their size.

But start by tracking all of this. Take 2-4 weeks, talk to your teams, see what is actually going on. Then make a plan to act. The reason for this is because it will help you make principled decisions on what you keep, what you remove and what you reduce. You can’t start simplifying your data infrastructure without knowing why you have all of it.

Reducing Your Total Data Analytics Cost In 2023

Just because your team might need to reduce their overall costs doesn’t mean your team can’t still use data. But it is important that your team rationalize the expenses. Make sure you’re aligning your data projects with the business and don’t simply spend on the fanciest tools because everyone else is.

Instead, take a moment, audit where spend is going and reduce, remove and resize. If you need help, then contact our team and we will hep reduce your data analytics costs.

The 9 ETL Best Practices Every Data Team Needs To Know

Databases Vs Data Warehouses Vs Data Lakes

​​How to Choose the Right ELT Tool

How To Build a Data-Based Business Strategy in 2022

Onboarding For Data Teams – How to set-up a streamlined onboarding experience