Why You Should Use Fivetran And How It Can Save You Time And Money – Data Consulting
Photo by Margaret Weir on Unsplash
Our team of data engineering and data science consultants has spent a lot of time working with and researching what is often referenced as the modern data stack.
Whether we are working with small companies or large companies the modern data stack has tools for everyone.
We will be taking a look at these tools and how they help alleviate many of the problems faced by companies of all sizes today.
Our first look at the data stack is Fivetran. We have recently started utilizing Fivetran for some of our consulting projects and wanted to share some of the benefits we found.
Fivetran is a highly comprehensive ELT tool that is becoming more popular every day.
You’re probably wondering, what is an ELT?
ELT stands for extract, load, transform.
These are the steps required to take your data from a system like Salesforce, Shopify, or Quickbooks and insert it into your centralized data storage system (often referred to as a data warehouse).
Basically, Fivetran allows for the efficient collection of business processes and customer data from related applications, websites, and servers. The data collected is then transferred to other tools for analytics, marketing, and data warehousing purposes.
How Can Fivetran Help Your Business?
Let’s go beyond just the technical jargon.
How can Fivetran help your business? Fivetran was developed to help reduce the amount of time required to get data from source systems to a centralized data warehouse.
Whether this end location is SnowflakeDB, Redshift, or some other modern cloud data warehouse.
This ability to quickly pull in data allows businesses of all sizes to not have to wait until complex pipelines get developed. General data warehouse development could take months maybe even a year. In a modern data-driven world, this is rarely sufficient.
Your company needs answers from your data yesterday.
Fivetran focuses on creating a software solution that is easy to set up, easy to maintain, and has a broad range of connectors providers the ability to become data-driven at any size.
In this article, we will discuss exactly how these benefits make a massive difference and how your company can consolidate its data teams by utilizing a more efficient data tool like Fivetran.
Quick To Set Up
Fivetran is a cloud-first software solution. Meaning it has been developed to quickly integrate with your other cloud software. Also, unlike some other options like Airflow and Luigi which require a lot of upfront time being spent just to correctly set it up. Fivetran can basically be utilized out-of-the-box.
Once you have a Fivetran account, setup takes anywhere from 5 minutes to 2 hours (depending on the complexity of your internal firewalls and integration setups).
In our experience with Fivetran consulting, the first step is to connect a data warehouse.
Following connect the data warehouse, next you can go to the main Dashboard, where you can add new connectors. Adding a new connector will lead you to the setup page with detailed notes on the screen and verification for the configuration.
If you don’t have permissions or access to a data source, it’s best to invite users who do to Fivetran – you can create as many Fivetran users as you want! Once your first connector is created, it will begin syncing immediately and once that is done you can start using that data in your data warehouse. You can add as many connectors as needed to complete your initial setup.
That’s it.
You could be pulling data into your data warehouse within the first day and analyzing it before the weekends.
None of this would be possible without a wide array of data connectors that reduce the amount of custom coding required to design your ELT.
Fivetran Offers Multiple Connectors
Being a data engineer, I can’t tell you the number of times I have written code to connect to Salesforce, Workday, or Asana repeatedly. For different companies and different projects.
Just to get raw data from the source to a raw table in a data warehouse. This is before even applying any logic, developing models, or any form of ROI producing activity.
This is why I like Fivetran.
It gets rid of a lot of the early heavy lifting that data engineers typically find themselves doing.
In my experience tools like Fivetran allow me to can spend more time developing dashboards and finding out ways your company can save money.
This is all because Fivetran offers well over 50 connectors. You name a third-party tool(Shopify, Zendesk, S3, etc) and they probably pull data from it.
They even have some event-driven data connectors which can provide a little more real-time analytics.
Overall,
Fivetran Makes ELTs Easy To Maintain
Fivetran isn’t just easy to use, it’s easy to maintain.
This is very important for small and medium-sized companies who are possibly looking to only have a part-time data engineer or analysts manage their data platform.
It also frees companies from needing to use complex systems utilizing Airflow or Informatica. Teams can take advantage of Fivetran’s integrations with dbt.
For those who are new to the concept of dbt.
dbt is open-source software that enables you to perform sophisticated data transformations in your destination using simple SQL statements. With dbt, you can take your SQL transformations and implement version control with your transformations. Also, dbt allows you to automatically create and share documentation about your dbt transformation and view data lineage graphs.
Here are some more of the features the dbt integration offers.
For those who aren’t accustomed to developing data platforms and data warehouses. The ability to automatically have your code set up with version control is a game changer. It saves time and makes data engineers lives easier. Also, the fact that dbt automatically manages documentation as well as data lineage graphs make your final product more maintainable.
This is because it is easy to roll back code when you make a bad change as well as it’s easy to track code thanks to data lineage.
Overall, all the benefits above, lead to one major benefit. A reduced amount of time for teams to go from raw data to analytics.
Fivetran Reduces The Number Of Developers/Time Required
When it comes to your typical data teams you will often need data scientists, a data engineer, and a software engineer.
Utilizing low-code/no-code options like Fivetran allow small and medium companies the ability to hire a small team of data consultants (if not a signal data consultant) to set up their data analytics architecture.
Having tools like Fivetran, Tableau, and cloud data warehouses allow companies to access the benefits of data analytics easy. Many of these tools cover that 60-80% of work that used to be custom code.
Allowing tools
Now, much of this can be reduced, and thus having a strong broadly skilled technical consultant can help your team benefit without the heavy cost of hiring 3 150k engineers.
Overall, Fivetran offers your team a lot of benefits. Whether it be through the consolidation of mundane work or speeding up your development process. Small and medium companies can now compete at the same level as larger tech-focused companies.
Before Looking To Code – Get A Free Consultation
Before deciding what your technical stack should look like, consider having a data science or software consultant scope out your work first.
Low code/no code ELTs work great in a lot of cases. However, they might not be for you. So making sure you have an expert look over your future data projects and needs is a must.
Our data science and engineering consultants have helped develop multiple companies’ entire data platform. Starting from raw data sources like Salesforce and Shopify to dashboards and machine learning models.
We would love to provide consulting for your data project. Please feel to set up a quick 30-minute scoping meeting to see how we can help you.
Whether you’re looking for Fivetran consulting or a broader range of data science consulting. Our team can help.
If you are interested in reading more about data science or data engineering, then read the articles below.
Developing A Data Analytics Strategy For Small Businesses And Start-ups
3 Ways To Improve Your Data Science Teams Efficiency
4 SQL Tips For Data Scientists
How To Improve Your Data-Driven Strategy
What Is A Data Warehouse And Why Use It
2 Responses
Interesting read. I think ELT tools are the future, they provide companies with the ability to reallocate staff to roles where they can innovate and help grow the business. The ability to allow non-technical staff to solve problems that formerly needed one or more technical staff members is amazing, it can both free up resources and speed up processes.
How do you think Fivetran compares to other ELT tools on the market like, Alteryx, Coefficient, and Kinetic Data?
– John R.
Co-founder Abada Solutions
Website: Abada Solutions
I also agree that ELT tools help reallocate resources. I do think you will still unavoidable need to have data engineers create data models for BI reporting as otherwise what ends up happening is report A will say something from report B. However, they can now spend less time creating the initial connector like they might have to if they are using a coding framework.
As far as those other tools. I feel like Altreyx is a great option for larger companies in particular. It might also be a little overwhelming for some smaller companies with all of its useful features. That’s why I generally prefer to build using Fivetran for smaller companies.
I feel like coefficient.io solves a different problem. It’s more similar to Power Query. E.g. just a bunch of connected spreadsheets. Great for automated analysis. Maybe a little less great for developing a robust reporting data storage system.
As for KD, this is the first time I have looked into it. Initial thoughts is it is another drag and drop tool that will likely eventually get taken over by Talend or Informatica.
That’s my quick two cents.
Comments are closed.