Rivery.io – What Is It and How It Can Help You Develop Your Data Pipelines
Photo by Martin Sanchez on Unsplash
There are plenty of clichés about data and its likeness to oil. But it’s far from easy to get value from data.
Companies are creating more data than ever before. At our current pace, 2.5 quintillion bytes of data are created every day. Companies went from pulling data solely from their CRM and ERP systems to pulling data from dozens or hundreds of data sources simultaneously.
Creating data pipelines for all of these data sources is expensive and difficult to maintain. Data engineers are often stretched thin, between constantly updating API connections, to building new pipelines for every data source. Without access to this data, analysts and data scientists cannot provide key insights for organizational stakeholders.
But, in the last decade, software solutions have emerged that can pull data from many different data sources without requiring companies to build infrastructure manually. These managed, low-code platforms increase efficiency and time-to-insights, while freeing engineers to work on more important projects.
Rivery counts itself among these platforms. I recently had the chance to test out Rivery for myself. Here’s what I found.
What Is Rivery?
Rivery is a SaaS DataOps platform that gives companies full access to their data quickly and securely. Rivery has been growing rapidly, adding several hundred customers over the last three years, because the company has a solution to a problem most companies have.
Rivery’s approach to data pipelining and data management incorporates automation and actionable logic into traditional data ETL/ELT processes. Rivery doesn’t just focus on getting data into a company’s data warehouse. The platform also manages the data modeling process, including through pre-built data model Kits (more on that in a moment).
Rivery’s focus on automation and actionable logic empower data engineers to develop a centralized data management system rapidly. The platform also allows companies to create custom data connections that run alongside standard pre-built data connectors, saving significant time and cost for both deployment and ongoing use. So overall, Rivery has a great framework for data ingestion.
Now let’s dig into the specifics of the Rivery platform.
Rivery: Key Features & Capabilities
Rivery is built on a DataOps framework that automates data ingestion, transformation, and orchestration. As a low-code ETL platform, Rivery offers many key features, from pre-built data connectors to instant data model Kits.
This ability to instantly create and replicate data models via ready-built Kits is something I look forward to seeing mature over the next year.
Like many ELT tools that exist today, Rivery offers over a hundred automated data connectors.
Rivery’s pre-built connectors run the full gamut of data sources, from standard SFTP connections, to specific niche connections for Marketo or Zendesk. With Rivery’s custom Rest API, companies can also create custom data connections that run alongside standard pre-built data connectors
For companies developing a centralized data warehouse, the ability to connect to data sources quickly is of the utmost importance. But in practice, this is not always the case. Data engineers over the past few decades have been forced to create these connectors manually as well as repeatedly. I have personally had to develop a Salesforce connector at least 3 times.
That’s why having an automated data connector is a must for any form of data pipelining tool, and Rivery more than passes the test.
However, data ingestion is only one layer in terms of the work required to process data in preparation for analysis. Getting the data into your data storage system is step one. Arguably the trickier work is modeling, cleaning up, and transforming the data correctly.
Rivery simplifies this process with instant data model Kits. Kits essentially act as pre-built data models – including all the logics and transformations – for predefined data sources. Kits are available for use cases such as B2C marketing analytics, business monitoring, and Netsuite income statements. Currently, there are a few dozen Kits that Rivery offers, andI am interested to see how Rivery will continue to grow this feature.
What Is My First Impression of Rivery
There are a whole host of low-code solutions in terms of data pipelines, automated data connectors, ELTs, and ETLs.
All of these tools are helping data engineers and analysts get data from data sources to data warehouses.
So here’s what stood out to me when I tested out Rivery.
Ease Of Use
By looking at the various data connectors, I can tell that Rivery was developed by engineers who understand the general workflows that data engineers follow.
For example, this might seem straightforward, but Rivery provides a flexible SFTP connector that can look for specific string patterns. Things like this are very helpful. Recently, I was working with another tool that utilizes automated connectors. The tool didn’t provide the ability to match patterns for file names. A small difference, but an impactful one.
On another note, Rivery’sBigQuery connector is very straightforward for the end-user. In another real-life example, I was working with another tool that required me to perform a lot of heavy-lifting in terms of actually digging into BigQuery’s API documentation. I’m an engineer, so this did not pose a technical challenge. But the whole point of low-code tools, in my opinion, is to reduce the amount of time and resources I spend on building infrastructure. Rivery gets this.
Otherwise, why is a company spending so much on a low-code tool?
UI
Rivery UI is clean and makes it easy for an end-user to understand how to get data from point A to point B.This is thanks to Rivery’s GUI, which is not too cluttered, and guides the end-user rather than hiding configuration options in various drop-downs. Personally, I have worked with SaaS solutions that seem to force developers to call up sales engineers on a daily basis just to find some nuanced configuration setting. With Rivery, that doesn’t happen.
Operational Benefits
Many low-code solutions do provide some sort of environmental, variable, and API token management in their overall process. But honestly, probably less than you think.
But Rivery has all of these options and treats data pipelines as deployment packages. This focus on DevOps/DataOps best practices is a unique feeling.
Many other data pipelining solutions are focused more on getting data from point A to point B, and perhaps provide some level of version control. But Rivery incorporates a data infrastructure component that can help manage a data engineer’s entire data workflow.
Conclusion
With the exponential growth of data in companies, data sources are becoming very difficult to manage without tools and technologies. Companies must hire large teams of data engineers to even begin to wrangle their data.
Amid these trends, Rivery stands out. It is helping companies go from data trapped in their various source systems into fully functional data warehouses and data lakes.
Rivery offers a low-code, GUI-based web solution that helps amplify a data engineer’s skill set. Instead of needing to hire 20 data engineers, companies can create a much more manageable data process with Rivery.
Thanks for reading! If you want to read more about data consulting, big data, and data science, then click below.
How Much Do Data Engineers Make: Data Engineer Salaries Vs Software Engineers
Building Your First Data Pipeline: How To Build A Task In Luigi Part 1
Greylock VC and 5 Data Analytics Companies It Invests In
How To Improve Your Data-Driven Strategy
What In The World Is Dremio And Why Is It Valued At 1 Billion Dollars?