Category: data engineering

Real-time Analytics Vs Stream Processing – What Is The Difference?

One of the holy grails that many data teams seem to chase is real-time data analytics. After all, if you can have real-time analytics, you can make better decisions faster. However, there often is a conflation between real-time data analytics and stream processing.  These are two different concepts that are crucial to understanding how to…
Read more


September 3, 2024 0
airbyte alternatives

4 ELT Alternatives To Airbyte – How To Ingest Your Data

Getting data out of source systems and into a data warehouse or data lake is one of the first steps in making it usable by analysts and data scientists. The question is how will your team do that? Will they write custom data connectors, pay for a data connector out of the box or perhaps…
Read more


May 8, 2024 0
change data capture time

Terms You Should Know If You’re Planning To Use Change Data Capture

If you’ve worked in data long enough, then you’ve likely come across the term change data capture. Often called CDC, change data capture involves tracking and recording changes in a database as they happen, and then transmitting these changes to designated targets. This can be crucial because some pipelines, in particular batch pipelines, don’t capture…
Read more


April 29, 2024 0
data engineering videos

10 Great Videos To Help You Learn Data Engineering

How data is structured, managed and processed will continue to grow in importance as the demand for AI and machine learning increase. It’s unavoidable that as businesses demand that their data teams implement AI, they will also realize that data engineers are a crucial piece of the data pipeline. That means, if you’re looking for…
Read more


April 20, 2024 0
apache druid architecture

Apache Druid’s Architecture – How Druid Processes Data In Real Time At Scale

Recently, I wrote an article diving into what Druid is and which companies are using it. Now I wanted to do a deeper dive into Apache Druid’s architecture. Apache Druid has several unique features that allow it to be used as a real-time OLAP. Everything from its various nodes and processes that each have unique…
Read more


March 11, 2024 0
ssis migration project

Alternatives to SSIS(SQL Server Integration Services) – How To Migrate Away From SSIS

SQL Server Integration Services (SSIS) comes with a lot of functionality useful for extracting, transforming, and loading data. It can also play important roles in application development and other projects. But SSIS is far from the only platform that can provide these services. You might seek alternatives to SSIS because you want a more agile…
Read more


February 27, 2024 0
data quality

Why Your Team Needs To Implement Data Quality For Your AI Strategy

Companies that range from start-ups to enterprises are looking to implement AI and ML into their data strategy. With that it’s important not to forget about data quality. Regardless of how fancy or sophisticated a company’s AI model might be, poor data quality will break it. It will make the outputs of these models useless…
Read more


February 12, 2024 0
cut data stack costs

Cutting Your Data Stack Costs: How To Approach It And Common Issues

I once had an engineer tell me that they essentially didn’t want to consider cost as they were building a solution. I was baffled. Don’t get me wrong, yes, when you’re building, you iterate and aim to improve your solutions cost. But from my perspective, I don’t think completely ignoring costs from day one is…
Read more


January 5, 2024 0
SSIS Consulting

What Is SSIS and Should You Use It?

SSIS, short for SQL Server Integration Service, is an essential data migration tool for modern businesses. As a key part of Microsoft’s SQL database software, It allows you to easily complete many complex tasks, including data extraction, merging data, loading and transformation, aggregating data, and more. It’s a comprehensive solution to your data management needs. In today’s business landscape,…
Read more


September 2, 2023 0

4 Alternatives to Fivetran: The Evolving Dynamics of the ETL & ELT Tool Market

The ETL & ELT tool market is experiencing continuous transformation, propelled by fluctuating pricing structures and the advent of inventive alternatives. This industry remains fiercely competitive due to these changing elements and a swiftly growing user base. In the following sections, we will explore four emerging alternatives to Fivetran. Of course, that is if you…
Read more


July 16, 2023 0