Tag: Consulting

Real-time Analytics Vs Stream Processing – What Is The Difference?

One of the holy grails that many data teams seem to chase is real-time data analytics. After all, if you can have real-time analytics, you can make better decisions faster. However, there often is a conflation between real-time data analytics and stream processing.  These are two different concepts that are crucial to understanding how to…
Read more


September 3, 2024 0
how to lead a data team

9 Habits Of Effective Data Managers – Running A Data Team

Running a successful data team is hard. Data teams are expected to juggle a combination of ad-hoc requests, big bet projects, migrations, etc. All while keeping up with the latest changes in technology. In the past few years I have gotten to work with dozens of teams and see how various directors and managers deal…
Read more


July 2, 2024 0
airbyte alternatives

4 ELT Alternatives To Airbyte – How To Ingest Your Data

Getting data out of source systems and into a data warehouse or data lake is one of the first steps in making it usable by analysts and data scientists. The question is how will your team do that? Will they write custom data connectors, pay for a data connector out of the box or perhaps…
Read more


May 8, 2024 0
change data capture time

Terms You Should Know If You’re Planning To Use Change Data Capture

If you’ve worked in data long enough, then you’ve likely come across the term change data capture. Often called CDC, change data capture involves tracking and recording changes in a database as they happen, and then transmitting these changes to designated targets. This can be crucial because some pipelines, in particular batch pipelines, don’t capture…
Read more


April 29, 2024 0
spark vs flink

Apache Spark Vs Apache Flink – What Is The Difference?

As data increased in volume, velocity, and variety, so, in turn, did the need for tools that could help process and manage those larger data sets coming at us at ever faster speeds. As a result, frameworks such as Apache Spark and Apache Flink became popular due to their abilities to handle big data processing…
Read more


April 25, 2024 0
data engineering project consulting

Common Pitfalls of Data Analytics Projects

Have you ever been part of a data or software project that seems stuck in a loop? Three weeks have passed, and although you arrive at work daily, exhausted, having tackled numerous issues, the project remains stagnant. Why? Then, suddenly, a new engineer or project manager steps in, reorganizes and prioritizes tasks, and just like…
Read more


April 12, 2024 0
real-time streaming consulting

5 Real-Time Data Processing and Analytics Technologies – And Where You Can Implement Them

No matter your industry, you’ll often need to make split-second business decisions in the digital age. Real-time data can help you do just that. It’s information that’s made available as soon as it’s created, meaning you don’t need to wait around for the insights you need. Real-time data processing can satisfy the ever-increasing demand for…
Read more


March 1, 2024 0
ssis migration project

Alternatives to SSIS(SQL Server Integration Services) – How To Migrate Away From SSIS

SQL Server Integration Services (SSIS) comes with a lot of functionality useful for extracting, transforming, and loading data. It can also play important roles in application development and other projects. But SSIS is far from the only platform that can provide these services. You might seek alternatives to SSIS because you want a more agile…
Read more


February 27, 2024 0
cut data stack costs

Cutting Your Data Stack Costs: How To Approach It And Common Issues

I once had an engineer tell me that they essentially didn’t want to consider cost as they were building a solution. I was baffled. Don’t get me wrong, yes, when you’re building, you iterate and aim to improve your solutions cost. But from my perspective, I don’t think completely ignoring costs from day one is…
Read more


January 5, 2024 0
apache airflow consulting

Common Pitfalls in Deploying Airflow for Data Teams

If you’re a data engineer, then you’ve likely at least heard of Airflow. Apache Airflow is one of the most popular open-source workflow orchestration solutions that gets used for data pipelines. This is what spurred me to write the article “Should You Use Airflow” because there are plenty of people who don’t enjoy Airflow or…
Read more


November 27, 2023 0