Automating daily tasks with Python is easy. With a combination of APIs and easy-to-understand libraries, you can easily set up systems that scrape websites, send emails, and manage data and analytics. One very common task you’ll need to automate in the corporate world is the ability to scrape public government data. This usually comes from…
Read more
As a developer, you will often need to create APIs to interact with various systems and integrations. Traditionally, this required a lot of work as far as developing the infrastructure and deploying the code on either an on-premise or cloud server like EC2. However, this method is slow and costly. Thus, the concept of serverless…
Read more
Photo by Mike Benna on Unsplash One of the main roles of a data engineer can be summed up as getting data from point A to point B. We often need to pull data out of one system and insert it into another. This could be for various purposes. This includes analytics, integrations, and machine learning. But in…
Read more
Photo by Levi Jones on Unsplash A vital part of the successful completion of any project is the selection of the right tools. When it comes to big data, sometimes the difficult choice here is picking the right streaming data service. For example should you use Apache Kafka or RabbitMQ. Both platforms feature several functionalities and use cases…
Read more
Photo by Nathan John on Unsplash Managing big data no longer means just buying bigger and faster servers It now also means needing to understand the concept of parallel computing. The list of tools and data systems that are helping manage this specific concept continues to grow on a yearly basis. Whether it be using AWS and querying…
Read more
Python really is everywhere at this point. Although many gatekeepers argue whether a person is really a software developer if they don’t code in a language more difficult than Python, it still is everywhere. It’s used to automate, manage websites, analyze data, and wrangle big data. As data grows, the way we manage it becomes more and…
Read more