Storing Data In The Cloud In 2020: Using RDS, S3 and Redshift

Photo by Taylor Vick on Unsplash

How companies manage their data is no longer limited to traditional relational databases.

Amazon Web Services (AWS) for example offers a diverse collection of options when it comes to storing data.

We recently wrote a piece solely focused on Redshift, but we wanted to introduce a few more options.

We now wanted to take a look now at the various options you have for storing your data with AWS.
In this post we will discuss S3, Redshift and RDS.

These platforms all offer solutions to a variety of different needs that make them unique and distinct. In Comparing Amazon s3 to Redshift to RDS, an in-depth look at exploring their key features and functions becomes useful. Hopefully, the comparison below would help identify which platform offers the best requirements to match your needs.

Amazon Simple Storage Service (Amazon S3)

Amazon’s Simple Storage Service (Amazon S3) is a cloud storage service comes that allows you to interface with your stored objects using REST and SOAP.
S3 provides access to a storage system that is fast, reliable, scalable, and inexpensive data storage infrastructure. Several client types, big or small, can make use of its services to storing and protecting data for different use cases.

Why Use It?

Amazon S3 offers an object (which are essentially files) storage service with features for integrating data, easy-to-use management and everything else cloud often offers. It can essentially act as a type of file server that can manage your companies content for your website like videos and photos or be used to develop a data layer for your analytics.

The platform makes data organization and configuration flexible through adjustable access controls to deliver tailored solutions. Overall, one of the biggest reasons many companies turn to S3 is because of its cost. It offers

S3 Data Lakes And Data Warehouses

S3 is not limited to only storing data like a file server but it can also be utilized as the data layer in a companies data lake or data warehouse. Companies like Snowflake for example have developed a cloud-only based data warehouse that uses a combination of S3 and EC2 to both improve computation as well as AWS charges.

Companies do this by separating the compute and data storage. Using S3 as the data storage layers provides the benefits of fast, scalable and effective storage without requiring an active database server running at all times.

If you chose to set this up internally then it does require a higher level of technical skill but can prove to be very effective.

But let’s go back to old school databases for a moment.

Amazon Relational Database Service (Amazon RDS)

Amazon RDS is a relational database with easy setup, operational management, and scalability. It provides cost-effective and resizable capacity solution which abstracts or automates away many administrative tasks.

RDS is created to overcome a variety of challenges facing today’s business experience who make use of database systems.

What Does it Really Do?

Amazon Relational Database Service offers a web solution that allows you to spin up a database with a click of a button.

No need to look into buying new servers and sizing them up, you can just spin up what you need.

RDS has six database engines you can use including Amazon Aurora, MariaDB, Microsoft SQL Server, MySQL , Oracle, and PostgreSQL.

The traditional database system server comes in a package that includes CPU, IOPs, memory, server, and storage. With Amazon RDS, these are separate parts that allow for independent scaling.

Amazon RDS patches the database automatically as well as automatically backing up, and storing databases. The platform enables developers to generate and handle relational databases as well as integrate its services using Amazon’s NoSQL database tool, SimpleDB, and other supportive applications having relational and non-relational databases.

Use Case for Data Warehouse And Applications

Being that RDS is really just some flavor of your favorite traditional databases it allows for all the use cases you might be accustom to. This includes developing applications that can scale easily or a data warehouse with the computational power required to handle a massive amount of analytical queries. This is up to the user to decide.

Amazon Redshift

Amazon Redshift is unique compared to the other two services because unlike the other two which have much more traditional counterparts. Redshift is a columnar database that has been developed to handle large amounts of data as well as computationally heavy queries. More can be read about that here.

The short of it is that Redshift features an outstandingly fast data loading and querying process through the use of Massively Parallel Processing (MPP) architecture. In addition, the fact that the system is set to store data in a columnar fashion vs. the standard row based store provides huge computational advantages.

From there Redshift offers similar advantages as any cloud service. It allows for several approaches to managing clusters on the cloud. A more interactive approach is the use of AWS Command Line Interface (AWS CLI) or Amazon Redshift console. You can also easily update how many nodes are being used with very little technical understanding(which can be good and bad)

For developers, the usage of Amazon Redshift Query API or the AWS SDK libraries aids in handling clusters.

Also, the usage of infrastructure Virtual Private Cloud (VPC) to launching Amazon Redshift clusters can aid in defining VPC security groups to restricting inbound or outbound accessibilities. The platform makes available a robust Access Control system which permits privileged access to selected users or maintaining availability to defined database groups, levels, and users.

Use Case for A Data Warehouse

Amazon Redshift offers a fully managed data warehouse service and enables data usage to acquire new insights for business processes. The use of this platform delivers a data warehouse solution that is wholly managed, fast, reliable, and scalable.

One could say that Redshift was developed to be a data warehouse first. Where as your standard database was set up for your standard transaction database.

In Conclusion

Completely managed database services are offering a variety of flexible options and can be tailored to suit any business process.

This includes developing data lakes, data warehouses or your standard transactional database. As someone who has helped develop all three each offers it’s own distinct advantages and you should think through your entire process before getting too committed to a specific one.

Many businesses will use a combination of all three. RDS to manage their application, S3 to hold onto scraped data and Redshift for their data warehouse.
Good luck with your cloud building!

If you enjoyed this post, consider reading some of these posts!
Healthcare Fraud Detection With Python
5 Uses Cases For DynamoDB
Learning Data Science And Machine Learning With Youtube And Coursera
Hadoop Vs Relational Database
How Algorithms Can Become Unethical and Biased
Top 10 Business Intelligence (BI) Implementation Tips​
5 Great Big Data Tools For The Future – From Hadoop To Cassandra