Cloud based storage and data transfer services in AWS
We’ve spent a lot of time looking at Amazon Web Services recently. We’ve done an overview of the entire suite of solutions, we’ve looked at Autoscaling in some detail, and we’ve looked at some of the lesser known business applications of AWS that you might want to consider using. But today we are going to look specifically at cloud based storage.
One of the most common services that has traditionally been delivered by an on-premise system is that of file storage. A physical server would sit in a server room somewhere in the office connected to the network and when individuals in the company wanted to save, access, or share a file it would live on that server.
However like many traditional methods of working that is no longer the way things tend to be done. With a physical server in your office, you have to pay for the whole thing, however much of it you use. With cloud based storage solutions you only pay for what you use.
It wouldn’t be one of my articles if I didn’t include an analogy, though for once I can’t come up with one that is based around food, but I hope you’ll forgive me.
The Obligatory Analogy
Imagine that you have been collecting books for years. It’s getting to the point where you can no longer fit them all in your house, but you aren’t willing to get rid of any.
You need more bookcases for them.
Using the traditional model you would have an extension built on the house. Then you’d box all the books up, take them through to the extension, and then put them out neatly on brand new bookcases. The whole extension is yours whether you are using it or not. You need to maintain it to make sure your books are safe, secure and accessible should you decide to retrieve one.
The cloud model is that you take the boxes of books to a local storage facility who will store them on their own bookcases. They charge you only for the shelf space the books take up. They will make sure the room they are in is dry, at a reasonable temperature, and secure. They also give you a key so you can get in and access them at any point should you wish to.
So now let’s put that to one side and go back to looking at your digital files and how we can use cloud to store them.
File storage is now one of the most common cloud services with a vast number of companies offering their own take on it. But it is safe to say that no-one has been doing it for longer than Amazon Web Services. However it isn’t just a single big cloud service you can upload files to and share them from.
There are many specialist storage solutions and utilities and I thought it might be interesting to look at some of them in a little more detail.
AWS originally launched with a very simple storage solution offering a few gigabytes of space in the cloud. However this has gradually evolved in the years since Amazon launched their web services. It is now an impressive suite of solutions suitable for most business requirements and scaling from the individual user to the large multinational.
The flagship product of AWS’s many storage options is the “S3” (Simple Storage Service) package. As the name would suggest it is a simple to use, high performance, low cost, fully scalable cloud storage option that is suitable for all kinds of data.
Of all the many AWS storage options, S3 certainly has the most in common with the packages AWS launched with many years ago. However it has kept up with progress and is often ahead of the curve, going so far as to innovate several new functions and facilities that are now considered essential for cloud-based storage systems.
S3 boasts an exceptional 99.99999999999% durability, and Amazon are keen to emphasise that this has been precisely calculated to eleven decimal places.
Due to its position as something of a market leader in cloud storage, Amazon have ensured that their S3 service is supported widely. They claim they are by far the most supported cloud storage service with many other software packages offering direct integration with your S3 account.
This ensures you can access your files from anywhere and make changes, additions and removals with the minimum of fuss however you run your business, or wherever you decide to work.
AWS S3 Glacier
Running alongside the standard S3 service is Amazon Web Services S3 Glacier.
This cloud-based storage solution is designed for archiving and backup systems, and with low costs, near perfect uptime, and huge storage capacities it is easy to understand why S3 Glacier has become one of the most popular archiving and backup systems on the market.
One of the most useful features of S3 Glacier is the ability to run analytics on your archived data. It can query it for certain filenames, extensions, phrases and more. For the majority of organisations a backup solution needs to be spacious, seamless and fast. However one area that many companies overlook when integrating a backup system is the speed of restoring your data.
Of course we all hope that we will never need to use our backups, but should the worst occur and you suddenly require your backed up information you will discover that productivity can be effectively eliminated whilst you wait days for a restore.
S3 Glacier boasts incredibly quick retrievals with data being returned to you in as little as a minute depending on the size of your backup.
AWS Data Transfer
It’s not just about having data in the cloud, it’s also about how you get it there. Maybe the size of the data transfer makes it impractical, or maybe the information is sensitive enough that you don’t want to risk uploading it over a standard connection. Either way Amazon supports its many storage options with the AWS Snow family of solutions which are designed to satisfy exactly those requirements…
These are designed to facilitate the transfer of large volumes of data at rapid rates, especially when dealing with amounts of data running into the petabyte range.
If you require the services of the AWS Snow family then Amazon will work directly with your organisation to provide a set of solutions tailored to your requirements. Often this will involve the use of physical storage devices and capacity points where data can be transferred directly at great speed.
Devices can be “booked” easily via the AWS Console, it is simple enough to then receive the device via express courier, load it with your data, and return it to Amazon where it will be uploaded to a secure cloud-based storage option.
To briefly revisit our earlier analogy this is the equivalent of the storage facility sending someone in a van to come and pick up your books for you so you don’t have to try and carry them yourself.
The devices that AWS Snow family utilise are tamper-proof and highly secure. You maintain ownership and management of the 256-bit encryption keys that are generated to keep your data safe throughout the transaction, and after a job has been completed all devices are erased according to NIST media sanitation guidelines.
AWS Snow Options
AWS Snow family devices start with Snowball, a small petabyte scaled box that allows your information to be directly transferred and shipped to Amazon to be uploaded to your S3 cloud. Snowball is ideal for video archiving, bulk image storage, corporate level backups, and securing analytics data for future use.
Snowball Edge is a mid-sized device with larger capacity than the standard Snowball, ideal for use in environments where internet connectivity is intermittent.
The pinnacle of the Snowball family is the AWS Snowmobile, a 45-foot long shipping container that has been converted for secure data storage. It can move up to 100 petabytes of data and as you would expect is fully temperature controlled, waterproof, fireproof, tamperproof and boasts layers upon layers of both digital and physical security.
Many companies are already committed to using Windows file systems. If this is the case for you then Amazon FSx is an ideal choice. It integrates entirely with Microsoft Windows ensuring a completely seamless experience; you can easily migrate your information and applications that require file storage to function rapidly with AWS.
It supports SMB and NTFS, ensuring it should function with 99.9% of Windows based computer systems starting from Windows Server 2008 and Windows 7.
FSx uses Solid State Drive cloud-based storage to be certain that you shouldn’t notice any lag or slowdown when saving to the cloud rather than locally. Amazon can support performance of up to 2 gigabytes per second when transferring files, far in excess of what most internet connections can utilise.
Your storage volumes can be created easily using the AWS Management Console and automatic daily backups can be scheduled making FSx an ideal solution for small or medium sized organisations.
As you would expect from Amazon, FSx is incredibly secure, your data is encrypted for transmission and also when it is idling in the cloud, and system administrators can use AWS CloudTrail to monitor any actions taken by users on Amazon FSx.
So whether you are looking for a cloud based storage solution to sit alongside your hosting architecture, a truly secure and robust data storage platform to aid with disaster recovery, or you simply want to replace your aging on-premise servers with a system that is versatile, fast, and simple to use Amazon has a storage solution that is perfect for you.
If you’d like to talk through the options that are available with absolutely no obligation why not give us a shout?