Jon Totham

SENIOR SYSTEMS ADMINISTRATOR WIREHIVE

Thoughts

AWS London Summit 2019 - Opening Address

Wednesday the 8th May saw the AWS London Summit, and our man on the ground Jon Totham was there.

Here is his summary of the opening of this amazing event.

I'm reporting on the Amazon Web Services Summit in London where the keynote speaker was Matt Garman, Vice President of Amazon Elastic Compute Cloud (EC2). He was initially joined on stage by Darren Mowry, Director of Business Developments at Amazon, along with various customers of the AWS Platform to help communicate how flexible and useful Amazon Web Services can be for all kinds of businesses. Various intriguing new developments were discussed, and some new products were officially unveiled that will help drive cloud computing forward throughout the next decade.

It is easy to understand why Amazon (and their competitors) have become so invaluable to modern businesses, be they big or small. A recent study by the London School of Economics was quoted during the opening of the summit and it concludes that the use of cloud computing has, on average, reduced time to market around the globe by up to 5000%. Even more interesting is the statement that 95% of UK start-ups that are currently in existence simply would not exist without the use of the cloud, it has allowed employers and employees alike to innovate in new ways and find much quicker routes to market their products.

Amazon started affairs by announcing continued development of the Web Services platform, of particular note was the AWS Restart programme, an innovative development that aims to introduce more people around the world to AWS by nurturing skilled and motivated individuals who are intimately familiar the platform; it is designed to allow people to get into Amazon Web Services by ensuring there is a global userbase who understand enough about the services and programs provided by Amazon to ensure continued development well into the future.

Legal_Pages_Hero-1030x589

AWS has expanded tremendously in thirteen years since it was first unveiled, and it is interesting to note that Amazon now consider the storage, compute and other aspects of their service as “basic” rather than fundamental.

It is a sign of shifting times that only a very short time ago any concept of the “cloud” primarily consisted of internet-based storage and it is fascinating to see how much more is included under the Amazon Web Services banner than just online disk space. Of course, storage is vital to everything in computing, but whilst it is essential to almost every conceivable operation it is now so easy to deploy and configure cloud based storage options with AWS that they are included as standard with almost everything you could wish to use Amazon Web Services for.

Amazon offer more options than their competitors and they are the only service that allow you to resize your storage requirements on-the-go with no downtime. It is a similar situation with databases, once considered an arcane realm that few people wanted to meddle with once they were up and running, Amazon have innovated with fourteen complete database service options for all types of users. It is worth noting that this is double what their major competitors offer, and their migration services onto the multitude of Amazon database options is second-to-none.

Amazon were also very keen to emphasise how secure all aspects of the AWS platform are, based around existing innovations that are constantly being refined and improved, along with new developments to security that are just around the corner waiting to be implemented in the next few years. When we consider the entire AWS suite of applications and solutions it is wonderful to note that over two-hundred of them are entirely compliant with government regulations for data security around the world and are entirely secure. You can choose to encrypt your data very easily at any point during many Amazon data transfers, and you now have full control over the keys used to implement your encryption.

Sainsburys_17a07-1030x773

Darren Mowry then turned the stage over to Clo Moriaty (CDO) and Phil Jordan (CIO) from the large retail chain Sainsbury’s. The organisation is a major user of the cloud via AWS, and as part of their 150th anniversary they discussed the new innovations and developments from Amazon in the past few years that have allowed them to entirely overhaul many of their digital systems.

At present they take over seventy-two million online orders, and turnover two billion in mobile sales via Argos, with 20% of the entire organisation’s transactions taking place on the internet. Infuriatingly for the business and their customers alike, until very recently, every browser and operating system would connect to an entirely different version of their website and e-commerce gateway. Now everything is consolidated into one streamlined system, and since gradually beginning to move to AWS they now have 30% less physical infrastructure, and they have seen between a 70% and 80% performance increase across all aspects of their online trading.

The vast majority of Sainsbury’s online functionality is with Amazon EC2, and Amazon have worked closely with Sainsbury’s to develop many custom solutions by using their apps to streamline Nectar card information. Until very recently Nectar cards were considered an analogue loyalty card with analytics being almost impossible to perform on the information gathered about customers. As Sainsbury’s migrated more of their information to the Amazon cloud this allowed Nectar card information to be integrated entirely with every aspect of their shopping experience. This is the first phase in what Sainsbury’s are talking about as being a huge transformation of Nectar card, however even the small changes made so far have resulted in a five-fold increase in customer engagement.

One surprising revelation was that until recently all online shopping orders that were delivered to a customer via a van utilised manual, Excel driven spreadsheets. This clearly caused something of a logistical nightmare, however efficiency has been vastly improved by migrating this information onto the cloud and vastly increasing automation. Sainsbury’s have just begun to develop their own applications on the AWS platform after utilising eighty-one different services from Amazon, and their experiences with the AWS platform have resulted in huge in cost savings, vastly improved customer experiences and the ability to run analytics on wealth of data harvested from many sources.

Darren Mowry returned to the stage to remind customers about the often-overlooked Snowball and Snowmobile services. These wonderfully named data transfer options are designed with a nod to the future, but a foot firmly planted in the here and now, and with the acknowledgement that the internet infrastructure can sometimes be lacking and that one sometimes need to resort to new takes on old fashioned methods to transfer data.

Amazon Snowball is a small, secure, physical device that you can order from direct Amazon. It is shipped to your business address, and you can rapidly upload your data to Snowball, which is then shipped back to Amazon and uploaded to the cloud on your behalf. For organisations with tremendous amounts of data there is the wonderful Snowmobile, a literal “data truck” that arrives at your locale and that can store up to 100PB of data that is then immediately driven to a high-speed Amazon data node to be uploaded.

These are innovative solutions to data transfer issues faced by organisations with lots of information in areas where the internet isn’t as quick as it should be. Amazon appear to be fully aware that whilst we are working towards a future that is 100% cloud based, the reality of the situation is that many customers utilise a hybrid model of local physical servers and cloud-based options. As well as Snowball and Snowmobile Amazon offer options like AWS Direct Connect that allows you to quickly integrate your own private networks with the cloud, and other options that allow you to integrate AWS hardware with your own data centre.

One thing we were reminded about is how well AWS integrates with Microsoft Windows, Amazon features the most comprehensive set of applications and services for Windows yet available with any cloud based provider. Amazon mentioned that they have seen Windows EC2 usage grow 400% in the last few years, and they commented that they now host twice as much Windows based workload as Microsoft itself does, a testament to the reliability and speed of the AWS platform. This is excellent news as Windows is often overlooked when we consider networking and cloud based applications, and it also serves to highlight how many smaller organisations that have only ever used Windows and are not tech-savvy enough to utilise other more niche operating systems  are joining AWS.

sea-sky-cloud-rock-monolith-1424971-pxhere_com_-1030x579

Amazon took the opportunity to address the issue of monolithic applications; obviously AWS has long been innovating by decentralising things to allow for the development and production of microservices, here they seemed to be urging other platforms, users and developers to take the same steps.

Monolithic applications are very much becoming obsolete, by having many processes running together as one large single service you will hinder your scalability, multiple developers will have issues implementing changes at the same time, and the whole operation of updates and deployments becomes much higher risk. Microservices are clearly the way forward, it is worth noting that Amazon integrated microservices into their infrastructure and phased out monolithic applications fifteen years ago and this served as a foundation of sorts upon which AWS is based.

Some new exciting announcements were made regarding the Compute platform, in the near future instances will be available with up to 24TB of RAM, adding to Amazon’s already extensive and powerful portfolio of Compute solutions. At present they offer over one-hundred and ninety instance types ranging from GPU based solutions through to CPU or storage intensive options, and even those specifically designed to deal with high I/O traffic.

The official announcement was also made for the EC2 I3en. This has very recently been released however it was celebrated to much fanfare today, it is the first NCMe SSD instance-based storage option available on AWS. It boasts up to 50% lower costs per GB compared to similar instances, with up to 100GBpbs networking bandwidth, and a full suite of Intel Xenon Scalable (Skylake) processors to choose from. The largest instance available boasts 60TB of storage space, 96 virtual CPU’s and vast amounts of memory.

Tom Read, chief DO and IO for the Ministry of Justice then took to the stage. He presented a fascinating case study of how the Ministry of Justice have begun to implement AWS solutions to fix their somewhat outdated IT systems and fully integrate their services with the cloud. Just a few years ago this large organisation with some seventy thousand employees and eight-hundred or so locations in the UK, and millions of on-line queries each day from UK residents utilised an on-site Oracle based estate.

They suffered regular service outages that denied the public access to their services, resulting in long delays and severe cost overruns and much general frustration. To compound this, they could only deploy updates to their already broken system infrequently, sometimes monthly but more usually quarterly. All their services have now migrated to the cloud via AWS after a failed attempt at moving some SPARC based systems to IAAS. This has resulted in huge cost savings in hosting, full automated patching and monitoring with no downtime, and vastly improved reliability. They are planning to continue their relationship with Amazon by moving to a Kubernetes stack and breaking down any legacy monolithic applications into microservices.

Darren Mowry returned to the stage to remind all present that Microsoft and Oracle databases, as proprietary, licensed applications, can sometimes cause more problems than they can solve. It is no surprise that many organisations are looking to switch to open-source solutions for databasing, however in the past these alternative options have offered suffered from poor performance.

Enter Amazon Aurora, a stunning cloud only database based on MySQL and PostgreSQL. It is five times faster than MySQL, three times faster than PostgreSQL, and it costs a tenth of the larger commercial licensed options currently on the market. As you would expect with an AWS database solution everything is automatically backed up to Amazon S3, and to date it has proved to be the fastest growing service in the history of AWS. It is worthy to note that with Amazon services database migration is entirely a two-way process, whilst you can easily migrate your database to AWS with ease thanks to the simple to use migration tools you can also send things in the other direction and migrate to other platforms should you wish.

Data lakes were also touched on briefly, Amazon seem keen to expand on the 10,000 or so data lakes currently deployed on S3. It is very easy to migrate all of your data from a huge variety of sources into a data lake ensuring that it is easy to audit your information appropriately, scale your requirements if you need to expand, and generate excellent analytics and insights.  The relatively new Lake Formation tool allows you to easily create your own data lake in a very short period, wherever the information currently resides Amazon Lake Formation will crawl, index, catalogue and then prepare everything for analytics of your choice.

Machine learning was a hot topic at the Amazon Summit London, with mention of how deeply integrated into our lives artificial intelligence is now becoming, being used for everything from conversational chat bots (that many people prefer to converse with than actual humans) through to being used to find human trafficking victims by comparing photographs and video footage to build up a trail of locations so that victims can be tracked down and returned safely to their families. We are living in the AI age, so it was only fitting that the last speaker was Angie Ma, CCO and founder of Faculty, an artificial intelligence company.

Faculty have worked in almost every industry sector to integrate AI into various organisations using AWS, from working with the Home Office to help use AI to stop the spread of terrorist propaganda online, through to working with the Fire Brigade to help identify commonality in hazards around the home to develop better fire safety advice.

Angie Ma was keen to point out that AI should be safe and responsible, everything should be based around fairness, robustness, and explainability, with humans being involved in many ways throughout the process and working alongside artificial intelligence. One example cited was that of car insurance companies now using machine learning and artificial intelligence to determine claim assessment results and insurance premiums for customers.

Obviously whilst AI is far more capable of crunching through the figures to generate results in a completely impartial and practical way, it is important that a human can justify precisely why a claim was awarded in a certain way or an insurance premium went up (or down!) For Faculty the explainability concept is vital to their work with AI, on average each instance of having to transfer and translate the systems results after it has ploughed through huge amounts of information takes 96 CPU’s and 393GB of RAM. It is easy to understand why Faculty are working closely with Amazon both in terms of AI applications and cloud-based hardware solutions to provide the required horsepower for their work.

This was a timely opportunity for Amazon to announce the launch of Inferentia, a custom machine learning chip available sometime this year. This new piece of hardware has been developed by Amazon specifically for machine learning and artificial intelligence applications, complete specifications are still being refined however it will certainly boast hundreds of teraflops of processing capability and will drive forward AI development in leaps and bounds.

The presentation was rounded off with some hardware announcements, firstly Amazon DeepLens, a custom designed deep learning enabled video camera for developers. It can process HD video in real time and is designed to make machine learning development more intuitive and inventive. Finally the rather quirky DeepRacer was announced, a 1/18th scale racing car designed to get people interested in machine learning. Amazon are running championships throughout the year, it is hoped that the simplicity of developing machine learning applications to enable DeepRacer to accomplish it’s simple task of staying on the track and winning the race will inspire many more to take up the challenge of working with machine learning and artificial intelligence in the future.

With seventy five billion devices set to be online by 2025 it is certainly an exciting time to be involved with AWS as Amazon continue to set trends and innovate in many exciting ways. It appears that the future is bright for Windows users on the Amazon platform and no doubt many of their competitors will follow suite soon ensuring that we gradually begin to transition to an entirely cloud based infrastructure. That said Amazon are still supporting the hybrid model of physical locations and cloud-based computing ensuring ease of entry to the cloud for those who are somewhat wary of abandoning their real-world servers immediately.

The newly released Amazon EC2 I3en instances make it easy even for those with high transaction workloads to migrate to the cloud, and the Amazon Restart program promises to bring AWS to the forefront of public consciousness in the future ensuring it continues to grow and flourish. With G4 instances available in the next few weeks, and the astounding Inferentia being released by Amazon this year we will no doubt see further integration of machine learning and artificial intelligence in all sorts of ways that will benefit our lives in the future. And those looking to migrate their databases to the cloud can do no better than Amazon Aurora, based on popular open source concepts but delivering the speed and reliability of proprietary licensed solutions it is easy to understand why it is rapidly becoming one of the most utilised facets of Amazon Web Services.