Here is a summary of our impressions of this year’s largest AWS event in Las Vegas. In particular we try to provide a perspective for the Atlassian/Jira/Mobile space and ecosystem. Re:Invent has grown rapidly and counted over 50,000 attendees this year. Keynotes, sessions, were spread out across several hotels on the Strip. Events filled up quickly and navigation between the different venues was challenging. These are the few trends that we observed during the conference:
Using Relational Databases for Everything is a Thing of the Past
To license a Relational Database (RDBMS) and then used it for any scenario seems to be less and less common. Many organizations favor using specialized data stores for specific use cases. For example, Amazon announced a timeseries database that specifically performs well for data structured as timestamps and attributes. Why would anyone use this over a flat relational table? Because it scales much better and provides better performance for large data sets. Due to the offer as a managed service, not much in-house knowledge is required. In addition Amazon announced scale-out Aurora (based on PostgreSQL or MySQL) and Quantum ledger databases, and additional features for Dynamo DB. Also Dynamo DB has become a choice for use cases where previously RDBMs were used. We have been using Dynamo DB for our cloud offering for several years, it is a very solid product for simple key/value data structures. For example our Jira Cloud Google Sheets Integration and Mobility for Jira Mobile solution are built on Dynamo DB. It is all about having the right tool for the right job. For Atlassian Server or Datacenter users, hosting Jira with AWS has become a true alternative to on-prem deployments and Jira Cloud. As a database backend you have a variety of choices, RDS based on Postgres or MySQL being the most cost effective ones. With the newly announced RDS features, Amazon claims a performance benefit between 3-5x compared to Postgres and MySQL respectively. Aurora offers similar performance as commercial databases but at 10th of the cost. If you are running your Atlassian stack on a commercial or open-source database, there may be some optimization potential.
Serverless Becomes More Mature
Server maintenance is time consuming, expensive, and requires the right skills. Amazon has several offerings like Lambda, serverless databases, other managed serverless services such as Amazon Managed Streaming for Kafka to spare a client from having to maintain a fleet of EC2 servers. Many announcements at re:Invent were around serverless computing to make it more mature and remove some constraints. Lambda functions are cloud native, event-driven, and based on RESTful microservices. Advanced organizations recommend to use serverless first, if not available use containers. Virtual Machines are considered legacy by these organizations. Serverless brings operational benefits and lower costs. As an Atlassian user you will not be able to use these services directly, however, if you rely on RESTful services to call your Jira instance or to update some external systems, this is obviously a choice for you.
Multi-cloud Strategy is a Myth
It makes sense to not lock yourself into one cloud vendor and keep your infrastructure cloud-agnostic. However, it turns out that the effort to achieve this is large and builds on the least common denominator. This means you will give a up a lot of the advantages in particular for managed services. Only large organizations or ones that have a strategic reason (for example the retail industry) not to go with another vendor will truly benefit from this. Amazon remains the leader of the pack with about half the market share. Although Microsoft Azure seems to be growing quickly, they are a distant second and they growth is dwarfed by Amazons.
All-Purpose Storage is Ubiquitous
S3 is a very fundamental building block of the AWS product offering and infrastructure. It is used for many different things. It is beautifully simple, powerful, and cost effective. Many third party services have been built on top of S3. For example, Snowflake built a high-performance elastic data warehouse on top of S3 that according to some sources even out performs Amazon’s Redshift MPP Cloud Datawarehouse. In addition, Amazon announced Deep Glacier, which will provide even lower cost for storage that is not accessed frequently. Data lakes are on the rise and Amazon claims that 10,000 Data Lakes are currently running on S3 so far. However, the difficulty of setting it up properly and managing security is significant. Therefore, Amazon introduced Data Lake Formation which allows template-based creation and management of data lakes. If your Atlassian data needs to be accessed by different systems or you are running some analytics on it this might be a beneficial approach for you.
Although Atlassian was present at re:Invent as well, their presence was mostly to promote their DevOps offerings of Jira Service Desk and the newly acquired OpsGenie. However, Atlassian Cloud products run on AWS and they were mentioned several times during keynotes so Atlassian seems to be an important AWS reference client. Assuming you are running Jira Server or Datacenter in the cloud, you may benefit from scale out Aurora and other services as well. If you are running Jira on VMware you have now options to expand this to the cloud with the collaboration between VMware and AWS, which may benefit your DR/BCP strategy. More practically there are cloud formation templates that builds Jira datacenter on AWS. This will speed up a potential deployment. Also if you have a need to load test Jira but don’t have the infrastructure in house, you can use use JPT which is built on AWS and fires up an EC2 based Jira Datacenter instance. AWS is at the forefront of cloud computing and its speed and release of new services and products is impressive.