The Ultimate Guide to Serverless Announcements @ AWS re:Invent 2017

Written by Alex DeBrieEdit this post

Last Updated: 11/30/17 @ 11:18AM PST

It's that time of the year, the Christmas-comes-early for cloud developers. Are you ready for AWS re:Invent?

This is where AWS Lambda was introduced in 2014, and we're expecting more huge announcements in the serverless realm this year. AWS has gone all-in on serverless, and we can't wait to see the improvements they're making.

This post is the place to understand what is announced and why it affects you, the intrepid Serverless developer. We'll be updating throughout the week with all of the latest announcements.

If you're attending re:Invent, be sure to check out the Serverless guide to re:Invent 2017.

#Overview:

#re:Invent announcements (newest to oldest):

#Pre-re:Invent announcements:

#Announcements:

#CloudTrail Logging for Lambda Invocations

What: Ability to send CloudTrail events for all Lambda function invocations

Why this matters: CloudTrail is a great tool for performing security auditing and compliance for your AWS resources. Previously, you could only create CloudTrail events for more management-like features of Lambda, such as creating or deleting a function. Now, you can get information on function invocations as well. This will enable you to record and react to invocations with much more detail.

#Golang support

What: You can now write your Lambda functions in Go.

Why it matters: Like concurrency but hate Node? A typed language without Java? Boom - Golang arrives on Lambda.

Note: This is a pre-release and won't be available for a few weeks yet. Patience you must have, my young Padawan.

#3GB Memory

What: Lambda memory limits upped to 3GB.

Why it matters: More RAM! This can help for heavier processing tasks or anything that would benefit from more memory. As Vlad points out, this also seems to apply to Lambda@Edge:

#Concurrency controls

What: Lets you manage invocation concurrency on a per-function basis.

Why it matters: AWS has account-wide limits on how many concurrent Lambdas you can have running—1000 by default, though you can raise that. It could lead to some of your functions getting throttled if one of them got hammered with a bunch of requests. But now, you can set limits so that one function doesn't result in throttles for other functions.

#VPC integration

What: VPC integration with API Gateway.

Why it matters: We'll let Ajay describe why this is important, we couldn't have said it better:

#Serverless App Repo

What: A repository for discovering serverless applications. Preview available here.

#Cloud9

What: IDE for writing, running, and debugging code. This is GA today!

Why it matters: A developer's editor is a sacred thing—from terminal junkies to full-fledged IDEs. For those less opinionated or that want to be less tethered to a particular development machine, the Cloud9 IDE is interesting. Develop directly in the cloud with an always-available environment. Pair programming is interesting, and something we've seen from VSCode and Atom in the last few weeks as well.

#AWS Greengrass

What: Lets you run machine learning at the edge.

Why it matters: It makes it easy to use your machine learning models at the edge, even without internet connectivity. You can do all your training in the cloud (maybe with SageMaker!), then push the model to your edge devices for inference.

#IoT Device Defender

What: Define and audit security policies across fleets of devices. Coming early 2018.

#IoT Device Management

What: Improves security of IoT implementations. Lets you remotely onboard and manage new IoT devices at scale.

#IoT 1-Click

What: Lets you set up a device to trigger a Lambda with a click.

#Amazon Transcribe

What: Speech to text

#Amazon Translate

What: Real-time language translation

#Amazon Comprehend

What: Natural language processing

#Rekognition Video

What: Vision analysis on video streams

#Kinesis Video Streams

What: Lets you ingest and store video streams.

#SageMaker

What: Service that assist with the heavy-lifting of machine learning. Helps you author, train, and host your models.

#AWS Neptune

What: Fully-managed Graph database. Good for graph relations such as social networks, recommendations, routes, etc. Available in preview.

Notes: Probably not serverless-friendly to begin with (e.g. not auto-scaling or pay-per-usage), but it probably won't be too long.

Want in? Sign up for the preview here.

#DynamoDB backup and restore

What: Much easier restore operations for DynamoDB. Includes the ability to backup to any second in the last 35 days!

#DynamoDB global tables

What: Multi-region, multi-master tables for having your tables everywhere. For more on why this is important, might want to refresh your memory on Jared Short's ServerlessConf talk.

#Serverless Aurora

What: Aurora is AWS's relational database in the cloud. It's cheaper and faster than MySQL or PostgreSQL databases. Serverless Aurora brings automatic scaling and per-second billing. MySQL available early 2018 and PostgreSQL later in 2018.

Why this matters: If done right, this is a dream come true. The data layer is an unsolved problem in the Serverless realm. If you want to use a traditional relational database, you're configuring a lot of network rules and VPC configuration -- just what you want to avoid with serverless architectures. You can avoid this pain by using DynamoDB, but the limited query patterns & hidden foot-guns can cause problems down the road.

Serverless Aurora could change all of that—a relational database that's accessible over HTTP, doesn't require complicated networking configuration, and uses IAM for authentication? Count me in.

#AWS Fargate

What: Run your containers directly without a cluster. Just pay for your compute resources.

Why this matters: This drastically lowers the barrier for running containers. You don't need to set up and maintain a cluster for deploying your containers. It's not serverless -- you're still paying for resources even when they're not actively serving requests -- but it does have some of the similar benefits of serverless architectures.

#AWS EKS

What: Managed Kubernetes on AWS.

Why this matters: AWS will run your Kubernetes for you. It's not a serverless solution by any means, but many companies are interested in containers over straight serverless. For running functions-as-a-service on Kubernetes, check out the Kubeless integration with the Serverless Framework.

#AWS AppSync

What: A platform for building data-rich apps with offline functionality.

Why this matters: This is an interesting, ambitious offering from AWS. Basically, it lets you set up a managed GraphQL endpoint over a variety of data sources. This endpoint can proxy to DynamoDB, ElasticSearch, or your custom Lambda functions. Further, it provides some nice functionality to keep your device synced when moving between online and offline modes.

I'd compare this to an AWS version of Firebase or Realm with the ability to have multiple different backing data sources. That's pretty powerful. One of the drawbacks to Firebase stems from the limitations of its data model. This offering sidesteps those issues.

The product page is live at https://aws.amazon.com/appsync/ and you can find docs here.

*credit: AWSreInvent main account.

#Weighted aliases for Lambda

What: This will let you send some percentage of traffic to one version of a Lambda, the rest to another version. This is GA today, try it out!

Why this matters: This will make it easier to confidently push new changes to production. When you push new versions, you can shift a small percentage of users to the new version and monitor for errors, performance metrics, etc. If you're happy with the results, you can gradually ramp up traffic so that all users see the new version.

#AWS CodeDeploy incremental deployment

What: CodeDeploy support for incremental deployment of serverless applications. This is GA today, try it out!

Why this matters: This update fits well with the addition of weighted aliases for Lambda. In your CodeDeploy configuration, you can use phased rollouts of your applications. For example, you could have CodeDeploy make a deploy to 25% of your production traffic at first, then roll it out to the remaining 75% 45 minutes later. If you discover problems during that 45 minutes, you can rollback the deploy to limit problems to your users.

*credit: @GenuineM7

#Canary management for API Gateway

What: This will allow you to send some percentage of API Gateway traffic to one source and the rest to another. This is GA today, try it out!

Why this matters: Same benefits as the weighted aliases for Lambda above -- more fine-grained rollouts of new code to production. This change is at the API Gateway level, rather than for an individual Lambda function. Canary deployments are a way to safely roll out new changes to customers.

#Lambda at Edge Improvements

What: Lambda@Edge increased memory limits, maximum package size, and function durations. It also allows for dynamic origin selection based on content and the ability to make remote calls in viewer-facing requests.

Why this matters: This is a big one. Previously, Lambda@Edge functions allowed you to run limited logic at the edge, such as rewriting headers or redirecting unauthenticated users to a login page. However, the functionality was limited, particularly if you wanted to integrate with other services in your architecture.

Now, you can run entire applications at the edge. You can make remote calls to your other services to get dynamic content. You can route requests to different origins based on the request path, making it easier to slowly migrate to Serverless architectures using the strangler pattern. This is a huge deal. It's mind-blowing that this came out before re:Invent—they must have some other amazing announcements in store.

#SNS Message Filtering

What: SNS Subscriptions can add a filter policy where they only receive certain messages rather than all messages published to a topic.

Why this matters: This announcement may be underrated, but it makes it much easier to build pub/sub architectures. Previously, you might make a "fat" topic with all messages published to it and required annoying filtering logic within your Lambda functions that subscribed to a topic. This would result in wasted Lambda invocations to SNS messages that your function didn't care about. Alternatively, you could create multiple, smaller topics with specific messages types, but that required the complexity of multiple subscriptions for different Lambdas.

With this new filter policy, you can use the fat topic pattern while only invoking your function for messages it cares about. This could be based on an event_type (e.g., I care about order_placed but not order_shipped) or other attributes (if a new User is created, trigger me when the user_type is admin). This can simplify your Lambda logic and lower your costs.

#API Gateway Access Logs

What: You can now enable detailed access logs from API Gateway, just like you could do with Apache, Nginx, or HAProxy.

Why this matters: This enables granular analytics on your web requests. You can feed these into your analytics systems or dump them into S3 to analyze with Athena.

About Alex DeBrie

Alex is a data engineer at Serverless. He is an ex-lawyer who loves Python, basketball, and his family.

Serverless Blog

The blog on serverless & event-driven compute

New to serverless?

To get started, pop open your terminal & run

npm install serverless -g

how? learn more

Subscribe

Join 12,000+ other serverless devs & keep up to speed on the latest serverless trends

Comments