This is awesome. If you're deploying stuff with Python on Lambda, I highly recommend checking out Chalice (https://github.com/awslabs/chalice) and also an extension to it that I wrote for using Lambdas on SNS/S3/Cloudwatch events or scheduled events - https://github.com/kislyuk/domovoi (which I now need to go update to support this as well).
AWS now has Sam which I hadn't heard of before. It's an extension of cloud formation specifically for deploying serverless code to lambda, building API gateways etc. Their command line tool is pretty slick, I'm wondering if these other third party frameworks are needed anymore.
No CloudFormation support for tagging and embedding python3.6 code in the templates - they really do their best to push us away from using CloudFormation!
It was strange that it ever supported Python 2 given how recent the AWS Lambda Python product is - they could have avoided the support burden of Python 2 entirely.
Maybe tangential but how do people feel about implementing a typical RESTful database API in Lambda & API Gateway?
Most of the prototypical examples of Lambda I see are for things like data processing pipelines. I know in theory Lambda should be able to handle just about any kind of request from a browser or mobile app short of a websocket connection, and Amazon does have some sample code and a brief case study on their site. But I'm wondering if it's really ready for this or if people have experiences going near-100% serverless for their apps.
Lambda excels at taking in arbitrary amounts of long-running jobs and feeding you its output. For example: Upload a png image to convert it to jpg. Zip a directory of S3 objects. Etc.
Lambda gets very costly and inconvenient when you're just taking in requests you could handle by a couple of load-balanced web servers. The whole "running a whole website on Lambda" craze does not actually yield any benefit and is more complex, harder to play with than a simple ec2 instance (which, with a good setup, needs very little "server" management at all).
Also, API Gateway is just horribly inflexible imho.
We did this at CloudSploit. Our entire service is written using Lambda-based APIs behind the API Gateway. The biggest challenge at first was the cold boot times. If our API went un-pinged for more than five minutes the next request would take > 5 seconds (!). This was solved with a CloudWatch event that invokes the function every 4 seconds (which is silly in my mind, but it works).
I've built a complete web application that is entirely serverless except for the database - I have an instance running Postgres. This is because Postgres I the only database I want to use because it always meets my needs.
Everything else is serverless however.
My application has:
-- AWS Cognito as the user management and auth
-- AWS Cloudfront serving a Reactjs front end as static files
-- AWS EC2 running Postgres as the database
-- AWS Lambda Python functions for the back end
-- AWS SES serverless email inbound and outbound processing
-- AWS SQS for email processing coordination
Oh I do in fact have another server which does spam checking of emails, PDF conversion of emails, text extraction of emails and parsing of emails - this works best on a server rather than serverless.
The most interesting thing I found along the way is that the API gateway (which I liked a whole lot by the way, and found to be powerful and easy to configure) is completely unnecessary. My application simply directly calls the Lambda functions to get and set the data that it needs - dramatic simplification.
Of further note is that I only have one Lambda function for the entire back end - this further reduces the need for layers of APIs and parameters. All my Python code goes into the one function which is structured as a complete application. An additional benefit of this is that AWS Lambda can be slow to first run a function unless it is "warm". If you have lots and lots of small Lambda functions then any given function is less likely to be warm. With everything in the one Lambda function, then all parts of my Lambda function are more likely to be warm.
So no API gateway completely rips out an entire layer of complexity, and then only one single Lambda function rips out another layer of complexity. It's very nice to be able to write front end functions in ReactJS that just call the back end function that they need. Making changes means I just change the functions and don't need to fiddle with all sorts of REST API layers or anything to accommodate the change.
I started with node.js as the Lambda back end but switched to Python because personally I find that Python with its synchronous programming model is much easier to reason about for back end stuff. I'm more than happy using ES2015 at the front end for ReactJS.
If you go to www.bustle.com or www.romper.com you get a JS app server side rendered by AWS Lambda and delivered through API Gateway (CDN in front of that). We do 50+ million unique visitors a month. Our team is super happy with the result and our AWS bill is smaller than it was on EC2. Cost was not a motivating factor for the switch but it is nice when the bill goes down.
I would be very interested in the answer to this question. If it can surpass Flask in terms of simplicity in creating a simple REST API it would be a major milestone.
Uh, the API Gateway can't handle binary requests/responses very well and it certainly cannot handle multipart requests/responses. Additionally it has a 30 second timeout so I would say it's definitely not something you could just replace your hosted web server with completely yet.
I'm running/wrote a flask-restfull app on lambda via Serverless with the wsgi plugin behind API Gateway. Not a huge fan of Python, but it's all working well enough.
Create a new alias with a unique ID for each lambda deployment. Then manage your application to point at the proper alias. Roll out just becomes updating the apps however you update them normally.
[+] [-] ak217|9 years ago|reply
[+] [-] Mizza|9 years ago|reply
https://github.com/Miserlou/Zappa
Comparison: https://blog.zappa.io/posts/comparison-zappa-verus-chalice
It also lets you build fully-fledged event-driven apps with a single line of code: https://blog.zappa.io/posts/zappa-introduces-seamless-asynch...
[+] [-] revicon|9 years ago|reply
[+] [-] jamiesonbecker|9 years ago|reply
1. https://blog.zappa.io/posts/comparison-zappa-verus-chalice
2. https://www.reddit.com/r/Python/comments/4hebys/cost_analysi...
3. https://news.ycombinator.com/item?id=14075634
[+] [-] scrollaway|9 years ago|reply
... I know what I'm doing tomorrow.
Edit: I know what I'm doing today.
[+] [-] abhirag|9 years ago|reply
[+] [-] Mizza|9 years ago|reply
Follow the progress here: https://github.com/Miserlou/Zappa/issues/793
[+] [-] ak217|9 years ago|reply
[+] [-] INTPenis|9 years ago|reply
Edit: Just noticed in my AWS account that there's 3.6 in Europe. This will get me away from GCP.
[+] [-] StreamBright|9 years ago|reply
[+] [-] wging|9 years ago|reply
[+] [-] nikolay|9 years ago|reply
[+] [-] braidenj|9 years ago|reply
[+] [-] svdgraaf|9 years ago|reply
[+] [-] movedx|9 years ago|reply
[+] [-] andrewstuart|9 years ago|reply
[+] [-] m23khan|9 years ago|reply
[+] [-] abalone|9 years ago|reply
Most of the prototypical examples of Lambda I see are for things like data processing pipelines. I know in theory Lambda should be able to handle just about any kind of request from a browser or mobile app short of a websocket connection, and Amazon does have some sample code and a brief case study on their site. But I'm wondering if it's really ready for this or if people have experiences going near-100% serverless for their apps.
[+] [-] scrollaway|9 years ago|reply
Lambda excels at taking in arbitrary amounts of long-running jobs and feeding you its output. For example: Upload a png image to convert it to jpg. Zip a directory of S3 objects. Etc.
Lambda gets very costly and inconvenient when you're just taking in requests you could handle by a couple of load-balanced web servers. The whole "running a whole website on Lambda" craze does not actually yield any benefit and is more complex, harder to play with than a simple ec2 instance (which, with a good setup, needs very little "server" management at all).
Also, API Gateway is just horribly inflexible imho.
[+] [-] cddotdotslash|9 years ago|reply
You can read more about our process here: https://blog.cloudsploit.com/we-made-the-whole-company-serve...
[+] [-] andrewstuart|9 years ago|reply
Everything else is serverless however.
My application has:
-- AWS Cognito as the user management and auth
-- AWS Cloudfront serving a Reactjs front end as static files
-- AWS EC2 running Postgres as the database
-- AWS Lambda Python functions for the back end
-- AWS SES serverless email inbound and outbound processing
-- AWS SQS for email processing coordination
Oh I do in fact have another server which does spam checking of emails, PDF conversion of emails, text extraction of emails and parsing of emails - this works best on a server rather than serverless.
The most interesting thing I found along the way is that the API gateway (which I liked a whole lot by the way, and found to be powerful and easy to configure) is completely unnecessary. My application simply directly calls the Lambda functions to get and set the data that it needs - dramatic simplification.
Of further note is that I only have one Lambda function for the entire back end - this further reduces the need for layers of APIs and parameters. All my Python code goes into the one function which is structured as a complete application. An additional benefit of this is that AWS Lambda can be slow to first run a function unless it is "warm". If you have lots and lots of small Lambda functions then any given function is less likely to be warm. With everything in the one Lambda function, then all parts of my Lambda function are more likely to be warm.
So no API gateway completely rips out an entire layer of complexity, and then only one single Lambda function rips out another layer of complexity. It's very nice to be able to write front end functions in ReactJS that just call the back end function that they need. Making changes means I just change the functions and don't need to fiddle with all sorts of REST API layers or anything to accommodate the change.
I started with node.js as the Lambda back end but switched to Python because personally I find that Python with its synchronous programming model is much easier to reason about for back end stuff. I'm more than happy using ES2015 at the front end for ReactJS.
[+] [-] southpolesteve|9 years ago|reply
It is most definitely ready. I gave a talk at Node Interactive last year that has some more detail https://www.youtube.com/watch?v=c4rvh_Iq6LE&index=2&list=PLf...
[+] [-] vidar|9 years ago|reply
[+] [-] idbehold|9 years ago|reply
[+] [-] Rapzid|9 years ago|reply
[+] [-] braidenj|9 years ago|reply
[+] [-] jgill|9 years ago|reply
[+] [-] kinghajj|9 years ago|reply
[+] [-] IanCal|9 years ago|reply
[+] [-] Rapzid|9 years ago|reply
[+] [-] wahnfrieden|9 years ago|reply
[+] [-] mason55|9 years ago|reply
[+] [-] QuinnyPig|9 years ago|reply
[+] [-] sctb|9 years ago|reply