Coming Soon – AWS SDK for Go(aws.amazon.com)
aws.amazon.com
Coming Soon – AWS SDK for Go
http://aws.amazon.com/blogs/aws/coming-soon-aws-sdk-for-go/
12 comments
> AWS' first-party client libraries are generally awful
Do you have some feedback that I can pass along to the AWS SDK team?
Do you have some feedback that I can pass along to the AWS SDK team?
The node.js library [1] is incredibly well done. It's easily the best-documented node.js library I've ever seen.
[1] https://github.com/aws/aws-sdk-js
[1] https://github.com/aws/aws-sdk-js
Sure, but going from memory on this :) As far as the Ruby SDK, my opinions are mostly based on how things were 2-3 years ago, when I wrote it off entirely after inheriting several large projects that made heavy use of it.
In the past, it was difficult (impossible?) to figure out how it worked by reading the code. That's highly unusual for a Ruby library. It felt like a generated library, not like Ruby. When something wasn't working as expected (which happened often), it was difficult to figure out what was going wrong and why.
As a heavy AWS user, I've felt much more comfortable using Fog, even though I only really use its AWS provider. That's true even though I didn't even get to take advantage of Fog's multi-provider abstractions (basically just using it to make raw AWS API calls). Hopefully that's all in the past and I should give it another look the next time I'm doing that stuff in Ruby. Fog also gave much better control over a lot of the details that are important when doing stuff in production, like timeouts and keep-alive settings.
The 2nd example that comes to mind is the AWS Flow framework for Ruby: https://github.com/aws/aws-flow-ruby
Flow does things like create global methods ( https://github.com/aws/aws-flow-ruby/blob/master/aws-flow/li... ) and its DSL abstraction just doesn't feel right. It tries too hard to make it feel like you're just running Ruby code, and it's tough to determine how it uses SWF under the hood. It's such a large abstraction on top of SWF that it basically means you can't write parts of your app in anything other than the Flow frameworks for Java or Ruby. But then, the other major SWF framework I've seen (https://github.com/sclasen/swf4go) also locks you in. So maybe it's just a shortcoming of SWF that it's not usable without a big abstraction layered on top of it?
From what others are saying, it sounds like the newer stuff is in much better shape. That's very encouraging!
Regardless, I think you've made a great decision in taking over aws-go. I have very high hopes for its future :)
In the past, it was difficult (impossible?) to figure out how it worked by reading the code. That's highly unusual for a Ruby library. It felt like a generated library, not like Ruby. When something wasn't working as expected (which happened often), it was difficult to figure out what was going wrong and why.
As a heavy AWS user, I've felt much more comfortable using Fog, even though I only really use its AWS provider. That's true even though I didn't even get to take advantage of Fog's multi-provider abstractions (basically just using it to make raw AWS API calls). Hopefully that's all in the past and I should give it another look the next time I'm doing that stuff in Ruby. Fog also gave much better control over a lot of the details that are important when doing stuff in production, like timeouts and keep-alive settings.
The 2nd example that comes to mind is the AWS Flow framework for Ruby: https://github.com/aws/aws-flow-ruby
Flow does things like create global methods ( https://github.com/aws/aws-flow-ruby/blob/master/aws-flow/li... ) and its DSL abstraction just doesn't feel right. It tries too hard to make it feel like you're just running Ruby code, and it's tough to determine how it uses SWF under the hood. It's such a large abstraction on top of SWF that it basically means you can't write parts of your app in anything other than the Flow frameworks for Java or Ruby. But then, the other major SWF framework I've seen (https://github.com/sclasen/swf4go) also locks you in. So maybe it's just a shortcoming of SWF that it's not usable without a big abstraction layered on top of it?
From what others are saying, it sounds like the newer stuff is in much better shape. That's very encouraging!
Regardless, I think you've made a great decision in taking over aws-go. I have very high hopes for its future :)
Thanks, this is incredibly helpful. I will pass it along to the team in the next 45 seconds.
How well do you know the SNS team? :) https://twitter.com/blakegentry/status/539873715446878209
I believe you need to enable "Raw Message Delivery" in the subscription attributes for that subscription for it to deliver raw JSON content instead of wrapping your message in an envelope like that.
Jeff is the chief evangelist at AWS & all around good guy over there. I'm sure he'd be able to get in touch with any team you wanted to provide feedback on!
The libraries are good.
The primary thing needed for ongoing success is relentless commitment to 100% coverage of the AWS feature set, with new AWS features supported when they become available, not "at some later date". Nothing worse than reading about some new (or old) AWS feature that effectively does not exist because it is not supported in the client libraries.
The primary thing needed for ongoing success is relentless commitment to 100% coverage of the AWS feature set, with new AWS features supported when they become available, not "at some later date". Nothing worse than reading about some new (or old) AWS feature that effectively does not exist because it is not supported in the client libraries.
I like the first party libraries personally, because I find them to be very full featured.
> AWS' first-party client libraries are generally awful
Not a reasonable comment.
Not a reasonable comment.
I will say that the new AWS ruby SDK is allright. It's a bit unruby-like in some places, but it's got absolutely stellar documentation, and is very complete, which is far more than I can say for a lot of projects.
Agreed. I'll take outstanding documentation over stylistic perfection any day.
You certainly may be commenting on outdated notions. I have used the .Net and the Ruby SDK's exetensively and have been satisfied with both(other than .Net being a bit behind Java with the KMS stuff).
I'm only familiar with their Ruby and Node libraries, but I enjoy both of them very much.
There exists a third party library called goamz which has been forked all over the place when people have needed to add random features not found in the original version. It works well but hopefully you don't need a set of features that require you to import multiple versions.
Hopefully Amazon's version will be comprehensive and idiomatic.
Hopefully Amazon's version will be comprehensive and idiomatic.
True. The "big" players that forked it couldn't be explicatived to work together so the project became seriously fractured. I'm hoping this will move swiftly through experimental to release, and that the generated nature doesn't produce yucky to use implementations.
the problem with goamz, when i forked it, was that merging changes into canonical's bazaar cvs was very hard to do. that was before github and pull requests became de-rigeur, so i'm a bit dated. still, very few of canonical's go projects are bzr-based anymore.
that's the reason you're seeing all those forks: we all added what we needed, couldn't get it in, and gave up.
that's the reason you're seeing all those forks: we all added what we needed, couldn't get it in, and gave up.
If this has better coverage of AWS features than their goamz fork, I hope Hashicorp will switch to it in Terraform. I'd love to use Terraform but too much of what I use Cloudformation for isn't there.
https://github.com/hashicorp/terraform/issues/28
https://github.com/hashicorp/terraform/issues/28
I'd love to see their CLI tool converted to Go - not because I have anything against Python, but because installing Go binaries is so easy.
(Would there need to be any dependencies to install?)
(Would there need to be any dependencies to install?)
Good point, I look forward to seeing how this fits together. It would definitely add to the value of AWS for me.
If you're just installing a binary, there shouldn't be.
This is great! I can now unify a lot of my workflow and not have to use ruby as a glue layer between applications and operations. It's great to see the Go ecosystem improving! I think in a few years, it may become the defacto tool of systems programmers.
Frankly go isn't my favorite glue language (and I'm one of the relatively few people who's been doing go full time for a couple years). I currently prefer python for that kind of stuff, or sometimes bash.
However it's certainly possible to use as a glue language. I've talked to some folks at DO who just install go on machines and `go run` scripts.
However it's certainly possible to use as a glue language. I've talked to some folks at DO who just install go on machines and `go run` scripts.
For small internal projects, I typically find that I end up writing the program and compiling it; It just builds up my developer toolchain from the command line to do my bidding. Also, these are all managed under source control, so my co-workers can also take advantage.
I'm going to go out on a limb and say that probably isn't true. All go at DigitalOcean is deployed using one of two mechanisms. Nothing is `go run` outside of a developer's workstation.
It's not going to be displacing low-level programming any time soon, unless radical changes are in store for 2.x and beyond, hypothetically.
That said, I can see it becoming a standard for daemons, utilities and low-level userspace in general due to its rather excellent POSIX and syscall support. I really find the (present) lack of this to be Rust's main deficiency, though the third-party crates are slowly picking up with bindings for things like prctl(), epoll() and so on.
EDIT: Why the downvotes? Rob Pike has stated that he regrets attaching the "system programming" label to Go, since he never intended it for OS kernels and the like. If it's about me being wrong on Rust's POSIX bindings or Go's suitability for daemons, do prove me wrong.
That said, I can see it becoming a standard for daemons, utilities and low-level userspace in general due to its rather excellent POSIX and syscall support. I really find the (present) lack of this to be Rust's main deficiency, though the third-party crates are slowly picking up with bindings for things like prctl(), epoll() and so on.
EDIT: Why the downvotes? Rob Pike has stated that he regrets attaching the "system programming" label to Go, since he never intended it for OS kernels and the like. If it's about me being wrong on Rust's POSIX bindings or Go's suitability for daemons, do prove me wrong.
Once upon a time that was a programming language called Cedar that was used at Xerox PARC to build an environment for strong typed languages similar to what Smalltalk and Interlisp-D environments offered. It used reference counting with GC for collecting cycles.
A certain swiss guy decided to create a graphical workstation based on the NS32000 processor, using a revised version of Modula-2, with a GC for systems programming, named Oberon.
The Oberon OS managed to provide a fully working graphics workstation capable of fulfilling many tasks at the ETHZ systems programming department. Specially the latest version, System 3 with the GUI Gadgets framework.
It had as descendents EthOS, BlueBottle AOS and its design influenced Plan9's ACME and Rio.
But the world was busy looking into UNIX being adopted by the industry and Oberon is no more.
A certain swiss guy decided to create a graphical workstation based on the NS32000 processor, using a revised version of Modula-2, with a GC for systems programming, named Oberon.
The Oberon OS managed to provide a fully working graphics workstation capable of fulfilling many tasks at the ETHZ systems programming department. Specially the latest version, System 3 with the GUI Gadgets framework.
It had as descendents EthOS, BlueBottle AOS and its design influenced Plan9's ACME and Rio.
But the world was busy looking into UNIX being adopted by the industry and Oberon is no more.
As I noted regarding Oberon, but you'll have a hard time trying to break the cultural bias against GC.
I learned long time ago that you just need to wait a few programmer generations, than the bias against whatever technology is solved by virtue of nature.
When I started programming, the bias was against any language higher level than Assembly for home computers.
When I started programming, the bias was against any language higher level than Assembly for home computers.
I believe this is why you are getting downvoted:
> It's not going to be displacing low-level programming any time soon, unless radical changes are in store for 2.x and beyond, hypothetically
What radical changes are you referring to, and why are they needed in order for Go to be a de-facto language for system programmers?
> It's not going to be displacing low-level programming any time soon, unless radical changes are in store for 2.x and beyond, hypothetically
What radical changes are you referring to, and why are they needed in order for Go to be a de-facto language for system programmers?
Drop the garbage collector. Systems programming cannot stop the world.
You can already do that: http://golang.org/pkg/runtime/
That's not useful for a kernel. You can disable garbage collection in go, but go doesn't provide the tools to disable creating garbage entirely. You can get really close to creating zero garbage through profiling and helping the compiler's escape analysis, but afaik the go spec makes no guarantees about stack vs heap allocation.
> That's not useful for a kernel.
Oberon Kernel.Mod, initial version 11.4.86
http://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources/...
Oberon Kernel.Mod, initial version 11.4.86
http://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources/...
I failed to see any references to the go runtime package in that link which leads me to believe you misunderstood my statement. Using the APIs documented in the go runtime package to disable garbage collection would not be an effective memory management strategy for a kernel.
The GC is more of a cultural issue than anything (what with Oberon being perfectly fine), but it will have an impact as a result. The runtime will be an overhead, though it does have a malleable library interface, apparently.
Wow that moved quick, I remember reading about this library a month or so back.
Glad to hear Amazon have formally adopted it as canonical!
Glad to hear Amazon have formally adopted it as canonical!
I maintain https://github.com/AdRoll/goamz FKA https://github.com/crowdmob/goamz , if anyone has any questions.
aws golang sdk's have been around for a while and I've found several other which are well tested and maintained such as https://github.com/goamz/goamz. It's awesome that aws finally decide which library they're officially adopting but I found frustrating that the library is poorly tested and it will take while until it becomes the official. It'd be great if all the other sdk's contributors start working on the same project.
The goamz project is an embarrassing mess, so many forks.
But anyone can fork any project on github willy-nilly, for whatever reason they want, it's not something that's really under the control of original project.
This is great news indeed. Why could this not have come 8 months ago? Back then, it was quite disheartening to see the replies from AWS team about Go support.
Recently I have heard so many good stories about Go although my impression of Go gradually decreases due to unstable development tools and unfriendly debugging tools. FYI, I use IntelliJ to write Go.
Sublime Text + GoSublime plugin is my favorite setup.
Sublime Text + GoSublime was my first try but then I kept getting //GsDoc: no docs found. At that time, my solution was using Sublime Text + GoSublime + Godef. I still could not find a good debugging tool though.
Godef: https://github.com/buaazp/Godef
Others also have this issue with GoSublime http://stackoverflow.com/questions/21832288/gosublime-go-to-...
Godef: https://github.com/buaazp/Godef
Others also have this issue with GoSublime http://stackoverflow.com/questions/21832288/gosublime-go-to-...
I used that. These days, I use Atom + Go-Plus.
I used to use IntelliJ with the Go plugin but I've recently switched to LiteIDE and I find it a significant improvement.
This is very promising for the Go future as more mature packages will be available from big companies
How about a Rust SDK for AWS?
Also, the current situation with goamz is truly horrific. There are multiple independent forks of the original Canonical Launchpad-hosted version, each with their own subtle differences in usage, interfaces and feature support. None of them feel like idiomatic Go.
It really would be unlikely for this now AWS-supported project to end up doing worse than the status quo.
[1] mainly Coda Hale from Stripe, where the project originated: https://github.com/stripe/aws-go