I looked for something like this for years, and could never find it, so I ended up writing it myself: take a look at Fundamentals of DevOps and Software Delivery [1]. It's a hands-on, step-by-step guide to all the tools and techniques for deploying and managing software in production, including hosting (cloud, on-prem, IaaS, PaaS), infrastructure as code (IaC), application orchestration (VMs, containers, serverless), version control, build systems, continuous integration (CI), continuous delivery (CD), networking, monitoring, observability, and so on.
It means that whether you can use Terraform at any future company you work for will be determined... by HashiCorp.
That's because the BSL license is intentionally vague. What does "competing" mean? What does "hosting or embedding" mean? Who decides?
In order to really know if you're a competitor, you have to reach out to HashiCorp (as the FAQ tells you to do). So whether your usage is valid is not controlled by the license terms, but is instead entirely at the whim of HashiCorp. So they switched from a permissive open source license to a HashiCorp decides license: they get to decide on a case by case basis now—and they can change their mind at any time.
That is very shaky footing on which to build anything.
And the legal team at every company you work for will have to take that into account before deciding you can or can't use Terraform.
How do you know you're not competing with HashiCorp?
That's not meant to be a redundant or snarky question. The key issue with the BSL and that FAQ is that the wording is intentionally vague. What does "competing" mean? What does "hosting or embedding" mean? Who decides?
In order to really know if you're a competitor, you have to reach out to HashiCorp (as the FAQ tells you to do). So whether your usage is valid is not controlled by the license terms, but is instead entirely at the whim of HashiCorp. So they switched from a permissive open source license to a HashiCorp decides license: they get to decide on a case by case basis now—and they can change their mind at any time.
That is very shaky footing on which to build anything.
We just moved the signatures to a table format, so you individuals can now add themselves to the table: just set the "type" column to "Individual." Thank you!
Imagine a future CTO trying to pick the IaC tools for their company. They see Terraform as an option, but then learn there are multiple forks, licensing questions, and a big battle happening in the community. What do they do? They are now way more likely to pick a different tool that is genuinely open source. The same is true of every dev considering where to build their career, every hobbyist, every open source enthusiast, every vendor, etc. In the end, no matter which fork wins, everyone will be worse off: the community will be smaller and more splintered.
So we opted to ask HashiCorp do the right thing first. If they choose to do the right thing, we can avoid a fork, and avoid splintering the community. We still think that's the best option. But if that doesn't work, then a foundation + fork it is.
This is precisely the problem with the new BSL license. Whether your usage of Terraform complies with the license isn’t determined by the legal terms, but instead is entirely at the whim of HashiCorp. And they can change their mind at any time. It makes it impossible to build anything on top of Terraform.
If you can use a Platform as a Service (PaaS) offering like Vercel, Netlify, Heroku, etc, you absolutely should! I always recommend those types of tools as the first stop tools to anyone building software these days.
But there are many use cases that don't fit into those neat PaaS molds: typically, as a software company grows beyond one team, one service, one database, etc, they start to hit limitations with the PaaS solutions out there. As you scale, you often find you need more control than you can get from a SaaS: you may need more control over the hardware (e.g., for performance or cost reasons), or networking (e.g., you need service discovery or a service mesh to allow microservices to communicate with each other), or security (e.g., to meet compliance standards), or a hundred other items.
That's when many companies find themselves migrating to an Infrastructure as a Service (IaaS) provider like AWS, Infrastructure as Code (IaC) tools like Terraform, orchestration tools like Kubernetes, and so on. I'm guessing every software company with more than 50-100 developers ends up moving from PaaS to IaaS, and that's when developers need to understand how to use the tools covered in this blog post series.
Perhaps, some day, the PaaS tools out there will be good enough that you never have to migrate off of them, regardless of scale or requirements, but we're not there yet, and probably won't be there for a while longer.
Author here. I tried to answer your question in the first two paragraphs. But to add some context, given the nature of my work, I hear from developers on a nearly daily basis who are struggling to get started with the technologies mentioned in this blog post series, which include not only Kubernetes, but also Docker, AWS, and Terraform. In part, they are struggling because they are too scared to ask for help, and comments like yours only make that worse: you seem to be implying that the materials out there for Kubernetes are so good, that if you don't get it, there must be something wrong with you. And yet, there are thousands of devs who don't get it, so maybe for different people, there are different ways to learn?
In discussions like this, I'm a fan of what Steve Yegge wrote about blogging [1]:
> This is an important thing to keep in mind when you're blogging. Each person in your audience is on a different clock, and all of them are ahead of you in some ways and behind you in others. The point of blogging is that we all agree to share where we're at, and not poke fun at people who seem to be behind us, because they may know other things that we won't truly understand for years, if ever.
That's why I write: to share what I know, from my particular perspective. Hopefully, that's useful to some people out there. If it's not useful to you, no problem!
And for the record, I agree the Kubernetes docs are great, including those interactive tutorials: if you read the series, you'd see I actually recommend those exact docs at the end of the post [2].
More of the latter. Don't put all your eggs in one basket. Don't create a single "module" (i.e., single deployable thing) with 100,000 lines of code and all your infrastructure in it. Break things up into small, reusable, composable pieces. This is what you typically do in any general purpose programming language, and it turns out it's a very good idea with infrastructure-as-code languages too.
Deleting prose and lines of code is still work. You have to read the text, think about it, figure out what would happen if you remove it, update all other places accordingly, and so on. The same goes for deleting code. To ignore that 160k lines were deleted as part of the writing process would be to ignore a massive amount of work.
Author here. I very much do understand that one line of "insertion" or "deletion" doesn't necessarily mean the entire line was rewritten. But as I wrote in the blog post, there are also many types of changes missing from the data:
1. I don't do a commit for every single line that I change. In fact, I may change a line 10 times, and commit only once.
2. This is actually even more pronounced for code. While doing a code-test cycle, I may change a few lines of code 50 times over, but only do one commit.
3. For my books, a lot of edit rounds and writing happened outside of Git (e.g., O'Reilly does copyediting in a PDF).
My guess is that these two factors roughly cancel out. It won't be exact, of course, and the actual ratio may be 8:1 or 12:1, but the order of magnitude is probably correct.
I'm one of the creators of Terratest. Happy to answer questions.
The main question I've seen so far seems to be how Terratest compares with various "spec" tools (e.g., inspec, serverspec). Most of the spec tools focus on checking the properties of a single server or resource. For example, is httpd installed and running? Terratest is largely for end-to-end, acceptance style testing, where you deploy your real infrastructure, in a real environment (e.g., AWS), and test the infrastructure actually works as expected.
For example, let's say you wanted to test a module for running Vault (https://github.com/hashicorp/terraform-aws-vault), which is a distributed secret store. With a spec tool, you might test a single Vault node to check that Vault is installed and the process is running. With Terratest, you'd check that the whole Vault cluster deployed correctly, bootstrapped itself (including auto-discovery of the other nodes), that you can initialize the cluster, unseal it store data, retrieve data, and so on.
At Gruntwork, we offer DevOps as a Service. That means we help companies figure out how to do all the steps that come after writing code. How do you test that code? How do you deploy it? How do you monitor it? How do you keep it secure?
It turns out that many, many companies are are struggling with these questions, and we need to grow our company to keep up with demand. We are distributed (work from anywhere), bootstrapped (no investor pressure), family-friendly (no crazy hours), trying to build a diverse team (people of all backgrounds, genders, races, cultures, and ethnicities are welcome), and profitable (we work with some of the biggest brands in the world).
We're looking for a seasoned developer with deep experience running production software ("operations", "DevOps"). If you're interested, email us at [email protected].
Our mission is to make it an order of magnitude easier to understand, develop, and deploy software. The first step in that direction is to take the thousands of hours we’ve spent building infrastructure and to condense all that experience and code into pre-assembled Infrastructure Packages. Each Infrastructure Package is a best-practices definition of a single piece of infrastructure (e.g. a Docker cluster or a relational database) that is reusable, battle-tested, documented, configurable, and supported (read more here: https://blog.gruntwork.io/gruntwork-infrastructure-packages-...).
In other words, we're like a DevOps team that has already built everything you need. We give customers 100% of the source code for our packages and we offer a subscription that includes support, package updates, access to the Gruntwork community, security alerts, proactive maintenance, and more.
We are looking for Full Stack Engineers, DevOps Engineers, and AWS experts. We make heavy use of Terraform, Docker, Packer, AWS, and Go. Currently, we have one project where we're looking for experts in the Windows environment (e.g. IIS, SQL Server, Active Directory) and one where we are looking for an expert on Kubernetes. We are a completely distributed company, so location doesn't matter, and the work is 100% remote.
Check out the talk Startup Ideas and Validation [1]. The key insights are:
* Innovative ideas are rarely, if ever, eureka moments. They usually evolve after spending a huge amount of time thinking about a specific problem.
* Ideas do not form out of nothing. You are not creating new knowledge so much as finding new ways to connect knowledge that you already have. So the more you know, the more ideas you can have.
* The environment plays a huge role in helping you combine your existing knowledge in novel ways. There are a number of tweaks you can make to your environment that make it easier to come up with great ideas, including keeping an idea journal (write down not only ideas, but problems you encounter on a daily basis), regularly letting your mind wander and relax (fixation can prevent creativity), adding constraints to the problem (constraints breed creativity), playing the "Wayne Gretzky Game" (live in the future and build what's missing), and seeking out pain (where there is pain, there is opportunity).
1. "The Adventures of Sherlock Holmes" by Arthur Conan Doyle.
This may sound like an odd one, but years ago, I almost never took the time to read. My girlfriend, who knew that I loved Sherlock Holmes books when I was younger, convinced me to try this book as an audiobook while I did my ~40 minute commute to work. I was skeptical, but within days, I was hooked. It made my work commute much more interesting (a British person was reading me Sherlock Holmes!); then I started listening to audiobooks during all my driving (instead of wasting time, I can learn!); then I got an iPod, put audiobooks on that, and started listening to them during all sorts of odd chores (e.g. cleaning, walking, biking); after that, I was so hooked on books, that I started making time to read them too. This had a profoundly transformative effect on my career.
2. "The Lean Startup" by Eric Ries.
I got a copy of this book when I went to a talk by Eric Ries. Eric seemed like a humble, down-to-earth person and helped dispel the notion that to be a successful entrepreneur, you need to be a prescient, superhero, god-like visionary. Instead, what you need to do is to treat your startup and product ideas as hypotheses and test them, as quickly and cheaply as you can (i.e. lean development, MVPs, etc). This fit very well with what I had seen in the real world and with how I thought about problem solving as a software engineer, and gave me a lot of confidence to try out many of my ideas. Since then, I've used these ideas to start a company (http://www.gruntwork.io/) and written quite a bit on what I learned, including an article on The Macro about MVPs (http://themacro.com/articles/2016/01/minimum-viable-product-...).
3. "On Writing Well" by William Zinsser.
If Conan Doyle taught me about the fun of reading, then William Zinsser taught me about the fun of writing. If you want to learn how to write, what it's like to write, or why you should write ("Writing is not a special language owned by the English teacher. Writing is thinking on paper."), it's hard to find a better guide. This book significantly improved my writing skills and even gave me the confidence to write a book (http://www.hello-startup.net/).
Before I decided to write the book, I talked to a number of friends who were authors and they all basically gave me the same advice:
* It's a massive amount of work, especially if it's your first book. If you're working a full-time job at the same time, depending on the length of the book you have in mind, expect it to take on the order of 2 years.
* It's a different type of work than programming or even blogging. In programming, you get feedback on a near constant basis at all levels: your IDE (sub-second), your compiler (seconds), your test suite (minutes), your co-workers (hours), and customers (days or weeks). This helps keep motivation high and gives you the info you need to improve your work. With a book, unless you make a massive effort to seek it out, you get more or less no feedback whatsoever for months or even years. For a project that takes such a long time, this can really sap your motivation. You have to come back and write a bit every single day, day after day, and yet on any individual day, it feels like you've hardly made any progress at all. You have to be very good at driving yourself and you need to put in an effort to give talks, to send out chapters to friends and family for feedback, to join a writing group, and anything else you can to get the feeling of tangible, incremental progress.
* You won't make much money from it. If your book hits the New York Times best seller list, sure, you can make money. But most tech books don't sell anywhere near that much, and even if you have "decent" sales, when you factor in the massive amount of work (see point #1), it's a comically small return, especially compared to a programmer salary.
* Despite that, every single author I talked to was writing their 2nd, 3rd, or even fourth book, and they all recommended doing it, subject to the caveats above.
As a result, I took the plunge, and I am very happy that I did. The real reasons to write a tech book are:
* It's an unbelievable learning experience. I used to think that experts became authors, but the reality is that authors become experts. I did a ton of research for my book, met a lot of interesting people along the way, read a huge number of books I had been meaning to for years (http://www.hello-startup.net/resources/recommended-reading/), improved my writing skills, got better at marketing, and so on.
* It's a great way to develop ideas. I started a company not long after writing my book (http://www.gruntwork.io/) and many of the ideas for that company came directly from what I learned during the writing process. As a bonus, the book is also a nice sales and marketing tool.
* It opens doors. People treat you just a little differently when they find out you are a "published author." They are more willing to listen. You get more opportunities for jobs, talks, meeting people, and so on.
* It feels good. I love teaching and sharing knowledge. I love seeing something that I created have a positive impact, even a tiny one, on someone's life. I love creating things. It feels wonderful to see positive reviews; to get emails from readers telling you how much the book meant to them; to hold your book in your your hands for the first time; to give your parents a copy; to find it on shelves at bookstores and famous libraries (my book is at Harvard, Oxford, etc!); and so on.
[1] https://www.fundamentals-of-devops.com/