Serious question. What value does Terraform provide?
Two years ago I looked into it and rather then having an abstraction from cloud providers it seemed to require to still target (and code against) each one specifically.
So, I was quite disappointed as I thought the value proposition was to not have to know x cloud provider specific terminologies.
Any insights much appreciated.
Edit: I was a little worried asking such a naive question but the comments are super useful! Thanks everyone for sharing your insights.
> Two years ago I looked into it and rather then having an abstraction from cloud providers
This is a misrepresentation I've seen multiple times and I don't know how it's come to be.
Terraform doesn't abstract resources. It simply supports all cloud providers and lets you intermix resources from different clouds inside a single project. Resources can depend on each other and use each other's attributes.
As an example, you can bring up a load balancer in AWS and create a DNS record for it on Cloudflare in a same Terraform project and maintain them together.
I believe the issue is the Terraform has been labelled: “Cloud agnostic”. That was why I believed that Terraform would abstract away the individual cloud providers.
It depends on your interpretation of the word “agnostic”. Personally I would say a more correct description would be: Support for multiple cloud providers.
While resources are fundamentally the same across clouds (i.e. they're all VMs, they all have firewalls etc), they are vastly different concepts and have different feature sets. It's almost impossible to do a like for like api call between two providers.
However, you can develop cloud agnostic modules that you can then consume, which allows for a decent cloud-agnostic experience.
this, different resourcers are named in different providers and I would find that HCL code would vary from one provider to another. This is the reason I have been a huge proponent of using container based applications that happen to get launched in a specific cloud, rather than using a base OS/ function app service
It makes things repeatable, organized, and most importantly checked into source control.
It is actually useable unlike cloudformation which is for a nightmare of unreadable, barely editable yaml files and fifty commands to then upload and apply the files. Lets not even get into debugging or unsticking cloudformation when it breaks, something that usually requires writing a support ticket.
Additionally you can build your own modules. I can have a module that is `ServiceFoo`. Pass in a param that causes it to switch between different backends. Yea I have to write the AWS and GCP part seperately, but then anything that needs `ServiceFoo` can just call the module and have the things split across both sides.
You can then also do things like have your DNS in AWS, but have nodes in both GCP and AWS. Use the settings pulled from GCP to input into Route53, etc.
Abstracting over Cloud vendors is not a use case for Terraform itself. The value it brings is that you get to specify your infrastructure 'as code', which means you'll be able to re-create it from code, and reliably deploy changes.
There's a lot more benefits, it depends on what you are comparing it against. Coming from a software development background, I'd like to compare it to a wordpress app vs webapp development from something like rails. The wordpress app works fine, is faster to write, but once it gets complex things fall apart. The rails app is maybe a bit more difficult to develop at first, some features might take longer, but it's more flexbile, powerful and when engineered well, it will not hit a ceiling where it just all falls apart.
I'll chime in. When I first used Terraform, it was described as this tool that would create resources in a cloud agnostic way. That's still possible, but not the main focal point.
Terraform just takes any API (called terraform providers) and applies the GitOps philosophy to it. That's it. Now you can easily recreate resources with a single command, modify whatever parameters you need, store those changes in Git, etc.
Yes, one long curl command would give you the same results but then you miss out on the concept of dependencies or API versioning or simple programming constructs like reading from a file or looping over values.
You're right, Terraform is not "write once, deploy infrastructure anywhere" tool. It does however allow you to reuse many portions of your infrastructure descriptions between cloud providers by using module composition[1].
Terraform can also be used to put your infrastructure under version control, which is a pretty big deal.
Writing infrastructure as code is quite often an exercise in:
a) define what I want
b) write an API call to find out whether what I want already exists
c) write an API call to create it if it doesn't exist
d) Sometimes people do stuff manually and your code should tolerate working around these manual changes (i.e. update in place when possible, tear down and recreate when not possible)
e) To be efficient, your code should run things in parallel when possible
Terraform allows you to write (a) and outsource the rest to a provider (b,c) usually maintained by the API provider themselves or Terraform itself (d,e)
Creating and destroying infra resources with the click of a button. Better visibility into what's provisioned and their configuration. If you need to main many individual pieces of infrastructure then it's nice to have a central codified manner to achieve that
I kinda expected to see many examples of this in this thread (whether they use Terraform or not). So, to ask explicitly: are there usable abstractions of the type "use-case-achievable-with-3-top-public-clouds"? Even something extremely simple like a bunch of linuxes behind a regional load balancer. I don't mean the lowest common denominator of all the clouds, but just a few popular ones, with an obvious intention to reduce vendor lock-in.
These would be probably very basic scenarios, but still the whole multi-cloud hype could have produced something decent-ish by now?
It's better than CloudFormation (or a bunch of home grown bash scripts) and you can also modify providers beyond just the cloud host (ie: datadog alerts, database users and permissions, etc, etc.).
Just to expand on your second point, because I think it's often missed, there are a ton of 'providers' available that extend management of their products (to varying, sometimes hilariously little extent). If you're running products from any of the vendors on this list, you might be able to use terraform to manage them as well:
There are 3 ways to allocate cloud resources:
1. Use GUI
2. Use Boto library with Python
3. Use devops tools like Terraform
As you go from 1 to 3, the programmability increases. 3 is additionally idempotent. As others mention, 3 has competing products and each has its warts.
Two years ago I looked into it and rather then having an abstraction from cloud providers it seemed to require to still target (and code against) each one specifically.
So, I was quite disappointed as I thought the value proposition was to not have to know x cloud provider specific terminologies.
Any insights much appreciated.
Edit: I was a little worried asking such a naive question but the comments are super useful! Thanks everyone for sharing your insights.