Introducing tfmetrics: Terraform Metrics for a Whole Org, Without Running Terraform
tfmetrics statically parses the Terraform in a directory, a git URL or every repo in a GitHub org, and emits one consistent set of metrics: what you have, who owns it, how far behind each module and provider is, and a 0 to 100 hygiene score. It never runs Terraform or reads state. who-uses lists every caller of a module with its file, line and owner, which is most of the work of an upgrade campaign.Every platform team eventually gets asked a question about Terraform that nobody can answer. How many repos still use the old EKS module? Which teams are on AWS provider 4.x? How many root modules don’t pin required_version? Is any of this getting better?
The answers are all sitting in .tf files, spread across a few hundred repos. The usual way to get at them is to clone everything and grep, which works once and gives you a number you can’t trust. The other way is to run terraform init and plan everywhere, which needs credentials for every account, takes hours, and tells you about your infrastructure when you only wanted to know about your code.
So I wrote tfmetrics. It parses the Terraform without running any of it, and turns an org’s worth of configuration into metrics you can put on a dashboard.
What You Get #
- Inventory. Modules (root and child), resources, data sources and module calls for every repo, labeled with an owner from
CODEOWNERSor a GitHub custom property. - Version drift. How many releases each module call, provider constraint and
required_versionis behind the latest, from registry and git tag listings. who-uses. Every caller of a module or provider, with the repo, file, line, version and owner.- A hygiene score from 0 to 100, made of a few structural checks you can weight however you like.
- Seven sinks. JSON, OpenMetrics, OTLP, CSV, Parquet, Markdown for a PR comment, and SARIF for code scanning.
- Nothing to configure in the cloud. It never runs Terraform, never touches state, and the only credential it needs is a GitHub token.
brew install rossedman/tap/tfmetricsQuick Start #
Point scan at a repo. It writes a facts file, which is everything tfmetrics learned about that repo:
➜ tfmetrics scan ./infra
scanned acme/infra: 5 modules -> facts/acme__infra.jsonexplain turns facts into a report a person can read:
➜ tfmetrics explain facts/ --offline
acme/infra @ 084d5f2f6d15
owners: acme/platform
modules 5 (2 root, 3 child)
resources 8 declared, 1 data sources
module calls 8 (1 remote unpinned)
lines 194
parse errors 1
hygiene score 65.3 / 100
Remote module calls pinned 5/6 weight 30
Variables and outputs described 4/8 weight 15
Providers with a version constraint 2/5 weight 15
Roots with required_version 1/2 weight 15
Variables with a type 3/4 weight 15
Modules that parse cleanly 4/5 weight 10And aggregate turns facts into metrics:
➜ tfmetrics aggregate facts/ --offline --sink openmetrics
aggregated 1 repos (1 with Terraform) into 36 series
➜ grep -E '^tf_(hygiene_score|repos_total)' metrics/metrics.prom
tf_hygiene_score{org="acme",owner="acme/platform",repo="acme/infra"} 65.3
tf_repos_total{org="acme"} 1.0That all runs offline in a few seconds. Drop --offline and tfmetrics also looks up the latest release of every module, provider and Terraform itself, so explain shows how far behind each call is and aggregate adds the *_versions_behind gauges.
How It Works #
There are two halves with one file format in the middle. Scanning turns repos into facts, and aggregation turns facts into metrics. Neither half runs Terraform.
- Discover
tfmetrics orglists the repos in your GitHub orgs and filters them by name, topic or custom property. Archived repos and forks are skipped. - Classify
Each repo is shallow-cloned with a sparse checkout of only
*.tf,*.tofuandCODEOWNERS. A directory with a backend, acloudblock or a configured provider is a root module; anything else is a child. - Parse
Modules load with HashiCorp’s
terraform-config-inspect, and the raw HCL is walked for the rest:lifecycle,count,for_each,movedandimportblocks. Nothing is evaluated. A file that doesn’t parse becomes a diagnostic and the scan keeps going. - Extract
One facts file per repo, cached by commit SHA. A repo whose
HEADhasn’t moved isn’t cloned again. - Aggregate
Facts become the metrics catalog and the hygiene score, written to every sink you’ve configured.
The facts file in the middle is what makes this flexible. Because aggregation only reads facts, you can scan once a night and re-aggregate with different weights or sinks without cloning anything. You can run scan in each repo’s CI and aggregate centrally. And you can diff two facts files to see exactly what changed between two commits.
Scanning a Whole Org #
For more than a handful of repos, describe the org in a tfmetrics.yaml:
source:
github:
token_env: GITHUB_TOKEN
orgs:
- name: acme
include:
topics: [terraform]
exclude:
names: ["sandbox-*"]
scan:
concurrency: 16
ignore_dirs: [.terraform, examples]
owners:
property: team
sinks:
- json: {path: metrics/}
- openmetrics: {path: metrics/metrics.prom}
- markdown: {}Then two commands scan the org and write every sink:
➜ tfmetrics org --out facts/
➜ tfmetrics aggregate facts/owners.property: team labels every metric with the value of a GitHub custom property, so a dashboard can break everything down by team. Without it, owners come from CODEOWNERS.
Finding Every Caller of a Module #
This is the command I wanted most. A module you depend on ships a new major version, or one of your own needs a breaking change, and the first question is always “who’s using it?”
➜ tfmetrics who-uses terraform-aws-modules/eks/aws facts/
registry.terraform.io/terraform-aws-modules/eks/aws: 6 calls in 3 repos
versions: ~> 21.20 (2), 17.24.0 (1), 18.31.2 (1), 19.11.0 (1), ~> 21.0 (1)
REPO OWNER MODULE VERSION PINNED BEHIND FILE
acme/k8s-infra acme/platform aws/prow-build-cluster ~> 21.20 range 0 aws/prow-build-cluster/eks.tf:21
acme/k8s-infra acme/platform aws/policy-staging 19.11.0 exact 78 aws/policy-staging/eks.tf:17
acme/cloud-platform acme/webops vpc/eks 18.31.2 exact 90 vpc/eks/cluster.tf:293
...BEHIND counts the major.minor release lines between the version a call selects and the latest. A constraint like ~> 21.20 resolves to the newest 21.x the way terraform init would, so it stays at 0 until 22.0 ships. FILE is the line with the module block, so you can open exactly what needs changing.
Add --behind to keep only the callers still to move, and --format csv to hand the list to the people who own them:
➜ tfmetrics who-uses eks/aws facts/ --behind 25 --format csv > eks-upgrades.csvRun it again after each wave of upgrades and the list gets shorter. --provider does the same thing for required_providers, so “who’s still on AWS provider 4.x?” is one command too.
On Every Pull Request #
scan . --sink markdown prints a report for the repo you’re in, which drops straight into a PR comment or a job summary:
on:
pull_request:
paths: ["**.tf", "**.tf.json"]
permissions:
contents: read
jobs:
report:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version: stable
- run: go install github.com/rossedman/tfmetrics@latest
- run: |
tfmetrics scan . --out "$RUNNER_TEMP/facts" --sink markdown > report.md
cat report.md >> "$GITHUB_STEP_SUMMARY" The PR check guide has the full workflow, with pinned actions and a step that keeps one comment up to date. tfmetrics reports rather than gates, so scan exits 0 whatever the score. If you do want to fail below a threshold, write the json sink too and compare tf_hygiene_score against it.
And for annotations right on the diff, the sarif sink turns parse errors, unpinned modules and git modules that follow a branch into code scanning alerts. That’s also how tfmetrics plugs into sarifix, which I’m announcing today as well.
The Metrics #
Every metric carries org, and per-repo metrics add repo and owner. A few of the ones I find myself looking at most:
| Metric | Meaning |
|---|---|
tf_module_versions_behind | Releases between a module call’s pinned version and the latest |
tf_provider_versions_behind | Releases between the provider version a constraint selects and the latest |
tf_module_calls_unpinned | Remote module calls with no version or ref |
tf_required_version | Root modules per Terraform core version constraint |
tf_backend | Root modules by backend type |
tf_prevent_destroy_resources | Resources with prevent_destroy = true |
tf_hygiene_score | The weighted 0 to 100 hygiene score |
There are 19 in all; the metrics catalog lists every one with its labels. The OpenMetrics and OTLP sinks feed Prometheus or anything that speaks OpenTelemetry, and Parquet and CSV are there for when someone wants to answer their own question in a notebook.
Try It #
brew install rossedman/tap/tfmetrics
tfmetrics scan ./your-terraform-repo
tfmetrics explain facts/It also installs with go install github.com/rossedman/tfmetrics@latest, and every release is signed with cosign and carries build provenance. The docs have tutorials for a first org audit, a nightly scan, a module upgrade campaign and scanning very large orgs.
It’s Apache-2.0 licensed and still pre-1.0, so the metric names and flags may still change. If you point it at your org and a number looks wrong, or there’s a question about your Terraform it can’t answer yet, open an issue on GitHub. I’d love to hear what you find.