Introducing tfmetrics: Terraform Metrics for a Whole Org, Without Running Terraform

TL;DR
tfmetrics statically parses the Terraform in a directory, a git URL or every repo in a GitHub org, and emits one consistent set of metrics: what you have, who owns it, how far behind each module and provider is, and a 0 to 100 hygiene score. It never runs Terraform or reads state. who-uses lists every caller of a module with its file, line and owner, which is most of the work of an upgrade campaign.
rossedman/tfmetrics on GitHub

Every platform team eventually gets asked a question about Terraform that nobody can answer. How many repos still use the old EKS module? Which teams are on AWS provider 4.x? How many root modules don’t pin required_version? Is any of this getting better?

The answers are all sitting in .tf files, spread across a few hundred repos. The usual way to get at them is to clone everything and grep, which works once and gives you a number you can’t trust. The other way is to run terraform init and plan everywhere, which needs credentials for every account, takes hours, and tells you about your infrastructure when you only wanted to know about your code.

So I wrote tfmetrics. It parses the Terraform without running any of it, and turns an org’s worth of configuration into metrics you can put on a dashboard.


What You Get #

  • Inventory. Modules (root and child), resources, data sources and module calls for every repo, labeled with an owner from CODEOWNERS or a GitHub custom property.
  • Version drift. How many releases each module call, provider constraint and required_version is behind the latest, from registry and git tag listings.
  • who-uses. Every caller of a module or provider, with the repo, file, line, version and owner.
  • A hygiene score from 0 to 100, made of a few structural checks you can weight however you like.
  • Seven sinks. JSON, OpenMetrics, OTLP, CSV, Parquet, Markdown for a PR comment, and SARIF for code scanning.
  • Nothing to configure in the cloud. It never runs Terraform, never touches state, and the only credential it needs is a GitHub token.
shell
brew install rossedman/tap/tfmetrics

Quick Start #

Point scan at a repo. It writes a facts file, which is everything tfmetrics learned about that repo:

shell
➜ tfmetrics scan ./infra
scanned acme/infra: 5 modules -> facts/acme__infra.json

explain turns facts into a report a person can read:

shell
➜ tfmetrics explain facts/ --offline
acme/infra @ 084d5f2f6d15
owners: acme/platform

modules      5 (2 root, 3 child)
resources    8 declared, 1 data sources
module calls 8 (1 remote unpinned)
lines        194
parse errors 1

hygiene score 65.3 / 100
  Remote module calls pinned           5/6  weight 30
  Variables and outputs described      4/8  weight 15
  Providers with a version constraint  2/5  weight 15
  Roots with required_version          1/2  weight 15
  Variables with a type                3/4  weight 15
  Modules that parse cleanly           4/5  weight 10

And aggregate turns facts into metrics:

shell
➜ tfmetrics aggregate facts/ --offline --sink openmetrics
aggregated 1 repos (1 with Terraform) into 36 series

➜ grep -E '^tf_(hygiene_score|repos_total)' metrics/metrics.prom
tf_hygiene_score{org="acme",owner="acme/platform",repo="acme/infra"} 65.3
tf_repos_total{org="acme"} 1.0

That all runs offline in a few seconds. Drop --offline and tfmetrics also looks up the latest release of every module, provider and Terraform itself, so explain shows how far behind each call is and aggregate adds the *_versions_behind gauges.


How It Works #

There are two halves with one file format in the middle. Scanning turns repos into facts, and aggregation turns facts into metrics. Neither half runs Terraform.

  1. Discover

    tfmetrics org lists the repos in your GitHub orgs and filters them by name, topic or custom property. Archived repos and forks are skipped.

  2. Classify

    Each repo is shallow-cloned with a sparse checkout of only *.tf, *.tofu and CODEOWNERS. A directory with a backend, a cloud block or a configured provider is a root module; anything else is a child.

  3. Parse

    Modules load with HashiCorp’s terraform-config-inspect, and the raw HCL is walked for the rest: lifecycle, count, for_each, moved and import blocks. Nothing is evaluated. A file that doesn’t parse becomes a diagnostic and the scan keeps going.

  4. Extract

    One facts file per repo, cached by commit SHA. A repo whose HEAD hasn’t moved isn’t cloned again.

  5. Aggregate

    Facts become the metrics catalog and the hygiene score, written to every sink you’ve configured.

The facts file in the middle is what makes this flexible. Because aggregation only reads facts, you can scan once a night and re-aggregate with different weights or sinks without cloning anything. You can run scan in each repo’s CI and aggregate centrally. And you can diff two facts files to see exactly what changed between two commits.


Scanning a Whole Org #

For more than a handful of repos, describe the org in a tfmetrics.yaml:

tfmetrics.yaml yaml
source:
  github:
    token_env: GITHUB_TOKEN
    orgs:
      - name: acme
        include:
          topics: [terraform]
        exclude:
          names: ["sandbox-*"]
scan:
  concurrency: 16
  ignore_dirs: [.terraform, examples]
owners:
  property: team
sinks:
  - json: {path: metrics/}
  - openmetrics: {path: metrics/metrics.prom}
  - markdown: {}

Then two commands scan the org and write every sink:

shell
➜ tfmetrics org --out facts/
➜ tfmetrics aggregate facts/

owners.property: team labels every metric with the value of a GitHub custom property, so a dashboard can break everything down by team. Without it, owners come from CODEOWNERS.


Finding Every Caller of a Module #

This is the command I wanted most. A module you depend on ships a new major version, or one of your own needs a breaking change, and the first question is always “who’s using it?”

shell
➜ tfmetrics who-uses terraform-aws-modules/eks/aws facts/
registry.terraform.io/terraform-aws-modules/eks/aws: 6 calls in 3 repos
versions: ~> 21.20 (2), 17.24.0 (1), 18.31.2 (1), 19.11.0 (1), ~> 21.0 (1)

REPO                 OWNER          MODULE                  VERSION   PINNED  BEHIND  FILE
acme/k8s-infra       acme/platform  aws/prow-build-cluster  ~> 21.20  range   0       aws/prow-build-cluster/eks.tf:21
acme/k8s-infra       acme/platform  aws/policy-staging      19.11.0   exact   78      aws/policy-staging/eks.tf:17
acme/cloud-platform  acme/webops    vpc/eks                 18.31.2   exact   90      vpc/eks/cluster.tf:293
...

BEHIND counts the major.minor release lines between the version a call selects and the latest. A constraint like ~> 21.20 resolves to the newest 21.x the way terraform init would, so it stays at 0 until 22.0 ships. FILE is the line with the module block, so you can open exactly what needs changing.

Add --behind to keep only the callers still to move, and --format csv to hand the list to the people who own them:

shell
➜ tfmetrics who-uses eks/aws facts/ --behind 25 --format csv > eks-upgrades.csv

Run it again after each wave of upgrades and the list gets shorter. --provider does the same thing for required_providers, so “who’s still on AWS provider 4.x?” is one command too.


On Every Pull Request #

scan . --sink markdown prints a report for the repo you’re in, which drops straight into a PR comment or a job summary:

.github/workflows/tfmetrics-pr.yml yaml
on:
  pull_request:
    paths: ["**.tf", "**.tf.json"]

permissions:
  contents: read

jobs:
  report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-go@v7
        with:
          go-version: stable
      - run: go install github.com/rossedman/tfmetrics@latest
      - run: |
          tfmetrics scan . --out "$RUNNER_TEMP/facts" --sink markdown > report.md
          cat report.md >> "$GITHUB_STEP_SUMMARY"          

The PR check guide has the full workflow, with pinned actions and a step that keeps one comment up to date. tfmetrics reports rather than gates, so scan exits 0 whatever the score. If you do want to fail below a threshold, write the json sink too and compare tf_hygiene_score against it.

And for annotations right on the diff, the sarif sink turns parse errors, unpinned modules and git modules that follow a branch into code scanning alerts. That’s also how tfmetrics plugs into sarifix, which I’m announcing today as well.


The Metrics #

Every metric carries org, and per-repo metrics add repo and owner. A few of the ones I find myself looking at most:

MetricMeaning
tf_module_versions_behindReleases between a module call’s pinned version and the latest
tf_provider_versions_behindReleases between the provider version a constraint selects and the latest
tf_module_calls_unpinnedRemote module calls with no version or ref
tf_required_versionRoot modules per Terraform core version constraint
tf_backendRoot modules by backend type
tf_prevent_destroy_resourcesResources with prevent_destroy = true
tf_hygiene_scoreThe weighted 0 to 100 hygiene score

There are 19 in all; the metrics catalog lists every one with its labels. The OpenMetrics and OTLP sinks feed Prometheus or anything that speaks OpenTelemetry, and Parquet and CSV are there for when someone wants to answer their own question in a notebook.


Try It #

shell
brew install rossedman/tap/tfmetrics
tfmetrics scan ./your-terraform-repo
tfmetrics explain facts/

It also installs with go install github.com/rossedman/tfmetrics@latest, and every release is signed with cosign and carries build provenance. The docs have tutorials for a first org audit, a nightly scan, a module upgrade campaign and scanning very large orgs.

rossedman/tfmetrics github.com · Org-wide Terraform metrics from static parsing

It’s Apache-2.0 licensed and still pre-1.0, so the metric names and flags may still change. If you point it at your org and a number looks wrong, or there’s a question about your Terraform it can’t answer yet, open an issue on GitHub. I’d love to hear what you find.