Performance¶
A run spends its time in three places: finding the targets on GitHub, getting each repo onto disk and running the change in it, and writing branches and PRs back. This page explains what each costs, how gh fleet keeps that low, and what to change when a run is slow. The details live in the pages linked from each section.
Finding out what's slow¶
Start here rather than guessing. --debug (or GH_FLEET_DEBUG=1) logs each step on stderr as it starts and finishes, and ends with a table of how long everything took:
The end of the log looks like this (an example run over 120 repos):
Timings (totals add up across parallel workers):
NAME COUNT TOTAL AVG MAX
total: plan 1 41s 41s 41s
run change in every repo 1 36s 36s 36s
repo: total 120 9m12s 4.6s 31s
repo: clone 120 3m40s 1.8s 12s
git fetch 120 1m2s 517ms 9s
...
Repos run in parallel, so per-repo, git and API rows add up across workers and can be far larger than the wall time. Compare the AVG and MAX columns, and run change in every repo against total.
| Row | What it measures | When it dominates |
|---|---|---|
resolve targets |
finding the targets: org listings, searches, lookups, file checks | see Targeting cheaply; the progress note says why an org was listed |
pick wave, index existing PRs |
looking up the campaign's PRs, in batched queries | grows with the number of targets; --repo limits it |
look up campaign branches |
where the campaign branch is in every target, 50 repos to a query | should stay small |
skip unchanged repos |
checking which repos the last run found nothing to change in | should stay small |
repo: clone |
getting one repo onto disk, from the git cache or GitHub | big repos; the first run, which fills the cache; --no-git-cache |
git clone |
a repo cloned straight from GitHub | appears when the cache is off or couldn't be used, which a note on stderr says |
git fetch |
bringing a repo's cache up to date, or fetching an existing campaign branch's history to rebase it | a slow network, or many existing campaign branches |
repo: step <name> |
your change script, per step | the script itself: npm install and the like |
repo: diff, repo: push, repo: sync PR |
the diff, the push and the PR's API calls | sync PR waits on the write rate |
cleanup (background) |
deleting a finished clone, off the worker's critical path | never blocks other repos; only the end of the run waits for it |
api core, api graphql, api search, api code_search |
each GitHub request, by API, with its method, path and status in the log | many requests: a big org listing or file checks |
api wait: rate limits |
time requests spent waiting on GitHub's limits or the write rate | see Rate limits; a large value means GitHub, not gh fleet, is the bottleneck |
api wait: retry backoff |
backoff before retrying a 5xx |
GitHub having trouble |
What a run costs¶
| Step | Cost | What keeps it down |
|---|---|---|
| Find targets | API reads | caches, searches and batched lookups: see Targeting cheaply |
Skip unchanged repos (apply) |
one query per 50 repos | repos the last run found nothing to change in aren't cloned at all |
| Look up campaign branches | one query per 50 repos | replaces a round trip to the git server per repo |
| Get each repo onto disk | git traffic, no API quota | the git cache: only what changed since the last run is fetched |
| Run the change | your script's time | --concurrency runs more repos at once |
Push and open or update PRs (apply) |
git push, plus a few API writes per new PR (create it, then labels and reviewers when the campaign has them), and none for a PR that's already right | GitHub's limits set the pace: see Writes |
Three caches¶
gh fleet keeps three caches. Together they make the usual loop of plan, plan again after editing the script, then apply much cheaper than three full runs.
| Cache | Kept for | Saves | Turn off or refresh |
|---|---|---|---|
| Org listings | 10 minutes | listing every repo of an org again; targets, plan and apply in a row cost one listing |
wait 10 minutes, or delete gh-fleet/repos/ under your user cache directory |
| Git cache | until deleted; garbage collected weekly | downloading each repo again: later runs fetch only what changed | --no-git-cache, or delete gh-fleet/git/ |
| Unchanged repos | a day | apply skips repos the last run found nothing to change in, without cloning them |
--recheck |
The git cache¶
Every repo a run touches gets a shallow copy of its base branch under gh-fleet/git/ in your user cache directory. A run brings that copy up to date with a fetch, which transfers only the commits since the last run, and the repo's working copy borrows its files instead of downloading them. Each working copy still has its own .git directory with origin pointing at GitHub, so your scripts, pushes and rebases behave as they would in a plain clone, and nothing a script writes reaches the cache.
The first run on a repo fills its cache, so it costs about what a clone did. After that, getting a repo onto disk takes a fraction of a full clone, and the bigger the repo, the bigger the difference. If the cache can't be used, for example when two runs fetch the same repo at the same moment, that repo is cloned from GitHub and a note says so; the cache never fails a run.
The cache holds about one shallow copy of each repo you've targeted. Deleting it is safe: the next run rebuilds it.
Targeting cheaply¶
How targets are written decides how much of an org gets read. The cheapest form names the repos:
| Targets | What gh fleet reads |
|---|---|
include.repos alone, with exact owner/name or bare names |
only those repos, 50 to a query, however big the org |
include.names with exact names |
only those repos in each org, then the other filters on them |
include.topics or include.languages |
GitHub's repo search for the matches, without listing the org; it lists instead when a listing is cached, the search matches over 1,000 repos, or the campaign uses custom properties |
include.search |
code search, then only the repos with hits |
include.files, include.properties, an include.names pattern, or no include filter |
every repo of each org, cached for 10 minutes |
--repo or --repo-file |
only the repos named, whatever else the campaign's targets would read; its filters still apply to them |
include.repos beside any other include filter adds its repos to what that filter selects, so the org is still read; targets, plan and apply print a note when a campaign does this. When an org is listed, the progress note says which filter caused it, for example Listing all of acme's repos: include.files is checked in every repo. See Targeting repositories for the full rules.
Concurrency¶
--concurrency (or limits.concurrency, default 16) is how many repos are cloned and run at once. Getting a repo onto disk goes through git and costs no API quota, and GitHub API requests are capped at 8 in flight however many workers there are. So raising --concurrency speeds up cloning and scripts without adding API load.
Raise it when repo: clone or your script steps dominate and your machine has cores, memory and disk to spare. Lower it when the change script is heavy, such as a full dependency install, and the machine is struggling.
Writes¶
Writes are paced at 1 per second across all workers by default, which is GitHub's own advice for mutating requests, and new PRs are also kept within GitHub's limit of 80 a minute and 500 an hour. gh fleet already sends as few writes as it can: labels and reviewers only when the campaign has them, and an existing PR only when its title, body or labels differ, so re-running apply on repos whose PRs are already right writes nothing.
When apply opens thousands of PRs, GitHub's limits set the pace, not gh fleet. Waves and retry let such a rollout span several sessions. See Rate limits for the details, and why raising --write-rate rarely helps.
Checklist for a slow run¶
- Run it with
--debugand look at the table's largestAVGandMAXrows. - If
resolve targetsis large, check the progress note for why an org was listed, and see Targeting cheaply. - If
repo: cloneis large, run again: the first run fills the git cache. Ifgit cloneappears, read the note on stderr for why the cache wasn't used. - If your script's steps dominate, make the script faster or raise
--concurrency. - If
api wait: rate limitsis large, GitHub is pacing the run; see Rate limits. - To iterate on one repo, use
--repo owner/name: everything above shrinks to that repo.