Shipping Your Own Docker Syntax With BuildKit

You have probably seen this line at the top of a Dockerfile and never thought about it much:

dockerfile

# syntax=docker/dockerfile:1

That line is an image reference. When BuildKit sees it, it pulls docker/dockerfile:1, runs it, and hands it your build file. That image is the thing that actually understands FROM, RUN and COPY. Docker calls it a frontend, and here is the fun part: you can write your own.

In this post we will build a frontend called gobuild. Instead of a Dockerfile, you give it this:

gobuild.yaml yaml

# syntax=ghcr.io/rossedman/gobuild:v1
go: "1.26"
main: ./cmd/hello
ldflags: -s -w

and docker build turns it into a 2.6 MB image with nothing in it but your binary and a CA bundle. It cross-compiles for other architectures without emulation, takes a VERSION build arg, and works with docker compose. Notice the # syntax= line is a YAML comment, so the file is still valid YAML. Let’s dive into it.

How # syntax Works

BuildKit reads the first line
Before parsing anything, BuildKit looks for a # syntax= directive at the top of the build file.
It pulls and starts the frontend image
The frontend is a normal container image. BuildKit runs it and talks to it over gRPC on stdin and stdout.
The frontend asks for files and returns a build graph
It reads your build file and build context through that connection, then answers with LLB, BuildKit’s low-level build graph: “take this image, run this command, copy this file”.
BuildKit runs the graph
Caching, parallelism, multi-platform output and pushing all come for free, because they live in BuildKit, not in the frontend.

So a frontend is really a compiler from your file format to LLB. Ours is under 200 lines of Go. All of this works with a plain docker build on any recent Docker, since BuildKit has been the default builder since Docker 23.

The Spec

First, the language itself. Three fields, and we are strict about them, because a typo in a build file should fail loudly instead of quietly building the wrong thing.

spec.go go

package main

import (
	"bytes"
	"errors"
	"fmt"
	"regexp"
	"strings"

	"gopkg.in/yaml.v3"
)

// Spec is the whole build language: a Go version, a package to build
// and optional linker flags.
type Spec struct {
	Go      string `yaml:"go"`
	Main    string `yaml:"main"`
	LDFlags string `yaml:"ldflags"`
}

var goVersion = regexp.MustCompile(`^1\.[0-9]+(\.[0-9]+)?$`)

func parseSpec(dt []byte) (*Spec, error) {
	var s Spec
	dec := yaml.NewDecoder(bytes.NewReader(dt))
	dec.KnownFields(true) // a typo like "mian:" is an error, not a silent default
	if err := dec.Decode(&s); err != nil {
		return nil, fmt.Errorf("parsing build file: %w", err)
	}
	if s.Go == "" {
		return nil, errors.New(`build file needs a "go" version, like go: "1.26"`)
	}
	if !goVersion.MatchString(s.Go) {
		return nil, fmt.Errorf("go version %q should look like 1.26 or 1.26.1", s.Go)
	}
	if s.Main == "" {
		s.Main = "."
	}
	if !strings.HasPrefix(s.Main, ".") {
		return nil, fmt.Errorf("main %q should be a relative package path like ./cmd/app", s.Main)
	}
	return &s, nil
}

The version check isn’t just tidiness. The go field ends up inside an image reference (golang:<go>-alpine), so something like 1.26-alpine@sha256:... must not sneak through. This is plain Go, so it gets plain unit tests too, which is a nice change from testing Dockerfiles.

The Frontend

Here is the whole thing, then we’ll walk through it. It uses BuildKit v0.34.

main.go go

package main

import (
	"context"
	"fmt"
	"os"
	"regexp"
	"strings"

	"github.com/moby/buildkit/client/llb"
	"github.com/moby/buildkit/frontend/dockerui"
	"github.com/moby/buildkit/frontend/gateway/client"
	"github.com/moby/buildkit/frontend/gateway/grpcclient"
	"github.com/moby/buildkit/util/appcontext"
	dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
	ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
)

// Build args that are passed through to `go build` as environment variables.
var goEnvArgs = []string{"GOPROXY", "GOPRIVATE", "GOFLAGS"}

var versionArg = regexp.MustCompile(`^[A-Za-z0-9._+-]+$`)

func main() {
	if err := grpcclient.RunFromEnvironment(appcontext.Context(), build); err != nil {
		fmt.Fprintf(os.Stderr, "error: %v\n", err)
		os.Exit(1)
	}
}

func build(ctx context.Context, c client.Client) (*client.Result, error) {
	bc, err := dockerui.NewClient(c)
	if err != nil {
		return nil, err
	}

	// Read the file that was passed with -f and turn it into a Spec.
	src, err := bc.ReadEntrypoint(ctx, "yaml")
	if err != nil {
		return nil, err
	}
	spec, err := parseSpec(src.Data)
	if err != nil {
		return nil, err
	}

	ldflags := spec.LDFlags
	if v, ok := bc.BuildArgs["VERSION"]; ok {
		// The linker splits -ldflags on spaces, so keep VERSION to one token.
		if !versionArg.MatchString(v) {
			return nil, fmt.Errorf("VERSION %q may only contain letters, digits and ._+-", v)
		}
		ldflags = strings.TrimSpace(ldflags + " -X main.version=" + v)
	}

	// The build context: the directory at the end of `docker build`.
	buildCtx, err := bc.MainContext(ctx)
	if err != nil {
		return nil, err
	}

	rb, err := bc.Build(ctx, func(ctx context.Context, platform *ocispecs.Platform, idx int) (*dockerui.BuildResult, error) {
		buildPlatform := bc.BuildPlatforms[0]
		target := buildPlatform
		if platform != nil {
			target = *platform
		}

		// Compile on the machine doing the build and let Go cross-compile
		// for the target. No emulation needed.
		golang := llb.Image("docker.io/library/golang:"+spec.Go+"-alpine",
			llb.Platform(buildPlatform),
			llb.WithMetaResolver(c),
		)
		runOpts := []llb.RunOption{
			llb.Args([]string{"go", "build", "-trimpath", "-ldflags", ldflags, "-o", "/out/app", spec.Main}),
			llb.Dir("/src"),
			llb.AddEnv("CGO_ENABLED", "0"),
			llb.AddEnv("GOOS", target.OS),
			llb.AddEnv("GOARCH", target.Architecture),
			llb.AddEnv("GOARM", strings.TrimPrefix(target.Variant, "v")),
			llb.AddMount("/src", *buildCtx, llb.Readonly),
			llb.AddMount("/root/.cache/go-build", llb.Scratch(), llb.AsPersistentCacheDir("gobuild-cache", llb.CacheMountShared)),
			llb.AddMount("/go/pkg/mod", llb.Scratch(), llb.AsPersistentCacheDir("gobuild-mod", llb.CacheMountShared)),
			llb.WithCustomNamef("go build %s (%s/%s)", spec.Main, target.OS, target.Architecture),
		}
		for _, name := range goEnvArgs {
			if v, ok := bc.BuildArgs[name]; ok {
				runOpts = append(runOpts, llb.AddEnv(name, v))
			}
		}
		run := golang.Run(runOpts...)
		out := run.AddMount("/out", llb.Scratch())

		// The final image is just the binary and CA certificates.
		final := llb.Scratch().
			File(llb.Copy(out, "/app", "/app"), llb.WithCustomName("copy binary")).
			File(llb.Copy(golang, "/etc/ssl/certs/ca-certificates.crt", "/etc/ssl/certs/ca-certificates.crt", &llb.CopyInfo{CreateDestPath: true}), llb.WithCustomName("copy CA certificates"))

		def, err := final.Marshal(ctx, llb.Platform(target))
		if err != nil {
			return nil, err
		}
		res, err := c.Solve(ctx, client.SolveRequest{Definition: def.ToPB()})
		if err != nil {
			return nil, err
		}
		ref, err := res.SingleRef()
		if err != nil {
			return nil, err
		}

		img := &dockerspec.DockerOCIImage{
			Image: ocispecs.Image{
				Platform: target,
				RootFS:   ocispecs.RootFS{Type: "layers"},
			},
			// DockerOCIImage has its own Config that shadows Image.Config.
			// Set this one or the image ends up with no entrypoint.
			Config: dockerspec.DockerOCIImageConfig{
				ImageConfig: ocispecs.ImageConfig{
					Entrypoint: []string{"/app"},
					User:       "65534:65534",
					Env:        []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"},
				},
			},
		}
		return &dockerui.BuildResult{Reference: ref, Image: img}, nil
	})
	if err != nil {
		return nil, err
	}
	return rb.Finalize()
}

That’s a lot of words for a “hello world”, so let’s break it down.

Talking to BuildKit

grpcclient.RunFromEnvironment sets up the gRPC connection to BuildKit and calls build. Inside, dockerui.NewClient is a helper from BuildKit itself (the Dockerfile frontend uses it too) that handles the boring parts: which file was passed with -f, build args, .dockerignore, target platforms. ReadEntrypoint gives us the bytes of gobuild.yaml, and MainContext gives us the build context as an LLB state we can mount.

The Build Graph

Nothing is actually built while this code runs. llb.Image, Run and Copy only describe steps, and BuildKit runs them when we call c.Solve. In plain words, our graph says:

Start from golang:1.26-alpine on the build machine’s platform.
Mount the build context read-only at /src, plus two cache mounts so Go’s build cache and module cache survive between builds.
Run go build with GOOS/GOARCH set to the target platform, writing to an empty /out mount.
Start a new image from scratch and copy in /out/app and the CA bundle from the Go image.

Steps 1 and 3 are the cross-compiling trick. The Go toolchain always runs natively and only the output is foreign, so --platform linux/arm64 doesn’t need QEMU at all.

We use llb.Args instead of a shell string, so nothing in ldflags or main is ever interpreted by a shell. llb.WithMetaResolver(c) matters too: it pulls in the Go image’s config, so things like PATH are set and go can actually be found.

The Image Config

Finally we describe the image: run /app, as user 65534 (nobody), with SSL_CERT_FILE pointing at our bundle so HTTPS works from scratch. bc.Build calls our function once per target platform and rb.Finalize() stitches the results into one image, or a multi-platform index if you asked for several.

Shipping It

A frontend is just an image whose entrypoint is our binary. It doesn’t need a shell or anything else, so scratch is perfect.

Dockerfile dockerfile

# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM golang:1.26-alpine AS build
ARG TARGETOS TARGETARCH
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -trimpath -ldflags="-s -w" -o /gobuild .

FROM scratch
COPY --from=build /gobuild /bin/gobuild
ENTRYPOINT ["/bin/gobuild"]

Ship it for both Intel and ARM so it works on everyone’s laptop. I keep a GitHub token in 1password and load it with direnv, which makes logging in to GitHub’s registry painless. The token needs the write:packages scope.

shell

➜ echo $GITHUB_TOKEN | docker login ghcr.io -u rossedman --password-stdin
➜ docker buildx build --platform linux/amd64,linux/arm64 \
    -t ghcr.io/rossedman/gobuild:v1 --push .

Using It

Here is the project we’re building. A tiny app that prints its platform and version:

cmd/hello/main.go go

package main

import (
	"fmt"
	"runtime"
)

var version = "dev"

func main() {
	fmt.Printf("hello from %s/%s, version %s\n", runtime.GOOS, runtime.GOARCH, version)
}

Put gobuild.yaml from the top of this post next to go.mod, and point docker build at it with -f:

shell

➜ docker build -f gobuild.yaml -t hello .
➜ docker run --rm hello
hello from linux/amd64, version dev

The build log shows BuildKit fetching our frontend first, then running the steps we described:

text

#1 [internal] load build definition from gobuild.yaml
#2 resolve image config for docker-image://ghcr.io/rossedman/gobuild:v1
...
#8 go build ./cmd/hello (linux/amd64)
#9 copy binary
#10 copy CA certificates

And the image is tiny:

shell

➜ docker image ls hello
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
hello        latest    ...            ...             2.64MB

Build Args

Our frontend reads VERSION and bakes it into the binary:

shell

➜ docker build -f gobuild.yaml --build-arg VERSION=1.4.0 -t hello:1.4.0 .
➜ docker run --rm hello:1.4.0
hello from linux/amd64, version 1.4.0

If your modules live somewhere private, GOPROXY, GOPRIVATE and GOFLAGS pass straight through to go build the same way.

Other Platforms

Because we cross-compile, other architectures are just a flag. Here’s an ARM binary built on an Intel machine, exported straight to disk:

shell

➜ docker buildx build -f gobuild.yaml --platform linux/arm64 -o type=local,dest=out .
➜ file out/app
out/app: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, ...

Or build both and push a multi-platform image in one go:

shell

➜ docker buildx build -f gobuild.yaml --platform linux/amd64,linux/arm64 \
    -t ghcr.io/rossedman/hello:v1 --push .

Docker Compose

Compose doesn’t need to know anything special. Point dockerfile at the YAML file:

compose.yaml yaml

services:
  hello:
    build:
      context: .
      dockerfile: gobuild.yaml
      args:
        VERSION: "2.0.0"
    image: hello:compose

Pinning The Frontend

A tag like :v1 can move. For builds you care about, pin the digest, exactly like you would with a base image. BuildKit refuses to run a frontend whose digest doesn’t match.

gobuild.yaml yaml

# syntax=ghcr.io/rossedman/gobuild:v1@sha256:<digest>
go: "1.26"
main: ./cmd/hello

docker buildx imagetools inspect ghcr.io/rossedman/gobuild:v1 prints the digest.

Without The Syntax Line

You can also choose the frontend from the command line with the BUILDKIT_SYNTAX build arg. It wins over the # syntax= line, which makes it handy for trying a new version without editing files:

shell

➜ docker build -f gobuild.yaml --build-arg BUILDKIT_SYNTAX=ghcr.io/rossedman/gobuild:v2 .

If there’s no syntax line and no build arg, BuildKit falls back to the regular Dockerfile parser, which has opinions about YAML:

text

ERROR: failed to build: failed to solve: dockerfile parse error on line 1: unknown instruction: go:

Errors Are Part Of The Language

Whatever error the frontend returns is exactly what the person running docker build sees. That’s why the spec is strict. Here’s a typo:

text

ERROR: failed to build: failed to solve: parsing build file: yaml: unmarshal errors:
  line 3: field mian not found in type main.Spec

And a VERSION with spaces in it, which would otherwise smuggle extra flags into the linker:

text

ERROR: failed to build: failed to solve: VERSION "1.2.3 -X main.evil=1" may only contain letters, digits and ._+-

Iterating Locally

You don’t want to push to ghcr.io every time you change a line. Run a registry locally and point the syntax line at it:

shell

➜ docker run -d --name registry -p 5000:5000 registry:2
➜ docker build -t localhost:5000/gobuild:dev . && docker push localhost:5000/gobuild:dev

gobuild.yaml yaml

# syntax=localhost:5000/gobuild:dev
go: "1.26"
main: ./cmd/hello

Docker trusts localhost registries over plain HTTP, so there’s nothing else to configure. Rebuild, push, run docker build again.

Conclusion

That’s it! The # syntax line turns out to be a plugin system for docker build, and a frontend is just a Go program that turns your file format into a build graph. Everything hard (caching, parallelism, multi-platform images, pushing) is still BuildKit’s job.

The full source, with tests and an example app, is on GitHub. Ours is deliberately small, but it’s the same mechanism behind docker/dockerfile:1 itself. Once you see it that way, a lot of ideas open up: a frontend for your team’s standard service layout, one that reads a package.json or pyproject.toml directly, or one that enforces your base images and labels. Happy hacking!