Shipping Your Own Docker Syntax With BuildKit

You have probably seen this line at the top of a Dockerfile and never thought about it much:

dockerfile

# syntax=docker/dockerfile:1

That line is an image reference. When BuildKit sees it, it pulls docker/dockerfile:1, runs it, and hands it your build file. That image is the thing that actually understands FROM, RUN and COPY. Docker calls it a frontend, and here is the fun part: you can write your own.

In this post we will build a frontend called gobuild. Instead of a Dockerfile, you give it this:

gobuild.yaml yaml

# syntax=ghcr.io/rossedman/gobuild:v1
go: "1.26"
main: ./cmd/hello
ldflags: -s -w

and docker build turns it into a 2.6 MB image with nothing in it but your binary and a CA bundle. It cross-compiles for other architectures without emulation, takes a VERSION build arg, and works with docker compose. Notice the # syntax= line is a YAML comment, so the file is still valid YAML. Let’s dive into it.

How # syntax Works #

BuildKit reads the first line
Before parsing anything, BuildKit looks for a # syntax= directive at the top of the build file.
It pulls and starts the frontend image
The frontend is a normal container image. BuildKit runs it and talks to it over gRPC on stdin and stdout.
The frontend asks for files and returns a build graph
It reads your build file and build context through that connection, then answers with LLB, BuildKit’s low-level build graph: “take this image, run this command, copy this file”.
BuildKit runs the graph
Caching, parallelism, multi-platform output and pushing all come for free, because they live in BuildKit, not in the frontend.

So a frontend is really a compiler from your file format to LLB. Ours is under 200 lines of Go. All of this works with a plain docker build on any recent Docker, since BuildKit has been the default builder since Docker 23.

The Spec #

First, the language itself. Three fields, and we are strict about them, because a typo in a build file should fail loudly instead of quietly building the wrong thing.

spec.go go

package main

import (
	"bytes"
	"errors"
	"fmt"
	"regexp"
	"strings"

	"gopkg.in/yaml.v3"
)

// Spec is the whole build language: a Go version, a package to build
// and optional linker flags.
type Spec struct {
	Go      string `yaml:"go"`
	Main    string `yaml:"main"`
	LDFlags string `yaml:"ldflags"`
}

var goVersion = regexp.MustCompile(`^1\.[0-9]+(\.[0-9]+)?$`)

func parseSpec(dt []byte) (*Spec, error) {
	var s Spec
	dec := yaml.NewDecoder(bytes.NewReader(dt))
	dec.KnownFields(true) // a typo like "mian:" is an error, not a silent default
	if err := dec.Decode(&s); err != nil {
		return nil, fmt.Errorf("parsing build file: %w", err)
	}
	if s.Go == "" {
		return nil, errors.New(`build file needs a "go" version, like go: "1.26"`)
	}
	if !goVersion.MatchString(s.Go) {
		return nil, fmt.Errorf("go version %q should look like 1.26 or 1.26.1", s.Go)
	}
	if s.Main == "" {
		s.Main = "."
	}
	if !strings.HasPrefix(s.Main, ".") {
		return nil, fmt.Errorf("main %q should be a relative package path like ./cmd/app", s.Main)
	}
	return &s, nil
}

The version check isn’t just tidiness. The go field ends up inside an image reference (golang:<go>-alpine), so something like 1.26-alpine@sha256:... must not sneak through. This is plain Go, so it gets plain unit tests too, which is a nice change from testing Dockerfiles.

The Frontend #

Here is the whole thing, then we’ll walk through it. It uses BuildKit v0.34.

main.go go

package main

import (
	"context"
	"fmt"
	"os"
	"regexp"
	"strings"

	"github.com/moby/buildkit/client/llb"
	"github.com/moby/buildkit/frontend/dockerui"
	"github.com/moby/buildkit/frontend/gateway/client"
	"github.com/moby/buildkit/frontend/gateway/grpcclient"
	"github.com/moby/buildkit/util/appcontext"
	dockerspec "github.com/moby/docker-image-spec/specs-go/v1"
	ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
)

// Build args that are passed through to `go build` as environment variables.
var goEnvArgs = []string{"GOPROXY", "GOPRIVATE", "GOFLAGS"}

var versionArg = regexp.MustCompile(`^[A-Za-z0-9._+-]+$`)

func main() {
	if err := grpcclient.RunFromEnvironment(appcontext.Context(), build); err != nil {
		fmt.Fprintf(os.Stderr, "error: %v\n", err)
		os.Exit(1)
	}
}

func build(ctx context.Context, c client.Client) (*client.Result, error) {
	bc, err := dockerui.NewClient(c)
	if err != nil {
		return nil, err
	}

	// Read the file that was passed with -f and turn it into a Spec.
	src, err := bc.ReadEntrypoint(ctx, "yaml")
	if err != nil {
		return nil, err
	}
	spec, err := parseSpec(src.Data)
	if err != nil {
		return nil, err
	}

	ldflags := spec.LDFlags
	if v, ok := bc.BuildArgs["VERSION"]; ok {
		// The linker splits -ldflags on spaces, so keep VERSION to one token.
		if !versionArg.MatchString(v) {
			return nil, fmt.Errorf("VERSION %q may only contain letters, digits and ._+-", v)
		}
		ldflags = strings.TrimSpace(ldflags + " -X main.version=" + v)
	}

	// The build context: the directory at the end of `docker build`.
	buildCtx, err := bc.MainContext(ctx)
	if err != nil {
		return nil, err
	}

	rb, err := bc.Build(ctx, func(ctx context.Context, platform *ocispecs.Platform, idx int) (*dockerui.BuildResult, error) {
		buildPlatform := bc.BuildPlatforms[0]
		target := buildPlatform
		if platform != nil {
			target = *platform
		}

		// Compile on the machine doing the build and let Go cross-compile
		// for the target. No emulation needed.
		golang := llb.Image("docker.io/library/golang:"+spec.Go+"-alpine",
			llb.Platform(buildPlatform),
			llb.WithMetaResolver(c),
		)
		runOpts := []llb.RunOption{
			llb.Args([]string{"go", "build", "-trimpath", "-ldflags", ldflags, "-o", "/out/app", spec.Main}),
			llb.Dir("/src"),
			llb.AddEnv("CGO_ENABLED", "0"),
			llb.AddEnv("GOOS", target.OS),
			llb.AddEnv("GOARCH", target.Architecture),
			llb.AddEnv("GOARM", strings.TrimPrefix(target.Variant, "v")),
			llb.AddMount("/src", *buildCtx, llb.Readonly),
			llb.AddMount("/root/.cache/go-build", llb.Scratch(), llb.AsPersistentCacheDir("gobuild-cache", llb.CacheMountShared)),
			llb.AddMount("/go/pkg/mod", llb.Scratch(), llb.AsPersistentCacheDir("gobuild-mod", llb.CacheMountShared)),
			llb.WithCustomNamef("go build %s (%s/%s)", spec.Main, target.OS, target.Architecture),
		}
		for _, name := range goEnvArgs {
			if v, ok := bc.BuildArgs[name]; ok {
				runOpts = append(runOpts, llb.AddEnv(name, v))
			}
		}
		run := golang.Run(runOpts...)
		out := run.AddMount("/out", llb.Scratch())

		// The final image is just the binary and CA certificates.
		final := llb.Scratch().
			File(llb.Copy(out, "/app", "/app"), llb.WithCustomName("copy binary")).
			File(llb.Copy(golang, "/etc/ssl/certs/ca-certificates.crt", "/etc/ssl/certs/ca-certificates.crt", &llb.CopyInfo{CreateDestPath: true}), llb.WithCustomName("copy CA certificates"))

		def, err := final.Marshal(ctx, llb.Platform(target))
		if err != nil {
			return nil, err
		}
		res, err := c.Solve(ctx, client.SolveRequest{Definition: def.ToPB()})
		if err != nil {
			return nil, err
		}
		ref, err := res.SingleRef()
		if err != nil {
			return nil, err
		}

		img := &dockerspec.DockerOCIImage{
			Image: ocispecs.Image{
				Platform: target,
				RootFS:   ocispecs.RootFS{Type: "layers"},
			},
			// DockerOCIImage has its own Config that shadows Image.Config.
			// Set this one or the image ends up with no entrypoint.
			Config: dockerspec.DockerOCIImageConfig{
				ImageConfig: ocispecs.ImageConfig{
					Entrypoint: []string{"/app"},
					User:       "65534:65534",
					Env:        []string{"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"},
				},
			},
		}
		return &dockerui.BuildResult{Reference: ref, Image: img}, nil
	})
	if err != nil {
		return nil, err
	}
	return rb.Finalize()
}

That’s a lot of words for a “hello world”, so let’s break it down.

Talking to BuildKit #

grpcclient.RunFromEnvironment sets up the gRPC connection to BuildKit and calls build. Inside, dockerui.NewClient is a helper from BuildKit itself (the Dockerfile frontend uses it too) that handles the boring parts: which file was passed with -f, build args, .dockerignore, target platforms. ReadEntrypoint gives us the bytes of gobuild.yaml, and MainContext gives us the build context as an LLB state we can mount.

The Build Graph #

Nothing is actually built while this code runs. llb.Image, Run and Copy only describe steps, and BuildKit runs them when we call c.Solve. In plain words, our graph says:

Start from golang:1.26-alpine on the build machine’s platform.
Mount the build context read-only at /src, plus two cache mounts so Go’s build cache and module cache survive between builds.
Run go build with GOOS/GOARCH set to the target platform, writing to an empty /out mount.
Start a new image from scratch and copy in /out/app and the CA bundle from the Go image.

Steps 1 and 3 are the cross-compiling trick. The Go toolchain always runs natively and only the output is foreign, so --platform linux/arm64 doesn’t need QEMU at all.

We use llb.Args instead of a shell string, so nothing in ldflags or main is ever interpreted by a shell. llb.WithMetaResolver(c) matters too: it pulls in the Go image’s config, so things like PATH are set and go can actually be found.

The Image Config #

Finally we describe the image: run /app, as user 65534 (nobody), with SSL_CERT_FILE pointing at our bundle so HTTPS works from scratch. bc.Build calls our function once per target platform and rb.Finalize() stitches the results into one image, or a multi-platform index if you asked for several.

Shipping It #

A frontend is just an image whose entrypoint is our binary. It doesn’t need a shell or anything else, so scratch is perfect.

Dockerfile dockerfile

# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM golang:1.26-alpine AS build
ARG TARGETOS TARGETARCH
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -trimpath -ldflags="-s -w" -o /gobuild .

FROM scratch
COPY --from=build /gobuild /bin/gobuild
ENTRYPOINT ["/bin/gobuild"]

Ship it for both Intel and ARM so it works on everyone’s laptop. I keep a GitHub token in 1password and load it with direnv, which makes logging in to GitHub’s registry painless. The token needs the write:packages scope.

shell

➜ echo $GITHUB_TOKEN | docker login ghcr.io -u rossedman --password-stdin
➜ docker buildx build --platform linux/amd64,linux/arm64 \
    -t ghcr.io/rossedman/gobuild:v1 --push .

Using It #

Here is the project we’re building. A tiny app that prints its platform and version:

cmd/hello/main.go go

package main

import (
	"fmt"
	"runtime"
)

var version = "dev"

func main() {
	fmt.Printf("hello from %s/%s, version %s\n", runtime.GOOS, runtime.GOARCH, version)
}

Put gobuild.yaml from the top of this post next to go.mod, and point docker build at it with -f:

shell

➜ docker build -f gobuild.yaml -t hello .
➜ docker run --rm hello
hello from linux/amd64, version dev

The build log shows BuildKit fetching our frontend first, then running the steps we described:

text

#1 [internal] load build definition from gobuild.yaml
#2 resolve image config for docker-image://ghcr.io/rossedman/gobuild:v1
...
#8 go build ./cmd/hello (linux/amd64)
#9 copy binary
#10 copy CA certificates

And the image is tiny:

shell

➜ docker image ls hello
REPOSITORY   TAG       IMAGE ID       CREATED         SIZE
hello        latest    ...            ...             2.64MB

Build Args #

Our frontend reads VERSION and bakes it into the binary:

shell

➜ docker build -f gobuild.yaml --build-arg VERSION=1.4.0 -t hello:1.4.0 .
➜ docker run --rm hello:1.4.0
hello from linux/amd64, version 1.4.0

If your modules live somewhere private, GOPROXY, GOPRIVATE and GOFLAGS pass straight through to go build the same way.

Other Platforms #

Because we cross-compile, other architectures are just a flag. Here’s an ARM binary built on an Intel machine, exported straight to disk:

shell

➜ docker buildx build -f gobuild.yaml --platform linux/arm64 -o type=local,dest=out .
➜ file out/app
out/app: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, ...

Or build both and push a multi-platform image in one go:

shell

➜ docker buildx build -f gobuild.yaml --platform linux/amd64,linux/arm64 \
    -t ghcr.io/rossedman/hello:v1 --push .

Docker Compose #

Compose doesn’t need to know anything special. Point dockerfile at the YAML file:

compose.yaml yaml

services:
  hello:
    build:
      context: .
      dockerfile: gobuild.yaml
      args:
        VERSION: "2.0.0"
    image: hello:compose

Pinning The Frontend #

A tag like :v1 can move. For builds you care about, pin the digest, exactly like you would with a base image. BuildKit refuses to run a frontend whose digest doesn’t match.

gobuild.yaml yaml

# syntax=ghcr.io/rossedman/gobuild:v1@sha256:<digest>
go: "1.26"
main: ./cmd/hello

docker buildx imagetools inspect ghcr.io/rossedman/gobuild:v1 prints the digest.

Without The Syntax Line #

You can also choose the frontend from the command line with the BUILDKIT_SYNTAX build arg. It wins over the # syntax= line, which makes it handy for trying a new version without editing files:

shell

➜ docker build -f gobuild.yaml --build-arg BUILDKIT_SYNTAX=ghcr.io/rossedman/gobuild:v2 .

If there’s no syntax line and no build arg, BuildKit falls back to the regular Dockerfile parser, which has opinions about YAML:

text

ERROR: failed to build: failed to solve: dockerfile parse error on line 1: unknown instruction: go:

Errors Are Part Of The Language #

Whatever error the frontend returns is exactly what the person running docker build sees. That’s why the spec is strict. Here’s a typo:

text

ERROR: failed to build: failed to solve: parsing build file: yaml: unmarshal errors:
  line 3: field mian not found in type main.Spec

And a VERSION with spaces in it, which would otherwise smuggle extra flags into the linker:

text

ERROR: failed to build: failed to solve: VERSION "1.2.3 -X main.evil=1" may only contain letters, digits and ._+-

Iterating Locally #

You don’t want to push to ghcr.io every time you change a line. Run a registry locally and point the syntax line at it:

shell

➜ docker run -d --name registry -p 5000:5000 registry:2
➜ docker build -t localhost:5000/gobuild:dev . && docker push localhost:5000/gobuild:dev

gobuild.yaml yaml

# syntax=localhost:5000/gobuild:dev
go: "1.26"
main: ./cmd/hello

Docker trusts localhost registries over plain HTTP, so there’s nothing else to configure. Rebuild, push, run docker build again.

Conclusion #

That’s it! The # syntax line turns out to be a plugin system for docker build, and a frontend is just a Go program that turns your file format into a build graph. Everything hard (caching, parallelism, multi-platform images, pushing) is still BuildKit’s job.

The full source, with tests and an example app, is on GitHub. Ours is deliberately small, but it’s the same mechanism behind docker/dockerfile:1 itself. Once you see it that way, a lot of ideas open up: a frontend for your team’s standard service layout, one that reads a package.json or pyproject.toml directly, or one that enforces your base images and labels. Happy hacking!

How # syntax Works #

The Spec #

The Frontend #

Talking to BuildKit #

The Build Graph #

The Image Config #

Shipping It #

Using It #

Build Args #

Other Platforms #

Docker Compose #

Pinning The Frontend #

Without The Syntax Line #

Errors Are Part Of The Language #

Iterating Locally #

Conclusion #

Related posts