Skip to content

Concurrent lookups

Source · io.systemd.Resolve · systemd v257+ · Unprivileged

Fan out DNS lookups across goroutines through one resolve.Client. Generated clients keep a pool of connections, so the lookups overlap instead of queueing behind each other on a single socket, where Varlink would answer them one at a time.

Run it

go run ./examples/concurrent-lookups github.com go.dev systemd.io kernel.org
Output (illustrative)
github.com           1 addresses in 21ms
go.dev               8 addresses in 24ms
systemd.io           8 addresses in 19ms
kernel.org           2 addresses in 22ms
4 lookups in 25ms

The code

examples/concurrent-lookups/main.go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
// Command concurrent-lookups resolves many names in parallel. Generated
// clients keep a pool of connections, so goroutines do not queue behind one
// another on a single socket.
//
//  go run ./examples/concurrent-lookups github.com go.dev systemd.io kernel.org
package main

import (
    "context"
    "fmt"
    "log"
    "os"
    "sync"
    "time"

    "github.com/rossedman/sdvarlink/systemd/v257/resolve"
)

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
    defer cancel()

    c, err := resolve.Dial(ctx)
    if err != nil {
        log.Fatal(err)
    }
    defer c.Close()

    names := os.Args[1:]
    results := make([]string, len(names))
    var wg sync.WaitGroup
    start := time.Now()
    for i, name := range names {
        wg.Add(1)
        go func() {
            defer wg.Done()
            t := time.Now()
            out, err := c.ResolveHostname(ctx, resolve.ResolveHostnameIn{Name: name})
            if err != nil {
                results[i] = fmt.Sprintf("%-20s error: %v", name, err)
                return
            }
            results[i] = fmt.Sprintf("%-20s %d addresses in %v", name, len(out.Addresses), time.Since(t).Round(time.Millisecond))
        }()
    }
    wg.Wait()
    for _, r := range results {
        fmt.Println(r)
    }
    fmt.Printf("%d lookups in %v\n", len(names), time.Since(start).Round(time.Millisecond))
}

How it works

  • One client, many goroutines. resolve.Client is safe for concurrent use. Each concurrent call borrows its own connection from the client's pool, and returns it when done.
  • The total is close to the slowest lookup, not the sum; that's the pool at work. With a single varlink.Conn, calls would serialize.
  • One context for all. The 15-second timeout bounds every lookup; when it expires, in-flight calls return context.DeadlineExceeded.
  • For thousands of names, bound concurrency with a semaphore; see Concurrency & pools.

See also