defer

defer is a frequently used keyword in Go's daily development. It executes functions associated with defer in a LIFO (Last In First Out) manner. We often use this mechanism for resource release operations, such as file closing.

fd, err := os.Open("/dev/stdin")
if err != nil{
    return err
}
defer fd.Close()
...

Given how frequently this keyword appears, it's necessary for us to understand the underlying structure behind it.

Structure

The defer keyword corresponds to the runtime._defer structure. Its structure is not complex:

type _defer struct {
  started bool
  heap    bool
  openDefer bool
  sp        uintptr // sp at time of defer
  pc        uintptr // pc at time of defer
  fn        func()  // can be nil for open-coded defers
  _panic    *_panic // panic that is running defer
  link      *_defer // next defer on G; can point to either heap or stack!
  fd   unsafe.Pointer // funcdata for the function associated with the frame
  varp uintptr        // value of varp for the stack frame
  framepc uintptr
}

The fn field is the function corresponding to the defer keyword, link represents the next linked defer, and sp and pc record the caller's function information, used to determine which function the defer belongs to. defer exists as a linked list at runtime, with the head of the list on goroutine G, so defer is actually directly associated with the goroutine.

type g struct {
    ...
  _panic    *_panic // innermost panic - offset known to liblink
  _defer    *_defer // innermost defer
    ...
}

When a goroutine executes a function, it sequentially adds the defer from the function to the head of the linked list.

defer fn1()
defer fn2()
defer fn3()

The code above corresponds to this diagram:

Besides goroutines, P also has a certain relationship with defer. In P's structure, there's a deferpool field, as shown below.

type p struct {
  ...
  deferpool    []*_defer // pool of available defer structs (see panic.go)
  deferpoolbuf [32]*_defer
    ...
}

The deferpool contains pre-allocated defer structures, used to allocate new defer structures for goroutines G associated with P, which can reduce overhead.

Allocation

In terms of syntax, when using the defer keyword, the compiler translates it into a call to the runtime.deferproc function. For example, if the Go code is written like this:

defer fn1(x, y)

The actual compiled code looks like this:

deferproc(func(){
  fn1(x, y)
})

So in reality, the function passed to defer has no parameters and no return values. The deferproc function code is as follows:

func deferproc(fn func()) {
  gp := getg()
  d := newdefer()
  d.link = gp._defer
  gp._defer = d
  d.fn = fn
  d.pc = getcallerpc()
  d.sp = getcallersp()
  return0()
}

This function is responsible for creating a defer structure and adding it to the head of goroutine G's linked list. The runtime.newdefer function attempts to get a pre-allocated defer structure from the deferpool in P.

if len(pp.deferpool) == 0 && sched.deferpool != nil {
    lock(&sched.deferlock)
    for len(pp.deferpool) < cap(pp.deferpool)/2 && sched.deferpool != nil {
        d := sched.deferpool
        sched.deferpool = d.link
        d.link = nil
        pp.deferpool = append(pp.deferpool, d)
    }
    unlock(&sched.deferlock)
}

It first fills half of the local deferpool from the global sched.deferpool, then tries to get from P's deferpool.

if n := len(pp.deferpool); n > 0 {
    d = pp.deferpool[n-1]
    pp.deferpool[n-1] = nil
    pp.deferpool = pp.deferpool[:n-1]
}

if d == nil {
    // Allocate new defer.
    d = new(_defer)
}
d.heap = true

Only when nothing can be found will it use manual allocation. Finally, you can see this code:

d.heap = true

This indicates that defer is allocated on the heap. Correspondingly, when it's false, it's allocated on the stack. Memory allocated on the stack is automatically reclaimed when returning, and its memory management efficiency is higher than on the heap. The factor that determines whether to allocate on the stack is the number of loop levels. This logic can be traced to the escape.goDeferStmt method in cmd/compile/ssagen:

func (e *escape) goDeferStmt(n *ir.GoDeferStmt) {
  ...
  if n.Op() == ir.ODEFER && e.loopDepth == 1 {
    k = e.later(e.discardHole())
    n.SetEsc(ir.EscNever)
  }
    ...
}

e.loopDepth represents the number of loop levels for the current statement. If the current defer statement is not in a loop, it will be allocated on the stack.

case ir.ODEFER:
    n := n.(*ir.GoDeferStmt)
    if s.hasOpenDefers {
      s.openDeferRecord(n.Call.(*ir.CallExpr))
    } else {
      d := callDefer
      if n.Esc() == ir.EscNever {
        d = callDeferStack
      }
      s.callResult(n.Call.(*ir.CallExpr), d)
    }

If allocated on the stack, the defer structure is created directly on the stack, and ultimately the runtime.deferprocStack function completes the defer structure creation.

if k == callDeferStack {
    // Make a defer struct d on the stack.
    if stksize != 0 {
      s.Fatalf("deferprocStack with non-zero stack size %d: %v", stksize, n)
    }

    t := deferstruct()
    ...
    // Call runtime.deferprocStack with pointer to _defer record.
    ACArgs = append(ACArgs, types.Types[types.TUINTPTR])
    aux := ssa.StaticAuxCall(ir.Syms.DeferprocStack, s.f.ABIDefault.ABIAnalyzeTypes(nil, ACArgs, ACResults))
    callArgs = append(callArgs, addr, s.mem())
    call = s.newValue0A(ssa.OpStaticLECall, aux.LateExpansionResultType(), aux)
    call.AddArgs(callArgs...)
    call.AuxInt = int64(types.PtrSize) // deferprocStack takes a *_defer arg

The signature of the deferprocStack function is as follows:

func deferprocStack(d *_defer)

The specific creation logic is not much different from deferproc. The main difference is that when allocating on the stack, the defer structure comes from a directly created struct, while the defer allocated on the heap comes from the new function.

Execution

When a function is about to return or when a panic occurs, it enters the runtime.deferreturn function, which is responsible for taking defer from the goroutine's linked list and executing it.

func deferreturn() {
  gp := getg()
  for {
    d := gp._defer
    sp := getcallersp()
    if d.sp != sp {
      return
    }
    fn := d.fn
    d.fn = nil
    gp._defer = d.link
    freedefer(d)
    fn()
  }
}

First, it uses getcallersp() to get the current function's stack frame and compares it with the sp in the defer structure to determine if the defer belongs to the current function. Then it takes the defer structure from the head of the linked list, executes the next defer using gp._defer = d.link, releases the defer structure back to the pool via the runtuime.freedefer function, and finally calls fn to execute. This continues in a loop until all defer belonging to the current function have been executed.

Open-Coded Defer

Using defer is not without cost. Although it provides convenience in syntax, it's not a direct function call and goes through a series of processes, so it does cause performance overhead. Therefore, Go officials later designed an optimization method—open-coded defer. It's an optimization for defer, originally named "open-coded", which is translated as "开放编码" in Chinese. Here "open" means to expand, i.e., to expand the defer function code into the current function code, similar to function inlining. This optimization method has the following restrictions:

1. The number of defer in a function cannot exceed 8
2. The product of the number of defer and return cannot exceed 15
3. defer cannot appear in a loop
4. Compiler optimization is not disabled
5. No manual call to os.Exit()
6. No need to copy parameters from the heap

This judgment logic can be traced to the following part of the cmd/compile/ssagen.buildssa function:

s.hasOpenDefers = base.Flag.N == 0 && s.hasdefer && !s.curfn.OpenCodedDeferDisallowed()
if s.hasOpenDefers && len(s.curfn.Exit) > 0 {
    s.hasOpenDefers = false
}
if s.hasOpenDefers {
    for _, f := range s.curfn.Type().Results().FieldSlice() {
        if !f.Nname.(*ir.Name).OnStack() {
            s.hasOpenDefers = false
            break
        }
    }
}
if s.hasOpenDefers && s.curfn.NumReturns*s.curfn.NumDefers > 15 {
    s.hasOpenDefers = false
}

Then Go creates an 8-bit integer variable deferBits in the current function to serve as a bitmap for marking defer. Each bit marks one defer. An 8-bit integer uint8 can represent at most 8 defers. If the corresponding bit is 1, the corresponding open-coded optimized defer will be executed when the function returns.