unsafe

Official documentation: unsafe package - unsafe - Go Packages

The unsafe standard library is an official library that enables low-level programming. The operations provided by this package can directly bypass Go's type system to read and write memory. This package may not be portable, and the official states that it is not protected by the Go 1 compatibility guidelines. Even so, unsafe is still used by a large number of projects, including official standard libraries.

TIP

The reason it's not portable is that the results of some operations depend on the operating system implementation, and different systems may have different results.

ArbitraryType

type ArbitraryType int

Arbitrary can be translated as "any," representing any type here, and is not equivalent to any. In fact, this type does not belong to the unsafe package; it appears here solely for documentation purposes.

IntegerType

type IntegerType int

IntegerType represents any integer type. In fact, this type does not belong to the unsafe package; it appears here solely for documentation purposes.

There's no need to pay too much attention to these two types above; they are merely representatives. When using unsafe package functions, the editor will even prompt you that the type doesn't match. Their actual type is the specific type you pass in.

Sizeof

func Sizeof(x ArbitraryType) uintptr

Returns the size of variable x in bytes, excluding the size of its referenced content. For example:

func main() {
  var ints byte = 1
  fmt.Println(unsafe.Sizeof(ints))

  var floats float32 = 1.0
  fmt.Println(unsafe.Sizeof(floats))

  var complexs complex128 = 1 + 2i
  fmt.Println(unsafe.Sizeof(complexs))

  var slice []int = make([]int, 100)
  fmt.Println(unsafe.Sizeof(slice))

  var mp map[string]int = make(map[string]int, 0)
  fmt.Println(unsafe.Sizeof(mp))

  type person struct {
    name string
    age  int
  }
  fmt.Println(unsafe.Sizeof(person{}))

  type man struct {
    name string
  }
  fmt.Println(unsafe.Sizeof(man{}))
}

1
4
16
24
8
24
16

Offsetof

func Offsetof(x ArbitraryType) uintptr

This function is used to represent the field offset within a struct, so x must be a struct field. In other words, the return value is the number of bytes between the starting address of the struct and the starting address of the field. For example:

func main() {
   type person struct {
      name string
      age  int
   }
   p := person{
      name: "aa",
      age:  11,
   }
   fmt.Println(unsafe.Sizeof(p))
   fmt.Println(unsafe.Offsetof(p.name))
   fmt.Println(unsafe.Sizeof(p.name))
   fmt.Println(unsafe.Offsetof(p.age))
   fmt.Println(unsafe.Sizeof(p.age))
}

24
0
16
16
8

Alignof

If you don't understand what memory alignment is, you can go to: Go Language Memory Alignment Explained

func Alignof(x ArbitraryType) uintptr

The alignment size is usually the minimum of the computer word length in bytes and Sizeof. For example, on an amd64 machine, the word length is 64 bits, which is 8 bytes. For example:

func main() {
  type person struct {
    name string
    age  int32
  }
  p := person{
    name: "aa",
    age:  11,
  }
  fmt.Println(unsafe.Alignof(p), unsafe.Sizeof(p))
  fmt.Println(unsafe.Alignof(p.name), unsafe.Sizeof(p.name))
  fmt.Println(unsafe.Alignof(p.age), unsafe.Sizeof(p.age))
}

8 24
8 16
4 4

Pointer

type Pointer *ArbitraryType

Pointer is a "pointer" that can point to any type, with the type *ArbitraryType. This type must be used in combination with uintptr to truly unleash the power of the unsafe package. In the official documentation description, unsafe.Pointer type can perform four special operations:

• Any type of pointer can be converted to unsafe.Pointer
• unsafe.Pointer can be converted to any type of pointer
• uintptr can be converted to unsafe.Pointer
• unsafe.Pointer can be converted to uintptr

These four special operations form the cornerstone of the entire unsafe package, and it is precisely these operations that enable writing code that can ignore the type system and directly read and write memory. It is recommended to be especially careful when using them.

TIP

unsafe.Pointer cannot be dereferenced, and similarly, its address cannot be taken.

(1) Convert *T1 to unsafe.Pointer and then to *T2

Given types *T1 and *T2, assuming T2 is not larger than T1 and both have equivalent memory layouts, it allows converting data of type T2 to T1. For example:

func main() {
   fmt.Println(Float64bits(12.3))
   fmt.Println(Float64frombits(Float64bits(12.3)))
}

func Float64bits(f float64) uint64 {
   return *(*uint64)(unsafe.Pointer(&f))
}

func Float64frombits(b uint64) float64 {
   return *(*float64)(unsafe.Pointer(&b))
}

4623113902481840538
12.3

These two functions are actually from the math package. The type changes during the process are as follows:

float64 -> *float64 -> unsafe.Pointer ->  *uint64 -> uint64 -> *uint64 -> unsafe.Pointer -> *float64 -> float64

(2) Convert unsafe.Pointer to uintptr

When converting unsafe.Pointer to uintptr, the address pointed to by the former becomes the value of the latter. uintptr stores an address; the difference is that the former is syntactically a pointer, a reference, while the latter is merely an integer value. For example:

func main() {
   num := 1
   fmt.Println(unsafe.Pointer(&num))
   fmt.Printf("0x%x", uintptr(unsafe.Pointer(&num)))
}

0xc00001c088
0xc00001c088

A bigger difference lies in garbage collection handling. Since unsafe.Pointer is a reference, it won't be collected when needed. The latter, merely being a value, naturally doesn't receive such special treatment. Another point to note is that when the address of the element pointed to by the pointer moves, GC will update the old address referenced by the pointer, but it won't update the value saved by uintptr. For example, the following code may cause problems:

func main() {
   num := 16
   address := uintptr(unsafe.Pointer(&num))
   np := (*int64)(unsafe.Pointer(address))
   fmt.Println(*np)
}

In some cases, after GC moves the variable, the address pointed to by address becomes invalid. Using that value to create a pointer will then cause a panic:

panic: runtime error: invalid memory address or nil pointer dereference

Therefore, it's not recommended to save the value after converting Pointer to uintptr.

(3) Convert uintptr to unsafe.Pointer

The following method can obtain a pointer from uintptr. As long as the pointer is valid, the invalid address situation from example two won't occur. Pointer and type pointers themselves don't support pointer arithmetic, but uintptr is just an integer value and can perform mathematical operations. Performing mathematical operations on uintptr and then converting to Pointer completes pointer arithmetic.

p = unsafe.Pointer(uintptr(p) + offset)

This way, you can access some internal elements of certain types through just one pointer, such as arrays and structs, regardless of whether their internal elements are exposed externally. For example:

func main() {
  type person struct {
    name string
    age  int32
  }
  p := &person{"jack", 18}
  pp := unsafe.Pointer(p)
  fmt.Println(*(*string)(unsafe.Pointer(uintptr(pp) + unsafe.Offsetof(p.name))))
  fmt.Println(*(*int32)(unsafe.Pointer(uintptr(pp) + unsafe.Offsetof(p.age))))

  s := []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
  ps := unsafe.Pointer(&s[0])
  fmt.Println(*(*int)(unsafe.Pointer(uintptr(ps) + 8)))
  fmt.Println(*(*int)(unsafe.Pointer(uintptr(ps) + 16)))
}

jack
18
2

Add

func Add(ptr Pointer, len IntegerType) Pointer

Add returns a Pointer updated with offset len, equivalent to Pointer(uintptr(ptr) + uintptr(len)):

Pointer(uintptr(ptr) + uintptr(len))

For example:

func main() {
   s := []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
   ps := unsafe.Pointer(&s[0])
   fmt.Println(*(*int)(unsafe.Add(ps, 8)))
   fmt.Println(*(*int)(unsafe.Add(ps, 16)))
}

2
3

SliceData

func SliceData(slice []ArbitraryType) *ArbitraryType

This function receives a slice and returns the starting address of its underlying array. Without using SliceData, you can only get the address of the underlying array by taking the pointer of its first element, as follows:

func main() {
  nums := []int{1, 2, 3, 4}
  for p, i := unsafe.Pointer(&nums[0]), 0; i < len(nums); p, i = unsafe.Add(p, unsafe.Sizeof(nums[0])), i+1 {
    num := *(*int)(p)
    fmt.Println(num)
  }
}

Of course, you can also get it through the reflect.SliceHeader type, but it has been deprecated since version 1.20. SliceData is designed to replace it. Here's an example using SliceData:

func main() {
  nums := []int{1, 2, 3, 4}
  for p, i := unsafe.Pointer(unsafe.SliceData(nums)), 0; i < len(nums); p, i = unsafe.Add(p, unsafe.Sizeof(int(0))), i+1 {
    num := *(*int)(p)
    fmt.Println(num)
  }
}

Slice

func Slice(ptr *ArbitraryType, len IntegerType) []ArbitraryType

The Slice function receives a pointer and a length offset. It returns the slice expression form of that memory segment. No memory copying is involved in the process. Modifying the slice will directly affect the data at that address, and vice versa. It's usually used in conjunction with SliceData.

func main() {
  nums := []int{1, 2, 3, 4}
  numsRef1 := unsafe.Slice(unsafe.SliceData(nums), len(nums))
  numsRef1[0] = 2
  fmt.Println(nums)
}

[2 2 3 4]

Modifying the data of the numsRef1 slice will cause the data of nums to change as well.

StringData

func StringData(str string) *byte

Same as the SliceData function, but because the demand for converting strings to byte slices is frequent, it's separated out. Here's a usage example:

func main() {
  str := "hello,world!"
  for ptr, i := unsafe.Pointer(unsafe.StringData(str)), 0; i < len(str); ptr, i = unsafe.Add(ptr, unsafe.Sizeof(byte(0))), i+1 {
    char := *(*byte)(ptr)
    fmt.Println(string(char))
  }
}

Since string literals are stored in the read-only segment of the process, if you try to modify the underlying data of the string here, the program will crash with a fatal error. However, for string variables stored on the heap or stack, modifying their underlying data at runtime is entirely feasible.

String

func String(ptr *byte, len IntegerType) string

Same as the Slice function, it receives a byte type pointer and its length offset, returning its string expression form without involving memory copying. Here's an example of converting a byte slice to a string:

func main() {
  bytes := []byte("hello world")
  str := unsafe.String(unsafe.SliceData(bytes), len(bytes))
  fmt.Println(str)
}

StringData and String don't involve memory copying during the conversion process between strings and byte slices, so their performance is better than direct type conversion. However, they're only suitable for read-only situations. If you plan to modify data, it's best not to use them.