Why appending to slice in Golang is dangerous? Common Slice Gotchas

There is a common misconception about how slices work in Golang. That leads to unexpected program behaviour which is suprising to many developers. In this blog post I will highlight the mechanics of a slice in Golang

Golang is full of surprises and uncommon (in reference to other programming languages) constructs and behaviours. One of them may be somehow misleading claim (by Go authors) that everything in Golang is passed by value... unless it is a pointer... or unless its a channel... and a map. Well there are couple of other exceptions to that rule, but one of least mentioned is that slices behave like pointer until they stop behaving and act as copied value.

Is it twisted enough? Doesn't make any sense?

Well I hope you are confused, because I was when I first encountered this behaviour. Lets look at the code to see what is going on.

arr := [...]int{1, 2, 3, 4, 5}
fmt.Println(arr) // [1 2 3 4 5]

There is no confusion here. We create a 5 element array on integers and print it. Let's go further

sl := arr[2:4]
fmt.Println(sl) // [3 4]

All good, no surprises here, we've created a slice from `arr` variable from index no. 2 until index no. 4.

sl[0] = 0
fmt.Println(sl) // [0 4]

We did a slice modification: set slice element at index 0 to value 0. Lets look at our original array:

fmt.Println(arr) // [1 2 0 4 5]

This is interesting, our original array which we took a slice from also change it's value...

It's because slices are references to underlying arrays and their name: slice is very much descriptive about their internals. When we create a slice from an array the Go runtime creates a struct which points to an array with additional fields: len and cap. "Len" stands for length of a slice which is a number of elements it currently holding and "cap" is a capacity that when reached (by len) will results in copying an array to new place in memory.

Take a look at the structure that describes slices in runtime.

 // https://golang.org/src/runtime/slice.go
type slice struct {
    array unsafe.Pointer
    len   int
    cap   int

So let's move on with our example, because there is one more "gotcha" about Golang slices. We now know that slices are references to underlying arrays. When we take a slice of an array, the slice struct is created which points to an array, but what happens when we start appending items to a slice (since it's totally legal in Golang - it's slice characteristics)? Lets take a look:

sl = append(sl, 9)
fmt.Println(sl) // [0 4 9]
fmt.Println(arr) // [1 2 0 4 9]

See what happens? We appened another item to a slice and it also changed our original array. This is another characteristics, until reaching a starting capacity, appending a slice will change items in original array. Let's go further and add another element:

sl = append(sl, 9)
fmt.Println(sl) // [0 4 9 9]
fmt.Println(arr) // [1 2 0 4 9]

After adding another element, it was added to a slice but not to original array, why? The answer is simple, remember about characteristics of an array in Go? It says that arrays are fixed-size lists which means that they cannot be resized. On the other hand we have a slice which is totally fine with appending. How is that possible?

When reaching capacity, slice is copying its "array slice" into new fragment of memory to fulfill this behaviour. It's capacity is doubled an we can start appending again.

When copying occurs the original array is left intact, but the previous modifications did to it stay in place. Newly allocated array for the slice is now serving as underlying array for slice. Lets go back and check those modifications with additional descriptions. To prove that the array was copied I will print an element address to show its changed

fmt.Printf("len: %v, cap: %v, %v\n") // len: 2 cap: 3 [0 4]

sl = append(sl, 9)

fmt.Printf("len: %v, cap: %v, %v\n") // len: 3 cap: 3 [0 4 9]
fmt.Printf("%p\n", &sl[0]) // 0x45e008 <---

sl = append(sl, 9)

fmt.Printf("len: %v, cap: %v, %v\n") // len: 4 cap: 8 [0 4 9 9]
fmt.Printf("%p\n", &sl[0]) // 0x45e080 <--- address of [0] element changed

Do you see the effects now? After second append the slice exceeded its capacity and must been copied into new place, this applies to all item in slice. On the other hand original array was left intact (besided previous modifications).

Thank you for your time spent reading the article and I hope it helped in understanding how slices work in Golang.

Code Fibers provides services for Golang development, consulting and training, I invite you to visit our website and contact also get to know how we work and what is our exprience.

These posts might be interesting for you:

  1. Why Golang Nil Is Not Always Nil? Nil Explained
  2. Handling Promise rejections in Express.js (Node.js) with ease
Author: Peter

I'm a backend programmer for over 10 years now, have hands on experience with Golang and Node.js as well as other technologies, DevOps and Architecture. I share my thoughts and knowledge on this blog.