Slices in go are not same as slices in other programming language i.e Python. Assigning one slice to another only makes a shallow copy of the slice and should be used cautiously if you want to create a new slice from existing slice.

Introduction

Golang is undoubtly one of the popular language to day. It is easier to undersntad and get started with but however, it has its intricacies. I was working on a project and came across this behaviour.

When working with programming language like python the manipulation of slices are simple and expectable as its a high level language. You create an array, slice then, reslice them, concatinate them and the expected answer in your head matches the answer in the terminal output.

Take an example of the below python code

#!/usr/bin/env python3

a = [0,1,2,3,4]
print(a)

array1 = a[:1]
print(array1)


array2 = a[2:]
print(array2)

array1 = array1 + array2
array1 = array1 + array2

print(array1)
print(a)

If you run above code, you will get output like this

➜  python3 main.py
[0, 1, 2, 3]
[0]
[2, 3]
[0, 2, 3, 2, 3]
[0, 1, 2, 3]

let me explain what is happening above code and its output if you are new to python

We created an array a = [0,1,2,3,4]
Then we slice ( take out ) value first value from original array a and store it in another array called array1. After this array1 has value of [0]
Then we again we slice ( take out ) second to last value from original array a and store it in another array called array2. After this array1 has value of [2,3]
We then concatinate array1 and array2 and store again into array1 resulting in our value of array1 to be [0,2,3]
We again concatinate array1 and array2 and store again into array1. Since value of array1 has been changed from last operation which is [0,2,3],. new value of array1 becomes [0,2,3,2,3] which seems logical as we conctinated [0,2,3] and [2,3]
Then we print final value of array1 which is [0,2,3,2,3] and also print original array a value which is [0, 1, 2, 3]

The thing happened above seems logical and understandable.

Now i had similar requirement where i had to manipulate slices and create new ones using Go. Thinking that it would be similar to other language, i followed the similar approach.

package main

import (
	"fmt"
)

func main(){
	slice := []int{0,1,2,3}
	fmt.Println(slice)

	newslice1 := slice[:1]
	fmt.Println(newslice1)

	newslice2 := slice[2:]
	fmt.Println(newslice2)

	newslice1 = append(newslice1, newslice2...)
	fmt.Println(newslice1)

	newslice1 = append(newslice1, newslice2...)
	fmt.Println(newslice1)

	fmt.Println(slice)

}

looking surfacely at the code you can see that, its just a simple array manipulation using slices. Same as above python, there is a array, you slice it, reslice it and concatinate by appending to original slice.

Now Take a moment and think what would be the output of the code. You might think its the same as above python code, which is

➜ go run main.go
[0, 1, 2, 3]
[0]
[2, 3]
[0, 2, 3, 2, 3]
[0, 1, 2, 3]

But wait, there is a gotcha. The output is completely different to what we expect and the actual output when you run the above code, it looks like this.

➜ go run main.go
[0 1 2 3]
[0]
[2 3]
[0 2 3]
[0 2 3 3 3]
[0 2 3 3]

If you noticed the output, it’s different to what we expect in our head and different to same implementation in python.

The puzzling bit is even our original slice has been changed when in fact, there was no operation done on the original slice.

Problem identification

After going through few rabbit holes, i found the answer. Turns out slice is go is not same as slice in python.

When you create a slice, its not actually storing any data, but instead it is a descriptor of an underlying array. Slice in go could be better called slice header or slice variable. slice variable is a data structure describing continuous section of an array stored separately from the slice variables itself. Slice that we created above is not an actually slice, but rather a description of an array which could be represented as ( theoritically )

type slice struct {
  Length  int
  Capacity int
  firstElement *int ( or pointer to underlying array )
}

We know that each slice, has three attribute, length, capacity, and pointer to the data.

When we create a new slice the original using newSlice1 := slice[:1], it creates a new data structure for which points to original slice could be represented as ( theoritically )

type newSlice1 struct {
  Length  1
  Capacity 4
  firstElement &underlyingArray[0]
}

and again When we create a new slice the original using newSlice2 := slice[2:], it creates a new data structure for which points to original slice could be represented as ( theoritically )

type newSlice2 struct {
  Length  2
  Capacity 4
  firstElement &underlyingArray[2]
}

In both of the case new slice created point to the same original slice and share values with original slice, so when newSlice1 is created its value is [0] and newSlice2 value is [2,3]. For both array, length is changing as we sliced the slice, but capacity remains the same i.e 4. For newSlice1 one slot is used and 3 remains where are for newSlice2 two slots are used and 2 remains.

After first append is called, the value of newSlice1 is changed to [0,2,3] with one free slot and length 3. Its data structure could be represented as

type newSlice1 struct {
  Length  3
  Capacity 4
  firstElement &array[0]
}

Since newSlice1 and original array slice share the same elements of array i.e. values as both are pointing to the underlying array, value of original slice is changed to [0,2,3,3] becase we had overridden values from underlying arrays from index 0 to 3 which ultimately changed the original slice.

This is why after the append to newSlice1 value is changed to [0 2 3 3 3] instead of [0, 2, 3, 2, 3].

Before appending

before_append

Afer appending

Here you can see the value of underlying array of the slice gets changed after the append.

after_append

This can also be verified by checking the memory address of the underlying array. Since both of the slice point to same underlying array memory address, any change in one affects the other as well.

➜  go run main.go
Memory location of first element of slice 0xc00001c0a0 <== Same
Memory location of first element of newslice1 0xc00001c0a0 <== Same
Memory location of first element of newslice2 0xc00001c0b0
Memory location of first element of newslice1 0xc00001c0a0
Memory location of first element of newslice1 0xc000018100
Memory location of first element of slice 0xc00001c0a0

In short, go append changes the underlying array of the slice when new slices are created and modified from it.

Solution

Solution to this would be to only use append when you actually want to append value to the already existing slice and not create a new one.

	newArray1 = append(newArray1, "items")

If we want to create new slice by re-slicing original array or slice then we should use copy function as shown below. copy performs a deep copy of the slice.

package main

import (
	"fmt"
)

func main(){
	slice := []int{0,1,2,3}
	fmt.Println(slice)

	newslice1 := make([]int, len(slice))
	copy(newslice1, slice)

	newslice1 = newslice1[:1]
	fmt.Println(newslice1)

	newslice2:= make([]int, len(slice))
	copy(newslice2, slice)

	newslice2 = newslice2[2:]
	fmt.Println(newslice2)

	newslice1 = append(newslice1, newslice2...)
	newslice1 = append(newslice1, newslice2...)

	fmt.Println(newslice1)
	fmt.Println(slice)

}

Now when you run the following code you will see the output as expected and same as output from python code.

➜ go run main.go
[0 1 2 3]
[0]
[2 3]
[0 2 3 2 3]
[0 1 2 3]

This can again be verified by checking the memory address of the underlying array

➜ go run main.go
Memory location of first element of slice 0xc00001c0a0 <== Not same
Memory location of first element of newslice1 0xc00001c0c0 <== Not same
Memory location of first element of newslice2 0xc00001c0f0
Memory location of first element of newslice1 0xc000018100
Memory location of first element of slice 0xc00001c0a0

Conclusion

Such intricacies are difficult to detect and even difficult to debug with working with go in production. I came across this when i am working on a voluneteering project for a company. Hope this article was able to clear out some caveats with slices in Go.

Please feel free to reach out to me if you have any questions.

Learn how appending slices are not same as other programming language ?

Introduction

Problem identification

Solution

Conclusion