Golang Generics¶
In the past, I have found it very frustrating to program in Go. To me, it feels like a language that heavily prioritizes quantity over quality. It is a bit safer and definitely more performant than Python, but nil pointers, a lack of proper ADTs, no tuples, ... I digress.
However, recent (and wildly contentious) language changes have made it a lot less of a headache to use! As of Go 1.18, the language supports generics. This is an awesome feature, but it needs to be wielded carefully.
When all you have is a hammer, everything is a nail.
What are generics¶
I am not a type theorist, so take this with a lot of hand-waving. 👋 👋 👋
If you have programmed in Python or Ruby, you should be quite familiar with generic types whether or not you realize it. Any time you write a function that works on a container, you are dealing with generics. Let's look at a Python example with type annotations:
T = TypeVar('T')
def count_items(objects: Iterable[T]) -> int:
count = 0
for object in objects:
count += 1
return count
I added type annotations to highlight what is happening conceptually.
count_items doesn't care what is inside the container.
It works at a higher level than the contents of a list.
It only cares about the collection itself.
The type parameter, T, is what is known as a generic type.
"Why does this matter?", you may ask.
Let's evaluate a common situation in Go.
package main
import (
"encoding/json"
"fmt"
)
type Dog struct {
Id int `json:"id"`
Says string `json:"says"`
}
type Cat struct {
Id int `json:"id"`
Says string `json:"says"`
TimesMisbehaved int `json:"incidents"`
}
func GetDog(id int) *Dog {
fmt.Printf("Getting Dog %d...\n", id)
newDog := &Dog{}
_ = json.Unmarshal([]byte(`{"says": "woof"}`), newDog)
newDog.Id = id
return newDog
}
func GetCat(id int) *Cat {
fmt.Printf("Getting Cat %d...\n", id)
newCat := &Cat{}
_ = json.Unmarshal([]byte(`{"says": "meow", "incidents": 12}`), newCat)
newCat.Id = id
return newCat
}
func main() {
fmt.Printf("%v\n", *GetDog(42))
fmt.Printf("%v\n", *GetCat(5))
}
Wouldn't it be nice to have a single GetById function?
Both of our "getters" take an ID, unmarshall some bytes, and return an object.
Raymond Hettinger is beating a podium somewhere exclaiming, "There must be a better way!".
Any proper Gopher will look at this and shout, "Use an interface!".
Fair point, let's explore that.
Interfaces¶
Go functions that leverage an interface usually have a signature along the lines of:
type CanSayHello interface {
Hello() string
}
func SayHello(helloObj CanSayHello) string {
return helloObj.Hello()
}
We are faced with a sort of chicken and egg problem.
Ideally, we want a function that calls an endpoint and returns an object based on that endpoint.
Something along the lines of Get("cats/", id) -> *Cat.
Keep in mind that you cannot pass types in Go as variables.
I want to see if we can extract the object-specific functionality into its own function.
To do this, we can use an interface, but the function must return some kind of interface as well.
package main
import (
"encoding/json"
"fmt"
)
type Marshallable interface {
Unmarshal([]byte) Marshallable
}
type Dog struct {
Id int `json:"id"`
Says string `json:"says"`
}
type Cat struct {
Id int `json:"id"`
Says string `json:"says"`
TimesMisbehaved int `json:"incidents"`
}
func (d *Dog) Unmarshal(input []byte) Marshallable {
_ = json.Unmarshal(input, d)
return d
}
func (c *Cat) Unmarshal(input []byte) Marshallable {
_ = json.Unmarshal(input, c)
return c
}
func GetAnimal(animal Marshallable, id int) Marshallable {
fmt.Printf("Getting Animal ID %d...\n", id)
switch animal.(type) {
case *Cat:
animal.Unmarshal([]byte(`{"says": "meow", "incidents": 12}`))
case *Dog:
animal.Unmarshal([]byte(`{"says": "woof"}`))
}
return animal
}
func main() {
newCat := &Cat{}
fmt.Printf("%v\n", GetAnimal(newCat, 5))
newDog := &Dog{}
fmt.Printf("%v\n", GetAnimal(newDog, 42))
}
The maintenance burden of this version is arguably a little lower, but we still end up with a ton of boilerplate around the object handler.
The above isn't a terribly satisfying solution, either.
We could have used an empty interface as the return type for Marshallable, and we are stuck switching on the type in either case.
Type assertions are evaluated at runtime, and this leaves us wondering if there is a safer way to perform our tricks.
Can we do better? 🧐 That's the whole point of this post! Generics to the rescue! 🥳🥳🥳
Generic version¶
package main
import (
"encoding/json"
"fmt"
)
type HasEndpoint interface {
GetEndpoint() string
}
type Dog struct {
Id int `json:"id"`
Says string `json:"says"`
}
type Cat struct {
Id int `json:"id"`
Says string `json:"says"`
TimesMisbehaved int `json:"incidents"`
}
func (_ Dog) GetEndpoint() string {
return "/api/dogs/"
}
func (_ Cat) GetEndpoint() string {
return "/api/cats/"
}
// mock function emulating an API call
func callEndpoint(endpoint string) []byte {
switch endpoint {
case "/api/dogs/":
return []byte(`{"says": "meow", "incidents": 12}`)
case "/api/cats/":
return []byte(`{"says": "woof"`)
default:
return []byte(`{}`)
}
}
func GetAnimal[A HasEndpoint](id int) A {
var animal A
fmt.Printf("Getting Animal ID %d at %s...\n", id, animal.GetEndpoint())
resp := callEndpoint(animal.GetEndpoint())
_ = json.Unmarshal(resp, &animal)
return animal
}
func main() {
fmt.Printf("%v\n", GetAnimal[Cat](5))
fmt.Printf("%v\n", GetAnimal[Dog](42))
}
This version eliminates the runtime switch.
The types are checked before runtime, and Go will yell at you if you mess something up with an unsupported type.
GetAnimal is where all the magic happens.
A is a type parameter that is restricted to the HasEndpoint interface.
The client code is pretty simple, only requiring a type parameter to be passed to the function.
For a thorough overview of the syntax and inner workings, please see the "More reading" at the end of this article.
A better Netbox client¶
My use case for generics is a Golang Netbox client that isn't a pain in the behind to maintain. There are libraries based on OpenAPI or you can generate your own only to find that the schema isn't correct and complete. You end up adopting thousands of lines of code that needs to be maintained when you only use three or four endpoints in your client.
Well, three or four endpoints grows to two dozen a few months down the road, and you are back to maintenance problem. I wanted a solution that would grow with the project as needed and be easy to maintain. As new endpoints are needed, a simple PR to add new structs is all that is needed. You get the client code for free!
Better abstractions¶
Another common pattern with Netbox is that several objects have unique constraints that are not the primary key.
When you query for an object, Netbox always returns a list.
Many times, you simply want to queryWithFilter->GetUnique->Process.
This is such a common use case that SQLAlchemy has a Query.one function that returns one item or raises an error.
func (r NetboxResult[O]) First() (O, error) {
var empty O
if len(r.Results) != 1 {
return empty, fmt.Errorf("expected 1 object, got %d", len(r.Results))
}
return r.Results[0], nil
}
Alas, this also demonstrates a limitation of generics. There is no way to know what the parameterized type is at runtime without using reflection. In other words, it is difficult to return an error of the form "expected 1 Rack, got 3". That needs to be handled from the call site. This is a challenge in Python as well.
Conclusion¶
Generics feel like a welcome addition to the Go language, and I have just scratched the surface. This feature does complicate your code to some degree, so it should be used sparingly. In my use case, I have isolated the client to its own module, and callers simply need to parameterize the function call.