<BlogImage
 src="/go-sdk/flipt-sdk.png"
 alt="Flipt's Go SDK"
 width={2400}
 height={1350}
>
 
 Photo by [Lenny Kuhne](https://unsplash.com/@lennykuhne) on Unsplash
 
</BlogImage>

# Generating the Flipt Go SDK

This blog post describes the journey we took to build our new Go SDK, and how we developed our `protoc` plugin in Go to achieve that. It first explores our motivations and then gets into some of the details of how we built it and the tools that go into the automation around it.

We're releasing the new Go SDK today! You should be able to fetch it using `go get go.flipt.io/flipt/sdk/go`.

## Getting Serious About Clients

Recently, we decided to revise and improve our API client offering, starting with our support for Go.

Go is the core language used to develop Flipt's backend service. It is something the team has a long history with, so this was an obvious place for us to start.

With the [recent addition of different authentication methods](/blog/enhancing-authentication#kubernetes-service-account-authentication) for Flipt, we wondered how we could better support integrating these authentication mechanisms into clients.

Our [documentation](/docs/authentication/using-tokens) explains how to manually wire static client tokens into a gRPC or HTTP client, however, we now support authenticating using Kubernetes service-account tokens. Doing so is a more involved process, with an extra API handshake in the middle, so this is something we ideally want our clients to do for you with minimal configuration.

## Status Quo

Flipt’s core service and APIs are defined using [gRPC](https://grpc.io/) and protobuf. This allows us to define our APIs as a set of RPCs and messages in the [proto](https://protobuf.dev/programming-guides/proto3/) language and then use a set of `protoc` tools and plugins to scaffold domain models, servers and clients in different languages. This is a superpower for us. It allows us to reach other programming ecosystems and languages without prior knowledge or experience in them, opening Flipt up to a wider audience.

However, gRPC alone is not for everyone or every ecosystem. While it has a big reach, not every language is catered to equally. In particular, the browser hasn’t had the best support so we also wanted an equivalent HTTP API. The API should also be understandable by developers, allowing them to write their clients using an HTTP library in their language of choice.

This is where [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) comes into the mix. Flipt has used this project since the beginning to present an equivalent REST-like API using JSON as the message serialization format. The grpc-gateway tooling supports server generation (not client) in Go.

This is where we were until very recently. We have a gRPC protobuf generated client for the Go community, with no additional bells or whistles, just a thin shim around the Flipt gRPC endpoints and no support for invoking the HTTP API in Go. While we’re uncertain of the use cases, it seemed a shame to mandate gRPC when the [HTTP API](/docs/reference/overview) is also available.

## Our Goals

- A single consistent SDK API, which abstracts the details of the transport from the caller. While we want our users to have the choice, we don’t think the transport details matter when it comes to consuming the client's functionality in your application code.
- Two implementations of the transport abstraction, one for gRPC and one for HTTP.
- To generate as much of this as possible from the original `proto` definitions. We invested in gRPC and protobuf for the tooling and automation. Having our SDK advance with these definitions will keep us up to date.
- Develop an abstraction around authenticating our SDK, which supports consumers’ needs to access Flipt’s APIs with minimal configuration.

<BlogImage
 src="/go-sdk/Flipt_SDK_Architecture.svg"
 alt="Flipt SDK Architecture"
 width={729}
 height={339}
 caption="Flipt SDK Architecture"
/>

The entry point to our new client is the `SDK` type. This type encapsulates the three sections of the Flipt RPC APIs known as `Flipt` (core API), `Auth` (authentication-related APIs) and `Meta` (server metadata APIs). All of these types are generated from the original `proto` definitions.

These SDK types are (mostly) abstracted away from the transport details. The specifics are hidden behind a set of `Transport` Go interfaces. Each of these interfaces has an equivalent implementation for both `grpc` and `http`.

```go
type Transport interface {
	AuthClient() AuthClient
	FliptClient() flipt.FliptClient
	MetaClient() meta.MetadataServiceClient
}
```

For the most part, the `grpc` transport implementation is identical to that of the `grpc-go` generated client. The packages contain some thin wrapping code to glue our SDK together. The crux of the code has been implemented and the transport interfaces are very close to these concrete `grpc` implementations. This was an intentional decision to minimize effort and generated code.

<StarCTA />

The `http` implementation on the other hand required some more work. For a start, `grpc-gateway` does not ship a generated HTTP Go client, so we had to implement this ourselves. To invoke the API generated by grpc-gateway, we need to produce code that can adapt our RPC messages into the appropriate HTTP requests. We also need to provide a Go implementation that conforms to the Transport interface definitions derived from gRPC.

The grpc-gateway ecosystem has a collection of additional annotations and metadata which describe how the RPC methods should be represented as HTTP requests, for example:

```yaml
http:
 rules:
	# ...
	- selector: flipt.Flipt.CreateVariant
	 post: /api/v1/flags/{flag_key}/variants
	 body: "*"
```

Our custom `protoc` plugin parses this additional metadata and uses it to generate Go code which can produce an equivalent HTTP request.

```go
func (x *FliptClient) CreateVariant(ctx context.Context, v *flipt.CreateVariantRequest, _ ...grpc.CallOption) (*flipt.Variant, error) {
	var body io.Reader
	var values url.Values
	reqData, err := protojson.Marshal(v)
	if err != nil {
		return nil, err
	}
	body = bytes.NewReader(reqData)
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, x.addr+fmt.Sprintf("/api/v1/flags/%v/variants", v.FlagKey), body)
	if err != nil {
		return nil, err
	}
	req.URL.RawQuery = values.Encode()
	resp, err := x.client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	var output flipt.Variant
	respData, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	if err := checkResponse(resp, respData); err != nil {
		return nil, err
	}
	if err := protojson.Unmarshal(respData, &output); err != nil {
		return nil, err
	}
	return &output, nil
}
```

I don’t suspect we have managed to support the full breadth of protobuf, grpc and grpc-gateway features. However, we believe we have implemented enough to cover the functionality Flipt takes advantage of.

## How does one write a protobuf Go code generator?

[https://pkg.go.dev/google.golang.org/genproto](https://pkg.go.dev/google.golang.org/genproto) is a Go package designed to aid in the development of `protoc` plugins. It is used by `grpc-gateway` and `grpc-go` in order to parse and generate their respective client and server implementations.

<BlogImage
 src="/go-sdk/protoc.svg"
 alt="Protoc"
 width={561}
 height={157}
/>

`protoc` is the entry point to all things protobuf generation. It parses and validates your protobuf files and invokes any configured plugins with a request to generate code on `stdin`.

[https://pkg.go.dev/google.golang.org/protobuf/types/pluginpb#CodeGeneratorRequest](https://pkg.go.dev/google.golang.org/protobuf/types/pluginpb#CodeGeneratorRequest)

The message you are provided is a protobuf serialized structure itself. Specifically, it is a `pluginpb.CodeGeneratorRequest`. This contains the parsed definitions of your protobuf files collected by `protoc`.

It is up to the plugin to interpret this request and produce a response. The response is written to `stdout` as a protobuf encoded `pluginpb.CodeGeneratorResponse`. The response contains all the files you have produced and where they should end up (their file names).

The `protogen` package provides a framework which hides much of the hairy details of writing a `protoc` plugin. Simplifying the wiring and providing useful utilities specifically for generating Go code.

There is an excellent [blog post](https://rotemtam.com/2021/03/22/creating-a-protoc-plugin-to-gen-go-code/) by Rotem Tamir, which laid the foundational knowledge we needed to get started on our custom SDK wrapper and HTTP client. If building something with `protogen` interests you, then I highly recommend reading this first.

### Some `protogen` pointers

I won’t repeat all the wonderful tidbits and details in that blog post, however, I do have a couple of extra pointers that I discovered along the way while developing our SDK generators.

Firstly, we use `proto` version `3` and in particular, we use the `optional` keyword on some message fields. This happens to be a feature of protobuf which was introduced after the initial release of version `3`. If you attempt to use `protogen` on `proto` files using the `optional` keyword without first enabling it as a “supported feature” in your plugin, then you will be confronted with a warning each time your plugin is invoked:

```go
Warning: plugin "go-flipt-sdk" does not support required features.
 Feature "proto3 optional" is required by 1 file(s):
 auth/auth.proto
```

It was a little unclear at first, but the trick is to set the optional feature *bit* on the `*protogen.Plugin.SupportedFeatures`:

```go
protogen.Options{}.Run(func(gen *protogen.Plugin) error {
		gen.SupportedFeatures |= uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
		// ....
}
```

`SupportedFeatures` is a bitmask and the nice, human-readable names for the feature bits can be found at [google.golang.org/protobuf/types/pluginpb](http://google.golang.org/protobuf/types/pluginpb).

Secondly, managing imports in your generated Go files can initially be unclear. It turns out that `protogen` has a really nice feature, which does a lot of heavy lifting with regard to managing the naming and qualification of those names in your generated output:

```go
for _, f := range gen.Files {
 if !f.Generate {
 continue
 }

 filename := string(file.GoPackageName) + ".sdk.gen.go"
 g := gen.NewGeneratedFile(filename, myTargetPackageName)

 // QualifiedGoIdent will handle adding the GoImportPath
 // to the target GeneratedFile (g) in the `import` declaration
 // at the top. It even handles when you reference two packages
 // with the same name by adding a numeric suffix to distinguish them.
 httpClient := g.QualifiedGoIdent(protogen.GoIdent{
 GoImportPath: protogen.GoImportPath("net/http"),
 GoName: "Client",
 })

 g.P("type MyClient struct {")
 g.P("client *", httpClient)
 g.P("}")
}
```

`protogen.GeneratedFile.QualifiedGoIdent` ensures that the path you’re referencing is correctly imported into the top of the resulting generated Go file. It also handles any imports with duplicate names by adding a numeric suffix (e.g. importing both `text/template` and `html/template` would resolve to `template "text/template"` and `template1 "html/template"`). Then the resulting import path is correctly prepended onto the type when the `GoIdent` produced by this function is passed into `g.P("...", httpClient)`.

The example generator code above would produce something similar to:

```go
package client

import http "net/http"

type MyClient struct {
	client *http.Client
}
```

### Putting it altogether

So Flipt now has a single `protoc` plugin, which produces three target packages.

The plugin lives [here](https://github.com/flipt-io/flipt/tree/main/internal/cmd/protoc-gen-go-flipt-sdk) and it generates the following three packages into the Flipt repository:

- `sdk/go/http` containing `HTTP` implementations of the `sdk.Transport` interfaces.
- `sdk/go/grpc` contained `gRPC` implementations of the `sdk.Transport` interfaces.
- `sdk/go` the core of our Go SDK.

To bring all of this together, we use a tool called [buf](https://buf.build/) to orchestrate the invocations of the `protoc` toolchain. The protobuf toolchain is a fiddly beast and `buf` greatly simplifies this process for us.

We have a single addition to our `buf.gen.yaml` which drives this tool. It looks like this:

```yaml
plugins:
# ...
- name: go-flipt-sdk
 out: sdk/go
 opt:
 - paths=source_relative
 strategy: all
```

Each of the entries under `plugins` is treated as a `protoc` plugin. These will be passed on invocations of the `protoc` tool by `buf`. `protoc` knows when it sees the name `go-flipt-sdk` to prepend `protoc-gen-` onto the front and look that binary in your `PATH`. In our case, it looks for `protoc-gen-go-flipt-sdk`.

## Where next?

We’re hoping this automation keeps us on a good footing with each change to Flipt’s API, allowing us to generate a client in lockstep with the server API, which has support for all the RPC definitions we create.

Beyond just generating the client we plan to add support for the Kubernetes authentication method, reducing that down to some configuration with sensible defaults.

Finally, we have begun the work to build out a new integration test suite to ensure our SDK and Flipt are behaving as we intend. Our next blog post on this topic will explore how we’re building out a new CI pipeline using [Dagger](https://dagger.io). Dagger is enabling us to define a pipeline we can execute anywhere, which is both fast and reproducible. We’re very excited about the future of Dagger and can’t wait to talk about how we plan to leverage it in the Flipt build process.

We hope you’ll find this new SDK useful, and we’re happy to answer any questions that you may have in our [community Discord](https://www.flipt.io/discord) or via GitHub issues/discussions.

We welcome all feedback on how to improve the SDK or Flipt as a whole, so feel free to reach out! You can find us on [GitHub](https://github.com/flipt-io), [Twitter](https://twitter.com/flipt_io), [Mastodon](https://hachyderm.io/@flipt), and [Discord](https://www.flipt.io/discord).

Generating the Flipt Go SDK

<BlogImage
 src="/hi-yoofi/yoofi.png"
 alt="Hello, Yoofi!"
 width={2400}
 height={1350}
>
 
 Photo by [Dejan Zakic](https://unsplash.com/@dejanzakic)
 on Unsplash
 
</BlogImage>

We are excited to announce a new addition to the Flipt team, [Yoofi Quansah](https://www.linkedin.com/in/redblacktree/) - a Senior Software Engineer with 6+ years of experience and a love of developer tooling. With his impressive background and skills, we are confident that he will play a significant role in advancing our mission of helping engineers everywhere deploy software to production without fear. 

Here's what Yoofi had to say about his background and decision to join the Flipt team:

> Hello 👋, my name is Yoofi Quansah. I've had quite the journey to finally find my way here at Flipt, but let me not bore you with details you probably don’t want to know and get to the fun stuff!
>
> Before writing code, I was actually 99% positive I was going to make it to the NFL (as we all were). I am writing this blog post now, so you know that didn't happen 🙂, but I am very happy with how my life has turned out so far.
> I started my career out building simple developer tools over at Honey. After I got over the hump of imposter syndrome and really got into my work, we started to build some really cool tools on the team. These tools included our own feature flagging service (foreshadowing), deployment platforms, and testing platforms to name a few.
> Everything was cool right? Not quite. I realized that even though we were building these cool tools, we were not really getting much recognition for it, and even worse the engineers were relatively slow to adapt. It was quite draining, to say the least.
>
> Enter July 2019... My life changed when I went to Gophercon.
> I found people who were building developer tools and directly interacting with customers with the tools they built. Wow. It was at that point that I knew I wanted to write Go, and work with a company that interacted directly with developers as their customers. After all, it is pretty nice to see the value of the products that you build for others to use directly. 
> The result of all this was me leaving my job at Honey, and joining InfluxData in August of 2020, where I met [George](/blog/welcome-george-macrorie)!
>
> InfluxData was an incredible experience. I met tons of smart people, which allowed me to hone my craft as an engineer. It also allowed me to realize exactly what I wanted out of my career, which was to build incredible features for engineers to use. Most of my time there was taken up building the next generation of the query engine, which solved a wealth of problems we were facing with the old one.
> George introduced me to the grand idea of Flipt, all Mark has already done already, and the future outlook/potential roadmap in early 2023. Even though I was quite happy at InfluxData, I fell in love with the challenge, and all that is being hoped to accomplish.
>
> Combined with my previous experience helping out building a feature flagging service at Honey, and a whole lot of issues I ran into with feature flags both at Honey and InfluxData, I believe there is a bright future for this critical piece of software development and delivery.
> I am happy to be here, and hack on!

Welcome Yoofi, we are lucky to have you! 🚀

Meet Yoofi Quansah

<BlogImage
 src="/enhancing-authentication/enhancing-authentication.png"
 alt="Enhancing Flipt Authentication"
 width={2400}
 height={1350}
>
 
 Photo by [Roon van der
 Meer](https://unsplash.com/@roonz_nl)
 on Unsplash
 
</BlogImage>

In the previous few posts, we went over the addition of one of our most asked-for features: User Authentication support. Just when you thought that might be the end of our authentication story, we improved it even more by adding three more authentication-related features:

1. API token configuration via the UI
1. Configurable bootstrap token
1. Kubernetes Service Account authentication

In this post we'll briefly go over each feature, discussing its use case, why it's an important addition to our overall auth story, as well as how to get started.

## API Token Configuration

We've had basic token authentication support for authenticating your services with Flipt since v1.15. Admittedly though, it was somewhat incomplete as you could only manage those tokens via the [API](https://www.flipt.io/docs/reference/authentication/create-token).

In this [latest release](https://github.com/flipt-io/flipt/releases/tag/v1.19.0), we added the ability to create and delete static tokens in the UI as well. This allows anyone authenticated with Flipt to see which tokens have been issued and when they expire. It also allows users to create and delete tokens as needed without having to write code to do so.

Enabling and using static token authentication is a good first step when it comes to securing the communication between Flipt and your own services. Making the management of access tokens as user-friendly as possible was one of the major acceptance criteria for this new functionality.

This feature also necessitated a new 'Settings' section of the application which we'll populate with more controls in the future.

<BlogImageGallery
 images={[
 {
 src: "/enhancing-authentication/static-tokens/list.png",
 description: "Settings Screen with Static Tokens View",
 },
 {
 src: "/enhancing-authentication/static-tokens/new.png",
 description: "Creating a New Token",
 },
 {
 src: "/enhancing-authentication/static-tokens/expiry.png",
 description: "Adding an Expiration to a Token",
 },
 {
 src: "/enhancing-authentication/static-tokens/created.png",
 description: "Token Created",
 },
 ]}
/>

After creating a token via the UI, you can easily copy it to your clipboard and store it in a secure location as its value will no longer be accessible from Flipt itself.

## Configurable Bootstrap Token

Another improvement that we added in this release is the ability to provide a configurable bootstrap token when first enabling token authentication. Since v1.15, if an operator enabled token authentication for the first time, we would create what we call an initial bootstrap token on their behalf, logging it to `STDOUT` to be captured and used.

```console
access token created	{"server": "grpc", "client_token": "ks8gyZNdU7jmB92kAIYOp4MEXTcCJbX1TdXJM01VKCs="}
```

This was necessary because all requests to the Flipt API would fail as soon as you enabled and required token authentication.

This meant, however, that there wasn't a great programmatic way to run Flipt with token authentication enabled for the first time, such as in the case of Kubernetes. Flipt operators would have to either:

1. Start Flipt with authentication enabled but not required, create a token via the API, and then restart Flipt with authentication required.
1. Start Flipt with both authentication enabled and required and try to capture the created bootstrap token from our logs.

This also presented a similar challenge when trying to run Flipt in CI, such as we do for our own integration tests, as there was no easy way to set up authentication and then restart Flipt in a test pipeline.

In this release, we added the ability for the user to optionally specify their own known bootstrap token as well as to configure it with an expiration.

<BlogImage
 src="/enhancing-authentication/bootstrap-token-expiration.png"
 alt="Bootstrap token created with expiration"
 width={2442}
 height={490}
 caption="Bootstrap token created with expiration"
 className="shadow-md"
/>

Here's an example of how to configure a bootstrap token with a known value and an expiration of 24 hours:

```yaml
authentication:
 methods:
 token:
 enabled: true
 bootstrap:
 token: "s3cr3t!"
 expiration: 24h
```

In our opinion, this provides the best of both worlds by allowing for the initialization of an authentication token with a potential known value that can expire automatically. This gives operators the ability to enable authentication from the get-go, use a known token to create more tokens, then allow the known token to quietly expire to prevent unwanted access.

More information on how to configure this feature can be found in our [documentation](https://www.flipt.io/docs/configuration/authentication/#token-authentication).

## Kubernetes Service Account Authentication

In most cases, static tokens are a manageable approach for getting services talking with Flipt. For security reasons, you may set an expiry on your tokens and periodically rotate them with new ones. Given you have more than one type or replica of your application(s) communicating with Flipt, you may even choose to create multiple tokens. It may be one per application type or even as many as one per replica.

For those deploying Flipt into Kubernetes environments, we have added the new `kubernetes` authentication method.
This method takes advantage of the [service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) mechanisms made available in Kubernetes.

By default Kubernetes periodically writes a service account token (encoded as a JWT) into a well-known path inside each pod's filesystem.
Service account tokens can be used to communicate and identify the caller with the Kubernetes API. With Flipt [v1.19](https://github.com/flipt-io/flipt/releases/tag/v1.19.0) you can exchange this service account token for a Flipt client token via Flipt's API.

A small amount of client-side glue code can keep your applications authenticated without having to manually create, rotate or distribute tokens.

Here's an example using `curl` and `jq` to exchange a Kubernetes service account token for a Flipt client token:

```bash
curl -s -X POST http://flipt:8080/auth/v1/method/kubernetes/serviceaccount \
 --data "{\"service_account_token\":\"$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\"}" | \
 jq .
{
 "clientToken": "pKkaEik40Nu4lJ7O37l92MNyyD38U8UlaagSmAfJoS0=",
 "authentication": {
 "id": "f191babc-57b8-4856-89b1-f324941403d7",
 "method": "METHOD_KUBERNETES",
 "expiresAt": "2024-02-20T13:39:14Z",
 "createdAt": "2023-02-20T14:11:28.962841Z",
 "updatedAt": "2023-02-20T14:11:28.962841Z",
 "metadata": {
 "io.flipt.auth.k8s.namespace": "default",
 "io.flipt.auth.k8s.pod.name": "someservice-586bfb5b6b-fmh8g",
 "io.flipt.auth.k8s.pod.uid": "b5217947-aeac-4b35-afd3-23f50d63eae9",
 "io.flipt.auth.k8s.serviceaccount.name": "default",
 "io.flipt.auth.k8s.serviceaccount.uid": "8aeb28ad-66f0-4884-bafc-e606e5eda149"
 }
 }
}
```

Enabling this method means Flipt implicitly trusts other services deployed into the same cluster.

While currently leveraging this new method will require some application code on the developers' part, we're actively working on a way to provide out-of-the-box support for our official clients.

As always, our [documentation](https://www.flipt.io/docs/configuration/authentication) has more information on how to configure these features.

Please let us know what you think of these new features and if you have any questions or feedback, feel free to reach out to us on [Discord](https://www.flipt.io/discord), [Twitter](https://twitter.com/flipt_io), or [Mastodon](https://hachyderm.io/@flipt)!

Building the future of Feature Flags

Deploy With Confidence

Works with your existing stack

Features

Ship Safely

Context Targeting

Self Hosted

Secure

Fast

Easy Integration

From the Blog

Generating the Flipt Go SDK

Meet Yoofi Quansah

Enhancing Authentication Support

Building the future of Feature Flags

Deploy With Confidence

Works with your existing stack

Features

Ship Safely

Context Targeting

Self Hosted

Secure

Fast

Easy Integration

Keep up with what we're building

From the Blog

Generating the Flipt Go SDK

Meet Yoofi Quansah

Enhancing Authentication Support