
In the most recent release of Flipt ([v1.31.0](https://github.com/flipt-io/flipt/releases/tag/v1.31.0)) we added support for OCI registries as a backend for feature flag data. The new backend follows in the footsteps of our other [declarative](https://www.flipt.io/docs/configuration/storage#declarative) file-based backends (`local`, `git`, and `object`). This blog post talks a bit about OCI, its growing popularity for non-container usecases and why and how you can leverage Flipt to distribute flags as OCI bundles.

## What is OCI?

OCI stands for the Open Container Initiative. Taking words directly from their landing page:

> The **Open Container Initiative** is an open governance
> structure for the express purpose of creating open industry standards
> around container formats and runtimes.

[https://opencontainers.org/](https://opencontainers.org/)

OCI currently declares and maintains three different specifications:

- [Runtime Specification](https://github.com/opencontainers/runtime-spec)
- [Image Specification](https://github.com/opencontainers/image-spec)
- [Distribution Specification](https://github.com/opencontainers/distribution-spec)

Each of these specifications come together to create a standard set of operating procedures for OCI-compliant container runtimes to fetch, isolate, and execute workloads.

It is probably important to clarify first that Flipt is **not** concerned with running container workloads. Flipt is concerned with the distribution specification (and tangentially with the image specification). OCI-compliant registries have proven extensible and valuable beyond just containerized applications. Their distribution specification includes extensibility points, which can be leveraged to store arbitrary contents with different artifact and media types.

These extensibility points make it possible for Flipt to package and publish its feature flag configuration contents via OCI compatible registries. Some OCI-compatible registries include:

- [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) (Fun fact: our own [Mark Phelps](https://www.linkedin.com/in/markphelps1/) was a lead engineer and architect of GHCR at GitHub)
- [DockerHub](https://hub.docker.com/) (the OG)
- [Google Artifact Registry](https://cloud.google.com/artifact-registry)
- [AWS Elastic Container Registry](https://aws.amazon.com/ecr/)
- [Azure Container Registry](https://azure.microsoft.com/en-gb/products/container-registry/)

To ease our integration into the OCI landscape, Flipt has built new capabilities on top of a project called [ORAS](https://oras.land/). To quote what ORAS is from their site:

> ORAS is the de facto tool for working with OCI Artifacts. It treats media types as a critical piece of the puzzle. Container images are never assumed to be the artifact in question. ORAS provides CLI and client libraries to distribute artifacts across OCI-compliant registries.
>

Big shout out to ORAS for making this possible in Flipt.

## Prior-art

We’re not the first and certainly won’t be the last project to leverage OCI for non-container artifact distribution. OCI has created a standard for registry APIs, which is broadly useful for an extensible set of artifact and media types. The APIs are also designed around content-addressability, which lends itself to some very desirable caching traits and security improvements.

We got the idea to leverage OCI from past colleagues (shoutout [David](https://x.com/rawkode) and [Marko](https://x.com/mmikulicic)) and further validation from a number of modern technologies already leveraging OCI for non-container image purposes. To name a few:

**Helm** - A package manager for Kubernetes

Since v3.8.0 Helm has supported using OCI registries as a distribution medium for Helm charts.

[https://helm.sh/docs/topics/registries/](https://helm.sh/docs/topics/registries/)

**Timoni** - Another package manager for Kubernetes

From inception, Timoni has leveraged OCI registries for distributing its modules.

[https://timoni.sh/bundle-distribution/](https://timoni.sh/bundle-distribution/)

**Acorn Labs** - A platform for running and sharing applications

Acorns are shareable bundles of configuration for containers, services, and everything in between, which are distributed via OCI-compliant registries.

[https://docs.acorn.io/advanced/publishing](https://docs.acorn.io/advanced/publishing)

**Nuon** - A BYOC platform

Nuon leverages OCI registries for distributing their configuration artifacts.

[https://docs.nuon.co/concepts/vendor-cloud#artifacts](https://docs.nuon.co/concepts/vendor-cloud#artifacts)

## Improving Scale

The [Git backend for Flipt](https://www.flipt.io/docs/configuration/storage#git) provides an excellent experience for enabling GitOps feature flag workflows, with minimal operational overhead. You simply need Flipt and your repository provider to get up and running (the relational backend becomes optional, depending on whether or not you need authentication). This is great for quickly validating Flipt and should scale reasonably well for small to medium-sized projects and teams. The primary bottleneck for scale becomes the size of the source repository storing your flag state.

When configured this way, Flipt needs to clone the entire target repository and keep it in sync. We want Flipt to be suitable for both multi and mono-repositories, however, the larger the source repository, the greater the initial clone becomes and therefore startup time. Additionally, the higher the contribution throughput, the more work Flipt has to do to stay up to date. Flipt has only really begun to scrape the surface with Git, and there are certainly more optimizations we intend to make to scale this backend. However, OCI registries present a neat solution to this scaling problem.

<StarCTA />

Using an OCI registry and repository for feature flag bundles means the initial fetched payload is kept to only the state that matters to Flipt. In order to support this, Flipt comes with a set of new subcommands (see `flipt bundle`) for building, pushing, and pulling feature flag state as OCI bundles. The bundle format is designed such that Flipt only needs to refetch *namespace* contents when it changes from one update to the next. This is down to how we use separate layers for separate namespaces in the OCI bundles themselves.

One further byproduct of this is regarding the scope of access needed by Flipt. In this mode, Flipt only needs read access at runtime to the repository contents itself. This is opposed to the entire contents of your Git repositories. This is a win for the security-conscious folks out there.

## Experimentation

Beyond these scale improvements, there are further benefits in the form of arbitrary OCI repository tags. If you’re familiar with tagging a Docker image `latest`, then you already know what I am talking about. The `flipt bundle` commands support arbitrary tagging, and Flipt can be configured to source feature flag state from any desired tag you choose.

An intended goal of the Git declarative backend was to unlock branching feature flag state. Now you have the power of Git branches and tags, you can leverage them for experimenting with feature flag changes in different deployment environments. Instead of building the concept of environments into Flipt's own domain models, we would lean into the benefits of a version controlled storage medium to facilitate this kind of validation. Want to experiment with a change to feature flag state in your PR? Then you could automatically deploy Flipt to track your branch in a preview environment. Need to validate a change to your targeting rules in staging first? Then you could have your staging instance of Flipt track a `staging` branch in Git instead of `main`.

This last suggestion however has one big downside, while branches in Git are appropriate for more ephemeral proposed changes (e.g. via pull-requests and preview environments), long-lived branches other than `main` are frowned upon. We’re strong believers in [trunk-based development](https://trunkbaseddevelopment.com/) at Flipt, and we fully intend to help you enable it via feature flags. Expecting you to create long-lived branches for environments in hindsight seems unreasonable.

Where this new approach comes in handy, is instead separate folders can be leveraged in your repositories to represent different environments. The `flipt bundle` command can be employed to create separately tagged bundles for each environment. This has the added benefit that you can see the difference between your environments in a single commit by simply comparing the contents of each directory.

<BlogImage
  src="/oci-backend/diagram.png"
  alt="Demonstration of how to use OCI bundles for feature flag transport"
  width={3162}
  height={1543}
  caption="OCI bundles for feature flag transport"
></BlogImage>

One last potential use case of note is shareable and ephemeral remote bundles using [https://ttl.sh](https://ttl.sh). This is an on-demand ephemeral OCI registry provider. With ttl.sh you simply follow their instructions and tag your bundle with the duration of its lifetime (e.g. `ttl.sh/b7e858c6-8462-41d7-b526-863ae96db14b:10m` for a 10 minute bundle). Then you can share this bundle with anyone and they can `flipt bundle pull` it locally or configure Flipt to serve from ttl.sh directly.

## A Brief Tour

We’re currently working on a guide that will take you through the steps of setting up Flipt with OCI, as well as how to unlock these kinds of long-lived and ephemeral feature-flag environments in your platform. However, here is a quick introduction to get you started.

For this walkthrough, you will need the latest version of Flipt (`v1.31.0`) installed on your path (head over to our [Installation Documentation](https://www.flipt.io/docs/installation) for more details).

Next, you will need a directory with some Flipt feature state configured in it. For this example, we will create a single file named `features.yml`. If you want to name your feature flag files something different, check out our documentation on how [Flipt locates flag state](https://www.flipt.io/docs/configuration/storage#locating-flag-state). The commands we will talk about in this section (`flipt bundle build`) use the same strategy for locating flag state as Flipt’s other declarative backends (`git`, `local`, and `object`).

```yaml
# contents of features.yml
namespace: "default"
flags:
- key: "excitingFeature"
  name: "Exciting Feature"
  type: "BOOLEAN_FLAG_TYPE"
  enabled: false
```

Next, we will create our first bundle based on this state. In the same directory as this new file, run the following:

```bash
flipt bundle build mybundle:latest
```

Given everything works as expected you should see the resulting digest of the bundle printed onto your terminal. Now you have a feature flag bundle created locally. To see all your bundles, run the following:

```bash
flipt bundle list
```

Now that you have built your bundle, we can actually run Flipt over the local bundle itself. This is a great way to quickly validate your flag state before pushing it to some production serving registry. Simply start Flipt like so:

```bash
FLIPT_STORAGE_TYPE="oci" \
  FLIPT_STORAGE_OCI_REPOSITORY="mybundle:latest" \
  flipt
```

This will start Flipt on `http://localhost:8080`, where you should immediately see your feature flag is available.

To publish this bundle, we can use the Flipt bundle push command. Here we’re using the ephemeral registry [ttl.sh](http://ttl.sh) to get going quickly:

```bash
export UUID="$(uuidgen | tr '[:upper:]' '[:lower:]')"

flipt push mybundle:latest ttl.sh/$UUID:10m

echo "ttl.sh/$UUID:10m"
```

This should push your local bundle to the remote registry and print the target repository which can then be used again when running Flipt. Simply take the command to start Flipt earlier and replace the local bundle with the remote one.

```bash
# you can manually substitute the repository with the one echoed in
# the previous command if you're in a fresh terminal
FLIPT_STORAGE_TYPE="oci" FLIPT_STORAGE_OCI_REPOSITORY="ttl.sh/$UUID:10m"; flipt
```

Navigating back to Flipt in your browser should again show your feature flag as it was before. Only this time, Flipt will have used the remote repository to get its state. Try changing your feature flag from `enabled: false` to `enabled: true` and then re-build and re-push to the same repository hosted on ttl.sh. Flipt should eventually update (polling interval defaults to 30 seconds) to reflect this change automatically.

## What’s next?

This new backend in Flipt is in its infancy, and we have a number of ideas to make the experience of leveraging it easier. The following is a short list of features we’re considering:

**More local `flipt bundle` management commands**

For example, commands to help maintain and clean up old bundles or introspect into their contents. Perhaps a little TUI for exploring bundle contents?

**A GitHub action to simplify CI integration**

A quick and easy GitHub action to drop into your workflows for building and pushing.

**Feature flag state attestation**

Now that your flag state is in an OCI artifact, you could use a tool like [cosign](https://docs.sigstore.dev/signing/quickstart/) to sign your feature flag state.

**Garbage collection**

It will be no surprise that at some point someone will want to clear outdated bundles. We think Flipt could eventually help with that too.

We’re excited to see how this resonates with the community and what it unlocks for feature management. OCI is proving to be an invaluable distribution mechanism for all sorts of configuration concerns. We’re just getting started with our own ideas around how to leverage them for Flipt and for broader configuration concerns.

We would love to hear from you about your feature and configuration needs, so come hang out with us on our [Discord](https://flipt.io/discord) or open an issue or discussion in our [repository](https://github.com/flipt-io/flipt).


Feature Flags and OCI


Edge computing has become a recent phenomenon for organizations to explore as they scale to meet the demands of a worldwide customer base. It is simply defined as hosting either devices or networks closer to the consumer. Implicitly, as this happens, the user can perceive a much better experience due to the interaction speed of the application to the devices and networks the application interfaces with.

For example, you can imagine that a user in the United Kingdom will benefit from having their data stored in the UK instead of anywhere else in the world. In broader terms, geographical distance from a consumer to a device or network can be significant at times and does not come for free.

In terms of feature flags, the ability to leverage edge computing suits its use case fairly well. Feature flags can often guard critical paths of an application where it is beneficial that the latency of flag evaluation is kept as low as possible.

At Flipt, we have been exploring many different ways to achieve this goal of minimizing latency for flag evaluations. We already have features such as [server-side caching](https://www.flipt.io/docs/operations/deployment#caching), hosting data in an embedded database such as SQLite, or serving flag state straight from a [filesystem](https://www.flipt.io/docs/configuration/storage#filesystem) to name a few, so the idea of edge computing fits the paradigm quite nicely.

In recent weeks, we have been exploring running Flipt on the edge, and the rest of this post will detail the experience we had running Flipt in front of a notable edge database provider, [Turso](https://turso.tech/).

## Turso/sqld/libSQL

Before detailing exactly what Turso provides as a service, let us briefly cover the open-source technologies and their functionalities that power Turso, [libSQL](https://github.com/tursodatabase/libsql) and [sqld](https://github.com/tursodatabase/libsql/tree/main/libsql-server).

### libSQL

libSQL is an open contribution fork of SQLite and thus is fully backward compatible with SQLite. You can actually run a SQLite client over a libSQL database instance. Since SQLite is famously [closed off to external contributors](https://www.sqlite.org/copyright.html), the fork (libSQL) was born, which tries to cater to some of the issues or feature requests that the community wants with SQLite. There are many various distinct features that libSQL provides from SQLite such as [randomized ROWID](https://github.com/tursodatabase/libsql/issues/12), [WebAssembly](https://blog.turso.tech/webassembly-functions-for-your-sqlite-compatible-database-7e1ad95a2aa7), and [many more](https://github.com/tursodatabase/libsql#features).

### sqld

[sqld](https://github.com/tursodatabase/libsql/tree/main/libsql-server) is libSQL running as a daemon in “server-mode”. It not only provides an HTTP interface over libSQL, but also replication abilities from one libSQL instance to another. When running sqld, you can declare one libSQL instance as the primary and other instances as replicas of the primary. sqld will then replicate data from the primary to the replicas. This is a very powerful feature that we will explore in more detail later in this post.

The libSQL repository provides a nice [docker-compose example](https://github.com/tursodatabase/libsql/blob/main/docker-compose/docker-compose.yml) of how to configure sqld, and how it works overall. In addition to simple replication, sqld also provides [bottomless replication](https://github.com/tursodatabase/libsql/tree/main/libsql-server#integration-with-s3-bottomless-replication) in which data from the primary is continuously replicated to an AWS S3 bucket or any other S3 compatible storage such as [MinIO](https://min.io).

### Turso

<BlogImage
  src="/edge-turso/turso-logo.svg"
  alt="Turso Logo"
  width={100}
  height={100}
/>

[Turso](https://turso.tech/) is a platform that provides managed libSQL with seamless replication of data to multiple regions around the world via sqld. With Turso, you can spin up a database in any region it supports as the primary, and this will be the instance that will accept all writes. Based on that primary instance, you can spin up several replicas in any other region which will serve as read-only copies and will be synced with the primary over time, providing the natural eventual consistency of data.

As we mentioned above, from the other features we already have implemented in Flipt, eventual consistency has been a constant (caching, polling of data), and is acceptable for most use cases. There are also disproportionately more reads than writes of data for most Flipt use cases, so the “extra hop” as Turso describes it to get a write into the primary is acceptable. Knowing these two characteristics, it was a no-brainer for us to at least explore this route of providing capability of running Flipt over Turso.

<StarCTA />

### Flipt and libSQL

Flipt can already be run in a variety of storage modes with a [slew of databases](https://www.flipt.io/docs/configuration/storage#databases). We support the most popular RDBMS' such as: MySQL, PostgreSQL, CockroachDB, and last but not least SQLite. The already existing support for SQLite made it a relatively simple lift to support Flipt with libSQL. The functionality for this is included in [v1.29.0](https://github.com/flipt-io/flipt/releases/tag/v1.29.0) release! This release has support for not only running Flipt against libSQL in flat-file, embedded mode, but also in server mode with sqld over HTTP.

Although relatively painless, we did run into a couple of snags to get sqld working behind Flipt. The repository explains that currently libSQL clients are [somewhat limited](https://github.com/tursodatabase/libsql#getting-started) in terms of breadth of support. For instance, only the Rust, and Javascript clients are stable and supported while the rest of the languages are all “experimental”.

For us, the problem with the [libsql-client-go](https://github.com/yquansah/libsql-client-go) client was that the `main` branch did not have functionality for scanning boolean types nor recognition of the `TIMESTAMP` data type and conversion functions from that type to a common string. We [forked](https://github.com/yquansah/libsql-client-go) the repository and wrote patches for the missing functionality to get it working for the time being. We hope to contribute more to the project in the future if support for running Flipt with libSQL/Turso becomes more popular.

### Flipt ‘on the edge’ with Turso

Along with the functionality of the `v1.29.0` release also came an [example](https://github.com/flipt-io/flipt/tree/main/examples/database/libsql) of how to run Flipt against libSQL with replication provided by sqld. This example contains two containers of Flipt running against two containers of sqld, one primary and one replica instance.

You can access both Flipt instances and make edits via the UI, and witness the replication happening in real time. Although this works nice locally for a proof of concept, it does not show the true power of edge computing, so we opted to do an experiment with each employee at Flipt running in front of a Turso instance.

As you create database instances on Turso, the service will give you a URL that you can use to access the logical database. Turso defines a logical database as a collection of libSQL databases each with a schema and data and access to a primary instance (the primary instance itself is included in the logical grouping). The URL is then mapped to the closest instance to the user asking for the data upon query. Again, writes are routed to the primary instance by Turso, but reads can be served from any replica.

#### Experimenting

For our internal experimenti, we each ran Flipt locally, using the URL that Turso provides as [configuration](https://www.flipt.io/docs/configuration/storage#remote). We spun up a primary instance in IAD (Washington D.C, US), which was the closest on average for all 3 employees, and replicas in DFW (Dallas, Texas US) and LHR (London, England).

<BlogImage
  src="/edge-turso/turso-rw.png"
  alt="Turso Requests"
  width={1776}
  height={1759}
  caption="Turso Requests"
/>

On each read of the data we encountered correctness of routing, as each employee was routed to the nearest libSQL instance. For latency, we experimented with the default state of having a replica in each region closest to an employee and took some instances away over time to see if we could witness an increase in latency. Sure enough, as we took away the replica in DFW, the employee closest saw an increase in latency, and the same for the employee near London when we took away the replica in LHR.

<BlogImage
  src="/edge-turso/worldmap.png"
  alt="Worldmap of two of our Turso Instances"
  width={1105}
  height={426}
  caption="Worldmap of two of our Turso Instances"
/>

Given Turso's ability to replicate data to any region in the world in seconds, we could have spun up replicas in any region that Turso supports, and we would have seen similar results. This is a very powerful feature that we are excited to explore more in the future.

We're also looking forward to trying out Turso's [embedded replica feature](https://blog.turso.tech/introducing-embedded-replicas-deploy-turso-anywhere-2085aa0dc242) which would allow you to access a Turso/libSQL database via the native [libSQL C binding](https://github.com/tursodatabase/libsql/blob/main/bindings/go/libsql.go) instead of over HTTP with sqld and the [libsql-client-go](https://github.com/libsql/libsql-client-go) client. This would remove the additional latency of handling HTTP requests/responses and the marshalling/unmarshalling of JSON.

## Conclusion

Turso provides an impressive experience for managing distributed SQLite compatible databases and we are excited to see how it evolves in the future.

At Flipt, we've been actively exploring how to make feature flag evaluation as fast as possible. We are very excited about the many opportunities/technologies that can aid in that mission, including running Flipt with your own data on the edge. In the [previous blog post](/blog/flipt-as-a-sidecar), we explored running Flipt as a sidecar which can work well in a Kubernetes environment, and now we have explored running Flipt on top of a managed edge database provider without Kubernetes.

We're continuing to explore alternative ways to run Flipt on the edge, and we're excited to share more about that going forward.

If you are interested in Flipt, edge data/computing, or how to leverage the power of feature flags in your organization, please feel free to reach out to us via [GitHub](https://github.com/flipt-io/flipt), [Discord](https://flipt.io/discord), [X](https://twitter.com/flipt_io), or [Mastodon](https://hachyderm.io/@flipt).

Until next time!


Experimenting on the Edge


As organizations choose to integrate a feature flag solution within their stack, they should consider several core characteristics carefully. Feature flags can be used in various ways, and often guard critical paths of functionality that an organization wants to slowly rollout, or quickly turn off in case of a disaster. This naturally lends itself to an organization asking about the consistency and availability of the feature flag solution and how fast they could get values from feature flag evaluations to their clients that are leveraging it.

At Flipt, we have had several [feature requests](https://github.com/flipt-io/flipt/issues/2122) for client-side evaluation of flags. Currently, Flipt allows only server-side evaluation, with the option of enabling cache to reduce round trips to the database. This works fine for a good amount of use cases that our users, however, client-side evaluation is still a heavily requested feature.

For client-side evaluation to work, a special client would have to be written in each desired language to asynchronously pull flag state from a central server and store the flag state in memory. Evaluations for flags in this case would certainly be faster because it removes network hops to receive an evaluation value on each evaluation request. This could be a great solution for us, however writing a custom client in each language that users may request takes time away from building other necessary features for the core product.

Acknowledging the above facts, we still want to develop a solution that works for us as a company and our users. For these reasons, we have decided to explore using Flipt as a [sidecar](https://www.oreilly.com/library/view/designing-distributed-systems/9781491983638/ch02.html), a very popular design pattern used in distributed systems.

The remainder of this blog post will focus on what the architecture looks like in this new world and how consumers can leverage it.

### Architecture

Before talking about the overall sidecar architecture, however, it is important to note that today Flipt can be run in a variety of configurations for a storage backend. The traditional way to run Flipt is in front of an RDBMS (SQLite, MySQL, Postgres, etc.). By default, if you were to run Flipt locally it runs with a SQLite database 'next to' the Flipt binary. In addition to traditional RDBMS, Flipt can also be run in [file system (FS) mode](https://www.flipt.io/docs/configuration/storage#filesystem). In this mode, Flipt will periodically fetch from the target FS backend and build an in-memory snapshot of all of it's state. This process is asynchronous from any actual evaluation requests as requests are serviced on a live snapshot of state. The snapshot is only replaced with the new one once it has been built entirely.

It turns out that this architecture is perfect for doing evaluations in a sidecar as it removes one of our network hops from the evaluation call path. For reference, the different FS implementations we support are local file system, Git repositories (check out our Git blog post [here](https://www.flipt.io/blog/gitops-announcement)), and AWS S3 buckets thanks to an [awesome contributor](https://github.com/flipt-io/flipt/pull/1900).

<BlogImage
  src="/sidecar/s3-replication.png"
  alt="S3 Replication"
  width={1972}
  height={2248}
></BlogImage>

This same diagram can be found in our [labs repository](https://github.com/flipt-io/labs/tree/main/sidecar#object-store), where we also demonstrate how to run the proposed architecture in a minikube cluster.

In this diagram, you can see that there is a `flipt-master` process which will be the main source of truth for all feature flag data. This `flipt-master` process can be backed by any storage mechanism. The data from `flipt-master` will be replicated into an S3 bucket at a configurable interval via a Kubernetes cron job. Within the Flipt binary, there is an [`export`](https://www.flipt.io/docs/cli/commands/export) subcommand that reads flag state data out of `flipt-master` and writes that data into a file. That subcommand is what is used for the cron task, and the resulting data uploaded into the bucket.

Any sidecar that will be deployed alongside a service will then run in FS read-only mode with the S3 bucket as the storage mechanism. As the service makes evaluation calls it can source the data from either the [gRPC](https://www.flipt.io/docs/integration/grpc) or [REST](https://www.flipt.io/docs/integration/rest) API. These evaluation calls will be served from the sidecar, which will use data in memory from the S3 object store instead of having to get the data from a database over the network.

### Sample Results

If you run the `minikube` example within the labs repository, there is also another custom application that is written in Go and serves HTML content to the client.

The HTML page allows you to make evaluation calls by selecting either the `sidecar` or the `master` process.

As the instructions state in the [README](https://github.com/flipt-io/labs/tree/main/sidecar), you can choose from `flag_001` - `flag_010` to make evaluations as these flags exist in the initial seed data to `flipt-master`.

<BlogImage
  src="/sidecar/latencies.png"
  alt="Flipt Sample Evaluations"
  width={1527}
  height={708}
  caption="Load test results against `sample-app`"
></BlogImage>

The image above was generated doing a short load test using the [vegeta](https://github.com/tsenart/vegeta) tool. It made requests to the `sample-app`'s `/cli` endpoint at a rate of 15 req/s for 10 seconds, and dumped latency measurements to a csv file which was read and plotted using the [matplotlib](https://matplotlib.org/) + [pandas](https://pandas.pydata.org/) Python libraries. As you can see, evaluations against the sidecar were much faster (up to an order of magnitude) than evaluations against `flipt-master`. Keep in mind that this is just `minikube` as well, which is ultimately a VM running on your machine simulating a Kubernetes environment. In a real distributed environment, the difference in latency between the sidecar and `flipt-master` would likely be much more pronounced.

### Where to Go From Here?

At Flipt, we aim to make evaluations of your feature flags as fast as possible. We realize that adding an external dependency such as Flipt has a cost and we want to minimize it. Because of this, we are devising ways to decrease evaluation time for the source as much as possible, and the sidecar approach is one way to make this happen.

However, we are not done just yet. We're experimenting with additional integrations, architectures, and deployment patterns in the coming weeks to aide in operating Flipt in a multitude of real-world environments. Stay tuned and keep an eye on our [deployment docs](https://www.flipt.io/docs/operations/deployment) as well as this blog for updates.

If any of this interests you or you have any questions, please feel free to reach out to us via [GitHub](https://github.com/flipt-io/flipt), [Discord](https://www.flipt.io/discord), [X](https://x.com/flipt_io), or [Mastodon](https://hachyderm.io/@flipt).


Flipt as a Sidecar


<BlogImage
  src="/typed/typed.png"
  alt="Introducing Typed"
  width={812}
  height={397}
  caption="Typesafe feature flags with Flipt and Typescript"
/>

Feature flags are amazing tools to help you ship your code safely by guarding your end users against incomplete features in production. Many teams use feature flags in this way, to ship code that makes up a larger feature, enabling testing by a group of internal or beta users without having to deploy the code all at once.

This ability to slowly roll out and guard functionality is especially helpful in environments where multiple applications (services) make up the larger end-user experience. Sometimes these services are owned by orthogonal teams, which makes the communication around and planning of feature releases all the more important.

We believe that feature flags give engineers superpowers, allowing them to change application functionality at runtime, without requiring re-deploys. But, as you may have heard, "with great power, comes great responsibility". This ability to modify a running application through the click of a button or external configuration change (see our post on [Flipt for GitOps](/blog/gitops-announcement) ❤️) also introduces risk.

Feature flags introduce a dependency on an external system, which means your application code is no longer guaranteed to work the same in all environments. At Flipt we believe that the power and flexibility gained through the use of feature flags in production is worth the risk they introduce, however, we do think there is more we can do to help mitigate this risk and remove some of the sharp edges.

## Flag Identifiers

In Flipt, flag keys are unique (within a namespace) identifiers that are assigned to each feature flag in order to differentiate them from one another. It is important to choose meaningful and descriptive flag keys that accurately represent the functionality they control. This will make it easier for developers and stakeholders to understand and manage the feature flags within the system.

When defining flag keys, we recommend to follow a consistent naming convention. This can help maintain a clear and organized system of flags, especially when dealing with a large number of flags. For example, using a prefix or a specific format for flag keys can provide better context and improve readability.

Because flag keys are simple strings in most feature flag domains, they are extremely flexible and can contain almost any value. However, as most programmers know, strings are also easily mistyped which can lead to errors. In the case of feature flags, these errors include incorrect or non-existent flags being referenced, leading to runtime exceptions or unexpected behavior.

To mitigate the risk associated with mistyped flag keys, one common approach is to use a constant to define and manage flag keys. This can provide type safety and help catch errors at compile time rather than runtime for compiled languages. Using constants for flag keys, however, does have its downsides. Nothing stops multiple constants being created for the same value, which means there is no guarantee that when searching your code for a specific constant identifier, that you will find all references for that flag key. There is also nothing stopping you from defining a constant with an invalid value in the first place.

Also, using constants does not give the end user flexibility to define their own type which can enforce limiting a field being of a specific set of values.

```typescript
interface EvaluationRequest {
  requestId?: string;
  flagKey: string; // we only want a subset of possible values here
  entityId: string;
  context?: Record<string, string | undefined>;
}

let req: EvaluationRequest = {
  flagKey: "foo", // this is allowed!
  entityId: "1234",
};
```

## Adding Type Safety

Recently, a user of Flipt [proposed adding type safety](https://github.com/flipt-io/flipt-node/issues/6) to our [Node/Typescript SDK](https://github.com/flipt-io/flipt-node). After some back and forth and some learning on our part, we concluded that we could leverage some of the pieces that we already have in Flipt and create a CLI to generate these types.

When it comes to generating code, you first need a source or specification that can be parsed and then used for writing the code. We realized that for our feature flag data we already have this source, our [features.yml](https://www.flipt.io/docs/configuration/storage#defining-flag-state) file format which we use for [imports/exports](https://www.flipt.io/docs/operations/import-export) and to power our GitOps, Local, and Object Storage data backends!

Again, that same amazing user created a prototype/proof of concept CLI tool that parses the above-mentioned `features.yml` and generates `Flag` and `Context` union types for Typescript. We just open-sourced and published this tool that we’re calling [Typed](https://github.com/flipt-io/typed)!

Usage info is in the README, but here’s how you could quickly generate type-safe Typescript code from your features data in Flipt.

1. Export your feature data from Flipt, either via our API or through a direct DB connection using our [export](https://www.flipt.io/docs/operations/import-export#export) command: `flipt export`
2. Run `npx @flipt-io/typed --lang ts --input /path/to/features.yml` to output the generated Typescript code, or pipe it to a file.
3. Import the types and use in your code!

```typescript
import { Flag } from "./types/flipt";

interface EvaluationRequest {
  requestId?: string;
  flagKey: Flag["key"]; // enforces we can only use valid flag keys that exist in our Flipt instance
  entityId: string;
  context?: Record<string, string | undefined>;
}

let invalid: EvaluationRequest = {
  flagKey: "foo", // this will now error!
  entityId: "1234",
};

let req: EvaluationRequest = {
  flagKey: "language", // correct!
  entityId: "1234",
};
```

Here's what the generated types look like:

```typescript
// Generated by @flipt-io/typed
export type Flag = {
  key: "language";
  value?: "es" | "en" | "fr";
};
```

You can see that the generated types are very simple, and only contain the information that is needed to enforce type safety. The `Flag` type contains a [union string type](https://www.typescriptlang.org/docs/handbook/literal-types.html#string-literal-types) for the flag key and value.

Typed also supports the generation of a `Context` type which is the union of all the possible contexts that are defined in the `features.yml` file. This means that the `Context` type will contain all the possible fields that can be used in the `EvaluationRequest` interface.

## Contributing

[Typed](https://github.com/flipt-io/typed) is still very early days, and considered experimental. There are plenty of features/support we want to add and rough DX edges we want to smooth over, but importantly, we wanted to get it out there and see if anyone finds it useful. We also want to support more languages than Typescript in the future, including but not limited to Go, Java, Rust, Python.. pretty much every language that we have an ‘official’ [SDK](https://www.flipt.io/docs/integration/overview) for.

We’d love it if you gave [Typed](https://github.com/flipt-io/typed) a try and helped us out by filing issues, feature or language requests, or even contributing code. No contribution is too small and we’d love it if the open source community helped drive our roadmap.

In future post we’ll discuss another project we’re working on to improve the developer experience of working with feature flags.. so stay tuned!

As always, feel free to reach out on GitHub or come chat in our [Discord](https://flipt.io/discord). We’d love to learn more about your specific workflow when it comes to adding and working with feature flags in your codebases.


Generating Types for Feature Flags


In this post, I will introduce you to a project called Cup that we have been experimenting with. But before we delve into Cup, let me take you through the journey we have
embarked on to reach this point.

For those in a rush, here are some quick takeaways:

- We need better APIs for reading and writing configuration to Git
- You don’t have time to sweat the Git details, as you should be focusing on your user's needs
- Our new project [Cup](https://github.com/flipt-io/cup) is attempting to solve the above problems

## Gitting Going with GitOps

Back in May of this year (2023) we [proposed our new filesystem backend](https://github.com/flipt-io/flipt/discussions/1652) for Flipt. These backend types aimed to enable a native GitOps workflow for feature flags and users of Flipt. Now Git can be configured as the source of truth for your feature flags.

We then proceeded to build and deliver this capability over the coming months. Fast forward to today, Flipt can be deployed in front of your [local directory](https://www.flipt.io/docs/configuration/storage#local), [a Git repository](https://www.flipt.io/docs/configuration/storage#git), or even [AWS S3](https://www.flipt.io/docs/configuration/storage#object) thanks to one of our [amazing contributors](https://github.com/flipt-io/flipt/pull/1900).

The main downside to using Flipt with a filesystem backend in its current form is that the UI enters into a read-only mode. With these backends, the feature flag state management happens through files committed to git, written to directories, or pushed to object storage. This process happens outside of our current UI’s purview.

Our UI is designed to work over Flipt’s existing management APIs. These APIs operate imperatively and are built around storing state in a relational database backend. While there are ways we could engineer these endpoints to write to storage files, add and commit them to Git, and push these changes, the end experience would still likely leave a lot to be desired.

An ideal solution we have imagined would be for the UI to support both Git and proposal mechanisms such as GitHub’s pull-requests, or GitLabs merge-requests. Changes in the UI could be composed into proposals and then made onto these Git providers. The UI would also then present and track the proposals made back to the user.

## Going in Circles

While spiking on some of the pieces required to make a process like this work, I ended up reflecting on a lot of previous components and glue projects I had built with my past team working on a GitOps CD pipeline. I was reminded of all the pain and frustration many engineers came to us with around our configuration repositories.

- Where should the configuration for X live?
- How do I configure an environment variable for service Y?
- What’s needed to define an entirely new service?
- Where’s my recent application change deployed to right now?

To my team, reconfiguring a service's environment variables was a walk in the park. We did it every single day. But for newcomers, it was a real headache learning the code base. So instead we built tools and automation to read and write the configuration for them. Some of the things we ended up building to help with these problems were:

- Automation to update image versions in our repository, as and when new versions become available (continuous delivery all the way from merging an application change).
- Automation to promote image versions from one deployment environment (staging, production, etc.) to the next (progressive delivery).
- Graphical UIs to explain what versions of things were deployed in which environments.
- Templates for bootstrapping new services and new entire clusters.

At the end of the day, our configuration repository was a means to an end. No internal developer or platform team sets out with GitOps as their goal. GitOps is a process and set of tools to enable the end goal. The end goal is to build a platform that allows application developers to self-serve the infrastructure they need to deploy and manage their applications and to track those changes in something versioned and immutable.

<div className="not-prose flex items-center justify-center">
  <Tweet id="1678118806517067776" />
</div>

The automation we built was designed to either make our configuration and Git more accessible, or hide the need to interact with it altogether. This usually involved adding APIs over the top of these repositories to materialize more meaningful insight, or automation to generate pull requests over GitHub with the desired change for the logical request (new service, updated image version, and so on).

Git is ubiquitous these days, and it has a lot of desirable properties out of the box (versioned and immutable). This is why GitOps is so appealing, but is Git the best interface we have to offer? Let’s face it, it is prohibitive to non-developer or less technical audiences. In my experience, failing to provide more meaningful abstractions around these problems to our engineers led to a couple of undesirable outcomes:

1. Engineers work around you to get what they want
2. Or, they fill your inbox with requests for you to do it for them

I want internal developer and platform teams to have to focus on their developer’s needs and for these manual processes to be automated and for these stakeholders to serve themselves.

## How to Bridge the Gap?

We believe that we need to be building more meaningful APIs over Git. APIs that are designed to serve our stakeholder's needs:

- New service creation
- Getting a database
- Adding a feature flag
- Continuous deployment

Ideally, if you’re in one of these teams, you use a tool that helps you focus on these problems and not the necessary but distracting details of Git.

You should only need to focus on the business logic of parsing the current state, and transforming your repository contents from one revision into another, based on some new desired state. The tool should handle branching, adding, committing, pushing, and opening pull or merge requests for you.

<StarCTA />

You’re likely going to have all sorts of configuration formats and languages written into your repositories. This means the tool needs to be very **extensible**. There is no way a single project can support all the possible formats out there, given the nature of what can and is currently being stored in Git configuration repositories (YAML, JSON, Terraform/HCL, Jsonnet, CUE to name a few).

Beyond just automating away fiddly Git details, we’re going to want to consume these APIs with command-line tools and graphical user interfaces. This is how we reach those users that need a lower barrier to entry. They don’t want to interact with just a raw API as much as they don’t want to interact with configuration in a Git repo. But who has time to build these things? Most smaller to medium organizations don’t have the luxury of dedicating engineering time to these kinds of tasks.

Ideally, there will exist some off-the-shelf command line tools and user interfaces that can interact with any project built with our theoretical Git hiding framework or tool.

For this to work, the APIs should ideally present a **declarative** contract to consumers. This will greatly simplify the complexity of more generically applicable clients. Equally, the contract should be **consistent** across the extensible surface area that the tool can support. Finally, the API needs some capacity to be **self-describing** for generic tooling to present a meaningful interface over its extensible contents.

Does any of this seem familiar? The Kubernetes API server does this very well already.

_Shoutout to [Marko Mikulicic](https://github.com/mkmik) for helping to clarify this vision. I brought an
early prototype to him and he suggested that the Kubernetes API server is a
perfect fit for this problem._

## Building Cup

Introducing [Cup](https://github.com/flipt-io/cup).

A few months back we set out to build a Kubernetes-like API server that facilitates reading from and proposing changes back to Git repositories. A tool with a declarative API for retrieving well-structured and consistent API responses. The API reflects the shape of the data you’ve configured inside of it, such that more general-purpose tools can be developed to access its state. Eventually, Cup should get you a lot of the way without needing to build too much yourself, while also giving you the escape hatch to build your solutions around it.

### Consistent

The API is consistent from one resource kind to the next.

Retrieving, searching, updating, and removing should be the same for a service as it is for a logical feature or database. Retrieval, update, and removal require instances of each kind to be uniquely addressable (group, version, kind, namespace, and name). Search and cataloging require a taxonomy on which to organize and facet discovery (groups, namespaces, and labels).

### Declarative

The API expresses resources through a logical representation of _what_ is contained in the repository, as opposed to _how_ it is stored. The _how_ is defined inside [controllers](https://cup.flipt.io/configuration/controllers), where you can extend Cup to suit your layout and format requirements.

Controllers combine the state of the Git repository at a particular reference, with a desired
state of a resource to calculate a difference to be proposed via a pull or merge request on a target SCM (GitHub, GitLab, Gitea, etc.).

### Self-describing

Each resource has a structured schema, which can be requested via the API. This schema can be used by external tools to present meaningful interfaces or enable further automation to be built.

## Where Do We Go from Here?

Cup is very much an experiment and very much in its infancy. We’re looking to see if it excites people, and what people want to start making with it.

### Status Quo

Out of the box Cup comes with a simple configuration-driven controller for organizing the Kubernetes-like API resources as JSON in your repositories. This has limited scope for customizability. However, it works quite well for raw Kubernetes-style configuration.

The following video does a tour of Cup and the template controller configuring an application deployed with ArgoCD:

<div style={{ position: "relative", paddingBottom: "56.25%", height: "0" }}>
  <iframe
    src="https://www.loom.com/embed/a6adca0a455e4d55991d261f081e4934?sid=7120117f-a4ef-4a2a-b9c9-2c78a46ca3a9"
    frameBorder="0"
    allowFullScreen
    style={{
      position: "absolute",
      top: "0",
      left: "0",
      width: "100%",
      height: "100%",
    }}
  ></iframe>
</div>

Additionally, Cup comes with a very extensible WASM-type controller thanks to the awesome folks building [Wazero](https://github.com/tetratelabs/wazero/) for Go. This controller takes programs compiled to WASM and WASI snapshot preview 1, with a documented command line protocol, and turns them into a controller engine for whatever you can dream up. All you have to do is read and write files to a local directory, Cup (and Wazero) will take care of the rest.

If you’re interested in seeing this in action, we developed a controller you can explore for [Flipt feature flag configuration](https://github.com/flipt-io/cup/tree/main/ext/controllers/flipt.io). This is a stepping stone we envision for creating a UI experience for Flipt over Git. The video on the [Cup GitHub project README](https://github.com/flipt-io/cup#-feedback) demonstrates the Flipt controller for Cup.

### Future Vision

We’ve thrown around a lot of potential projects that could be built on top of Cup and Git in the future. Including but not limited to:

- Internal developer platforms
- Continuous delivery automation and observability
- Automatic dependency updates
- Progressive delivery
- End consumer self-service of infrastructure (let your customer's actions invoke configuration change)
- General purpose configuration API for your applications (why not read from Cup directly in your applications)

To get there we’re going to need to build some more features. Our current roadmap of ideas includes:

- Proposal status directly in the Cup API (see what desired state has been requested)
- Authentication and authorization (fortunately we have a consistent API to hang policies on)
- First-class clients in different programming languages
- Automatic replay of desired state changes as track branches progress (allowing for proposal to stay accurate as Git commits are merged ahead of your desired changes)
- Better management and packaging for controller implementations (support pulling controllers directly from OCI registries)

There is a lot to do to prove (or disprove) the idea, but the project is out there and we’re committed to exploring it. We could really do with your expertise in evolving this idea. Is there something you could automate with the help of Cup? We want to hear about it and to make Cup more appropriate for you.

Come check out the project at [https://github.com/flipt-io/cup](https://github.com/flipt-io/cup) and open an issue or discussion. Alternatively, come speak to us in realtime on our [Discord](https://flipt.io/discord).


GitOps Is a Means, Where Is the End?


Feature flags have become ubiquitous in the modern software development lifecycle for organizations across the world. Decoupling features from deployment, A/B testing, and gradual rollouts of features are a few of the benefits that come along with using feature flags.

Check out our [usescases](https://www.flipt.io/docs/usecases) documentation for more information on how feature flags can be used in your organization.

Feature flags allow for safety and ease of mind when rolling out features, but what happens when this new functionality behind feature flags are ready to be deployed fully? What tools do you need in place to monitor for unexpected application behavior? How can the organization correlate flag changes with regressions, or erratic results in their application behavior?

These questions demonstrate how there is a need for tighter coupling of feature flags to obvservability tooling. As a feature flaging tool at our core, we believe that we can introduce functionality to make this easier on our users and integrators to achieve tighter correlation with what can be observed while running in production. This blog post will highlight some of the features we have recently implemented to streamline that correlation.

### Webhooks

In the [v1.21.0](https://github.com/flipt-io/flipt/releases/tag/v1.21.0) release, Flipt introduced the [Audit Sink](https://www.flipt.io/docs/configuration/observability#audit-events) functionality where users can configure events in Flipt to be sent to a sink for auditability.

That release came with implementation for the inaugural sink which was a log file. These events are all CUD (create, update, delete) operations on Flipt entities (`flags`, `variants`, `segments`, `constraints`, `rules`, `rollouts`, `distributions`, `tokens`).

The log file was easy enough to get started, but hard to centralize that data to make any use out of it. The user would have to have additional tooling, such as [Grafana Loki](https://github.com/flipt-io/flipt/tree/50edfb1b78e02081b8c6038129f2f7903d39646c/examples/audit), to gather the logs in a centralized location since the logs can be distributed amongst many Flipt instances.

<BlogImage
  src="/observability/loki.png"
  alt="Audit Events Aggregated in Grafana Loki"
  width={1776}
  height={1304}
  className="shadow-md"
  caption="Audit Events Aggregated in Grafana Loki"
/>

As of [v1.27.0](https://github.com/flipt-io/flipt/releases/tag/v1.27.0), we have released functionality for a sending audit event payloads to a webhook sink. 🎉

Webhooks are a common way for applications to send data to a URL when an event happens. If you are familiar with GitHub, you may have seen the [webhooks](https://docs.github.com/en/developers/webhooks-and-events/about-webhooks) functionality that allows you to configure a URL to send events to when certain actions happen on a repository. This is the same concept in Flipt.

<StarCTA />

You can configure a URL with the [`audit.sinks.webhook.url`](https://www.flipt.io/docs/configuration/overview#audit-events-webhook) value and start receiving events as they happen to the web server that the URL is pointing to.

Use cases include: get notified in a Slack channel when a new flag is created, or transform and post that event to an [Events API](https://docs.datadoghq.com/api/latest/events/#post-an-event) in your observability solution (ie: Datadog). The webhook sink also can be configured with a signing secret to allow users to validate the event originated from somewhere appropriate.

<BlogImage
  src="/observability/webhook.png"
  alt="An Audit Event sent to a Webhook Sink"
  width={2880}
  height={1800}
  className="shadow-md"
  caption="An Audit Event sent to a RequestBin Webhook Sink"
/>

You can start using this functionality today! Check out the [documentation](https://www.flipt.io/docs/configuration/observability#webhooks) for more information as well as [an example](https://github.com/flipt-io/flipt/tree/main/examples/audit/webhook) in our main repository.

### Datadog

Datadog has long established themselves as a giant in the observability SaaS space. Knowing this, we figured that our users would benefit from tighter integrations with Datadog. The following sections will highlight some of the features we have implemented to allow for an improved observability experience if using Flipt with Datadog.

#### Exporting OTLP Traces

Flipt already has the ability to emit [OTLP](https://opentelemetry.io/docs/specs/otel/protocol/) traces to a configured collector. This depends on you spinning up the OTLP collector in your stack, and using the URL for that collector as configuration for Flipt. Until now however, we havent done a great job documenting how you could send your OTLP traces to third party tool such as Datadog.

We have recently published [an example](https://github.com/flipt-io/flipt/blob/cac1e65026add36cce51196fa9007eb4d235a5ae/examples/tracing/otlp/README.md#datadog-ui) on how to configure tracing with Datadog through the OTLP collector. Once the traces arrive in Datadog, you can leverage built-in features to [correlate traces and logs](https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces/), and [logs with metrics](https://docs.datadoghq.com/logs/guide/correlate-logs-with-metrics/), a nice two-way correlation. As things regress or errant things happen in your application, you can use these features to quickly track down a problem, and possibly correlate them to a switch in a feature flag.

<BlogImage
  src="/observability/datadog.png"
  alt="Flipt Traces in Datadog"
  width={1691}
  height={746}
  className="shadow-md"
  caption="Flipt Traces in Datadog"
/>

#### RUM

Many times organizations want a better view of not only what occurs server-side in their applications, but also on the front-end. For example, you might want to see exactly what steps a user took during a session, where the user might have navigated away (bounced), and a variety of other metrics. Organizations can use feature flags to supplement this especially when they are rolling out new UI functionality to see how a subset of users might react without rolling it out completely to their entire userbase.

[Datadog Real User Monitoring (RUM)](https://www.datadoghq.com/dg/real-user-monitoring/benefits-na/) is a drop in client that you can use within your front-end to capture the metrics that you care about during a user's session. The RUM client also has functionality for tracking the values of feature flags and correlating that with the metrics that the RUM client already keeps track of.

As of [v0.2.11](https://github.com/flipt-io/flipt-node/releases/tag/0.2.11) of our [Node/TypeScript client](https://github.com/flipt-io/flipt-node), we now have the ability to integrate with Datadog RUM natively.

Here's an example of how you can use the client to send evaluated values of feature flags to Datadog RUM:

```typescript
const metricsClient = new FliptMetrics(
  new FliptApiClient({
    environment: "http://localhost:8080",
    auth: {
      credentials: {
        username: "YOUR_USERNAME",
        password: "YOUR_PASSWORD",
      },
    },
  }).evaluation,
  datadogRum
);

const response = await metricsClient.variant({
  namespaceKey: "default",
  flagKey: "hello-this",
  entityId: uuidv4(),
  context: {},
});
```

The [README](https://github.com/flipt-io/flipt-node#metrics) in the repository goes a bit more in depth of how you can enable that functionality.

<BlogImage
  src="/observability/datadog-rum.png"
  alt="Datadog RUM with Flipt"
  width={2704}
  height={1560}
  className="shadow-md"
  caption="Datadog RUM with Flipt"
/>

The above image shows an instance where the flag `hello-this` in the `default` namespace was `enabled` during the entirety of a user's session. On the same Datadog UI page, you can click on the `Session & Replays` tab to see how that new functionality behind the feature flag may have impacted the user session compared to users who the feature flag was off or evaluated `false` for.

## Conclusion

With these new observability focused features, we hope to allow our users to gain more visibility into how feature flags are affecting their application/platform as a whole.

If you have any questions, other tools you'd like us to integrate with, or additional features you'd like to see added, please reach out to us on [GitHub](https://github.com/flipt-io), [X](https://twitter.com/flipt_io), or [Discord](https://www.flipt.io/discord).

We hope you enjoy these new features, and we look forward to hearing from you!


Improving Observability


Last week at Flipt, we took a break from our regular projects to kick off our first company-wide Hack Week. If you're unfamiliar with the concept of Hack Weeks, they are dedicated periods during which teams can work on creative and innovative projects outside their regular responsibilities. It was an excellent opportunity for everyone to stretch their creative muscles and explore new possibilities.

This post will cover the projects we developed, the insights we gained, and how these innovations might fit into our broader company vision.

We had three days to work on our hack projects, starting on Tuesday and culminating in a demo day on Friday. I'm incredibly proud of what the team accomplished in such a short period of time. Without further ado, let's dive into the projects.

## Project 1: Flipt Migrator

[Mark Phelps](https://www.linkedin.com/in/markphelps1/)

<BlogImage
  src="/hackweek/migrate.gif"
  alt="Flipt Migrator"
  width={1070}
  height={934}
  className="shadow-md"
/>

I wanted to simplify the process for users migrating from existing solutions to Flipt. One of the biggest challenges teams face after deciding to switch tools is the absence of a straightforward migration path. Teams often either:

1. Run the new solution in parallel with the existing one, leading to inconsistent data between the two systems.
2. Spend considerable time and effort writing custom migration code, sacrificing productivity that could be better spent on core business activities.

To address these issues, I created [Flipt Migrator](https://github.com/flipt-io/migrator), a CLI tool that automates the migration process. The tool extracts data from the existing solution, transforms it into a Flipt-compatible format, and allows teams to validate and import the data.

Above is a short .gif demonstrating the migrator, where I migrate data from a competitor into a format compatible with Flipt and validate it using our new `flipt validate` command.

The tool, written in Python, currently supports only one source but we open-sourced it to encourage community contributions. Check it out on [GitHub](https://github.com/flipt-io/migrator)!

It leverages the [Pydantic library](https://docs.pydantic.dev/latest/) for type validation in Python3, making data transformation more reliable and intuitive.

Our goal is to make Flipt the best solution for feature management, and that includes making it easy to migrate from other solutions. We're excited to see how this tool evolves over time.

## Project 2: Slack Bot

[Yoofi Quansah](https://www.linkedin.com/in/redblacktree/)

<BlogImage
  src="/hackweek/slack-1.gif"
  alt="Flipt Slack Bot Create Feature Flag"
  width={1510}
  height={844}
  className="shadow-md"
/>

Yoofi's project was a Slack bot designed to centralize communication and auditing for feature flag changes. In organizations where multiple teams communicate via Slack, feature flags often require cross-team coordination. Existing solutions lacked a centralized system for auditing and tracking these changes. Our Slack bot fills this gap by integrating feature flag management directly into Slack.

The bot operates through a custom server-side application that interfaces with Slack. Users configure Slack to send specific event notifications to this application. The application then performs various tasks, such as serving block kit forms back to Slack for creating new feature flags or listing existing flags within a specific namespace.

The Slack bot has the potential to become an indispensable tool for broader platform management within Flipt. We're exploring additional API functionalities and the possibility of enhanced authorization through OAuth or OIDC providers.

Outside of Slack, we've also got some new auditing features in the works that go beyond our existing [file-based event sink](https://www.flipt.io/docs/configuration/observability#audit-events), so stay tuned for more updates!

## Project 3: Ready - Feature Flight Plans

[George MacRorie](https://www.linkedin.com/in/geokeaton/)

<BlogImage
  src="/hackweek/ready.gif"
  alt="Ready Feature Flight Plans"
  width={1068}
  height={720}
  className="shadow-md"
/>

Developed by George MacRorie, our third project is "Ready," an app designed to standardize the way engineers plan and execute feature releases.

The journey from code completion to general availability is complex and varies from one organization to another. It often relies on word-of-mouth knowledge or outdated internal documentation. Ready aims to define this process, integrate all relevant tools, and build an executable flight plan for releases.

Ready is a task list focused on feature releases, offering rich integrations with essential production tools. It guides developers through a tailored process, allowing plans to be created, templated, and shared within the organization. A central dashboard provides visibility into active features and their current state.

Over time, Ready could become the go-to tool for getting new features into production. Some ideas include integrating more source control management systems, observability tools, and deployment solutions. Ultimately Ready could evolve to include autopilot features to automate actions at each step, allowing for a more streamlined and less error-prone process.

George also prepared a longer Loom video demonstrating the app. Check it out [here](https://www.loom.com/share/3d86a9d9864a43228e21a99ed5d5a0b3) to see Ready in action!

We're super exicted about the potential of this project and how it could help teams streamline their release processes beyond feature flags.

## Until Next Time

This first Hack Week was a great success, and we're already looking forward to the next one.

We hope you enjoyed this brief overview of our projects. If you have any questions, feedback, or want to try any of them out for yourself, please reach out to us on [Twitter (X)](https://twitter.com/flipt_io) or [Discord](https://www.flipt.io/discord). We'd love to hear from you!

Also, signup for our monthly [newsletter](https://www.flipt.io/#newsletter) to stay up to date with all things Flipt including new features, blog posts, and more!


Flipt's Inaugural Hack Week


Today we’re happy to announce the release of [v1.24.0](https://github.com/flipt-io/flipt/releases/tag/v1.24.0) of Flipt including a few major features including:

- A new `boolean` flag type that can be used in combination with [flag rollouts](/docs/concepts#rollouts) allowing you to enable a flag over a percentage of your user base and more
- A new, more straightforward set of evaluation API routes that you can also [enable/disable authentication](/docs/configuration/authentication#exclusions) separate from the existing ‘management’ APIs
- The ability to add user email addresses to [an allowlist](/docs/configuration/authentication#email-matches) when setting up OIDC authentication
- Improved performance and efficiency in flag evaluation with cache enabled
- New [Amazon S3 filesystem backend support](/docs/experimental/filesystem-backends#object) to serve your flags directly from an S3 bucket

## Boolean Flag Type and Rollouts

Previously, Flipt was designed with only a single flag ‘type’ in mind, a flag which can be enabled/disabled and optionally have variants that can be returned during evaluation time. While this worked well for many use cases, we recognized the need for more flexibility. With the introduction of the new `boolean` flag type, you can now enable a flag over a percentage of your user base more easily. This allows for better experimentation and targeted rollouts.

The `variant` flag type is still the default flag type, allowing you to use flags in the same way as before. The new `boolean` flag type adds an extra layer of versatility. Boolean flags work hand in hand with a new abstraction that we call 'rollouts' or 'rollout rules'.

[Rollouts](/docs/concepts#rollouts) work similarly to rules used during evaluation of a `variant` flag in that the first rule that matches ‘wins’, however there are a few key differences:

1. Rollouts are only allowed for `boolean` flag types and allow you to return a different value for a flag’s `enabled` property.
2. There are currently two types of rollouts: **threshold** and **segment**. Segment rollouts work just like previous rules in that you specify a segment that you want the rule to match. Thresholds, in contrast, are used to return a different result to a percentage of requests.
3. Rollouts allow you to specify the value to return for a boolean flag, either `true` or `false`.

It’s perhaps easier to show an example of the problem that boolean flags and rollouts were added to solve, by showing how you would have had to configure Flipt pre-v1.24.0 without rollout support and then how you would do the same now.

**Imagine the following scenario:**

You have a feature in your application that you want to roll out in stages to all users. Before the release of v1.24.0 of Flipt, you would have had to:

1. Create a flag with `true` and `false` variants, or `on`/ `off` or similar
2. Manually handle these variants in your application code, converting them to `booleans`, since `variants` can only be strings.
3. Configure a segment with constraints that matches the segment you want to target (ie: `all-users`)
4. Create an evaluation rule which returns the `true` variant to X% of requests, and the `false` to the remaining percentage.

<BlogImageGallery
  images={[
    {
      src: `/rollouts/rollout-variants-1.png`,
      alt: "Creating the Flag and Variants",
    },
    {
      src: `/rollouts/rollout-variants-2.png`,
      alt: "Creating the Segment to Match",
    },
    {
      src: `/rollouts/rollout-variants-3.png`,
      alt: "Setting up the Evaluation Rule",
    },
    {
      src: `/rollouts/rollout-variants-4.png`,
      alt: "The Flag with Variants and Evaluation Rule",
    },
  ]}
/>

As you can see , managing rollouts with boolean flags in Flipt used to be quite complex. However, with the release of [v1.24.0](https://github.com/flipt-io/flipt/releases/tag/v1.24.0), the process has become much simpler and more intuitive.

Now, to achieve the same rollout scenario, you can follow these steps:

1. Create a `boolean` type flag. This flag will represent the feature you want to roll out.
2. Configure a threshold-based rollout, specifying the percentage of requests that should receive the flag enabled. The remaining percentage will receive the flag disabled.
3. Thats it! 🎉 You can now use the flag in your application code as a boolean value.

<BlogImageGallery
  images={[
    {
      src: `/rollouts/rollout-boolean-1.png`,
      alt: "Creating the Boolean Flag and Rollout",
    },
    {
      src: `/rollouts/rollout-boolean-2.png`,
      alt: "The Boolean Flag and Rollout Rule",
    },
  ]}
/>

Not only are boolean flags easier to setup for this scenario, they are also easier to update in place in the UI. You can now simply drag the slider or manually input the percentage in the rollout input box and click update to increase or decrease the percentage over time!

<StarCTA />

## Evaluation API Routes and Security Configuration

While designing and implementing boolean flags, we also took the time to see how we could make evaluation more intuitive to support both flag types. We decided to create a new set of evaluation API routes, separate from the existing `/api/v1` prefix that can work with both variant and boolean flags.

Several users have previously asked how to separate the ‘management’ API (created/edit/delete flags and other resources) from the ‘evaluation’ API as they serve two distinct audiences. Normally the ‘management’ API is used by team members to setup Flipt resources and should not be exposed publicly. Depending on a team’s application architecture however, they may need to evaluate flags from ‘untrusted’ and unauthenticated requests, such as when using feature flags directly from a browser based application.

Before this release, our only answer was to suggest putting Flipt behind a reverse proxy such as Caddy or Nginx and configuring access rules there. While we still think reverse proxies are a good idea for handling things such as request routing, rate limiting, and more, v1.24.0 of Flipt now separates the ‘management’ and ‘evaluation’ APIs completely and allows the operator to enable or disable authentication for each independently. We think that this new set of evaluation APIs and authentication configuration will allow Flipt to work for a larger variety of applications and use cases than before.

Checkout the new [evaluation API documentation](/docs/reference/overview#v1-24-0) as well as all of our updated clients which now also support these new endpoints!

We also added the ability to configure `allowed_emails` when setting up OIDC Authentication for Flipt. This addition was driven by a [user request](https://github.com/flipt-io/flipt/issues/1872) to be able to configure specific users within their organization to have access to Flipt when configuring authentication with Google.

**Note:** in order for this feature to work properly, make sure you specify the `email` scope when [configuring your OIDC setup](/docs/configuration/authentication#email-matches) for Flipt.

## Improved Performance, S3 Backend and more

Finally, we've merged two amazing user-contributed PRs which made significant improvements to flag evaluation performance and efficiency as well as added support for serving flag state straight from an S3 bucket!

### Faster Evaluation

With cache enabled, the evaluation of variant flags has become even quicker as we now cache potential constraints that would match a request at the storage layer, meaning they can be used across different `entityIds`.

From the [PR](https://github.com/flipt-io/flipt/pull/1910):

> after some investigation we found the main cause of delays in evaluation requests were the rules query, in our case running Flipt with a PostgreSQL DB on AWS RDS we were getting up to 1s of delays, with this fix it reduced to a few milliseconds

We consistently aim to keep Flipt fast for our users, and thus your users so we very much appreciate any contributions that help improve performance!

### Flag State Not Just for Git Anymore

Last but certainly not least, Flipt now supports the use of [Amazon S3 as a filesystem backend](/docs/experimental/filesystem-backends#object) to serve your flags directly from an S3 bucket or set of buckets.

In [v1.23.0](https://github.com/flipt-io/flipt/releases/tag/v1.23.0) of Flipt we released a new set of experimental features allowing our users to manage their Flipt state. We strongly believe that feature flags should be treated like any other application configuration in our your infrastructure, which means you should also be able to use industry best practices such as managing configuration via CI/CD pipelines with GitOps. We also recognize however that every organization is different and so we’re delighted to now support Amazon’s S3 as an alternative filesystem backend!

This was another amazing user contribution. Here’s the reasoning for the feature straight from the [PR](https://github.com/flipt-io/flipt/pull/1900):

> In an AWS environment, this would allow deploying a readonly container and not require communication to a git repository outside of the deployment environment.

This feature opens up the door for us to implement other object stores such as Azure Blob Store and Google Cloud Storage as well, so please [open an issue](https://github.com/flipt-io/flipt/issues/new/choose) if you’d like us to support those backends as well!

## That’s a Wrap

This release is our most feature packed yet! We’re delighted to add functionality that makes Flipt better for all of our users, and love the community contributions we’ve gotten so far this summer.

As always, feel free to reach out on [Twitter](https://twitter.com/flipt_io) (X?), [Discord](/discord), or open a [GitHub issue](https://github.com/flipt-io/flipt/issues/new) for any functionality that you’d like to see implemented. We’d also love to answer any questions or just chat about software delivery in general.

Checkout the full [changelog](https://github.com/flipt-io/flipt/releases/tag/v1.24.0), and give the latest release a look and let us know what you think!


Boolean Flags and Rollout Functionality


<BlogImage
  src="/labs/flipt-labs.png"
  alt="Announcing Flipt Labs"
  width={2400}
  height={1350}
>
  <span>
    Image generated by <a href="https://www.midjourney.com/">MidJourney</a>
  </span>
</BlogImage>

Today we’re excited to show off a new project we’ve been working on. We’re calling it [Flipt Labs](https://github.com/flipt-io/labs). This project aims to show off everything you can do with Flipt by providing examples of real-life applications and how they can integrate with Flipt.

## Applications as Tutorials

Creating high-quality tutorials takes effort and dedication. We believe that good tutorials can make a big difference in helping users fully leverage the power of Flipt. That's why we decided to create Flipt Labs, where we can showcase different applications and really put ourselves in our user’s shoes.

One of the challenges in demonstrating an application such as Flipt is that it's not entirely useful on its own. To see the real benefit of using feature flags, it's necessary to integrate them within applications to demonstrate practical use cases.

Our first full application in the Labs repository is an interactive tutorial that steps the user through several ways of using Flipt:

1. Using flags for basic on/off global release
2. Using segmentation and evaluation to serve different responses based on request context
3. Using our new [GitOps backend](/blog/gitops-announcement) to support managing your feature flags in your Git repository

We decided that the tutorial would be based around the scenario of an engineer wanting to experiment with and eventually release a chatbot to their end users. We got a bit cheeky and chose to sprinkle in some LLM/AI magic as well to keep things interesting. ✨

## Try it Out

<BlogImage
  src="/labs/chatbot.png"
  alt="Flipt Chatbot Application"
  width={1692}
  height={1124}
></BlogImage>

Check out our [first full demo application](https://github.com/flipt-io/labs/blob/main/chatbot/README.md) and tutorial on GitHub. You can run it locally via NPM + Docker Compose, or in a GitHub Codespace. We would love to hear your feedback, so please let us know what you think!

We're working on another post that will go into more detail about how we built a working chatbot with context about Flipt itself! In the mean time, check out the code to see examples of using both the [Flipt Node SDK](https://github.com/flipt-io/flipt-node) for the frontend and the [Flipt Python SDK](https://github.com/flipt-io/flipt-python) for the backend, as well as how our [GitOps Backend](https://www.flipt.io/docs/experimental/filesystem-backends#git) can integrate into your existing workflow.

## Onward

We believe that this project will highlight the potential of Flipt and provide inspiration for its integration in various contexts. By showcasing practical examples, Flipt Labs aims to empower users to unlock the full potential of Flipt in their own projects. Through this initiative, we hope to encourage creativity and innovation in utilizing Flipt's features and capabilities.

If you have any ideas for applications you would like to see, please let us know! Feel free to open an issue on the [Flipt Labs](https://github.com/flipt-io/labs) repository or reach out to us on [Discord](https://www.flipt.io/discord) or [Twitter](https://twitter.com/flipt_io).

Stay tuned for updates as we release new projects and tutorials! Happy experimenting 🧪.


Announcing Flipt Labs


<BlogImage
  src="/gitops/flipt-gitops.png"
  alt="Flipt for GitOps"
  width={2400}
  height={1350}
>
  <span>
    Image generated by <a href="https://www.midjourney.com/">MidJourney</a>
  </span>
</BlogImage>

Today we’re excited to announce that we’re exploring a new type of backend for Flipt!

The team has been discussing and exploring ways Flipt could be better positioned to support GitOps-style workflows. For those unfamiliar with GitOps practices, we recommend the WeaveWorks [Guide To GitOps](https://www.weave.works/technologies/gitops/).

Our new [filesystem backends proposal](https://github.com/flipt-io/flipt/discussions/1652) adds two ways to configure Flipt to serve flag state content encoded as yaml stored in files, `local` and `git`.

We’re keen to understand if this idea resonates with you and your use case, this will help us shape the implementation, which is currently underway.

## Problem

We feel that many feature flags offerings (including Flipt today) work at odds with GitOps practices. GitOps prescribes that you store all your configuration in one or more Git repositories. This gives your entire system the benefits of version control. Feature flag configuration is just like any other configuration in that it adjusts the behaviour of your applications at runtime. However, we find that it is rarely stored and tracked within Git itself.

If you’re using GitOps practices for your wider configuration needs, but your feature flag solution is completely isolated, then you’re subverting the benefits and traceability of Git. When the alerts start firing you'll have to correlate your configuration changes in Git with the auditing features of your flagging solution.

Alternatively, some existing feature flag solutions support GitOps by bolting a declarative syntax onto their existing imperative API. For example, some offerings provide a [Terraform](https://www.terraform.io/) provider which is an entirely legitimate way to create a GitOps-style contribution experience. In fact, we’re still considering having one for Flipt and our existing relational backends. However, this approach does have drawbacks, primarily, there are now two sources of truth.

Reconciling from your Git repository into your online feature flag service also has to be performed. This process is often performed during some kind of continuous delivery step (e.g., via a GitHub action), which likely pushes changes into your feature flag system. You may have to manually intervene when this process fails for intermittent reasons. Your environment must also have credentials and access to your application repository and your feature flag solution. Ideally, your feature flag solution provides tooling to ease the extra work required to set up this CI/CD process and make it resilient.

In this scenario, given you’re using a self-hosted solution (such as Flipt), you now have the following set of applications to run and configurations to manage:

- The feature flag service itself (e.g. the `flipt` binary)
- Backing database for storing flag state (e.g. SQLite, PostgreSQL etc)
- Flag state configuration (e.g. in Terraform HCL or via some other process such as `flipt import`)
- CI processes for reconciliation (plus monitoring)
- For solutions such as Terraform, storage of reconciliation state (`tfstate` files)

This is all achievable, but care must be taken when building the reconciliation process to ensure a predictable end state.

Ultimately, feature flag offerings have to build a host of features you’re already getting from your SCM (e.g. GitHub or Gitlab). Version control, peer review, access control and backup to name a few.

## How We Intend to Serve GitOps

We intend to have Flipt pull feature flag state directly from Git at runtime.

We want Flipt to be able to drop its dependency on a database for flag state and go directly to the source. This removes the need for a reconciliation process altogether. By adopting a pull model, we embrace eventual consistency and allow Flipt to recover automatically during transitive errors without intervention.

You'll simply be able to configure Flipt with access to your repository and with some additional configuration, you will be able to control how and where to locate your flag state files. Your repository could be used solely for Flipt, it could be your existing configuration repository, or it could be stored within a monorepo.

<BlogImage
  src="/gitops/gitops-scm.png"
  alt="Flipt for GitOps - SCM"
  width={800}
  height={600}
/>

Flipt’s evaluation and read APIs won't change, meaning your existing interactions with the Flipt SDKs can remain the same. The existing [`flipt export`](https://www.flipt.io/docs/configuration/storage#export) command produces configuration files valid for committing straight to your repositories for ease of transition.

Flipt will be configured to serve a particular branch for a target repository. This branch doesn’t always have to be `main`, meaning you can deploy Flipt in multiple ways to support validating flag state changes in arbitrary Git branches or tags.

You'll also be able to run Flipt over a local directory. In this configuration, Flipt will simply serve your flag configuration without the need to go through Git. Updates to the local directory will update Flipt automatically. There is no need to access an external running instance of Flipt to replicate or alter your production feature flag state in your _local_ environment. Simply check out your repository and run `flipt` to begin experimenting.

<BlogImage
  src="/gitops/gitops-local.png"
  alt="Flipt for GitOps - Local"
  width={800}
  height={600}
/>

## Flipt in CI

Alongside building a new core backend for Flipt, we’re also developing a suite of tools for validating your use of Flipt during your existing continuous integration workflows.

Before Flipt can serve your configuration files, it is important to validate they are fit for consumption. Failure to do so could lead to runtime errors or unexpected behaviour. `flipt validate` is a new subcommand currently being added to Flipt. This will be useful for local validation or running directly in a CI pipeline.

We’re also developing a GitHub action which simplifies running `flipt validate` which will make it trivial to add alongside your existing GitHub workflows.

## Just the Beginning

We’re just getting started turning our proof-of-concept into something ready for production Flipt usage. While we have a good feeling that the experience will resonate with users, there are still a number of interesting technical challenges we expect to have along the way. This is why we wanted to get this plan out early in order to hear your feedback.

Help come shape the future of this solution on [GitHub](https://github.com/flipt-io/flipt/discussions/1652), or in our [Discord](https://www.flipt.io/discord).

Also, check out our new landing page for more of what's possible when combining Flipt + Git: [flipt.io/gitops](https://www.flipt.io/gitops).


How's about Git?


<BlogImage
  src="/audit-events/audit-events.png"
  alt="Events for Flipt"
  width={2400}
  height={1350}
>
  <span>
    Image generated by <a href="https://www.midjourney.com/">MidJourney</a>
  </span>
</BlogImage>

Events and logging are vital for many organizations to either keep track of history within the application/platform or to actually take action on things once they happen. Two brief examples include:

1. An organization keeping track of payment events that have happened on their platform
2. A platform triggering a job to purge data somewhere if an event is of a particular nature

In Flipt's case it may prove essential for some organizations to know when feature flags are mutated in some fashion as they can affect the real-time behavior of the consumer’s application.

It's also important to know who made the change and when it was made, especially when it comes to trying to determine the root cause of an issue in the consuming application. This is where audit events come in.

We're happy to announce that support for these type of audit events are now implemented as of [v1.21](https://github.com/flipt-io/flipt/releases/tag/v1.21.0) of Flipt! 🎉

## Ideal Implementation

Before diving into how the feature was implemented, it is worth discussing what an ideal scenario looks like for Flipt and audit events. Many providers, whether SaaS or open source, leave it strictly up to the consumer to deal with events as they please. In most cases, this is done through a webhook URL that you provide to the platform which sends a request to that URL when an event occurs.

An application that is listening to the endpoint of the webhook URL can then do whatever they want with the event. For instance, they can choose to send the message to a Kafka broker, do some other sort of pub/sub, or send that audit event to Slack in a specialized channel, so this is more of the laissez-faire approach.

What we decided on for an ideal implementation is to have this concept of first-class support for ‘native sinks’. In this case, Flipt will house the logic necessary to interact directly with the sinks that the user provides configuration for. The idea is that messages can go straight to a Kafka broker, pub/sub system, Slack channel, etc, as a direct form of communication. This allows users to avoid having to write their own backend to communicate with these systems and avoid the extra network hop.

The implementations for communicating with these native sinks would ideally have an abstraction point so that users can make easy sense of it to contribute to Flipt and write their own implementations for native sinks. One of the downsides of this native approach, however, is that as more and more sinks become implemented, it can lead to version churn for the Flipt application. To combat this potential for churn and to make contributions more accessible, we are currently looking into a way to make the sink implementations pluggable in their own repositories, potentially using a plugin system like Hashicorp's [go-plugin](https://github.com/hashicorp/go-plugin).

## Implementation

Implementing any event-driven system comes with many considerations and trade-offs. On the producer side of the system, an ‘event’ usually represents things in the system that have already happened to then notify the consumer. The benefit for the consumer(s) is that once the event is received on its end it knows the event is complete, and therefore does not have to poll and waste resources getting the complete event from the source.

While the definition of an event-driven system seems simple enough, there are many considerations to keep in mind. In this case particularly, what happens to the producer or consumer during high-traffic scenarios? Or what if the consumer becomes unavailable for a portion of the time? These represent just a few of many considerations, and we sought to address those in our implementation.

<BlogImage
  src="/audit-events/event-diagram.png"
  alt="OTEL Event Diagram"
  width={300}
  height={261}
  caption="The two basic components of an event system. One has to consider where to implement the functionality to deal with the considerations described above."
/>

Our first stab at an implementation was a basic homegrown publisher which contains abilities to batch events and send batches asynchronously to different sinks that are configured. This is done so the consumer is not burdened with a whole bunch of writes, and it frees up the publisher to do other work without worry of communication failures with the consumer (addressing some of those considerations mentioned above).

In addition to a batch size, there should also be configuration for a flush period to avoid messages laying around in memory for an indefinite amount of time. Considering all of this, it was a pretty straightforward implementation, but halfway through we realized that this problem has been solved several times for a lot of different use cases, leading us to pause the publisher implementation and seek out a more tried and true solution.

### Enter OpenTelemetry

<BlogImage
  src="/audit-events/otel.png"
  alt="OpenTelemetry"
  width={1000}
  height={524}
/>

For the sake of brevity, [OpenTelemetry](https://opentelemetry.io/) will not be discussed in this article, but simply how it was used in the Flipt application to achieve desired functionality for audit events. For trace data, OpenTelemetry (OTEL) not only provides configurable batching, and flushing abilities but also allows users to provide implementations of [`Exporters`](https://opentelemetry.io/docs/concepts/glossary/#exporter) which OTEL uses to send trace data to whatever the implementation specifies. In simpler words... OTEL does everything already implemented in the homegrown publisher implementation, and it already exists as a dependency in the Flipt source code for exporting trace data to [Jaeger](https://www.jaegertracing.io/), [Zipkin](https://zipkin.io/), or [OTLP](https://opentelemetry.io/docs/reference/specification/protocol/).

There might be some lack of context as to what a `Trace` or `Span` here is and how it ties into audit events for Flipt. Essentially, a `Trace` represents a unique transaction of a collection of operations throughout an application. A `Span` represents a single unit within that `Trace` or a single operation. Spans allow for additions of [Events](https://opentelemetry.io/docs/concepts/signals/traces/#span-events) which are viewable through any tracing provider client. The addition of `Event`s is the bit that we leveraged for audit events.

Specifically in the code, we’ve implemented a middleware that will intercept the request, and determine if the event is auditable, if it is, the code will add the details of that audit event to the Span via the `Event` described above. When a batch size or flush period is reached for the `SpanProcessor`, events are pulled off of the `Span`(s) and converted into a data structure housed in our code to then send it off to the native sinks that the user has configured on their Flipt instance. In addition to viewing the audit events in their sink implementation, an added benefit of using OTEL is the span events will show up in the tracing provider client for users to view.

<BlogImage
  src="/audit-events/jaeger-ui.png"
  alt="Jaeger UI with Event"
  width={3024}
  height={1542}
  caption="The audit event added to the flipt.Flipt/CreateConstraint span. The flipt* properties represent the audit event."
  className="shadow-md"
/>

## What's Released

With this initial release, there is only functionality for audit event logging to a file on disk. Users can specify the name of that log file, and Flipt will write the audit events to it in a JSON-encoded format.

If you were to `tail` the logs of your audit event log file for orthodox Flipt usage you might see something like the following:

```json
{"version":"0.1","type":"flag","action":"created","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"description":"your favorite NBA team","enabled":true,"key":"scenter","name":"scenter","namespace_key":"default"},"timestamp":"2023-05-01T14:01:12-05:00"}
{"version":"0.1","type":"variant","action":"created","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"attachment":"","description":"","flag_key":"scenter","id":"41e84846-bfbd-426f-afa9-3358d0d40875","key":"lakers","name":"","namespace_key":"default"},"timestamp":"2023-05-01T14:01:12-05:00"}
{"version":"0.1","type":"segment","action":"created","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"constraints":[],"description":"Segments for evaluation NBA teams","key":"postclavicula","match_type":"ANY_MATCH_TYPE","name":"postclavicula","namespace_key":"default"},"timestamp":"2023-05-01T14:01:12-05:00"}
{"version":"0.1","type":"constraint","action":"created","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"id":"a7ccde21-870e-4e36-953c-4d5849d96a77","namespace_key":"default","operator":"eq","property":"championship","segment_key":"postclavicula","type":"STRING_COMPARISON_TYPE","value":"most"},"timestamp":"2023-05-01T14:01:12-05:00"}
{"version":"0.1","type":"flag","action":"updated","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"description":"your favorite NBA team","enabled":false,"key":"eyepoint","name":"eyepoint","namespace_key":"default"},"timestamp":"2023-05-01T14:01:37-05:00"}
{"version":"0.1","type":"token","action":"created","metadata":{"actor":{"authentication":"none","ip":"127.0.0.1"}},"payload":{"io.flipt.auth.token.description":"hello","io.flipt.auth.token.name":"worldh"},"timestamp":"2023-05-01T14:02:45-05:00"}
```

Currently, the auditable events are all CRUD operations (sans Read) upon the following entities:

- [Flag](https://www.flipt.io/docs/concepts#flags)
- [Variant](https://www.flipt.io/docs/concepts#variants)
- [Distribution](https://www.flipt.io/docs/concepts#distributions)
- [Segment](https://www.flipt.io/docs/concepts#segments)
- [Constraint](https://www.flipt.io/docs/concepts#constraints)
- [Rule](https://www.flipt.io/docs/concepts#rules)
- [Token](https://www.flipt.io/docs/authentication/methods#static-token)
- [Namespace](https://www.flipt.io/docs/concepts#namespaces)

The event payload which is logged has the following fields for each event:

- `type`: the type of entity changing
- `action`: the action taken upon the entity
- `metadata`: extra information about the event such as identity of the subject who initiated the event
- `payload`: the actual payload of the request on Flipt
- `timestamp`: the timestamp of when the event was created

Any combination of actions on those entities will result in an event with the above fields being logged out to the file you specify via configuration:

```yaml
audit:
  sinks:
    log:
      enabled: true
      file: /tmp/flipt/audit.log
```

We also have an example of how you could potentially collect, parse, visualize, and query these events using Grafana's [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) + [Loki](https://grafana.com/docs/loki/latest/) in our [examples](https://github.com/flipt-io/flipt/tree/main/examples/audit) on GitHub. This would allow you to query the audit events in a similar fashion to how you would query logs in a traditional logging system, even when Flipt is running in a distributed fashion.

<BlogImage
  src="/audit-events/loki.png"
  alt="Grafana Loki Query"
  width={2880}
  height={1800}
  caption="Querying Flipt audit events using Grafana Loki"
  className="shadow-md"
/>

## What's Next

We are excited for this release to see just how users will interact with this feature, and also excited for community discussion on potential event sinks that have not been added yet.

[Contributions](https://github.com/flipt-io/flipt#contributing) are always appreciated! This [document](https://github.com/flipt-io/flipt/tree/643ccc1d541e6a1e79cfeeb548df6b53a09e9914/internal/server/audit) describes how you can implement your own audit event sink.

We hope you found this post useful. As always, you can find us on [GitHub](https://github.com/flipt-io), [Discord](https://www.flipt.io/discord), [Twitter](https://twitter.com/flipt_io), or [Mastodon](https://hachyderm.io/@flipt).


Audit Events for Flipt


<BlogImage
  src="/dagger/embracing-dagger.png"
  alt="Flipt loves Dagger."
  width={2400}
  height={1350}
>
  <span>
    Background by [Paul Volkmer](https://unsplash.com/@laup) on Unsplash
  </span>
</BlogImage>

In a [previous blog post](/blog/generating-the-flipt-go-sdk), I briefly mentioned our adoption of Dagger.
I wanted to take a moment to introduce Dagger and explain how we’re slowly but surely adapting our build, test (and one-day release) pipeline to leverage it.

> Dagger is a programmable CI/CD engine that runs your pipelines in containers.

[Dagger](https://dagger.io) lets you describe your build, test and release pipelines as code, often alongside the application you’re developing it around. They support an expressive set of SDKs in multiple programming languages. At Flipt we’re using Dagger's (you guessed it) [Go SDK](https://github.com/dagger/dagger/tree/main/sdk/go).

Perhaps unsurprisingly, in Dagger, almost everything is a container. I say unsurprisingly since a lot of the team also gave us Docker, the technology which put Linux containers truly on the map. Each of the Dagger SDKs exposes a builder pattern interface with the primary function of describing a graph of instructions. These instructions are remarkably similar to those found in a Dockerfile, which is not a coincidence. These instructions also produce layers in resulting OCI images, which can then be published or exported from your Dagger pipeline.

I don’t know about you, but I do love a DAG and so does Dagger. The DAG (Directed Acyclic Graph) is intrinsic to what makes it such a neat solution. No points for guessing where they got their name.

<BlogImage
  src="/dagger/dag.svg"
  alt="A DAG"
  width={509}
  height={144}
  caption="A DAG"
/>

A DAG is a set of vertices (the boxes) and edges (the arrows). The edges express dependencies from one vertex to the next. Before I can copy my Flipt binary into an appropriate location for execution, I first need to build it from the source. In order to run `go build` I first need to install Go.

DAGs allow us to express the dependent order in which operations must occur. Where this differs from a plain old sequential list, is that each vertex (entry) can have more than one (potentially) independent inputs. When you describe your workflow in this way, an execution engine (such as Dagger) can leverage the fact that fully _independent_ operations can be executed concurrently. If you’re running your workflow on a multicore machine you may even see these operations run in parallel.

In the rough illustration above, our Go and Node components can be built independently from one another. At this stage there are no interdependencies, so they can be run concurrently. Once both operations have been completed, we bring the results together.

### How does it work?

Dagger itself is comprised of a server and client architecture. The server (known as the engine) runs in a container on a target Docker host. Consumers (like Flipt) use one of the Dagger-generated and managed SDKs in one of the supported languages. These SDKs managed to initialize the Dagger engine container for you, as well making all the GraphQL API calls. Dagger has chosen to embrace GraphQL, which is a particularly good fit when you’re talking in directed acyclic graphs.

Having a GraphQL API means consumers are not strictly bound to only the SDKs developed by the Dagger team. Anyone can take the GraphQL docs and bake their own client in the language of their choice. This is similar to how we at Flipt use protobuf and gRPC and generate our managed clients from these definitions.

<StarCTA />

Under the bonnet Dagger leverages [Buildkit](https://github.com/moby/buildkit/). It was originally created by [Tõnis Tiigi](https://github.com/tonistiigi) and now lives under the Moby (Docker) brand. It powers most of `docker buildx`, where Dockerfile’s are actually transpiled into `llb` instructions (A Buildkit structure for describing DAGs) and then _solved_ (evaluated) by a Buildkit daemon.

If you’re familiar with the concept of `git` and its _plumbing_ vs _porcelain_ commands, I would liken the same to Dagger compared with Buildkit. Buildkit is the plumbing and Dagger is the porcelain. Buildkit exposes a fine-grained API for solving DAGs of instructions, which produce OCI images and layers. Dagger is a fast-evolving, developer-friendly layer on top. A layer focussed on solving the higher-level problem of expressing build, test and deployment pipelines in code.

I could talk about these pieces of tech all day, but for now, here is how we’re leveraging them at Flipt to build and ship quickly with confidence.

## Flipt’s Build, Test, Release Pipeline

<BlogImage
  src="/dagger/build-test-deploy.png"
  alt="The classic build, test, and release pipeline"
  width={2570}
  height={694}
  caption="Build, Test, and Release"
/>

Using Dagger we’re developing the classic build, test, and release sequence.

### Build Phase

The Build phase involves sourcing all the raw materials and putting the puzzle together. Effectively we perform the following five-step process:

1. Establish a base image containing all the build and test tools necessary.
2. Fetch all of our source code from GitHub.
3. Download all the Go and Javascript libraries we need.
4. Build the thing(s).
5. Copy the resulting Flipt binary into its own image with only the necessary _runtime_ dependencies.

If you’ve had experience writing a `Dockerfile` in the past, you may be familiar with some tricks folks employ to make better use of the Docker build cache. For example, with Go you might first add just your `go.mod` and `go.sum` files to your container before the rest of your application. Next, you will likely run `go mod download` in order to populate the module cache. Once this is complete then you add the rest of the application source and perform the build. You might also employ Docker's newer `RUN` cache features and mount a cache volume.

```docker
FROM golang:1.20

WORKDIR /src

ADD go.mod .
ADD go.sum .

RUN --mount=type=cache,target=/go/pkg/mod go mod download

ADD . .

RUN --mount=type=cache,target=/go/pkg/mod go build ./...
```

The same can be achieved with Dagger and the experience is familiar. The following is an illustration of using the Go SDK in Dagger to layer our graph to make the best use of caching for Go applications.

```go
func GoApp(ctx context.Context, client *dagger.Client) (*dagger.Container) {
	deps := client.Host().Directory(".", dagger.HostDirectoryOpts{
		Includes: []string{
			"./go.mod",
			"./go.sum",
		},
	})

	return client.Container().
		From("golang:1.20").
		WithWorkDir("/src").
		WithMountedDirectory("/src", deps).
		WithMountedCache("/go/pkg/mod", client.CacheVolume("go-mod-cache")).
		WithExec([]string{"go", "mod", "download"}).
		WithMountedDirectory("/src", client.Host().Directory("."))
		WithExec([]string{"go", "build", "."})
}
```

Our pipeline for building Flipt ultimately leverages this same process, which is to use layers effectively and take advantage of the engine's layer caching, and mounted cache volumes for finer control ensuring fetched or computed assets are preserved between builds. As with Go, the same applies for Node and its dependencies. First add the dependency manifests (`package.json`), install the dependencies and then add the source code.

Beyond this, we have some other details which are a little more specific to how we build Flipt:

1. The Flipt repo is a Go multi-module repository and uses a [Go workspace](https://go.dev/doc/tutorial/workspaces) to tie it all together. We read the contents of the `go.work` file to identify the multiple sets of `go.mod` and `go.sum` file pairs through the repository and ensure they’re all added before downloading dependencies.
2. `client.CacheVolume(...)` in Dagger takes a string to identify the cache volumes key. We build this key based on a hash of the `go.sum` contents. This invalidates our cache each time our sum file changes. The idea is to keep our cache fresh over time and stop it from growing indefinitely.
3. We initially exclude adding the `ui` directory containing our frontend code from the Go base. Instead, we mount this directory into its own container based on a `node` image and produce a static directory containing all of our frontend assets. This resulting static assets directory is copied into the base Go container and we use Go’s embedding to bake this into the binary.

This process is defined in the function we call `Base(...)` and the result is a `*dagger.Container`. Once all the dependencies have been built and added appropriately, we build the `flipt` binary into a well-known path inside the last layer of the container.

There is another function called `Flipt(...)` which takes the resulting container from `Base(...)` and copies the `flipt` binary into a fresh, empty `alpine` Linux image. This image is prepared with just the runtime dependencies needed for Flipt and gives us a thin distributable image for release.

### Test Phase

There are two important artifacts out of the Build phase which become inputs into our Test phase. The first is the base image (the result of steps 1, 2 and 3) with all the build and test tooling and source code. The second is the distributable Flipt OCI image (the thinner distributable image based on `alpine`).

In the testing phase, we first use the base image again to run our suite of Go unit tests. We run our unit test suite against each of the databases Flipt supports (SQLite, Postgres, MySQL and CockroachDB). The unit test suite requires Go to be installed, along with access to all the source code, which is why we use the base image.

The release of Dagger `v0.4.0` came with support for service containers. If you’re familiar with `docker-compose` and how it supports multiple containers being networked together, then this feature will be familiar. It supports the ability to run containers which are used as dependencies for your workflows as network addressable services. For example, a backing database, or your very own service that you want to test over the network. This feature was crucial for connecting the various databases Flipt supports during unit testing. It also allowed us to further build out our integration testing pipeline.

In a [previous blog post](/blog/generating-the-flipt-go-sdk), we discussed building our new Go SDK. In order to gain confidence in Flipt, its API and our SDK, we leverage the full power Dagger gives us with defining our pipelines in Go.

We first defined a suite of test cases using the Go SDK, which exercises a common set of Flipt operations: create a namespace, create a flag, define its variants and segments, evaluate it, etc. The core functional requirements of Flipt as a feature flagging system.

Given how we built our SDK to abstract away details such as the transport used and authentication, we could then easily extract those (non-functional requirement) details as arguments for running our test suite. This means that we could run our suite against the HTTP or gRPC API with or without authentication enabled. With the introduction of namespaces in our [latest release](https://github.com/flipt-io/flipt/releases/tag/v1.20.0), we further enhanced the suite to run across a configurable namespace. Now that our test suite can be run in a number of configurations, we have an n-dimensional matrix of possibilities.

<BlogImage
  src="/dagger/test-matrix.png"
  alt="Our test matrix"
  width={444}
  height={394}
  caption="Our test matrix"
/>

By simply using a set of nested loops in Go, we have a tidy way of expressing the multitude of possible ways Flipt could be run, giving us confidence that we’re not breaking something each time we add or change some functionality. Couple this with the ability to execute each possible configuration combination in a goroutine and we have a fast, concurrent way of executing the many permutations.

```go
func TestAll(base, flipt *dagger.Container) error {
	var g errgroup.Group
	for _, auth := range []bool{true, false} {
		for _, namespace := range []string{"", "some-namespace"} {
			for _, transport := range []string{"http", "grpc"} {
				g.Go(testFlipt(base, flipt, auth, namespace, transport))
			}
		}
	}
	return g.Wait()
}
```

Our pipeline is now fairly large and relatively complex. Lots of layers and concurrent operations in a web of directed edges and vertices are used to describe both build and test functionality. As with all big and complex CI/CD pipelines, they can start to slow down the more you add. This is where the advanced caching capabilities of Buildkit and Dagger come into play.

The beauty of the DAG in the Buildkit world is that each computed vertex (node) in the graph and its outputs are stored in content-addressable storage. Buildkit takes advantage of when vertices whose definitions and their inputs do not change and avoids recomputing any of these outputs. Instead, it retrieves previously computed results and uses those instead.

Locally, we have seen a full 10-minute end-to-end Flipt build pipeline with subsequent integration suite go from 10 minutes with an empty cache to **17 seconds** on a subsequent run. Flipt currently references 3.2GB **(!)** of Go module dependencies. When you compile Flipt with an empty build cache, a fresh run will produce 1GB of build-cache entries. All this takes a considerable amount of time to fetch and compile. Dagger makes a lot of this work trivial to compute once and reuse on subsequent runs.

Buildkit has the functionality to export its cache storage to a number of remote backends so that you can persist all the hard work done computing layers and cache volumes in CI/CD environments and reuse them between separate triggers of your pipeline. Dagger today (`v0.6.0`) does not expose a stable API for this feature. However, the team are hard at work sweating the details so we won't have to. They’re working hard to find a DX that works for many. At Flipt we’re looking forward to when this lands, as we expect the gains in CI will be very noticeable.

### Release Stage

Once Flipt is packaged and tested sufficiently, it is ready for release. Today we still use the excellent [GoReleaser](https://goreleaser.com/) project to package and ship the final Docker images and binaries to their various destinations. We do suspect though, that this will move to be more Dagger driven as time goes on, ultimately publishing the artifacts produced by the pipelines that we have developed using Dagger.

All the code to define our Dagger-based pipeline can be found in the [build](https://github.com/flipt-io/flipt/tree/main/build) directory of the Flipt repository. We use [mage](https://magefile.org/) both in the root of the Flipt repository and in our Dagger pipeline code as the CLI wrapper. We’re big fans of `mage` and so is Dagger. For those wanting a `make`-like interface which is defined in Go, it is a nice intersection of the two. It also marries nicely with all of our pipeline code being defined using the Dagger Go SDK.

### Dagger All The Way

Dagger is shaping up to be the solution I always wanted for CI/CD. Call me strange, but I am actually excited to develop and maintain these pipelines and I am hopeful it is going to be a sharp tool in our belts here at Flipt.

If you’re interested in Dagger then I highly recommend checking out their [site](https://dagger.io/), [GitHub project](https://github.com/dagger/dagger) and in particular [Discord](https://discord.gg/dagger-io).

Equally, we really want to understand your needs in building and shipping software here at [Flipt](/). So please do come and stop by our [Discord](https://www.flipt.io/discord) and say hi too.


Dagger, a ❤️  story


<BlogImage
  src="/go-sdk/flipt-sdk.png"
  alt="Flipt's Go SDK"
  width={2400}
  height={1350}
>
  <span>
    Photo by [Lenny Kuhne](https://unsplash.com/@lennykuhne) on Unsplash
  </span>
</BlogImage>

# Generating the Flipt Go SDK

This blog post describes the journey we took to build our new Go SDK, and how we developed our `protoc` plugin in Go to achieve that. It first explores our motivations and then gets into some of the details of how we built it and the tools that go into the automation around it.

We're releasing the new Go SDK today! You should be able to fetch it using `go get go.flipt.io/flipt/sdk/go`.

## Getting Serious About Clients

Recently, we decided to revise and improve our API client offering, starting with our support for Go.

Go is the core language used to develop Flipt's backend service. It is something the team has a long history with, so this was an obvious place for us to start.

With the [recent addition of different authentication methods](/blog/enhancing-authentication#kubernetes-service-account-authentication) for Flipt, we wondered how we could better support integrating these authentication mechanisms into clients.

Our [documentation](/docs/authentication/using-tokens) explains how to manually wire static client tokens into a gRPC or HTTP client, however, we now support authenticating using Kubernetes service-account tokens. Doing so is a more involved process, with an extra API handshake in the middle, so this is something we ideally want our clients to do for you with minimal configuration.

## Status Quo

Flipt’s core service and APIs are defined using [gRPC](https://grpc.io/) and protobuf. This allows us to define our APIs as a set of RPCs and messages in the [proto](https://protobuf.dev/programming-guides/proto3/) language and then use a set of `protoc` tools and plugins to scaffold domain models, servers and clients in different languages. This is a superpower for us. It allows us to reach other programming ecosystems and languages without prior knowledge or experience in them, opening Flipt up to a wider audience.

However, gRPC alone is not for everyone or every ecosystem. While it has a big reach, not every language is catered to equally. In particular, the browser hasn’t had the best support so we also wanted an equivalent HTTP API. The API should also be understandable by developers, allowing them to write their clients using an HTTP library in their language of choice.

This is where [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) comes into the mix. Flipt has used this project since the beginning to present an equivalent REST-like API using JSON as the message serialization format. The grpc-gateway tooling supports server generation (not client) in Go.

This is where we were until very recently. We have a gRPC protobuf generated client for the Go community, with no additional bells or whistles, just a thin shim around the Flipt gRPC endpoints and no support for invoking the HTTP API in Go. While we’re uncertain of the use cases, it seemed a shame to mandate gRPC when the [HTTP API](/docs/reference/overview) is also available.

## Our Goals

- A single consistent SDK API, which abstracts the details of the transport from the caller. While we want our users to have the choice, we don’t think the transport details matter when it comes to consuming the client's functionality in your application code.
- Two implementations of the transport abstraction, one for gRPC and one for HTTP.
- To generate as much of this as possible from the original `proto` definitions. We invested in gRPC and protobuf for the tooling and automation. Having our SDK advance with these definitions will keep us up to date.
- Develop an abstraction around authenticating our SDK, which supports consumers’ needs to access Flipt’s APIs with minimal configuration.

<BlogImage
  src="/go-sdk/Flipt_SDK_Architecture.svg"
  alt="Flipt SDK Architecture"
  width={729}
  height={339}
  caption="Flipt SDK Architecture"
/>

The entry point to our new client is the `SDK` type. This type encapsulates the three sections of the Flipt RPC APIs known as `Flipt` (core API), `Auth` (authentication-related APIs) and `Meta` (server metadata APIs). All of these types are generated from the original `proto` definitions.

These SDK types are (mostly) abstracted away from the transport details. The specifics are hidden behind a set of `Transport` Go interfaces. Each of these interfaces has an equivalent implementation for both `grpc` and `http`.

```go
type Transport interface {
	AuthClient() AuthClient
	FliptClient() flipt.FliptClient
	MetaClient() meta.MetadataServiceClient
}
```

For the most part, the `grpc` transport implementation is identical to that of the `grpc-go` generated client. The packages contain some thin wrapping code to glue our SDK together. The crux of the code has been implemented and the transport interfaces are very close to these concrete `grpc` implementations. This was an intentional decision to minimize effort and generated code.

<StarCTA />

The `http` implementation on the other hand required some more work. For a start, `grpc-gateway` does not ship a generated HTTP Go client, so we had to implement this ourselves. To invoke the API generated by grpc-gateway, we need to produce code that can adapt our RPC messages into the appropriate HTTP requests. We also need to provide a Go implementation that conforms to the Transport interface definitions derived from gRPC.

The grpc-gateway ecosystem has a collection of additional annotations and metadata which describe how the RPC methods should be represented as HTTP requests, for example:

```yaml
http:
  rules:
	# ...
	- selector: flipt.Flipt.CreateVariant
	   post: /api/v1/flags/{flag_key}/variants
	   body: "*"
```

Our custom `protoc` plugin parses this additional metadata and uses it to generate Go code which can produce an equivalent HTTP request.

```go
func (x *FliptClient) CreateVariant(ctx context.Context, v *flipt.CreateVariantRequest, _ ...grpc.CallOption) (*flipt.Variant, error) {
	var body io.Reader
	var values url.Values
	reqData, err := protojson.Marshal(v)
	if err != nil {
		return nil, err
	}
	body = bytes.NewReader(reqData)
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, x.addr+fmt.Sprintf("/api/v1/flags/%v/variants", v.FlagKey), body)
	if err != nil {
		return nil, err
	}
	req.URL.RawQuery = values.Encode()
	resp, err := x.client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	var output flipt.Variant
	respData, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	if err := checkResponse(resp, respData); err != nil {
		return nil, err
	}
	if err := protojson.Unmarshal(respData, &output); err != nil {
		return nil, err
	}
	return &output, nil
}
```

I don’t suspect we have managed to support the full breadth of protobuf, grpc and grpc-gateway features. However, we believe we have implemented enough to cover the functionality Flipt takes advantage of.

## How does one write a protobuf Go code generator?

[https://pkg.go.dev/google.golang.org/genproto](https://pkg.go.dev/google.golang.org/genproto) is a Go package designed to aid in the development of `protoc` plugins. It is used by `grpc-gateway` and `grpc-go` in order to parse and generate their respective client and server implementations.

<BlogImage src="/go-sdk/protoc.svg" alt="Protoc" width={561} height={157} />

`protoc` is the entry point to all things protobuf generation. It parses and validates your protobuf files and invokes any configured plugins with a request to generate code on `stdin`.

[https://pkg.go.dev/google.golang.org/protobuf/types/pluginpb#CodeGeneratorRequest](https://pkg.go.dev/google.golang.org/protobuf/types/pluginpb#CodeGeneratorRequest)

The message you are provided is a protobuf serialized structure itself. Specifically, it is a `pluginpb.CodeGeneratorRequest`. This contains the parsed definitions of your protobuf files collected by `protoc`.

It is up to the plugin to interpret this request and produce a response. The response is written to `stdout` as a protobuf encoded `pluginpb.CodeGeneratorResponse`. The response contains all the files you have produced and where they should end up (their file names).

The `protogen` package provides a framework which hides much of the hairy details of writing a `protoc` plugin. Simplifying the wiring and providing useful utilities specifically for generating Go code.

There is an excellent [blog post](https://rotemtam.com/2021/03/22/creating-a-protoc-plugin-to-gen-go-code/) by Rotem Tamir, which laid the foundational knowledge we needed to get started on our custom SDK wrapper and HTTP client. If building something with `protogen` interests you, then I highly recommend reading this first.

### Some `protogen` pointers

I won’t repeat all the wonderful tidbits and details in that blog post, however, I do have a couple of extra pointers that I discovered along the way while developing our SDK generators.

Firstly, we use `proto` version `3` and in particular, we use the `optional` keyword on some message fields. This happens to be a feature of protobuf which was introduced after the initial release of version `3`. If you attempt to use `protogen` on `proto` files using the `optional` keyword without first enabling it as a “supported feature” in your plugin, then you will be confronted with a warning each time your plugin is invoked:

```go
Warning: plugin "go-flipt-sdk" does not support required features.
  Feature "proto3 optional" is required by 1 file(s):
    auth/auth.proto
```

It was a little unclear at first, but the trick is to set the optional feature _bit_ on the `*protogen.Plugin.SupportedFeatures`:

```go
protogen.Options{}.Run(func(gen *protogen.Plugin) error {
		gen.SupportedFeatures |= uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
		// ....
}
```

`SupportedFeatures` is a bitmask and the nice, human-readable names for the feature bits can be found at [google.golang.org/protobuf/types/pluginpb](http://google.golang.org/protobuf/types/pluginpb).

Secondly, managing imports in your generated Go files can initially be unclear. It turns out that `protogen` has a really nice feature, which does a lot of heavy lifting with regard to managing the naming and qualification of those names in your generated output:

```go
for _, f := range gen.Files {
  if !f.Generate {
    continue
  }

  filename := string(file.GoPackageName) + ".sdk.gen.go"
  g := gen.NewGeneratedFile(filename, myTargetPackageName)

  // QualifiedGoIdent will handle adding the GoImportPath
  // to the target GeneratedFile (g) in the `import` declaration
  // at the top. It even handles when you reference two packages
  // with the same name by adding a numeric suffix to distinguish them.
  httpClient := g.QualifiedGoIdent(protogen.GoIdent{
    GoImportPath: protogen.GoImportPath("net/http"),
    GoName:       "Client",
  })

  g.P("type MyClient struct {")
  g.P("client *", httpClient)
  g.P("}")
}
```

`protogen.GeneratedFile.QualifiedGoIdent` ensures that the path you’re referencing is correctly imported into the top of the resulting generated Go file. It also handles any imports with duplicate names by adding a numeric suffix (e.g. importing both `text/template` and `html/template` would resolve to `template "text/template"` and `template1 "html/template"`). Then the resulting import path is correctly prepended onto the type when the `GoIdent` produced by this function is passed into `g.P("...", httpClient)`.

The example generator code above would produce something similar to:

```go
package client

import http "net/http"

type MyClient struct {
	client *http.Client
}
```

### Putting it altogether

So Flipt now has a single `protoc` plugin, which produces three target packages.

The plugin lives [here](https://github.com/flipt-io/flipt/tree/main/internal/cmd/protoc-gen-go-flipt-sdk) and it generates the following three packages into the Flipt repository:

- `sdk/go/http` containing `HTTP` implementations of the `sdk.Transport` interfaces.
- `sdk/go/grpc` contained `gRPC` implementations of the `sdk.Transport` interfaces.
- `sdk/go` the core of our Go SDK.

To bring all of this together, we use a tool called [buf](https://buf.build/) to orchestrate the invocations of the `protoc` toolchain. The protobuf toolchain is a fiddly beast and `buf` greatly simplifies this process for us.

We have a single addition to our `buf.gen.yaml` which drives this tool. It looks like this:

```yaml
plugins:
  # ...
  - name: go-flipt-sdk
    out: sdk/go
    opt:
      - paths=source_relative
    strategy: all
```

Each of the entries under `plugins` is treated as a `protoc` plugin. These will be passed on invocations of the `protoc` tool by `buf`. `protoc` knows when it sees the name `go-flipt-sdk` to prepend `protoc-gen-` onto the front and look that binary in your `PATH`. In our case, it looks for `protoc-gen-go-flipt-sdk`.

## Where next?

We’re hoping this automation keeps us on a good footing with each change to Flipt’s API, allowing us to generate a client in lockstep with the server API, which has support for all the RPC definitions we create.

Beyond just generating the client we plan to add support for the Kubernetes authentication method, reducing that down to some configuration with sensible defaults.

Finally, we have begun the work to build out a new integration test suite to ensure our SDK and Flipt are behaving as we intend. Our next blog post on this topic will explore how we’re building out a new CI pipeline using [Dagger](https://dagger.io). Dagger is enabling us to define a pipeline we can execute anywhere, which is both fast and reproducible. We’re very excited about the future of Dagger and can’t wait to talk about how we plan to leverage it in the Flipt build process.

We hope you’ll find this new SDK useful, and we’re happy to answer any questions that you may have in our [community Discord](https://www.flipt.io/discord) or via GitHub issues/discussions.

We welcome all feedback on how to improve the SDK or Flipt as a whole, so feel free to reach out! You can find us on [GitHub](https://github.com/flipt-io), [Twitter](https://twitter.com/flipt_io), [Mastodon](https://hachyderm.io/@flipt), and [Discord](https://www.flipt.io/discord).


Generating the Flipt Go SDK


<BlogImage
  src="/hi-yoofi/yoofi.png"
  alt="Hello, Yoofi!"
  width={2400}
  height={1350}
>
  <span>
    Photo by [Dejan Zakic](https://unsplash.com/@dejanzakic) on Unsplash
  </span>
</BlogImage>

We are excited to announce a new addition to the Flipt team, [Yoofi Quansah](https://www.linkedin.com/in/redblacktree/) - a Senior Software Engineer with 6+ years of experience and a love of developer tooling. With his impressive background and skills, we are confident that he will play a significant role in advancing our mission of helping engineers everywhere deploy software to production without fear.

Here's what Yoofi had to say about his background and decision to join the Flipt team:

> Hello 👋, my name is Yoofi Quansah. I've had quite the journey to finally find my way here at Flipt, but let me not bore you with details you probably don’t want to know and get to the fun stuff!
>
> Before writing code, I was actually 99% positive I was going to make it to the NFL (as we all were). I am writing this blog post now, so you know that didn't happen 🙂, but I am very happy with how my life has turned out so far.
> I started my career out building simple developer tools over at Honey. After I got over the hump of imposter syndrome and really got into my work, we started to build some really cool tools on the team. These tools included our own feature flagging service (foreshadowing), deployment platforms, and testing platforms to name a few.
> Everything was cool right? Not quite. I realized that even though we were building these cool tools, we were not really getting much recognition for it, and even worse the engineers were relatively slow to adapt. It was quite draining, to say the least.
>
> Enter July 2019... My life changed when I went to Gophercon.
> I found people who were building developer tools and directly interacting with customers with the tools they built. Wow. It was at that point that I knew I wanted to write Go, and work with a company that interacted directly with developers as their customers. After all, it is pretty nice to see the value of the products that you build for others to use directly.
> The result of all this was me leaving my job at Honey, and joining InfluxData in August of 2020, where I met [George](/blog/welcome-george-macrorie)!
>
> InfluxData was an incredible experience. I met tons of smart people, which allowed me to hone my craft as an engineer. It also allowed me to realize exactly what I wanted out of my career, which was to build incredible features for engineers to use. Most of my time there was taken up building the next generation of the query engine, which solved a wealth of problems we were facing with the old one.
> George introduced me to the grand idea of Flipt, all Mark has already done already, and the future outlook/potential roadmap in early 2023. Even though I was quite happy at InfluxData, I fell in love with the challenge, and all that is being hoped to accomplish.
>
> Combined with my previous experience helping out building a feature flagging service at Honey, and a whole lot of issues I ran into with feature flags both at Honey and InfluxData, I believe there is a bright future for this critical piece of software development and delivery.
> I am happy to be here, and hack on!

Welcome Yoofi, we are lucky to have you! 🚀


Meet Yoofi Quansah


<BlogImage
  src="/enhancing-authentication/enhancing-authentication.png"
  alt="Enhancing Flipt Authentication"
  width={2400}
  height={1350}
>
  <span>
    Photo by [Roon van der Meer](https://unsplash.com/@roonz_nl) on Unsplash
  </span>
</BlogImage>

In the previous few posts, we went over the addition of one of our most asked-for features: User Authentication support. Just when you thought that might be the end of our authentication story, we improved it even more by adding three more authentication-related features:

1. API token configuration via the UI
1. Configurable bootstrap token
1. Kubernetes Service Account authentication

In this post we'll briefly go over each feature, discussing its use case, why it's an important addition to our overall auth story, as well as how to get started.

## API Token Configuration

We've had basic token authentication support for authenticating your services with Flipt since v1.15. Admittedly though, it was somewhat incomplete as you could only manage those tokens via the [API](https://www.flipt.io/docs/reference/authentication/create-token).

In this [latest release](https://github.com/flipt-io/flipt/releases/tag/v1.19.0), we added the ability to create and delete static tokens in the UI as well. This allows anyone authenticated with Flipt to see which tokens have been issued and when they expire. It also allows users to create and delete tokens as needed without having to write code to do so.

Enabling and using static token authentication is a good first step when it comes to securing the communication between Flipt and your own services. Making the management of access tokens as user-friendly as possible was one of the major acceptance criteria for this new functionality.

This feature also necessitated a new 'Settings' section of the application which we'll populate with more controls in the future.

<BlogImageGallery
  images={[
    {
      src: "/enhancing-authentication/static-tokens/list.png",
      alt: "Settings Screen with Static Tokens View",
    },
    {
      src: "/enhancing-authentication/static-tokens/new.png",
      alt: "Creating a New Token",
    },
    {
      src: "/enhancing-authentication/static-tokens/expiry.png",
      alt: "Adding an Expiration to a Token",
    },
    {
      src: "/enhancing-authentication/static-tokens/created.png",
      alt: "Token Created",
    },
  ]}
/>

After creating a token via the UI, you can easily copy it to your clipboard and store it in a secure location as its value will no longer be accessible from Flipt itself.

## Configurable Bootstrap Token

Another improvement that we added in this release is the ability to provide a configurable bootstrap token when first enabling token authentication. Since v1.15, if an operator enabled token authentication for the first time, we would create what we call an initial bootstrap token on their behalf, logging it to `STDOUT` to be captured and used.

```console
access token created	{"server": "grpc", "client_token": "ks8gyZNdU7jmB92kAIYOp4MEXTcCJbX1TdXJM01VKCs="}
```

This was necessary because all requests to the Flipt API would fail as soon as you enabled and required token authentication.

This meant, however, that there wasn't a great programmatic way to run Flipt with token authentication enabled for the first time, such as in the case of Kubernetes. Flipt operators would have to either:

1. Start Flipt with authentication enabled but not required, create a token via the API, and then restart Flipt with authentication required.
1. Start Flipt with both authentication enabled and required and try to capture the created bootstrap token from our logs.

This also presented a similar challenge when trying to run Flipt in CI, such as we do for our own integration tests, as there was no easy way to set up authentication and then restart Flipt in a test pipeline.

In this release, we added the ability for the user to optionally specify their own known bootstrap token as well as to configure it with an expiration.

<BlogImage
  src="/enhancing-authentication/bootstrap-token-expiration.png"
  alt="Bootstrap token created with expiration"
  width={2442}
  height={490}
  caption="Bootstrap token created with expiration"
  className="shadow-md"
/>

Here's an example of how to configure a bootstrap token with a known value and an expiration of 24 hours:

```yaml
authentication:
  methods:
    token:
      enabled: true
      bootstrap:
        token: "s3cr3t!"
        expiration: 24h
```

In our opinion, this provides the best of both worlds by allowing for the initialization of an authentication token with a potential known value that can expire automatically. This gives operators the ability to enable authentication from the get-go, use a known token to create more tokens, then allow the known token to quietly expire to prevent unwanted access.

More information on how to configure this feature can be found in our [documentation](https://www.flipt.io/docs/configuration/authentication/#token-authentication).

## Kubernetes Service Account Authentication

In most cases, static tokens are a manageable approach for getting services talking with Flipt. For security reasons, you may set an expiry on your tokens and periodically rotate them with new ones. Given you have more than one type or replica of your application(s) communicating with Flipt, you may even choose to create multiple tokens. It may be one per application type or even as many as one per replica.

For those deploying Flipt into Kubernetes environments, we have added the new `kubernetes` authentication method.
This method takes advantage of the [service account](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) mechanisms made available in Kubernetes.

By default Kubernetes periodically writes a service account token (encoded as a JWT) into a well-known path inside each pod's filesystem.
Service account tokens can be used to communicate and identify the caller with the Kubernetes API. With Flipt [v1.19](https://github.com/flipt-io/flipt/releases/tag/v1.19.0) you can exchange this service account token for a Flipt client token via Flipt's API.

A small amount of client-side glue code can keep your applications authenticated without having to manually create, rotate or distribute tokens.

Here's an example using `curl` and `jq` to exchange a Kubernetes service account token for a Flipt client token:

```bash
curl -s -X POST http://flipt:8080/auth/v1/method/kubernetes/serviceaccount \
  --data "{\"service_account_token\":\"$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\"}" | \
  jq .
{
  "clientToken": "pKkaEik40Nu4lJ7O37l92MNyyD38U8UlaagSmAfJoS0=",
  "authentication": {
    "id": "f191babc-57b8-4856-89b1-f324941403d7",
    "method": "METHOD_KUBERNETES",
    "expiresAt": "2024-02-20T13:39:14Z",
    "createdAt": "2023-02-20T14:11:28.962841Z",
    "updatedAt": "2023-02-20T14:11:28.962841Z",
    "metadata": {
      "io.flipt.auth.k8s.namespace": "default",
      "io.flipt.auth.k8s.pod.name": "someservice-586bfb5b6b-fmh8g",
      "io.flipt.auth.k8s.pod.uid": "b5217947-aeac-4b35-afd3-23f50d63eae9",
      "io.flipt.auth.k8s.serviceaccount.name": "default",
      "io.flipt.auth.k8s.serviceaccount.uid": "8aeb28ad-66f0-4884-bafc-e606e5eda149"
    }
  }
}
```

Enabling this method means Flipt implicitly trusts other services deployed into the same cluster.

While currently leveraging this new method will require some application code on the developers' part, we're actively working on a way to provide out-of-the-box support for our official clients.

As always, our [documentation](https://www.flipt.io/docs/configuration/authentication) has more information on how to configure these features.

Please let us know what you think of these new features and if you have any questions or feedback, feel free to reach out to us on [Discord](https://www.flipt.io/discord), [Twitter](https://twitter.com/flipt_io), or [Mastodon](https://hachyderm.io/@flipt)!


Enhancing Authentication Support


<BlogImage
  src="/new-look-and-oidc/flipt-v1.18.0.png"
  alt="Flipt v1.18.0"
  width={2400}
  height={1350}
>
  <span>Photo by [Jr Korpa](https://unsplash.com/@jrkorpa) on Unsplash</span>
</BlogImage>

I'm happy to announce that there's a [new version of Flipt, v1.18.0](https://github.com/flipt-io/flipt/releases/tag/v1.18.0) which has a few important changes.

- The whole UI has been rewritten in [ReactJS](https://reactjs.org/) and styled with [TailwindCSS](https://tailwindcss.com/). Technically this happened in v1.17, but v1.18 has a few new features that we want to highlight, so we decided to wait and officially announce both at the same time.
- New authentication options — You can now authenticate into Flipt using Google, GitLab, Auth0, Dex, or any [OpenID Connect](https://openid.net/connect/)-compliant service.

## Refreshed User Interface

In the [previous post](/blog/embedding-react-in-go), we went over the technical challenges and decisions we made to integrate our new UI, written in ReactJS, with the existing Flipt backend, written in Go. In this post, we'll quickly cover the why behind this initiative, the result of this effort, and explain how our new UI helps us to improve your experience as an end user of Flipt.

Our previous UI was written in [VueJS](https://vuejs.org/) and it had started showing its age. We had tried to keep it up to date as far as library and dependency versions, however, there was no getting around the fact that it had been written for our needs a few years ago and wasn't as flexible as we needed it to be going forward.

First off, we were using VueJS 2, which was a major version behind the current version of VueJS, 3. While it might not have been too difficult to update our application to the latest version of Vue, we theorized that it would end up requiring a lot more work than if we started fresh by rethinking the user experience and then picking a new framework.

In fact, the component library that we were using in our Vue2 app, [Buefy](https://buefy.org/), has [explicitly stated](https://github.com/buefy/buefy/issues/2505#issuecomment-997000720) that they would not support Vue3 going forward. While this is completely understandable and not a knock on Buefy, it was a concern for us as it meant we were 'stuck' on Vue2.

The second major issue with our front-end application was that it was written in plain JavaScript, so we weren't able to take advantage of all the nice type safety and tooling that you get with TypeScript. This ended up being another supporting reason as to why made the switch from VueJS to React since we knew we wanted to move to TypeScript and would have to effectively re-write the entire application anyways.

The final reason we decided to start from scratch with the UI was that we wanted to create a new, aesthetically pleasing look while enabling our small team to continue to work effectively and be able to ship new features for the front end. Thank goodness libraries such as TailwindCSS exist, allowing even us non-designers to quickly create an attractive UI that we could be proud of.

We were able to accomplish all of these things by switching over from VueJS to React, and the end result was a UI that looked much better and was easier for our team members to develop and maintain. We also decided that React would be a better choice for our application long term because of React's popularity, which makes it easier to find contributors and team members who enjoy working with it.

While we probably could have tackled some of these concerns by better componentizing the existing UI, or upgrading to Vue3 without Buefy, in the end, I think we made the right choice.

The result of our work is a brand-new UI that is responsive, easy to navigate, and looks great on all devices. Here are a few screenshots of the UI in action:

<BlogImageGallery
  images={[
    {
      src: `/new-look-and-oidc/screenshots/flags.png`,
      alt: "Flag List",
    },
    {
      src: "/new-look-and-oidc/screenshots/variants.png",
      alt: "Add Variant",
    },
    {
      src: "/new-look-and-oidc/screenshots/segments.png",
      alt: "Segment List",
    },
    {
      src: "/new-look-and-oidc/screenshots/distributions.png",
      alt: "Configure Distributions",
    },
    {
      src: "/new-look-and-oidc/screenshots/evaluation_console.png",
      alt: "Evaluation Console",
    },
  ]}
/>

## New Authentication Options

We’re excited to announce that Flipt now supports additional authentication options besides the static token authentication that we added in [v1.15.0](https://www.flipt.io/blog/authenticating-flipt). This new, session-enabled, authentication is powered by the [OIDC (Open ID Connect)](https://openid.net/connect/) protocol, which allows you to use your existing identity providers like Google, GitLab, Auth0, [Dex](https://dexidp.io/), or Okta to log in to Flipt.

In the past we didn't have a great answer for how to secure the Flipt user interface from the outside world, other than to simply:

- Not expose it by putting it behind a VPN
- Put a reverse proxy in front and delegate the authentication to the proxy.

As noted above, in v1.15.0 we added static token support so that you could secure your integrating service-to-service communication with Flipt, but this still did not solve the problem of an end user wanting to authenticate with the UI.

Since OIDC is a popular standard for authentication, enabling it allows Flipt to support a variety of different identity providers using just one login flow, which makes it easier for you to secure your Flipt UI from the outside world.

So what does this mean for end users? If you configure an OIDC provider, Flipt will present it as a login option when you visit your Flipt dashboard. Once the end user authenticates with the provider, they will be returned to the Flipt dashboard. Flipt will establish a client token and set it in a protected cookie. The token has a configurable lifetime and will be rejected once that lifetime expires.

<BlogImage
  src="/new-look-and-oidc/login.png"
  alt="Flipt Login Screen"
  width={441}
  height={444}
  caption="Login Page with OIDC Providers"
  className="shadow-md"
/>

There are several new configuration settings that you can use to set up and customize your OIDC integration, and they are thoroughly documented in our updated docs. You can read more about them here:

- [OIDC Configuration Settings](https://www.flipt.io/docs/configuration/authentication#method-oidc)
- [OIDC Method Overview](https://www.flipt.io/docs/authentication/methods#oidc)

## Wrap Up

In future updates we'll be adding support for other non-OIDC-based authentication such as [GitHub authentication](https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps) (via OAuth2) as well as providing UI functionality to manage your static tokens and more.

We hope you’ll find this update useful, and we’re happy to answer any questions that you may have in our [community Discord](https://www.flipt.io/discord) or via GitHub issues/discussions.

We also welcome all feedback on how to improve Flipt as a whole, so feel free to reach out! You can find us on [GitHub](https://github.com/flipt-io), [Twitter](https://twitter.com/flipt_io), [Mastodon](https://hachyderm.io/@flipt), and [Discord](https://www.flipt.io/discord).


New Look + Authentication Options


<BlogImage
  src="/embedding-react-in-go/react-go.png"
  alt="Flipt React + Go"
  width={2416}
  height={1366}
>
  <span>
    Photo by [Jason Leung](https://unsplash.com/@ninjason) on Unsplash
  </span>
</BlogImage>

We’re not far off the 1.17 release of Flipt. It comes with a fresh coat of paint and some foundational changes to the Flipt user interface. We wanted to take a moment and share part of the engineering journey taken to get to this release.

## TL;DR

- We’re moving from Vue to React 🎉
- We experimented with Next.js 🧪
- We struggled with client-side routing 🗺️
- We settled on plain-old React + hash-based routing ⚛️

As we neared the end of 2022, we were settling new authentication features we had been building. The ability to support login in the UI was a primary motivator for the backend changes we were making, so naturally, we were closing in on building login functionality in the frontend. Since we had to tackle some reasonably sized frontend changes anyway, we took a moment to reconsider the state of our frontend architecture.

When Mark first laid the foundations of the current UI back in February 2019, Vue.js presented a quick and easy way to do so. Particularly, in such a way that could be easily embedded into Flipt.
Flipt is a single binary built primarily in Go, and it serves both the HTTP and gRPC protocols.

The UI is served over HTTP directly from the binary using Go’s fantastic [http.FileServer](https://pkg.go.dev/net/http#FileServer) implementation from the standard library. Sometime around February 2021 was the Go 1.16 release, which came with [native embedding](https://github.com/golang/go/issues/41191) of files into Go binaries. Flipt quickly took advantage of this feature for bundling the static UI assets directly into the binary. Go has a handy wrapping function that adapts the `embed.FS` into the required `http.FileSystem` interface to integrate it seamlessly into an `http.Server`.
You can find the current (`v1.16.0`) embed directive in Flipt [here](https://github.com/flipt-io/flipt/blob/9e0ba7d839ef452526815f71ed6e1207d6aa4997/ui/embed.go#L8-L9).

<BlogImage
  src="/embedding-react-in-go/ui-build-process.png"
  alt="UI to Go embedding build process"
  width={1646}
  height={503}
  caption="UI to Go embedding build process"
/>

Fast-forward back to today, and we're reviewing the future of the UI. It appears that React has dominated the world of frontend frameworks since Flipt inception. As such, we have speculated that React could have a broader appeal for Flipt contributors, so we have decided to take the plunge now before we get any more invested in frontend features.

The frontend is not our natural habitat at Flipt. We’re learning by doing. When we decided to take the first steps to transition from Vue to React, we looked around for what is the current hot stuff on the market.
This is where [Next.js](https://nextjs.org/) caught our attention. We had some initial familiarity with it, as this blog and our landing page is built using it.
Next has plenty of batteries included tooling and how-to material to sink our teeth into, so we got to work spiking a prototype.

It didn’t take long for us to get a prototype using Next.js together. Where things started to get tricky was getting Next.js to fit in our Go binary, where the existing UI currently resided.

## Challenges

### Challenge 1: Bundling our Next.js application into a Go binary

Our first goal was to try to replicate what we had already. Easy right? Last time we just:

1. Ran a build process in the UI project to produce a directory of assets.
2. Embedded those assets using Go’s embed tooling.

This first attempt didn’t take too long to replicate. A little digging around Next’s documentation led us to this page: https://nextjs.org/docs/advanced-features/static-html-export. After building the site with `npm run build`, a subsequent `next export` appeared equivalent to how we produced and consumed our Vue.js application. A little more digging to get the export CLI flags right, and it was as simple as `next export -o $FLIPT/ui/dist`.

Now we have all our assets (HTML, JS, CSS etc) in the correct directory. Without changing anything else, we booted Flipt and took a look. At first glance, everything seemed like it might have just worked.

### Challenge 2: Missing Assets

However, things were not quite what they seemed. Interactions weren’t working as expected. A brief look under the hood (via developer tools), and sure enough, there were a bunch of errors.

<BlogImage
  src="/embedding-react-in-go/console-errors.png"
  alt="Errors in the browser console"
  width={2024}
  height={920}
  caption="Errors in the browser console"
/>

The errors were a little misleading at first. What has this got to do with `X-Content-Type-Options: nosniff`? Why is `text/plain` being returned as the content-type for JS assets by our file server? Popping open the network tab was a little more enlightening.

<BlogImage
  src="/embedding-react-in-go/console-network-requests.png"
  alt="Console network tab requests"
  width={2024}
  height={920}
  caption="Console network tab requests"
/>

These failing assets were returning 404 (Not Found) status codes, but not everything was returning a 404.
Looking at the responses a little closer shined a light on our weird `Content-Type` and “blocked” error messages. Our Go static file server was presenting a generic 404 response page (Content-Type: text/plain) for our missing assets. These assets were being requested via `script` tags, and their security mechanisms were kicking in when they got back a content-type they weren’t expecting.

Long-story short, our Go file server was not serving some subset of our assets.
A quick check of the asset folder we created earlier, and the assets were actually where they should be.

```
➜  tree ui/dist
.
├── 404.html
├── _next
│   ├── E5aRj9aVYvtSy47LN7sYl
│   └── static
│           ├── E5aRj9aVYvtSy47LN7sYl
│           │   ├── _buildManifest.js
│           │   └── _ssgManifest.js
// trimmed output for brevity
```

So, what was it that these particular assets had in common, separate from the assets which were being served successfully?

<BlogImage
  src="/embedding-react-in-go/console-network-requests-ordered.png"
  alt="Console network tab requests ordered"
  width={2056}
  height={950}
  caption="Console network tab requests ordered"
/>

Could it be that they start with an underscore? A bit of digging into Go’s [embed](https://pkg.go.dev/embed#hdr-Directives) standard library turned up just what we were looking for.

```
If a pattern names a directory, all files in the subtree rooted at that directory are embedded (recursively), except that files with names beginning with ‘.’ or ‘_’ are excluded.
…

If a pattern begins with the prefix ‘all:’, then the rule for walking directories is changed to include those files beginning with ‘.’ or ‘_’. For example, ‘all:image’ embeds both ‘image/.tempfile’ and ‘image/dir/.tempfile’.
```

It appears that all we needed was to update our call to the embed directive to the following:

```diff
-// go:embed dist/*
+// go:embed all:dist/*
var UI embed.FS
```

This did the trick. At-least, the errors in the console were gone. All assets were returning status code 200.

### Challenge 3: Right-Click, Open New Tab

It didn’t take long to stumble onto the next set of issues and subsequent can of worms.
When attempting to open a link in a new tab, or visiting it in a new window, we were encountering errors.

Some actions returned 404s, and some listed links to other pages with unexpected content.

<BlogImage
  src="/embedding-react-in-go/segments-page-listing-contents.png"
  alt="Segments page returning directory contents"
  width={2378}
  height={1774}
  caption="Segments page returning directory contents"
/>

What is `[pid].html`? Why are we seeing 404s attempting to create a new flag? I thought this was simply another single-page app? Are we actually navigating to non-existent pages?

Comparing old and new side-by-side, we spotted something we had not considered during the transition. The routing appeared to be different.

<BlogImage
  src="/embedding-react-in-go/current-flipt-ui.png"
  alt="Current Flipt UI Routing Contains Hash"
  width={2040}
  height={1602}
  caption="Current Flipt UI Routing Contains Hash"
/>

We spotted the `#` before the path in our existing Vue.js application. The hash in the URL path (known as a URI fragment) is never sent as part of a server-side request. It is purely there to serve the frontend. When added to a URL, most browsers will attempt to locate an equivalent HTML ID attribute in the page and scroll to that location in the view. However, it has since become popular to leverage the hash value to drive single-page application paths.

In reality, only the root `index.html` (the file server for `/` path) file is being served.

In hindsight, that makes sense. It is a single-page application.
However, in our new application, we have real paths in the URL. These appear to work as expected when you stay within the same window, but the moment you navigate from a new context, we see attempts to serve these paths from the Go file server.

This is where the differences in the transition really started to become clear.
We put on our thinking caps to explore a path forward.

_Unsurprisingly, as the dust settled, some of these problems and solutions started to present themselves in other documentation and blog posts. That was after some failed attempts to Google the problem and come up with our own hacks solutions. Fortunately, we were on the right path._

We explored and prototyped three different paths forward.

1. Update our Go file server to only serve `index.html` for all (HTML asset) paths.
2. Teach our Go file server how to handle Next.js [File-System Routing](https://nextjs.org/docs/routing/introduction).
3. Change back to hash-based routing in our frontend application.

#### Serving `index.html` for all HTML requests

One quick trick (which we later found outlined in [Vue’s documentation](https://v3.router.vuejs.org/guide/essentials/history-mode.html#example-server-configurations) on a similar subject) is to serve the `index.html` page generated by Next, whenever any request for HTML is made.

| path requested | currently rendered | desired page |
| -------------- | ------------------ | ------------ |
| /flags         | /flags.html        | index.html   |
| /segments/one  | /segments/one.html | index.html   |
| /console       | /console.html      | index.html   |

You get the idea.

Though this isn’t the full picture as one small additional change is required.
By default, it appears that the Next.js router doesn’t attempt to reconcile the actual browser path (window.location.path) with the routers' initialization value for the path.
When we render `index.html` for say the `/segments` page, the router initializes with the path `/`.
I suspect our particular use case is something Next.js isn’t trying to cater for.

In order to get around this problem, we employed a quick hack.
On page load, we forcibly update the router's path if it was not as expected via a [`useEffect`](https://reactjs.org/docs/hooks-effect.html).
This causes a little visual hop from the root page to the desired destination... but it works.
If you know of a better way of doing it, come nerd-snipe us in our [Discord](https://www.flipt.io/discord).

#### Implementing Next.js File-System Routing in Go

This approach was a little heavier on the Go side. The crux of the solution is to support Next.js ability to serve dynamic pages via an on-disk naming convention using square brackets in the filename (This is our friend `[pid].html`).

Firstly, there is one-quick trick we did with `next export` functionality, that saved us some further complexity in our implementation. Set the `trailingSlash` property in the Next configuration to true.
https://nextjs.org/docs/api-reference/next.config.js/trailing-slash

This changes the exported output structure of the static HTML slightly.

- `segments.html` becomes `segments/index.html`
- `[pid].html` becomes `[pid]/index.html`.

This happens to be more congruent with how the Go file server handles particular request paths.
When a path happens to refer to a directory, like `/segments` is in Flipt, Go will render an HTML snippet containing a list of links to the directories contents.
This is what led to the screenshot earlier. Containing the following files:

- [pid].html
- new.html

When that directory happens to contain an `index.html`, it serves that instead.

This gets us a lot of the way there, however, the final challenge is dynamic routes in Next. For example, when requesting the path `/flags/my_flag_key`, the standard Go file server will attempt to open the following two files:

- /flags/my_flag_key
- /flags/my_flag_key/index.html

Neither of these files exists, as the `my_flag_key` refers to a flag key stored within the Flipt backend system. With next.js, what we really want to serve is `/flags/[pid]/index.html`. This page knows how to handle extracting the key from the path and requesting it from the backend.

We need a way to map all dynamically routed paths to their respective dynamic files “on-disk”.
I say “on-disk” in inverted quotes because remember, we embed these files into our Go binary earlier. The assets are not really on-disk. They are exposed via a filesystem abstraction (an interface) in Go.

Go’s [embed.FS](https://pkg.go.dev/embed#FS) is a structure that implements the [fs.FS](https://pkg.go.dev/io/fs#FS) interface.
This interface requires a single `Open(name string) (fs.File, error)` method.

Given that we know and can explore the contents of this directory on initialization of Flipt; we do so to derive the expected dynamic paths. Then we implemented a decorator of the `fs.FS` interface, which decides whether an incoming file path needs to be re-directed and open one of the dynamic paths.

```go
type nextFS struct {
  // we will slot our embed.FS instance in here
  fs.FS

  // prefix tree of potential paths
  tree *node
}

func newNextFS(embedded fs.FS) fs.FS {
  var root *node
  // walk provided filesystem "embedded" to derive
  // potential dynamic paths and build a tree.

  return &nextFS{
    FS:   embedded,
    tree: root,
  }
}

func (*nextFS) Open(name string) (fs.FS, error) {
  var adjustedPath string

  // walk tree in order to replace any dynamic path
  // parametes with the name of the dynamic paths on disk
  // building and adjusted path to the desired file.

  return n.FS.Open(adjustedPath)
}
```

Our first pass implementation of this can be found [here](https://github.com/flipt-io/flipt/blob/88ce30f342075fa216c1b64b76a05b86ae0009cb/internal/cmd/ui.go#L59).
The TL;DR is we built a crude prefix tree based on the paths in the embedded filesystem, with the dynamic files annotated within the tree.

When a requested file matches a dynamic path in the prefix tree, we re-write the path name before handing it on to the embedded `fs.FS` implementation.

Now `/flags/my_flag` serves the contents of `/flags/[pid]`.

#### Embracing Hash Routing

Finally, hash-based routing. This is what our current UI does with Vue.js.
Re-instating this routing capability as the primary approach to client-side routing brings us back in line with how Flipt works today.
It fully encapsulates Flipt as a single-page application, meaning, we can easily embed it into a simple, vanilla file server implementation in Go.
Furthermore, this approach is backward compatible with existing Flipt URLs (without additional changes).

As this potential strategy dawned on us, we also decided to review our decision to use Next.js.
We still feel that adopting React is a worthwhile bet for the future of Flipt, however, we don’t have to use Next.js to do so. In fact, we suspect vanilla React will be more broadly understood than Next.js right now.
Next.js brings plenty of extra opinions, which in hindsight are not necessarily congruent with where Flipt is going.

Finally, React Router has an out-of-the-box implementation of [hash routing](https://reactrouter.com/en/main/routers/create-hash-router) right there for the taking.

#### The Decision

After exploring all of the above, we settled on approach (3), vanilla React and hash-based routing.

_Sorry, perhaps next time Next.js. You’re very cool and shiny. But React is all we require right now._

The product of this choice is no change to the Go side of the equation.
The frontend application involves a simple npm-driven build process, which outputs a “dist” folder containing all the necessary assets.
This folder contains a single HTML file (`index.html`), along with several Javascript and CSS files.
We embed the folder exactly as we had before, using the Go standard library to both embed and serve it.

## Final Words

We hope that moving from Vue to React will attract more contributions from a wider pool of developers.
Flipt is an open-source solution, and we want to build an active community around it.

The new UI will be part of the `v1.17.0` release of Flipt.
We hope you will take a moment to give it a test drive.
Hot on this release's heels will be login functionality in the UI.

We intend to publish another blog post around `v1.18.0` talking more about the journey to this new UI, the new authentication related features and where we’re going next.


Embedding Our New React UI in Go


<BlogImage
  src="/grafana/announcing-dashboards.png"
  alt="Grafana Dashboards"
  width={2400}
  height={1350}
>
  <span>Photo by [Jay R](https://unsplash.com/@justjthings) on Unsplash</span>
</BlogImage>

We recently started developing a collection of Grafana Dashboards to provide visibility
into both the health of your Flipt deployment and the state of the evaluation system.

In [Flipt v1.16.0](https://github.com/flipt-io/flipt/releases/tag/v1.16.0) we have made some alterations to the metrics that we expose. This includes some measurement naming changes, along with the addition of some new ones, specifically around the evaluation engine.

These new dashboards depend on the names and new metrics added in [v1.16.0](https://github.com/flipt-io/flipt/releases/tag/v1.16.0). Upgrading to this version is a requirement before attempting to use these new dashboards.
Metrics are exposed as Prometheus-style metrics. Our dashboards have been developed against a target Prometheus instance. See [configuring metrics](https://www.flipt.io/docs/configuration#metrics) for documentation on configuring metrics in Flipt.

Currently, there are two separate dashboards—one for system overview and the other for flag evaluation state.

Flipt System is the name of the system dashboard. It includes operational details of Flipt, including RPC request throughput, response codes, and latency, as well as details relating to interactions with the backing SQL database.

<BlogImage
  src="/grafana/flipt-system-dash.png"
  alt="Grafana Flipt System Dashboard"
  width={3336}
  height={1416}
  caption="System Dashboard"
/>

Flipt Flag Evaluations dashboard includes details specific to each flag. Selecting a specific flag will present historical details on the resulting matches, match reasons, segments, and values produced.

<BlogImage
  src="/grafana/flipt-flag-dash.png"
  alt="Grafana Flipt Flag Evaluation Dashboard"
  width={3354}
  height={1926}
  caption="Flag Evaluation Dashboard"
/>

The system dashboard is more focused on operators trying to understand the health of a Flipt deployment, whereas the evaluations dashboard is focused on Flipt users who want to understand what Flipt is returning and why. It is critical as an end user of a feature flag system to understand how your flags are being evaluated in a production context so that you can answer questions like "Is my flag being referenced?", "Is it safe to delete this flag yet?" or "How often is variant A returned compared with variant B?".

This is our first pass at building reusable dashboards for Flipt. Our dashboards are open-source and we invite anyone who would like to contribute and improve them. We intend to keep them maintained and preserve a level of compatibility going forward with respect to measurement naming.

Head to our [Grafana dashboards repository](https://github.com/flipt-io/grafana-dashboards) to try them out or contribute.

As always, if you have any questions or feedback, please reach out to us on [Discord](https://www.flipt.io/discord) or [Twitter](https://twitter.com/flipt_io).

Cheers and happy graphing 📈!


Grafana Dashboards


<BlogImage
  src="/auth-v1/auth.png"
  alt="Flipt Authentication"
  width={2400}
  height={1350}
>
  <span>Photo by [FLY:D](https://unsplash.com/@flyd2069) on Unsplash</span>
</BlogImage>

Last week we cut the [v1.15.0 release](https://github.com/flipt-io/flipt/releases/tag/v1.15.0) of Flipt. The primary feature of this release was authentication.
As of v1.15.0, Flipt has support for static client token-based authentication.
This is our first step to bringing authentication and authorization features to Flipt.

Currently, the way to obtain a client token is to enable the “token” authentication method in Flipt’s configuration.
Once enabled, a section of the API ([/auth/v1/method/token](/docs/reference/authentication/create-token)) is mounted and available to create new access tokens.

Note that this endpoint is also protected when authentication is set to required, meaning prior authentication is needed to create tokens.

```yaml
authentication:
  methods:
    token:
      enabled: true
```

Check out our [token authentication configuration](/docs/configuration#token) documentation for more specific details.

If no tokens exist in the backing store then one will be created the first time you enable the token authentication method.
This way, an initial token can be bootstrapped so that you don’t lock yourself out of Flipt (authentication can also be set to not required in this situation).
The bootstrapped token will be output in the logs:

```
…
2022-11-22T12:02:10Z    INFO    access token created    {"server": "grpc", "client_token": "nCq9QO-FeEAS91LV1LFD2qOpUASXl8eljlXokeorbyY="}
…
```

By default Flipt remains open, however, with a slight adjustment to your instance configuration, Flipt’s API can be secured.

```yaml
authentication:
  required: true
```

Attempts to perform API operations will require a client token to be presented via the Authorization header:

```sh
curl localhost:8080/api/v1/flags \
   -H 'Authorization: Bearer nCq9QO-FeEAS91LV1LFD2qOpUASXl8eljlXokeorbyY=' \
   -H 'Content-Type: application/json'
```

Putting this altogether we get a secured API with the ability for prior authenticated clients to create additional credentials:

```yaml
authentication:
  required: true
  methods:
    token:
      enabled: true
```

Head over to the new [Authentication](/docs/authentication) section of our documentation to learn more about interacting with Flipt when authentication is required.

As mentioned, this is our first step to securing Flipt. Once authentication is enabled, the UI becomes unusable since we have not yet implemented the features required to authenticate browser sessions.

In the next release, we expect to add Google as the first OAuth login provider, followed by other authentication providers in subsequent releases.

Equally, we’re working on automated processes for managing expired client tokens.
With v1.15.0 it is currently on the operator to use Flipt’s APIs to delete outdated tokens, however, we plan for this mechanism to be built-in, automated, and configurable in the future.

On top of all this, we’re actively designing authorization features for Flipt, so that you can decide who can do what when interacting with the API and UI.

Join us in our [Discord](https://flipt.io/discord) to chat about your needs and to help influence the future of authentication and authorization with Flipt.


Authenticating Flipt


<BlogImage
  src="/openfeature-go/openfeature.png"
  alt="Flipt + OpenFeature"
  width={2400}
  height={1350}
/>

## What is OpenFeature?

[OpenFeature](https://openfeature.dev/) is a [CNCF Sandbox project](https://www.cncf.io/projects/openfeature/) that is developing an open standard for feature flag management.

> OpenFeature provides a unified API and SDK, and a developer-first, cloud-native implementation, with extensibility for open source and commercial offerings.

In short, OpenFeature provides a specification and implementation of a unified API to query and evaluate feature flags which abstract away the specific details of the backend responsible for storing and managing the flags themselves.

This allows the developers who are adding feature flags in their applications to use the OpenFeature SDK in their code and then register their feature flag management solution of choice as the 'provider', abstracting it away. The benefits of this are twofold:

1. It guards against 'vendor lock-in' which would require a large code refactor if a team were to decide to switch from one feature flag solution to another down the road.
1. It allows for common functionality to be shared and maintained by the community, which reduces the amount of duplication required as well as generally increases the quality and velocity of development. More on this later.

## SDKs and Providers

Before we get into more details of how OpenFeature works, it's important to define the key terminology and develop a shared model.

As mentioned above, OpenFeature has defined a [specification](https://docs.openfeature.dev/docs/specification/) that serves as a guide for how to think about OpenFeature from an end user perspective as well as how to implement a vendor specific library that OpenFeature can integrate with.

The specification defines '[providers](https://docs.openfeature.dev/docs/reference/concepts/provider)' as the adaptive layer that translates to and from feature flag solutions, such as Flipt and other open source and commercial solutions. It's up to the community or the feature flag vendors/projects themselves to build these providers that conform to the [OpenFeature Provider specification](https://docs.openfeature.dev/docs/specification/sections/providers).

<BlogImage
  src="/openfeature-go/openfeature-provider.png"
  alt="OpenFeature Provider"
  width={1654}
  height={340}
/>

An OpenFeature SDK is available for many popular programming languages such as **Go, JavaScript, .NET and Java**. The OpenFeature team just recently announced that these SDKs [have all reached a 1.0 version](https://docs.openfeature.dev/blog/hardening-and-1-0-sdks/), representing a certain maturity and resistance to breaking changes. They've also announced that more SDKs will be made available in the future for languages such as **PHP, Python, and Ruby**.

## Evaluation API

The [Evaluation API](https://docs.openfeature.dev/docs/specification/sections/flag-evaluation/) is the primary interface to access and interact with feature flags in the OpenFeature ecosystem. The OpenFeature Evaluation API allows you to query feature flags given a `flagKey`, default value, as well as an optional [evaluation context](https://docs.openfeature.dev/docs/specification/sections/evaluation-context). The evaluation context allows for the feature flag solution (provider) to make decisions based on configured rules, for example end-user targeting or percentage-based rollouts.

This maps nicely to how we think about [evaluation at Flipt](/docs/concepts#evaluation), where the integrator can provide as much or as little information as they need to make a decision for the purpose of flag evaluation.

## Announcing The Flipt OpenFeature Provider for Go

Today we are happy to announce that we've **open sourced** the initial version of the [OpenFeature Go Provider for Flipt](https://github.com/flipt-io/flipt-openfeature-provider-go)! It supports both HTTP(s) and GRPC transports to connect to your Flipt instance and is highly configurable allowing you to bring your own HTTP or GRPC clients.

To ensure that we conform to the OpenFeature Provider specification, we even decided to pull in OpenFeature's own [integration test suite](https://github.com/open-feature/test-harness/tree/5153eac71dc47413da4326bdb49a94e573b99578) into the repository. This move actually helped us find several bugs in our implementation, while exercising it as an end-user would.

To see our provider in action, checkout the [example in our main repository](https://github.com/flipt-io/flipt/tree/main/examples/openfeature) which shows how to integrate OpenFeature + our provider into an example REST API project. Stay tuned for updated documentation, more examples, and future providers written in other languages. Also, if you are itching to write your own provider for Flipt in your language of choice, feel free to reach out in our [Discord](https://discord.com/invite/kRhEqG2TEZ), we'd love to help!

## Hooks and the Power of a Common API

Before wrapping up, I'd like to quickly go over one of what we think is the most powerful features of OpenFeature, the [Hooks API](https://docs.openfeature.dev/docs/specification/sections/hooks). Hooks allow you to use a common API for performing actions at different stages of the evaluation lifecycle. By implementing Hooks that use the OpenFeature data model, the community can take advantage of shared implementations, preventing the need for duplicating effort for each provider. Some of the use cases for hooks include things like adding metrics and telemetry, request retries and circuit breaking, performing validation on data, and performing client side caching.

The OpenFeature team has released their own [OpenTelemetry Hook for Go](https://github.com/open-feature/go-sdk-contrib/tree/main/hooks/open-telemetry), allowing any integrator of the Go SDK to get [OpenTelemetry](https://opentelemetry.io/) traces with the evaluated feature flag data for free. In our [example application](https://github.com/flipt-io/flipt/tree/main/examples/openfeature) for integrating the Flipt OpenFeature provider we wired up the OpenTelemetry hook with a few lines of code, resulting in some cool visualizations of our feature flag data:

<BlogImage
  src="/openfeature-go/jaeger-openfeature-otel-trace.jpg"
  alt="OpenFeature OpenTelemetry Trace"
  width={1582}
  height={399}
/>

<BlogImage
  src="/openfeature-go/jaeger-openfeature-otel.jpg"
  alt="OpenFeature OpenTelemetry Trace"
  width={1305}
  height={390}
/>

We are looking forward to seeing what the community comes up with by leveraging hooks to provide common functionality for all implementers of the OpenFeature SDKs!

## Committed to Helping Organizations Unlock the Power of Feature Flags

We love feature flags here at Flipt and think that we have just started to scratch the surface of what can be accomplished with their use in the software development lifecycle. With the introduction of the OpenFeature project, we hope that teams can begin to realize the full potential of feature flags in a way that is easy to implement and use. We're also very excited to see how the OpenFeature project evolves in regards to the CloudNative space, something we're also particularly interested in here at Flipt.

If you have any feedback on our provider, want us to prioritize specific language support, or just to learn more about how Flipt can help your organization get started with feature flags, please reach out either by joining our [Discord](https://discord.com/invite/kRhEqG2TEZ) or send us an email at [hello@flipt.io](mailto://hello@flipt.io).


Announcing the Flipt OpenFeature Provider for Go


<BlogImage
  src="/hi-george/geo.png"
  alt="Hello, George!"
  width={2400}
  height={1350}
>
  <span>
    Photo by [Lampos Aritonang](https://unsplash.com/@lamposaritonang) on
    Unsplash
  </span>
</BlogImage>

I'm excited to announce that [George MacRorie](https://www.linkedin.com/in/geokeaton/) has joined Flipt as our first Founding Software Engineer! George is a seasoned engineer with over 10 years of experience building and scaling software products. George and I first met at CodeShip, where we worked together for 3 years. George is based in the UK 🇬🇧 and will be key to developing and scaling Flipt as well as helping to build out our engineering team and remote first culture. George is an amazing engineer and a great person and friend, and I'm excited to have him on the team!

I asked George to write a bit about himself and his background, and here's what he had to say:

> My name is George. I’ve been bashing the internet pipes in exchange for money for over a decade. I cut my teeth as a junior engineer in the world of Java (and a lot of XML) in 2012. Before abandoning it all for my new love, the Go programming language, in 2014. Somewhere along the way I discovered the root of all my frustrations didn’t come from within, it was with my tools. Yeah, that's right, the tools were the problem. Thus, a developer tooling engineer was born.
>
> For me, It all started with CI/CD. I don’t think it is a coincidence that I embraced baldness around that time. Hair pulling was a known side effect of the available solutions back then. I joined CodeShip to build the alternative of our dreams. The CI/CD SaaS market was heating up at the time and CircleCI was already out in front, a position they’ve held well. In the end, our little ship didn’t quite cut the mustard. We didn’t get to fulfil our dreams of a CI/CD revolution.
>
> Yet, the best part about my time there was the people. We met many times in person (we were remote) all over Europe and North America. We shared ideas, laughs, and one hair-raising e-bike ride around the city of Lisbon. An experience which went down like Marmite. It was during this time that I met Mark. It was also during this time that Flipt came to be.
>
> When it all came to an end, Mark went off to continue the good developer tooling fight at GitHub. I and a few other ex-shippers figured we had solved CI/CD, so we should do databases next (InfluxDB). At Influx, I soon fell into my old habits of trying to solve developer tooling problems. They let me attempt to lead the Deployments team. A team which tackles many things’ platform engineering, despite the name.
>
> Feature flags at Influx are critical to how engineers develop in production. I wasn’t personally involved in the implementation of feature flags for Influx Cloud; it was another member of the old CodeShip crew who unlocked this capability. Unquestionably, they delivered on enabling the team to safely ship changes.
>
> Over time, our flagging libraries and frameworks accrued a little kludge. Some flags became entrenched and forgotten in code. Complexities in targeting logic grew. Local development and remote environments diverged in flag state configuration. Developers became suspicious if their flags were working as expected. One uncompleted large-scale refactor left the system fractured.
>
> Along the way, I got to see how the frustration played out, and I experienced it first-hand. I can’t say that has left me with all the answers. But, I would like to think it has left me with empathy. Something I intend to bring to Flipt.
>
> When Mark reached out to me and said, “want to come and build the future of feature flags?” I felt very ready for the challenge.

Welcome to the team, George! 🎉 🎉 🎉


Meet George MacRorie


<BlogImage
  src="/all-in/hello-world.png"
  alt="Hello World"
  width={1792}
  height={1024}
>
  <span>
    Art by [Alejandro Ortiz](https://unsplash.com/@alejandroortiz) on Unsplash
  </span>
</BlogImage>

## The Origins of Flipt

When I started [Flipt](https://github.com/flipt-io/flipt) almost 4 years ago, I started it to scratch my own itch. Throughout the previous 10 years of my career as a software engineer, I noticed that every company I'd worked at that used feature flags seemed to run into similar issues.

From the outside, the idea of feature flags seems simple enough. At their most basic, they are boolean values used to make decisions in your code that can change 'remotely' at runtime. Because of this perceived simplicity, most engineering organizations that I'd been a part of had decided to roll their own solution. This usually took the form of either an internal library developed in-house or an open-source library that was integrated into the main application and added to haphazardly over the years.

This approach works fine when feature flags are used sparingly and the team and application architecture are small enough. Once the organization and architecture grow, however, the simple, in-house solutions start to show their cracks. In most cases, the solution that works when there is a monolithic application no longer scales with the move to multiple services and programming languages. As the team begins to build out more complex features that require many different moving parts and dependencies, it becomes necessary for access to feature flags to be shared between these different services and teams.

At the time, I was aware of existing solutions that aimed to solve this same problem of distributed feature flags, however, they were all mostly SaaS-based. To me, this was not an ideal solution as I believed that most teams would want to avoid the lock-in, cost, security, and latency concerns associated with these solutions. Back in 2018, [gRPC](https://grpc.io/) was still a relatively new technology but it seemed like a promising solution for creating a reliable remote access layer that could be used to solve the above-mentioned problem of distributed access to feature flags in a multitude of programming languages.

Thus, Flipt was born, a stand-alone, self-hosted, feature flag service that allows developers to make use of feature flags in any programming language or framework while being focused on low latency and data residency from the start.

## Going All In

Over these past several years, myself and an amazing group of [contributors](https://github.com/flipt-io/flipt/graphs/contributors) have continued to make improvements to Flipt, adding features and support for new technologies and use cases. Over this same time, I've also talked with engineers, managers, and engineering leaders that use Flipt in production at their companies, helping provide real value to their end-users.

While this has been incredibly motivating and gratifying, it's also presented a bit of a challenge to keep up with all the new things I want to support and features to add while only being able to work on Flipt in my spare time. Thankfully, I've recently been given the opportunity to be able to work on Flipt full-time and also build out a small team to help make shipping software easier and less error-prone for developers everywhere.

The software industry has also come a long way over these past several years. More and more companies have fully embraced the CloudNative software development model, which has allowed us to build systems that are more resilient and performant than ever before. There's also been an explosion of new open source projects for developers to use as building blocks for their applications, which has made it much easier to get started with many common tasks such as testing, logging and monitoring. All of this has helped to make software development a more enjoyable, productive experience. However, there's still one major area where things haven't changed as much: shipping code safely. Despite all the advances we've seen in other areas, it remains just as difficult for developers to push their code changes into production while being assured that their changes will not cause errors.

This is the area where we want to focus our efforts with Flipt. We want to make it as easy and stress-free for developers to ship software as confidently as possible, regardless of their development environment, language, or platform.

We also want to continue to stay true to the project's core principles so that Flipt:

- Remains open-source and self-service.
- Stays flexible enough to support various modes of experimentation.
- Is seamless to configure and secure in modern cloud environments.
- Fits into your existing stack.
- Continues to be fast and consistent.

## Looking Forward

We're working on some really exciting features right now that we'll be releasing over the next few months, so keep an eye out for those!

I'm incredibly excited to take on this new challenge, embrace this opportunity, and build a team of like-minded individuals who are passionate about shipping code safely and securely.

**Takeaway**: We want to build the future of modern software deployment mechanisms, and we'd love to have you along for the ride. Flipt is not going anywhere and will remain open-source and community-driven. We are committed to making Flipt even better and would love your help! Please join us on our [GitHub](https://github.com/flipt-io/flipt) and in our [Discord](https://discord.com/invite/kRhEqG2TEZ) to learn more and help build the future of Flipt.

Thank you for your support, and we can't wait to see what you build with Flipt!
