
Feature flags have become ubiquitous in the modern software development lifecycle for organizations across the world. Decoupling features from deployment, A/B testing, and gradual rollouts of features are a few of the benefits that come along with using feature flags.

Check out our [usescases](https://www.flipt.io/docs/usecases) documentation for more information on how feature flags can be used in your organization.

Feature flags allow for safety and ease of mind when rolling out features, but what happens when this new functionality behind feature flags are ready to be deployed fully? What tools do you need in place to monitor for unexpected application behavior? How can the organization correlate flag changes with regressions, or erratic results in their application behavior?

These questions demonstrate how there is a need for tighter coupling of feature flags to obvservability tooling. As a feature flaging tool at our core, we believe that we can introduce functionality to make this easier on our users and integrators to achieve tighter correlation with what can be observed while running in production. This blog post will highlight some of the features we have recently implemented to streamline that correlation.

### Webhooks

In the [v1.21.0](https://github.com/flipt-io/flipt/releases/tag/v1.21.0) release, Flipt introduced the [Audit Sink](https://www.flipt.io/docs/configuration/observability#audit-events) functionality where users can configure events in Flipt to be sent to a sink for auditability.

That release came with implementation for the inaugural sink which was a log file. These events are all CUD (create, update, delete) operations on Flipt entities (`flags`, `variants`, `segments`, `constraints`, `rules`, `rollouts`, `distributions`, `tokens`).

The log file was easy enough to get started, but hard to centralize that data to make any use out of it. The user would have to have additional tooling, such as [Grafana Loki](https://github.com/flipt-io/flipt/tree/50edfb1b78e02081b8c6038129f2f7903d39646c/examples/audit), to gather the logs in a centralized location since the logs can be distributed amongst many Flipt instances.

<BlogImage
  src="/observability/loki.png"
  alt="Audit Events Aggregated in Grafana Loki"
  width={1776}
  height={1304}
  className="shadow-md"
  caption="Audit Events Aggregated in Grafana Loki"
/>

As of [v1.27.0](https://github.com/flipt-io/flipt/releases/tag/v1.27.0), we have released functionality for a sending audit event payloads to a webhook sink. 🎉

Webhooks are a common way for applications to send data to a URL when an event happens. If you are familiar with GitHub, you may have seen the [webhooks](https://docs.github.com/en/developers/webhooks-and-events/about-webhooks) functionality that allows you to configure a URL to send events to when certain actions happen on a repository. This is the same concept in Flipt.

<StarCTA />

You can configure a URL with the [`audit.sinks.webhook.url`](https://www.flipt.io/docs/configuration/overview#audit-events-webhook) value and start receiving events as they happen to the web server that the URL is pointing to.

Use cases include: get notified in a Slack channel when a new flag is created, or transform and post that event to an [Events API](https://docs.datadoghq.com/api/latest/events/#post-an-event) in your observability solution (ie: Datadog). The webhook sink also can be configured with a signing secret to allow users to validate the event originated from somewhere appropriate.

<BlogImage
  src="/observability/webhook.png"
  alt="An Audit Event sent to a Webhook Sink"
  width={2880}
  height={1800}
  className="shadow-md"
  caption="An Audit Event sent to a RequestBin Webhook Sink"
/>

You can start using this functionality today! Check out the [documentation](https://www.flipt.io/docs/configuration/observability#webhooks) for more information as well as [an example](https://github.com/flipt-io/flipt/tree/main/examples/audit/webhook) in our main repository.

### Datadog

Datadog has long established themselves as a giant in the observability SaaS space. Knowing this, we figured that our users would benefit from tighter integrations with Datadog. The following sections will highlight some of the features we have implemented to allow for an improved observability experience if using Flipt with Datadog.

#### Exporting OTLP Traces

Flipt already has the ability to emit [OTLP](https://opentelemetry.io/docs/specs/otel/protocol/) traces to a configured collector. This depends on you spinning up the OTLP collector in your stack, and using the URL for that collector as configuration for Flipt. Until now however, we havent done a great job documenting how you could send your OTLP traces to third party tool such as Datadog.

We have recently published [an example](https://github.com/flipt-io/flipt/blob/cac1e65026add36cce51196fa9007eb4d235a5ae/examples/tracing/otlp/README.md#datadog-ui) on how to configure tracing with Datadog through the OTLP collector. Once the traces arrive in Datadog, you can leverage built-in features to [correlate traces and logs](https://docs.datadoghq.com/tracing/other_telemetry/connect_logs_and_traces/), and [logs with metrics](https://docs.datadoghq.com/logs/guide/correlate-logs-with-metrics/), a nice two-way correlation. As things regress or errant things happen in your application, you can use these features to quickly track down a problem, and possibly correlate them to a switch in a feature flag.

<BlogImage
  src="/observability/datadog.png"
  alt="Flipt Traces in Datadog"
  width={1691}
  height={746}
  className="shadow-md"
  caption="Flipt Traces in Datadog"
/>

#### RUM

Many times organizations want a better view of not only what occurs server-side in their applications, but also on the front-end. For example, you might want to see exactly what steps a user took during a session, where the user might have navigated away (bounced), and a variety of other metrics. Organizations can use feature flags to supplement this especially when they are rolling out new UI functionality to see how a subset of users might react without rolling it out completely to their entire userbase.

[Datadog Real User Monitoring (RUM)](https://www.datadoghq.com/dg/real-user-monitoring/benefits-na/) is a drop in client that you can use within your front-end to capture the metrics that you care about during a user's session. The RUM client also has functionality for tracking the values of feature flags and correlating that with the metrics that the RUM client already keeps track of.

As of [v0.2.11](https://github.com/flipt-io/flipt-node/releases/tag/0.2.11) of our [Node/TypeScript client](https://github.com/flipt-io/flipt-node), we now have the ability to integrate with Datadog RUM natively.

Here's an example of how you can use the client to send evaluated values of feature flags to Datadog RUM:

```typescript
const metricsClient = new FliptMetrics(
  new FliptApiClient({
    environment: "http://localhost:8080",
    auth: {
      credentials: {
        username: "YOUR_USERNAME",
        password: "YOUR_PASSWORD",
      },
    },
  }).evaluation,
  datadogRum
);

const response = await metricsClient.variant({
  namespaceKey: "default",
  flagKey: "hello-this",
  entityId: uuidv4(),
  context: {},
});
```

The [README](https://github.com/flipt-io/flipt-node#metrics) in the repository goes a bit more in depth of how you can enable that functionality.

<BlogImage
  src="/observability/datadog-rum.png"
  alt="Datadog RUM with Flipt"
  width={2704}
  height={1560}
  className="shadow-md"
  caption="Datadog RUM with Flipt"
/>

The above image shows an instance where the flag `hello-this` in the `default` namespace was `enabled` during the entirety of a user's session. On the same Datadog UI page, you can click on the `Session & Replays` tab to see how that new functionality behind the feature flag may have impacted the user session compared to users who the feature flag was off or evaluated `false` for.

## Conclusion

With these new observability focused features, we hope to allow our users to gain more visibility into how feature flags are affecting their application/platform as a whole.

If you have any questions, other tools you'd like us to integrate with, or additional features you'd like to see added, please reach out to us on [GitHub](https://github.com/flipt-io), [X](https://twitter.com/flipt_io), or [Discord](https://www.flipt.io/discord).

We hope you enjoy these new features, and we look forward to hearing from you!


Improving Observability


Last week at Flipt, we took a break from our regular projects to kick off our first company-wide Hack Week. If you're unfamiliar with the concept of Hack Weeks, they are dedicated periods during which teams can work on creative and innovative projects outside their regular responsibilities. It was an excellent opportunity for everyone to stretch their creative muscles and explore new possibilities.

This post will cover the projects we developed, the insights we gained, and how these innovations might fit into our broader company vision.

We had three days to work on our hack projects, starting on Tuesday and culminating in a demo day on Friday. I'm incredibly proud of what the team accomplished in such a short period of time. Without further ado, let's dive into the projects.

## Project 1: Flipt Migrator

[Mark Phelps](https://www.linkedin.com/in/markphelps1/)

<BlogImage
  src="/hackweek/migrate.gif"
  alt="Flipt Migrator"
  width={1070}
  height={934}
  className="shadow-md"
/>

I wanted to simplify the process for users migrating from existing solutions to Flipt. One of the biggest challenges teams face after deciding to switch tools is the absence of a straightforward migration path. Teams often either:

1. Run the new solution in parallel with the existing one, leading to inconsistent data between the two systems.
2. Spend considerable time and effort writing custom migration code, sacrificing productivity that could be better spent on core business activities.

To address these issues, I created [Flipt Migrator](https://github.com/flipt-io/migrator), a CLI tool that automates the migration process. The tool extracts data from the existing solution, transforms it into a Flipt-compatible format, and allows teams to validate and import the data.

Above is a short .gif demonstrating the migrator, where I migrate data from a competitor into a format compatible with Flipt and validate it using our new `flipt validate` command.

The tool, written in Python, currently supports only one source but we open-sourced it to encourage community contributions. Check it out on [GitHub](https://github.com/flipt-io/migrator)!

It leverages the [Pydantic library](https://docs.pydantic.dev/latest/) for type validation in Python3, making data transformation more reliable and intuitive.

Our goal is to make Flipt the best solution for feature management, and that includes making it easy to migrate from other solutions. We're excited to see how this tool evolves over time.

## Project 2: Slack Bot

[Yoofi Quansah](https://www.linkedin.com/in/redblacktree/)

<BlogImage
  src="/hackweek/slack-1.gif"
  alt="Flipt Slack Bot Create Feature Flag"
  width={1510}
  height={844}
  className="shadow-md"
/>

Yoofi's project was a Slack bot designed to centralize communication and auditing for feature flag changes. In organizations where multiple teams communicate via Slack, feature flags often require cross-team coordination. Existing solutions lacked a centralized system for auditing and tracking these changes. Our Slack bot fills this gap by integrating feature flag management directly into Slack.

The bot operates through a custom server-side application that interfaces with Slack. Users configure Slack to send specific event notifications to this application. The application then performs various tasks, such as serving block kit forms back to Slack for creating new feature flags or listing existing flags within a specific namespace.

The Slack bot has the potential to become an indispensable tool for broader platform management within Flipt. We're exploring additional API functionalities and the possibility of enhanced authorization through OAuth or OIDC providers.

Outside of Slack, we've also got some new auditing features in the works that go beyond our existing [file-based event sink](https://www.flipt.io/docs/configuration/observability#audit-events), so stay tuned for more updates!

## Project 3: Ready - Feature Flight Plans

[George MacRorie](https://www.linkedin.com/in/geokeaton/)

<BlogImage
  src="/hackweek/ready.gif"
  alt="Ready Feature Flight Plans"
  width={1068}
  height={720}
  className="shadow-md"
/>

Developed by George MacRorie, our third project is "Ready," an app designed to standardize the way engineers plan and execute feature releases.

The journey from code completion to general availability is complex and varies from one organization to another. It often relies on word-of-mouth knowledge or outdated internal documentation. Ready aims to define this process, integrate all relevant tools, and build an executable flight plan for releases.

Ready is a task list focused on feature releases, offering rich integrations with essential production tools. It guides developers through a tailored process, allowing plans to be created, templated, and shared within the organization. A central dashboard provides visibility into active features and their current state.

Over time, Ready could become the go-to tool for getting new features into production. Some ideas include integrating more source control management systems, observability tools, and deployment solutions. Ultimately Ready could evolve to include autopilot features to automate actions at each step, allowing for a more streamlined and less error-prone process.

George also prepared a longer Loom video demonstrating the app. Check it out [here](https://www.loom.com/share/3d86a9d9864a43228e21a99ed5d5a0b3) to see Ready in action!

We're super exicted about the potential of this project and how it could help teams streamline their release processes beyond feature flags.

## Until Next Time

This first Hack Week was a great success, and we're already looking forward to the next one.

We hope you enjoyed this brief overview of our projects. If you have any questions, feedback, or want to try any of them out for yourself, please reach out to us on [Twitter (X)](https://twitter.com/flipt_io) or [Discord](https://www.flipt.io/discord). We'd love to hear from you!

Also, signup for our monthly [newsletter](https://www.flipt.io/#newsletter) to stay up to date with all things Flipt including new features, blog posts, and more!


Flipt's Inaugural Hack Week


Today we’re happy to announce the release of [v1.24.0](https://github.com/flipt-io/flipt/releases/tag/v1.24.0) of Flipt including a few major features including:

- A new `boolean` flag type that can be used in combination with [flag rollouts](/docs/concepts#rollouts) allowing you to enable a flag over a percentage of your user base and more
- A new, more straightforward set of evaluation API routes that you can also [enable/disable authentication](/docs/configuration/authentication#exclusions) separate from the existing ‘management’ APIs
- The ability to add user email addresses to [an allowlist](/docs/configuration/authentication#email-matches) when setting up OIDC authentication
- Improved performance and efficiency in flag evaluation with cache enabled
- New [Amazon S3 filesystem backend support](/docs/experimental/filesystem-backends#object) to serve your flags directly from an S3 bucket

## Boolean Flag Type and Rollouts

Previously, Flipt was designed with only a single flag ‘type’ in mind, a flag which can be enabled/disabled and optionally have variants that can be returned during evaluation time. While this worked well for many use cases, we recognized the need for more flexibility. With the introduction of the new `boolean` flag type, you can now enable a flag over a percentage of your user base more easily. This allows for better experimentation and targeted rollouts.

The `variant` flag type is still the default flag type, allowing you to use flags in the same way as before. The new `boolean` flag type adds an extra layer of versatility. Boolean flags work hand in hand with a new abstraction that we call 'rollouts' or 'rollout rules'.

[Rollouts](/docs/concepts#rollouts) work similarly to rules used during evaluation of a `variant` flag in that the first rule that matches ‘wins’, however there are a few key differences:

1. Rollouts are only allowed for `boolean` flag types and allow you to return a different value for a flag’s `enabled` property.
2. There are currently two types of rollouts: **threshold** and **segment**. Segment rollouts work just like previous rules in that you specify a segment that you want the rule to match. Thresholds, in contrast, are used to return a different result to a percentage of requests.
3. Rollouts allow you to specify the value to return for a boolean flag, either `true` or `false`.

It’s perhaps easier to show an example of the problem that boolean flags and rollouts were added to solve, by showing how you would have had to configure Flipt pre-v1.24.0 without rollout support and then how you would do the same now.

**Imagine the following scenario:**

You have a feature in your application that you want to roll out in stages to all users. Before the release of v1.24.0 of Flipt, you would have had to:

1. Create a flag with `true` and `false` variants, or `on`/ `off` or similar
2. Manually handle these variants in your application code, converting them to `booleans`, since `variants` can only be strings.
3. Configure a segment with constraints that matches the segment you want to target (ie: `all-users`)
4. Create an evaluation rule which returns the `true` variant to X% of requests, and the `false` to the remaining percentage.

<BlogImageGallery
  images={[
    {
      src: `/rollouts/rollout-variants-1.png`,
      alt: "Creating the Flag and Variants",
    },
    {
      src: `/rollouts/rollout-variants-2.png`,
      alt: "Creating the Segment to Match",
    },
    {
      src: `/rollouts/rollout-variants-3.png`,
      alt: "Setting up the Evaluation Rule",
    },
    {
      src: `/rollouts/rollout-variants-4.png`,
      alt: "The Flag with Variants and Evaluation Rule",
    },
  ]}
/>

As you can see , managing rollouts with boolean flags in Flipt used to be quite complex. However, with the release of [v1.24.0](https://github.com/flipt-io/flipt/releases/tag/v1.24.0), the process has become much simpler and more intuitive.

Now, to achieve the same rollout scenario, you can follow these steps:

1. Create a `boolean` type flag. This flag will represent the feature you want to roll out.
2. Configure a threshold-based rollout, specifying the percentage of requests that should receive the flag enabled. The remaining percentage will receive the flag disabled.
3. Thats it! 🎉 You can now use the flag in your application code as a boolean value.

<BlogImageGallery
  images={[
    {
      src: `/rollouts/rollout-boolean-1.png`,
      alt: "Creating the Boolean Flag and Rollout",
    },
    {
      src: `/rollouts/rollout-boolean-2.png`,
      alt: "The Boolean Flag and Rollout Rule",
    },
  ]}
/>

Not only are boolean flags easier to setup for this scenario, they are also easier to update in place in the UI. You can now simply drag the slider or manually input the percentage in the rollout input box and click update to increase or decrease the percentage over time!

<StarCTA />

## Evaluation API Routes and Security Configuration

While designing and implementing boolean flags, we also took the time to see how we could make evaluation more intuitive to support both flag types. We decided to create a new set of evaluation API routes, separate from the existing `/api/v1` prefix that can work with both variant and boolean flags.

Several users have previously asked how to separate the ‘management’ API (created/edit/delete flags and other resources) from the ‘evaluation’ API as they serve two distinct audiences. Normally the ‘management’ API is used by team members to setup Flipt resources and should not be exposed publicly. Depending on a team’s application architecture however, they may need to evaluate flags from ‘untrusted’ and unauthenticated requests, such as when using feature flags directly from a browser based application.

Before this release, our only answer was to suggest putting Flipt behind a reverse proxy such as Caddy or Nginx and configuring access rules there. While we still think reverse proxies are a good idea for handling things such as request routing, rate limiting, and more, v1.24.0 of Flipt now separates the ‘management’ and ‘evaluation’ APIs completely and allows the operator to enable or disable authentication for each independently. We think that this new set of evaluation APIs and authentication configuration will allow Flipt to work for a larger variety of applications and use cases than before.

Checkout the new [evaluation API documentation](/docs/reference/overview#v1-24-0) as well as all of our updated clients which now also support these new endpoints!

We also added the ability to configure `allowed_emails` when setting up OIDC Authentication for Flipt. This addition was driven by a [user request](https://github.com/flipt-io/flipt/issues/1872) to be able to configure specific users within their organization to have access to Flipt when configuring authentication with Google.

**Note:** in order for this feature to work properly, make sure you specify the `email` scope when [configuring your OIDC setup](/docs/configuration/authentication#email-matches) for Flipt.

## Improved Performance, S3 Backend and more

Finally, we've merged two amazing user-contributed PRs which made significant improvements to flag evaluation performance and efficiency as well as added support for serving flag state straight from an S3 bucket!

### Faster Evaluation

With cache enabled, the evaluation of variant flags has become even quicker as we now cache potential constraints that would match a request at the storage layer, meaning they can be used across different `entityIds`.

From the [PR](https://github.com/flipt-io/flipt/pull/1910):

> after some investigation we found the main cause of delays in evaluation requests were the rules query, in our case running Flipt with a PostgreSQL DB on AWS RDS we were getting up to 1s of delays, with this fix it reduced to a few milliseconds

We consistently aim to keep Flipt fast for our users, and thus your users so we very much appreciate any contributions that help improve performance!

### Flag State Not Just for Git Anymore

Last but certainly not least, Flipt now supports the use of [Amazon S3 as a filesystem backend](/docs/experimental/filesystem-backends#object) to serve your flags directly from an S3 bucket or set of buckets.

In [v1.23.0](https://github.com/flipt-io/flipt/releases/tag/v1.23.0) of Flipt we released a new set of experimental features allowing our users to manage their Flipt state. We strongly believe that feature flags should be treated like any other application configuration in our your infrastructure, which means you should also be able to use industry best practices such as managing configuration via CI/CD pipelines with GitOps. We also recognize however that every organization is different and so we’re delighted to now support Amazon’s S3 as an alternative filesystem backend!

This was another amazing user contribution. Here’s the reasoning for the feature straight from the [PR](https://github.com/flipt-io/flipt/pull/1900):

> In an AWS environment, this would allow deploying a readonly container and not require communication to a git repository outside of the deployment environment.

This feature opens up the door for us to implement other object stores such as Azure Blob Store and Google Cloud Storage as well, so please [open an issue](https://github.com/flipt-io/flipt/issues/new/choose) if you’d like us to support those backends as well!

## That’s a Wrap

This release is our most feature packed yet! We’re delighted to add functionality that makes Flipt better for all of our users, and love the community contributions we’ve gotten so far this summer.

As always, feel free to reach out on [Twitter](https://twitter.com/flipt_io) (X?), [Discord](/discord), or open a [GitHub issue](https://github.com/flipt-io/flipt/issues/new) for any functionality that you’d like to see implemented. We’d also love to answer any questions or just chat about software delivery in general.

Checkout the full [changelog](https://github.com/flipt-io/flipt/releases/tag/v1.24.0), and give the latest release a look and let us know what you think!


Building the future of Feature Flags

Deploy With Confidence

Works with your existing stack

Trusted by the world’s most innovative teams

Introducing GitOps for Feature Flags

Features

Ship Safely

Context Targeting

Self Hosted

Secure

Fast

Easy Integration

Integrations

From the Blog

Improving Observability

Flipt's Inaugural Hack Week

Boolean Flags and Rollout Functionality

Building the future of Feature Flags

Deploy With Confidence

Works with your existing stack

Trusted by the world’s most innovative teams

Introducing GitOps for Feature Flags

Features

Ship Safely

Context Targeting

Self Hosted

Secure

Fast

Easy Integration

Integrations

Keep up with what we're building

From the Blog

Improving Observability

Flipt's Inaugural Hack Week

Boolean Flags and Rollout Functionality