From chaos to clarity: structured event reporting in Rails

Hi! If you've been working with Rails for a while, you've surely felt this familiar sensation: your app grows, logs multiply, and suddenly you find yourself trying to locate a needle in a haystack when something breaks in production.

Logs quickly become unreadable. And in production, with thousands of events this is like trying to decipher the Matrix. And I'm not Neo!

Today, we're going to talk about an elegant solution to this problem: the structured events in Rails.

This approach, developed by Shopify's infrastructure team, turns the chaos of unstructured logs into clear, actionable data. You'll discover why it's important, how it works, and especially how to implement it in your Rails projects.

What is structured event reporting?

Imagine your Rails app as a big party. Without structured events, it's a bit like all the guests talking at once in a dark room – you hear noise, but it's impossible to understand who is saying what!

Structured events bring order to this chaos. Instead of scattered logs like :

You get organized events with consistent metadata:

The Shopify approach: solving problems at scale

The Shopify infrastructure team faces daily challenges that most of us can only imagine. When you run a platform that handles millions of transactions per day, the slightest logging problem can become a nightmare.

Why Shopify Invests in Rails

Shopify is not just a user of Rails – the company actively acts as a guardian of the framework. This philosophy consists of:

1. Using Rails the 'right way' internally
2. Contributing to improvements of the open-source framework
3. Ensuring Rails remains viable for years to come
4. Solving problems at Shopify's scale and then sharing them with the community

That's exactly what happens with structured events: a solution born from a real need at Shopify, then shared with the Rails ecosystem. Their commit #55334 will be available natively in Rails 8.1.

Before proposing this solution, they had tested a package internally. They concluded that one of the best ways to do this would be to improve Rails' native logging system rather than trying to build analytics layers on top or ancillary logging systems via gems.

Their objectives:

1. Develop a unified API system for all events in Rails
2. Make it quicker for developers to implement
3. Improve the quality of event data and their consistency.
4. Produce a stack-agnostic system that will manage downstream data.
5. Facilitate the ability to annotate events with all information we would deem relevant.

The proposed solution ActiveSupport::EventReporter

To emit structured events, you'll go through ActiveSupport::EventReporter. This interface will provide the tools to emit structured events to subscribers.

Integration with ActiveSupport::Notifications

For a bit of history, Rails already offers a similar service: ActiveSupport::Notifications. This is a notification system that allows you to instrument a block of code. It is then possible to subscribe to this instrumentation and be notified when the block of code in question runs.

They can then perform a specific action based on this information or use the instrumentation data for web-based logic.

By contrast, EventReporter is a pure API for reporting. It is designed for telemetry, nothing more. But that gives it dedicated features like the ability to retrieve context or the ability to export events following a specific data schema.

The two APIs are complementary.

Emitting events with Event Reporter

With Event Reporter, it will be very easy to emit events natively, as shown here when creating a new user. Note well the additional information that can be added to capture maximum context, such as the path to the source file, the line of code, and the method in which the event is emitted.

And all of this in a single, simple and efficient line of code.

During emission you will receive an event with the following keys

You can then subscribe to this event and process the transmitted information. Imagine we have a class LogSubscriber. In the example, we will:

1. Validate the data carried by the event (here, the name and the payload)
2. Encode the event in JSON (or another format)
3. Export this event in its encoded form

Here it's fairly simple but you can imagine a much more advanced process such as:

1. Persist to a database
2. Send an email to the developers
3. etc...

This obviously depends on the business logic you want to implement. Sky's the limit :)

The benefits of structured events

Better error traceability

No more spending hours grepping through logs! With structured events, every important action in your application generates an event with a predictable format.

Monitoring and alerts more effective

You can easily configure alerts based on precise criteria:

Analytics and business metrics

Structured events aren't just for debugging - they become a valuable data source for understanding user behavior. We're talking about a "business" event

At Shopify, which is primarily an e-commerce platform, they also want to track purchase events, for example. They will have a structure and a schema that is often more complex:

Encoding information via Protobuf

At Shopify they've decided to standardize all events from their Rails monolith with Protobuf to have structured, coherent, and easy-to-exploit data.

Ok but what exactly is Protobuf?

What is Protobuf?

Protobuf (Protocol Buffers) is a data serialization format developed by Google. To put it simply:

1. It's a schema definition language: you define the structure of your data (like a contract)
2. It's a serialization format: it transforms your objects into compact binary data for transport
3. It's multi-language: you can share data between Ruby, Python, Java, etc.

Simple analogy

Imagine you want to send a package (your data):

1. JSON = a transparent cardboard box with a readable label
2. Protobuf = a compact, optimized box, with a strict manual

In comparison:

When you’re a company as big as Shopify and you process an enormous data volume, such a big performance gain adds up fast! Moving to Protobuf also improves reliability and maintainability.

Conclusion

In the future with native integration of Event Reporter into Rails, this will allow:

1. To have structured events across Rails: framework hooks (SQL queries, jobs, etc.) emitted as structured events
2. Better integration with Rails.logger: a default-provided subscriber that transforms events into JSON for small apps
3. Better developer experience: readable logs in development, structured in production

Structured events represent a natural evolution in how we manage logging and monitoring in our Rails applications. By turning the chaos of traditional logs into structured, actionable data, this approach lets us better understand, debug, and optimize our applications.

Shopify’s commitment as the guardian of Rails shows how large companies can positively contribute to the open-source ecosystem. The solutions developed to solve problems at scale ultimately benefit the entire community.

Don’t hesitate to subscribe so you don’t miss anything!

Below is the original presentation video from Rails World 2025: