RapidProto - fast, header-only Protobuf decoders for C++

~7× faster than protoc + Arena when materializing a full message tree, and faster than protozero when streaming fields - with wire validation that never compiles out (benchmarks).

RapidProto compiles a .proto schema into header-only C++ decoders with no third-party dependencies. One CLI, rapidprotoc, turns your schema into headers you #include; there is nothing to link. A single schema gives you two decode models, and you pick whichever fits the job:

Arena. decode() materializes the whole message into a read-only object tree in a bump arena, which you navigate with accessors (person->name()) in any order, as many times as you like.
Streaming. decode() walks the wire once and hands each field’s typed value to a callback you supply. Nothing is materialized, and there’s zero allocation.

Both models are decode-only - no serialization, no JSON codec, no reflection - and validate untrusted wire input without crashing on malformed bytes. They cover proto2, proto3, and the editions format (2023/2024), including groups, maps, and oneofs. An optional --dump emitter adds a JSON-like debug dumper for inspection.

For how the decoders got fast, see the walkthrough; to start using them, jump to the quick start.

Why RapidProto

protoc’s C++ runtime is a linked library that builds a full mutable message object per decode; zero-copy pull parsers like protozero drop that allocation but leave you to hand-write the read loop. RapidProto generates the decoder - specialized to your schema at compile time, header-only - and gives you both shapes.

Faster decode. On a realistic mixed payload the arena decoder materializes a full object tree ~7× faster than protoc + google::protobuf::Arena, and the streaming decoder - materializing nothing - is faster still, beating protozero, the zero-copy yardstick, on the realistic payload and most microbenchmark shapes. See benchmarks.
Less memory. The arena tree holds ~half of protoc’s (both payload bytes and total allocation): strings, bytes, and raw payloads are borrowed as views into the input rather than copied, so the arena carries only a read-only bump-allocated tree with no per-field object overhead.
Compile-time typed. Dispatch is entirely compile-time - no std::function, no virtual calls, and a wrong value type or a renamed field is a compile error rather than a silent bug.

RapidProto is decode-only (no serialization, no JSON codec, no reflection) - the one exception being an opt-in debug dumper that emits JSON-like inspection text - the arena tree is read-only, it decodes enums as open even when the schema declares them closed, and it does not validate string UTF-8 (it accepts string bytes protoc would reject). If you also need to produce or mutate messages, keep protoc for that side and use RapidProto for the hot decode path.

Quick start

Requirements: C++17 and a recent GCC, Clang (AppleClang included), or MSVC - Linux, macOS, and Windows are all CI-covered (platform notes).

Get rapidprotoc. The quickest path is a prebuilt binary for Linux, macOS, or Windows from the releases page (license files included). To wire generation into a CMake build instead, the rapidproto_generate() helper does it in a few lines - via FetchContent, an installed package, the Conan recipe, or the vcpkg overlay port (package managers). Or build the tool from source:

cmake --preset release                               # system compiler, optimized
cmake --build --preset release --target rapidprotoc
# binary: build/release/rapidprotoc

The prebuilt binaries are unsigned: on macOS clear the quarantine flag if Gatekeeper blocks it (xattr -d com.apple.quarantine rapidprotoc); on Windows, dismiss the SmartScreen prompt (More info -> Run anyway). This section then drives the tool by hand so each step is visible.

Given person.proto:

syntax = "proto3";
package example;

message Person {
  string name = 1;
  uint32 id = 2;
  repeated string email = 3;   // repeated: navigable array
  Address address = 4;         // sub-message
}

message Address {
  string city = 1;
  string country = 2;
}

1. Generate the arena decoder (the default model) and a self-contained copy of the runtime, into out/:

./build/release/rapidprotoc -I. --out-dir=out person.proto   # add -v to log each written file
# out/person.rp.hpp + out/person.rp.common.hpp + out/rapidproto/{runtime,arena_runtime}.hpp

2. Decode. You supply the serialized message bytes (from a file, socket, database, …) as a rapidproto::ByteView (an alias for std::string_view, so a non-owning view; for a std::uint8_t buffer, rapidproto::byte_view(ptr, size) builds one without a manual cast). Create an Arena, call decode(), then navigate the returned tree:

#include "person.rp.hpp"

namespace ex = rp::arena::example;   // generated types live under rp::arena; alias it once

std::string buf = /* the serialized Person bytes */;

rapidproto::Arena arena;
rapidproto::ArenaDecodeError err;
const ex::Person* p = ex::Person::decode(rapidproto::ByteView(buf), arena, &err);
if (p == nullptr) { /* malformed input: see err.code / err.wire / err.offset */ }

std::uint32_t id = p->id();                        // scalar, by value
std::string_view name = p->name();                 // string, a view into the input buffer
if (const ex::Address* a = p->address())           // sub-message: a pointer (nullptr if absent)
    std::string_view city = a->city();

Test bytes: encode some with protoc - protoc --encode=example.Person -I. person.proto < values.txt > person.bin

3. Compile with only the output directory on the include path:

g++ -std=c++17 -Iout my_consumer.cpp -o my_consumer

Repeated, map and oneof fields, and optional, read differently - see the arena model before writing much against them.

To stream instead, pass --stream (or --arena --stream for both) and use the callback API.

Choosing a model

	Arena	Streaming
What you get	a materialized object tree you read by accessor	a callback fired per field, in wire order
Allocation	one bump arena (you own it)	none
Random access / re-reading	yes: any field, any order, repeatedly	no: a single forward pass
Memory	the whole decoded tree	only what your callbacks keep
`#include`	`<stem>.rp.hpp`	`<stem>.rp.stream.hpp`
Best for	needing the message as a navigable object; a faster/lighter `protoc`+`Arena`	extracting a few fields, stream-processing, lowest overhead

You can use both models for one schema in one translation unit; see using both models.

Documentation

The user manual lives in docs/ (also published at https://veaac.github.io/rapidproto/), one page per topic:

Page	What’s in it
docs/arena.md	The arena decoder: accessors, the `Arena`, `decode_owned`, error handling
docs/streaming.md	The streaming decoder: field tags, the three consumption patterns, aborting
docs/dumper.md	The `--dump` debug dumper: JSON-like inspection text, `DumpOptions`
docs/semantics.md	The shared rules: lifetimes, validation & trust, presence/defaults, open enums, duplicate fields, thread-safety
docs/using-both-models.md	Both models in one TU, the mid-decode hybrid, coexisting with protoc
docs/profiles.md	Decode profiles (`drop` / `raw`) and unknown-field detection (arena)
docs/integration.md	The `rapidprotoc` CLI reference and the CMake helper (incl. cross-compiling)
docs/osm-pbf.md	Real-world walkthrough: OpenStreetMap’s planet format, decoded with both models
docs/optimizations.md	How the decoders got fast: seven optimizations, re-enabled one at a time and measured
docs/benchmarks.md	The numbers, how they’re measured, and how to reproduce them
CHANGELOG.md	Notable user-visible changes per release (SemVer-0: the MINOR version is the breaking axis)
architecture.md	Internals and design rationale, for contributors

A runnable end-to-end example (one schema, both models in one TU, a decode profile) is in examples/consumer.

Compatibility & stability

Versioning is SemVer-0 until 1.0: the MINOR version is the breaking axis - expect breaking changes between 0.x and 0.(x+1), never within a patch, each listed in the CHANGELOG. At 1.0 the promise flips, and the stable surface is everything a consumer binds to: the generated API (names, accessor shapes, callback signatures), the decode-profile file format, the CLI flags, the rapidproto_generate() contract, and the runtime headers’ shape. Only a major release may break or remove anything on that surface. A minor may deprecate (announced in the CHANGELOG under a Deprecated heading, the spelling still working) as advance notice of what the next major removes.

Supported platforms are what CI covers: Linux, macOS, and Windows, with GCC, Clang, AppleClang, and MSVC. Windows/MSVC builds and passes the test suite on every CI run. Prebuilt rapidprotoc binaries for all three platforms ship on the releases page, and a Conan recipe and a vcpkg overlay port are available (package managers). One Windows caveat: the recursion-depth caps that make deeply nested input fail cleanly instead of overflowing the stack are sized for an 8 MB stack (the Linux and macOS default). Windows defaults an executable to a 1 MB stack, so a program that decodes untrusted, pathologically nested input on Windows should link with a larger stack (/STACK:8388608) to keep that rejection stack-safe.

Contributing

See CONTRIBUTING.md for building, the ./check.sh quality gate, and how the golden tests work. The design and the invariants a change must preserve are in architecture.md.

Security

See SECURITY.md for the threat model and how to report a vulnerability.

License

RapidProto is licensed under the Apache License 2.0; see LICENSE, with attributions in NOTICE and THIRD_PARTY_NOTICES.md.

The vendored runtimes (rapidproto/runtime.hpp, rapidproto/arena_runtime.hpp, and rapidproto/dump_runtime.hpp) carry the same Apache-2.0 license, so the headers rapidprotoc drops into your out-dir are usable under those terms. The decoder code generated from your schema is your own work product, and RapidProto claims no rights over it. The embedded Protocol Buffers well-known-type definitions are Copyright 2008 Google Inc., licensed 3-Clause BSD. Catch2 and protozero are development-time dependencies and are not distributed.