Firedancer Conformance: Matching Agave on the Bank Hash

Firedancer can be fast and still be wrong. A second client also has to match Agave's consensus behavior when it processes the same accepted input.

This article looks at that question from the Frankendancer seam: how public conformance tooling compares execution paths against Agave, where those comparisons happen, and what a bank-hash mismatch actually proves.

TL;DR

Frankendancer uses Firedancer for networking and block production while Agave still handles runtime execution and consensus. Firedancer does not need to recreate the block an Agave leader might have built. Given the same accepted block, transaction, instruction, or VM input under the same feature state, the implementations need compatible execution effects and the same consensus state.

The public conformance model uses differential testing: replay fixtures through an Agave shared object and a Firedancer shared object, compare harness-defined effects, and use fuzzing to generate more cases. A bank-hash mismatch proves disagreement in the consensus-committed result; the cause may be state, serialization, or hash computation. The harness helps find where the behavior first split.

1. The Frankendancer seam is a surface, not one socket

Frankendancer is a hybrid, not a clean second stack. Firedancer takes over networking and block production. An Agave child process still handles the runtime and consensus functionality that Firedancer has not reimplemented.

Treating that boundary as one API call understates it. Current code maps named shared-memory workspaces into the Agave child process and connects them to Firedancer's bank and PoH topology.

The public architecture docs draw the line like this:

• Firedancer supplies QUIC and UDP ingress, kernel-bypass networking, signature verification, transaction packing, and block-distribution networking.
• Agave retains the remaining functionality, including the runtime that tracks account state and executes transactions, and runs as a child process.
• Frankendancer is compatible with Agave's ledger and snapshot formats.

The exact message layouts and ownership rules still have to be traced per link, and the topology can change as Firedancer evolves. Format compatibility for ledgers and snapshots is a separate property from this live handoff.

The consensus-critical requirement is narrower than byte-for-byte equivalence across the entire pipeline. Alternative ingress timing, transaction selection, and packing strategies are allowed. Once a block or execution input is accepted, the clients must agree on the effects that feed committed consensus state.

2. Agave is the practical reference implementation

For executable runtime conformance, the public suite treats Agave's behavior as the reference rather than relying solely on a prose specification.

That does not mean Firedancer ignores protocol documentation or separately stated invariants. It means this particular test system designates the Agave shared object as its ground-truth target.

By default, conformance compares each harness's Effects messages. Some instruction and transaction paths also provide a consensus-mode comparator that can normalize or ignore differences not designated consensus-critical. Relevant effects include:

• account data
• lamport balances
• transaction status
• fee and compute behavior
• the account-state hash that feeds the bank hash, which may include the account-delta hash, the accounts lthash, or both depending on the release and active feature gates

A difference that never touches those fields may not cause bank-hash divergence, but it can still matter for operators, tooling, debugging, or liveness. A difference that does touch them can become a consensus bug.

Handwritten tests cannot cover that surface. The runtime has many feature-gated branches, and the SBPF virtual machine executes arbitrary bytecode. The practical approach combines handwritten fixtures, minimized divergences preserved as regressions, and cases produced by fuzzing campaigns.

3. Differential testing: one input, two shared objects, diffed effects

For the standard execution harnesses, each target is a shared object that exposes an entry point, takes a Context message, and returns an Effects message. Agave is wrapped as libsolfuzz_agave.so; Firedancer is built as libfd_exec_sol_compat.so.

The same Context goes into both targets, and the resulting Effects messages are compared. Newer gossip and direct ELF paths use raw-byte or FlatBuffers inputs, but keep the same reference-versus-target model. The implementations do not need to take the same internal path; they need to expose compatible results.

The harness does not prove full semantic equivalence across every possible input. It provides a scalable way to find divergences before they reach production.

The public solana-conformance suite was archived read-only on April 16, 2026. Public sources do not identify its replacement or establish the configuration of any non-public tooling.

The comparison itself is four lines:

context       = fixture.input
agave_effects = run(libsolfuzz_agave.so, context)       # SOLFUZZ_TARGET
fd_effects    = run(libfd_exec_sol_compat.so, context)  # FIREDANCER_TARGET
diff(agave_effects, fd_effects)                         # match | logged mismatch

Two properties make this more than a unit test:

1. The public runner explicitly supports ASAN-aware loading and sanitizer-coverage dependencies. Firedancer separately supports ASAN and UBSAN builds in broader testing, but public workflows do not establish that the conformance-vector job itself runs under UBSAN.
2. The runner replays a fixture corpus containing fuzz-generated cases, handwritten tests, and fixed mismatches preserved as regressions. Separate fuzzing campaigns generate candidate inputs and add minimized divergences to that corpus.

The suite compares behavior at several semantic layers, from a single instruction to an entire block. That helps localize where a mismatch first becomes reproducible.

A logged mismatch means the two Effects messages differed. It does not automatically mean consensus diverged. A difference in logs or error formatting is a lead to investigate; only differences that reach committed state or the bank hash are consensus-relevant.

The final archived source registered these harnesses:

The final source extended the same reference-versus-target model beyond execution into gossip decoding. It also retained a direct FlatBuffers ELF comparison path after the registered ElfLoaderHarness was removed.

Logs are not bank state, but mismatched logs and errors can still break debugging, automation, and incident response. They belong in the same testing discipline even when they do not affect consensus.

4. What the validation work is trying to catch

Firedancer's runtime known-issues tracker provides a useful historical map of validation risks. It was an Immunefi contest disclosure against a pinned audit snapshot, not a log of bugs found by the public differential harness and not proof that every listed issue changed a bank hash.

The table mixes three kinds of evidence:

1. Differential conformance findings show that the same input or feature state can produce different client behavior. Public disclosures do not always reveal which Effects fields differed.
2. Snapshot integrity checks sit outside the public shared-object Effects harness but still protect restored bank state.
3. Sanitizer findings are not semantic mismatches. They show why sanitizer-instrumented fuzzing complements effect comparison.

Feature gates compound these risks. A sound release process should test the Agave versions and activation states it intends to support rather than treating conformance as a one-time result.

ELF loading still needs explicit cross-client review even though ElfLoaderHarness was removed before the public suite was archived. Auditors flagged a sh_offset versus sh_addr difference in read-only-data sizing; the split-out tracker marks that specific discrepancy not exploitable and asks for a clarifying code comment.

5. Bank-hash divergence, worked

The bank hash is where some execution mismatches become visible to consensus. A leader records ordered transaction batches in PoH entries; each transaction entry mixes a Merkle root of the transactions' signatures into the PoH chain. Validators verify that stream, replay the included transactions, and freeze the resulting bank.

At the commit involved in the incident below, Agave's Tower path passed the frozen bank hash into its vote update. A validator cannot delete a transaction from the published entry stream and keep the same PoH history. It replays every included transaction under the protocol rules; an instruction-level failure can itself be the valid result, while a block-invalid entry or transaction causes the slot to be rejected rather than edited locally.

Agave issue #5454 is a clear public example. It happened between two versions of Agave itself, which is exactly the point: this class of bug does not require a second client.

A change to which fee-related value was recorded in the blockhash queue, not the fee charged to transactions, altered one account update:

// before
bank.fee_rate_governor.lamports_per_signature

// after (Agave PR #5219)
last_lamports_per_signature

The values usually matched. Under high Testnet throughput, the current governor value became higher. Transaction fees and fee calculation did not change, but the deprecated RecentBlockhashes sysvar account was serialized differently. At that release's feature state, the change propagated into the bank hash:

different lamports_per_signature recorded with the recent blockhash
  -> different RecentBlockhashes sysvar data
  -> different account-delta hash
  -> different bank hash
  -> canary freezes the slot with a local hash that differs from the cluster hash
  -> duplicate-slot purge and repair loop
  -> after more than ten unsuccessful repairs, the canary panics and exits

The issue records both bank hashes for the slot:

expected: 59nGB3tQAoyKiwfog6wUoy9QRKo52tqkq7kcbes3tQQH
computed: 8Ce6XM2i6GyJ8hwxpQybDafRjftxhyt4UwA4BfUn59ZK

No malicious input and no exotic opcode were required, only a routine refactor in Agave PR #5219 on a path that touched a hashed account. An Agave maintainer confirmed that the RecentBlockhashes update fed the account-delta hash, which fed the bank hash in that release. The issue establishes that the canary exited; it does not report a Testnet-wide stall.

PR #5219 was reverted in PR #5455. Independently, the already accepted SIMD-0223 change later removed the account-delta hash from the bank hash and was activated across public clusters. A separate feature-gated attempt to reintroduce the fee-governor cleanup, PR #5749, closed without merging.

This is the kind of divergence semantic comparison is meant to catch before release. A bank-hash mismatch proves disagreement in the consensus-committed result. It does not, by itself, tell you whether the cause was account state, transaction counting, the PoH-derived last blockhash, ancestry, serialization, hash computation, or another component of that result.

Conclusion

Conformance checks should gate Firedancer and Agave release combinations. They should cover the feature states each release supports, preserve past mismatches as regression fixtures, and pair semantic comparison with sanitizer-instrumented fuzzing.

For auditors reviewing client behavior, the bank hash is a high-signal symptom. The harness output shows where the mismatch first became observable.

The bank hash tells you something diverged. The harness tells you where.