Skip to content

tx-generator 3.0: sub-transactions, reference scripts, Plutus V4, compatible API - #6776

Draft
Russoul wants to merge 28 commits into
masterfrom
russoul/dijkstra-bench
Draft

Russoul wants to merge 28 commits into
masterfrom
russoul/dijkstra-bench

Conversation

@Russoul

@Russoul Russoul commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Description

Supersedes #6602, whose work is included here, co-authored with @carbolymer.

tx-generator 3.0 and the benchmarking profiles for the Dijkstra era: sub-transactions, reference scripts and Plutus V4. See bench/tx-generator/CHANGELOG.md for details.

tx-generator

  • Transactions are built with Cardano.Api.Compatible (from Update tx-generator to not use deprecated cardano-api API #6602) instead of the deprecated createTransactionBody and signShelleyTransaction, in every Shelley-based era. Pre-Conway eras are supported through the compatible API but are not tested; whether to add per-era tests is left to review.
  • Sub-transactions: a new NestedNtoM generator, whose transactions each carry sub-transactions (Dijkstra onwards). The compatible API has no sub-transactions yet, so these transactions are built with the experimental API by genNestedTx, which shall go once the compatible API supports them. Until then, a test checks that without sub-transactions genNestedTx builds the same bytes as genTx.
  • Script variants and reference scripts: a Plutus workload can spend variants of its script (distinct hashes, the same behaviour and cost), by reference or in the witnesses, so that the ledger's cache of decoded scripts doesn't hide the cost of decoding them.
  • Plutus V4: execution units of V4 scripts, a new LoopV4 script, and calibrate-script in the era of the protocol version.
  • Fee estimates equal the ledger's minimum fee. Fix: the reference script fee parameters were lost on the way through ProtocolParameters, which made reference scripts free in every estimate.
  • Transactions are Exp.SignedTx at the submission boundary (from Update tx-generator to not use deprecated cardano-api API #6602).
  • Unused dependencies and -Wno-deprecations dropped (from Update tx-generator to not use deprecated cardano-api API #6602).

cardano-profile, workbench

  • ci-test-v12-nested, forge-stress-pre-v12 and forge-stress-pre-v12-nested: flat and nested transactions on the v12 preview.
  • LoopV3 profiles comparing scripts by reference with scripts in the witnesses; plutuscall-voltv12-loopv{3,4}, LoopV3 and LoopV4 on the v12 preview.
  • plutuscall-volt-refscript-1mib*: blocks at the limit of 1 MiB of reference scripts on mainnet parameters, over a range of variant group sizes.
  • Nodes ask a new peer for transactions after 1 s rather than 60 s. In a fresh cluster a setup transaction otherwise stays in one node's mempool for a minute, and is lost if that node's block loses a slot battle.
  • supervisord: the run's config is passed to every supervisorctl call (from Update tx-generator to not use deprecated cardano-api API #6602).

plutus-scripts-bench: LoopV4, LoopV3's loop as a Plutus V4 script.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. These may include:
    • golden tests
    • property tests
    • roundtrip tests
    • integration tests
      See Running tests for more details
  • Any changes are noted in the CHANGELOG.md for affected package
  • The version bounds in .cabal files are updated
  • CI passes. See note on CI. The following CI checks are required:
    • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
    • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
    • Code builds on Linux, MacOS and Windows for ghc-9.6 and ghc-9.12
  • Self-reviewed the diff

Transactions are now built, witnessed and signed with
Cardano.Api.Experimental (makeUnsignedTx, makeKeyWitness, signTx) instead
of createTransactionBody / signShelleyTransaction; the signed ledger
transaction is still handed to the submission code as an old-API Tx.
The resulting transactions are byte-identical to the previous ones.

- genTx, the genesis spend, the sized-metadata measurements and
  calibrate-script's dummy transaction all go through the new API; fee
  estimates use Exp.evaluateTransactionFee (estimateTxFee).
- Outputs are ledger outputs (Exp.TxOut); fund witnesses are
  Exp.AnyWitness, with Plutus spending witnesses converted once via
  legacyWitnessConversion (toAnyTxInWitness); a fund's value is a Coin.
- Only Conway and Dijkstra are supported, as by the experimental API:
  withEra rejects other eras, and the selftest, PureExample and the
  metadata cost tests use Conway and Dijkstra.
- getFundWitness compares eras with testEquality, so funds can be spent
  in Dijkstra.
Add a NestedNtoM generator: an NtoM transaction that also carries K
sub-transactions (Dijkstra onwards), each spending its own n funds into its own
m outputs, balanced on its own. The top-level transaction pays the fee for the
whole transaction and carries the metadata; with no sub-transactions it is the
same transaction as NtoM's.

- genNestedTx builds and signs the sub-transactions with the experimental
  cardano-api (makeUnsignedSubTx, makeSubTxKeyWitness, signSubTx) and attaches
  them to the top-level transaction, built and signed like genTx's.
- The outputs of a sub-transaction are stored under its own id.
- A Tx preview warns (TraceBenchTxSubWarning) if a transaction would exceed
  maxTxSize or need more than the configured fee, which the nodes would
  reject.
- The optional sub_txs setting ({ count, inputs, outputs }) makes the compiler
  split enough funds for the sub-transactions and generate NestedNtoM instead
  of NtoM; it requires the Dijkstra era and is not combined with Plutus yet.
…actions

Add a sub_txs generator field to cardano-profile (left out of the JSON when
unset, so no other profile changes) and the matching tx-generator service
option, and a ci-test-v12-nested profile: the ci-test-v12 value workload whose
transactions each carry 4 sub-transactions of 2 inputs and 2 outputs. It runs
at 0.2 TPS, so that the UTxO setup fits before the nodes shut down at block 8
and the benchmarking phase gets to run.
By default a node waits 60 s (ouroboros-network's
defaultTxSubmissionInitDelay) before asking a new peer for transactions,
to temper peers that disconnect soon after connecting. In a fresh
benchmarking cluster every connection is new: for a minute a submitted
transaction stays in the mempool of the node it was submitted to, and
when the block that node puts it in loses a slot battle, nothing returns
it to a mempool. A setup transaction lost that way breaks the generator's
setup chain, and every benchmarking transaction after it is rejected.

The base node configuration now sets TxSubmissionInitDelay to 1 s for all
profiles.
…tions

Run ci-test-v12-nested at ci-test's 15 TPS with a fixed count of 300
transactions instead of 0.2 TPS. Splitting the 3000 UTxOs they spend takes
about 2 blocks, and the remaining blocks before the shutdown at block 8
are full (45 nested transactions each). For this, add the txCount
primitive. The blocks fill because the nodes take a new peer's
transactions after 1 s: with the nodes' default of 60 s, about 3 of them
were empty.

Count sub-transaction inputs in derived.utxo_generated.
forge-stress-pre in the Dijkstra era (the v11 and v12 preview overlays on
the same 64k-block genesis), and the same again with transactions
carrying 4 sub-transactions of 2 inputs and 2 outputs each, to compare a
full block of nested transactions with a full block of flat ones.

The nested profile submits 7200 transactions rather than 36000, so that
the generator prepares the same 72000 UTxOs (10 rather than 2 per
transaction) and its splitting phase ends at the same point as in the
flat profiles; all three stay saturated until shutdown.
As ci-test, with the Conway and the v12 genesis, but no generator starts
and the nodes run until stopped: a cluster to run a generator script of
one's own against, such as one using actions the compiler does not
produce yet.
Low-level support for transactions that spend outputs locked by many
distinct Plutus scripts, with the scripts either in the witnesses or
referenced from reference script outputs.

The ledger caches decoded scripts by hash for the duration of a block, so
transactions that all use one script measure decoding once per block.
Script sets make every transaction use its own script:

* DefineScriptSet: variants of a Plutus script with distinct hashes and a
  chosen size (Cardano.TxGenerator.ScriptVariants). A variant applies the
  base script to a delayed, never evaluated padding term, so all variants
  behave like the base script at the same cost.
* PublishScripts: a generator that puts each variant of a set into a
  reference script output, packed into as few transactions as fit.
* PayToScriptSet: a pay mode that locks outputs at the variants'
  addresses, cycling through them in groups, and stores each fund with
  the witness for spending it, by reference or with the script in the
  witnesses.

Spending a fund by reference adds its reference input to the transaction,
and NtoM accounts for the reference scripts' size in its fee estimate.

ProtocolParameters now carries minFeeRefScriptCostPerByte, and for
Dijkstra the reference script cost stride and multiplier: they used to be
lost on the way from the queried ledger parameters and back, which made
reference scripts free in every fee estimate.
A transaction's outputs are now planned before it is built, as 'Output's:
the ledger output together with how the fund it becomes is spent. Once the
transaction's id is known, 'outputFund' turns output i into the fund at
TxIn txId (TxIx i).

Pay modes become 'Destination's: the wallet that keeps the funds, and the
output paying a value as the j-th output paid to the destination. 'payTo'
plans outputs to a destination, and 'storeOutputs' stores the funds they
become and counts them as paid, so previews and draft builds no longer skip
script set variants. The variant index of NestedNtoM now continues across
the top-level transaction and its sub-transactions.

This replaces the per-generator 'mangle'/'cyclingOutputs' plumbing and the
'CreateAndStore' closures in Cardano.TxGenerator.Tx, along with
'sourceToStoreTransaction' and friends.
'VariantIx' numbers the variants of a script set, in place of 'Int' in
'scriptVariant', the script set's reference outputs, and the PublishScripts
and PayToScriptSet setup.
…oads

A Plutus workload can now spend variants of its script instead of the script
itself: generator.plutus.variants = { count, size, source }, with the same
JSON in the profile, the Nix service and the tx-generator's options. The
variants are size bytes each, or as small as they can be without a size, and
the benchmarking transactions find them in their witnesses ("witness") or by
reference ("reference").

With variants, the compiler defines them as a script set after the
collaterals and, to be spent by reference, publishes them with funds from the
genesis wallet, at the address of a new ReferenceScripts key. The last
splitting step then locks the benchmarking inputs at the variants in turn,
as many consecutive inputs at one variant as a benchmarking transaction
spends. Without variants, the compiled script is unchanged.

ci-test-v12-plutus-ref and ci-test-v12-plutus-inline run LoopV3 with 20
variants of 4 KiB, by reference and in the witness respectively.
…e estimates

FixedLoop runs a Plutus loop script with the loop count its redeemer gives,
at the execution units of one run, which the compiler budgets with the new
MeasuredScript. Unlike the calibrated loop types, its execution stays small
and fixed, for benchmarks that vary something else.

plutus.variants.group is the number of consecutive benchmarking inputs locked
at one variant of the script: by default as many as a transaction spends,
with 1 a variant of its own for each input.

Fee estimates count the key witnesses a signed transaction carries, rather
than one per input plus one for the collateral, which made them exact: the
estimate equals the ledger's minimum fee. NtoM now warns of a configured fee
below the estimate for any transaction, not only those with reference
scripts.
The setup publishes the variants of a Plutus script as reference scripts
also when the benchmarking transactions carry them in their witnesses, so
that a profile with plutus.variants.source witness and one with reference
differ in nothing but how the benchmarking transactions discharge the
scripts.
Profiles that compare reference scripts with scripts in the witnesses,
alike in all else. LoopV3 runs as 200 variants of 15 KB, each benchmarking
input locked at one, its script in the witnesses (*script15k*) or by
reference (*refscript15k*). The variants' padding is never run, so they
calibrate to the execution of LoopV3 alone. All pay the 2 ada fee of the
dearer witness discharge and analyse without a size filter, since their
blocks differ in size by the scripts.

  * plutusv3-volt-{script,refscript}15k-nomadperf: plutusv3-volt-nomadperf
    on the perf cluster;
  * plutuscall-volt-loopv3{,-script15k,-refscript15k}: plutuscall-volt-loop,
    6 nodes on loopback, with LoopV3 alone as a baseline for the cost of
    the script bytes.
LoopV4 runs LoopV3's loop as a Plutus V4 script. The V4 script context
is encoded as a list, [txInfo, redeemer, scriptInfo, scriptHash], rather
than as a constructor, so the script reads the redeemer as the second
element of that list; it decodes nothing past the redeemer, as LoopV3
does. gen-plutus lists it and prints V4 scripts as text envelopes of type
PlutusScriptV4.
preExecutePlutusScript counts the execution units of Plutus V4 scripts, as
it does for V1 to V3, against an empty V4 script context, which adds the
hash of the executing script and new transaction info fields, and is
encoded as a list. The generator's automatic budgeting and calibrate-script
both go through it.

data/protocol-parameters-v12.json is the v12 preview for that: the v11
parameters with protocol version 12, the PlutusV4 cost model of the
v12-preview genesis overlay, and the fees of reference scripts.
scripts-fallback/LoopV4.plutus is LoopV4 as gen-plutus prints it, and a
test evaluates it: it runs on a redeemer of 1,000,000 or more and fails
below, which it can only do reading the redeemer from the V4 context.
calibrate-script finds LoopV4 by name, with data/LoopV4.redeemer.json,
and builds the transaction it approximates size and fee with in Dijkstra
for protocol version 12 or later, else in Conway as before: a Plutus V4
script can only be witnessed in Dijkstra. With
data/protocol-parameters-v12.json, LoopV4 calibrates to 8667 loops for the
transaction memory budget, at 4 transactions per block.
plutuscall-voltv12-loopv3 and plutuscall-voltv12-loopv4 are
plutuscall-volt-loopv3 with the v11 and v12 preview overlays (Dijkstra,
protocol version 12, the PlutusV4 cost model), running LoopV3 and LoopV4,
the same loop as a Plutus V3 and a Plutus V4 script. Nothing else differs
between them. They need ERA=dijkstra.
@Russoul
Russoul force-pushed the russoul/dijkstra-bench branch from fd7368f to db31fec Compare November 19, 2026 13:40
Russoul and others added 10 commits November 19, 2026 17:51
…ck limit

plutuscall-volt-refscript-1mib fills blocks to the limit of 1 MiB of
reference scripts per block, on mainnet parameters (Conway, epoch 537),
with next to no execution. Every transaction spends one input locked at a
variant of LoopV3 of its own, 15,900 bytes large, within the first tier of
the reference script fee; 65 of them fill a block. FixedLoop runs no loop
iterations. 128 variants keep the transactions of a block from sharing
one. 6 nodes on loopback, 4 epochs, analysed from slot 600 (epoch2+).

P.analysisEpoch2Plus and the epoch2+ chain filter analyse from the second
epoch.
Script variants can now be padded with a spine of applications over error
terms, besides the mixed pieces of program they were padded with so far.
At 4 bits a term, the spine makes a decoder build two terms per byte:
deserialising a 15,900-byte LoopV3 variant takes 89 ns a byte with it,
against 27 with the mixed padding and well under 1 with a bytestring. The
padding is never run, so the variants behave and cost the same as before.

ScriptSizeFixed carries the padding; plutus.variants.padding, "mixed" by
default or "error-spine", selects it, and P.plutusVariantPadding sets it in
a profile. plutuscall-volt-refscript-1mib-dense is
plutuscall-volt-refscript-1mib with the spine: its blocks take a peer about
2.6 times as long to adopt as with the mixed padding.
…up size

plutuscall-volt-refscript-1mib{,-dense}-g{2,4,8,16,32}: g consecutive
transactions reference the same variant, so a block decodes fewer distinct
scripts for the same reference script fees.
- The workbench passes the supervisord config to every supervisorctl call.
- TxGenError's Error instance prints PlutusError and TxGenError messages,
  instead of an empty string.
- The selftest and the pure example keep the decoding error of their
  hard-coded signing keys.
- A stray blank line in .gitignore, and sorted imports.

Co-authored-by: Mateusz Galazyn <mateusz.galazyn@blockpqr.io>
Transactions are built with cardano-api's compatible API (createCompatibleTx)
instead of the experimental one, as in #6602, so tx-generator runs in every
Shelley-based era again rather than Conway and Dijkstra only.

- genTx builds with createCompatibleTx and Compatible.addWitnesses, and takes
  a TxMetadataInEra. Reference inputs come from the witnesses.
- genNestedTx keeps the experimental API, in Dijkstra: the compatible API has
  no sub-transactions yet. It shall go once it does. Without sub-transactions
  it builds the same bytes as genTx, which NestedTxTest checks for key inputs,
  metadata, a reference script output, and LoopV3 and LoopV4 spent by
  reference and in the witnesses.
- Transactions are Exp.SignedTx from the generators to submission, the Ogmios
  endpoint and the dumps.
- withEra dispatches on every Shelley-based era. Script outputs set the datum
  hash per era (Alonzo, Babbage onwards), reference script outputs need
  Babbage onwards, and NestedNtoM checks for Dijkstra.
- The Plutus spending witness is #6602's toAnyPlutusScript and
  mkPlutusSpendingWitness, which can now also spend by reference; the legacy
  witness conversion goes.
- Fee estimates use the ledger's estimateMinFeeTx in every era.
- The genesis transaction, SizedMetadata and calibrate-script are #6602's:
  calibrate-script builds in the era of the protocol version, with Plutus V4
  in Dijkstra, as preExecutePlutusScript runs V4.
- The selftest and the pure example use protocol-parameters-v10.json.
- A fund's value stays a Coin, and TxEnvironment's metadata a TxMetadataInEra.

Co-authored-by: Mateusz Galazyn <mateusz.galazyn@blockpqr.io>
As in #6602: the library no longer needs cardano-crypto-wrapper,
cardano-ledger-alonzo, cardano-ledger-byron, constraints-extras, formatting,
generic-monoid, ghc-prim and unordered-containers, calibrate-script and the
API test no longer need transformers and transformers-except, nor the API
test cardano-cli. Nothing uses deprecated API any more, so -Wno-deprecations
goes. cardano-strict-containers stays, for UTxO's StrictMaybe.

Co-authored-by: Mateusz Galazyn <mateusz.galazyn@blockpqr.io>
The idle profiles are no longer needed.
wait-generator-stopped still asked the default supervisord for the
generator's status; every other supervisorctl call names the run's
supervisord.conf.
…epoch3+

plutuscall-volt-refscript-1mib* run for 5 epochs rather than 4, and are
analysed from epoch3+ like the other plutuscall profiles; the epoch2+
chain filter they alone used is removed. The measured part is still the
last 30 minutes.
@Russoul
Russoul force-pushed the russoul/dijkstra-bench branch from db31fec to 845ebe1 Compare November 19, 2026 14:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant