Repetitions and evaluations
The unit that matters for both accuracy and billing is the scoring evaluation. One scoring evaluation runs three independent repetitions. Each repetition scores a two-model pair, and the three pair scores are averaged. That means a single evaluation makes six generator samples while consuming one ranking credit.Credits are deducted per successful scoring evaluation — not per repetition, not per model
call. Three repetitions cost the same as one.
Score layers
An asset is scored at one of two layers, reported asscore_layer:
image-only
The default. The persona sees the creative with no ad copy attached.
full-ad-package
Used when you send
copy_context. The persona sees the creative and the words.The image-only ablation and copy lift
When you attachcopy_context, Kettio does not just score the full package. It also re-scores
the same creative with the copy stripped out, then reports the difference.
This is why a copy-context asset can cost two evaluations instead of one: the ablation is a
second evaluation. If the ablation fails,
image_only_score and copy_lift are absent and the
asset costs one.
What a score is not
This is the part worth being precise about, because it determines whether you build something useful or something misleading.Not a conversion probability
Not a conversion probability
A score of 4.24 does not mean 4.24% of anything, and does not map to a purchase rate.
Not comparable across batches
Not comparable across batches
Scores are calibrated relative to the assets in the submitted batch. Two assets that scored
4.1 in different batches did not necessarily perform equally.
Not comparable across goals or audiences
Not comparable across goals or audiences
Changing
goal or audience changes what is being measured. Hold both fixed for any
comparison you intend to act on.Not a forecast
Not a forecast
Kettio does not predict CPA, ROAS, or causal lift. It is a prioritization signal for
deciding what to test, not a substitute for testing.
Related
Reading confidence
Deciding when a gap between two scores is real.
Limits and billing
How evaluations translate into credits and rate limits.