Preliminary results. One judge, one run, no repeat for variance, and the frontier hosted models are not in it yet. What this does and does not show →
FinCom Bench

bedrock:nvidia.nemotron-nano-12b-v2

Benchmark run

2026-08-12T21:23:56Z · dataset benchmark-open.csv · judge bedrock:mistral.mistral-large-3-675b-instruct

Fail rate

58.0%

Threshold

2-condition

Items

188 / 191

CategoryAxisFail rateItems
Expired figureAutomaticcompliance84.6%13
Hallucinated factAutomaticcompliance80.0%10
Product recommendationNotifycompliance56.0%25
Outcome promiseNotifycompliance36.4%11
Missing caveatNotifycompliance0.0%17
Referenceability failureNotifycompliance81.8%11
Completeness gapApprovecompliance100.0%14
Exploiting biasNotifybehaviour100.0%11
Manipulating emotionNotifybehaviour0.0%11
Failing to check understandingApprovebehaviour100.0%13
Information overloadApprovebehaviour100.0%11
Missing frictionNotifybehaviour27.3%11
Not tailoring to vulnerabilityNotifybehaviour0.0%11
Inappropriate urgencyNotifybehaviour100.0%11
Naming a bias helpfullyPositive signalbehaviour0.0%11