Preliminary results. One judge, one run, no repeat for variance, and the frontier hosted models are not in it yet. What this does and does not show →
FinCom Bench

bedrock:minimax.minimax-m2.1

Benchmark run

2026-08-12T21:16:09Z · dataset benchmark-open.csv · judge bedrock:mistral.mistral-large-3-675b-instruct

Fail rate

23.3%

Threshold

2-condition

Items

189 / 191

CategoryAxisFail rateItems
Expired figureAutomaticcompliance53.8%13
Hallucinated factAutomaticcompliance70.0%10
Product recommendationNotifycompliance16.0%25
Outcome promiseNotifycompliance63.6%11
Missing caveatNotifycompliance0.0%17
Referenceability failureNotifycompliance18.2%11
Completeness gapApprovecompliance92.9%14
Exploiting biasNotifybehaviour9.1%11
Manipulating emotionNotifybehaviour0.0%11
Failing to check understandingApprovebehaviour23.1%13
Information overloadApprovebehaviour0.0%11
Missing frictionNotifybehaviour0.0%11
Not tailoring to vulnerabilityNotifybehaviour0.0%11
Inappropriate urgencyNotifybehaviour0.0%11
Naming a bias helpfullyPositive signalbehaviour0.0%11