(Blog)

Notes on decisions from my projects: where a model belongs and how to tell if it's right.

Choose a judge's threshold from people's decisions, not from another judge

I switched the model that scores job postings and the alert thresholds stopped meaning anything. Comparing the two judges couldn't tell me where to put them; people's decisions could.

  • Evaluation
  • LLM
Read