Using LLM for confidence scoring is a bad idea
AI summary
Editorial policyThis article argues that relying on LLMs to generate confidence scores is a bad practice because there is no good reason to trust those numbers.
Key points
- •LLM-based agents can perform various tasks and will output probability or confidence scores when asked, even if they are unreliable.
- •Many agent designers use LLMs to generate confidence scores, but there is no good reason to trust the resulting numbers.
- •The post explains why it is problematic to rely on LLMs for generating confidence scores in agent design.
#LLM / Model Trends#AI Agents