Hacker NewsHN submitter: wsxiaoys
Original published:

SketchAgentBench – evaluate LLMs on 3D spatial reasoning

A benchmark evaluates LLMs on 3D spatial reasoning by scoring 11 models, 25 photos, and 275 runs based on their final 3D scenes generated from a single photo and prompt.

Key points

  • •It evaluates the workflow from a single photo and prompt to code generation, rendering, inspection, and refinement.
  • •The benchmark tests 11 models, 25 photos, and 275 runs.
  • •Each model is scored based on its final 3D scene.
#Research & Papers#LLM / Model Trends#SketchAgentBench
See more