omegause-officeval-18acf28a·1 events·first seen Aliases: OmegaUse-OfficeVal
A new arXiv preprint introduces OmegaUse-OfficeVal, a 100-task benchmark for evaluating LLM agents on long-horizon office-suite workflows derived from real practitioner requests. Each task is paired with economic signals — human labor time (averaging 2.32 hours per task) and a task price proxy — enabling direct cost comparisons between human workers and LLM inference. Frontier LLMs evaluated against a human baseline are found to be substantially cheaper and faster but have not yet reached human-level deliverable quality. Code and dataset are fully open-sourced.