SemPlan: Benchmarking Structured Semantic Planning for LLM-Based Queries over Enterprise Data
arXiv cs.AIen
arXiv:2608.13612v1 Announce Type: new Abstract: Natural-language interfaces to enterprise data must translate underspecified requests into governed, executable behavior while controlling invalid queries, policy failures, cost, and nondeterminism. SemPlan Benchmark evaluates this architectural design space with a deterministic synthetic bilingual benchmark containing 1,800 cases in English and Brazilian Portuguese; 1,200 cases form the frozen scientific evaluation subset. Four architectures are compared under the same model configuration: direct SQL generation (A1), a bounded tool-agent baseline (A2), structured semantic-request generation followed by deterministic planning and execution (A3)
This is a short summary published by AI Global Wire. The full article is owned and hosted by arXiv cs.AI — open it there to read it in full.
Read the full story at arXiv cs.AI- Verktyg
- Forskning
- Agenter
- Reglering
Related AI news
- Tekoälypomo antoi potkut ihmistyöntekijälle – 17 myöhästymisen jälkeenTivi · August 17, 2026
- Qwen 3.8 27B shows a 17GB open-weight general purpose model can have long context, effective tool calling, strong vision ability, and competent code generation (Simon Willison/Simon Willison's Weblog)Techmeme · August 17, 2026
- Modular Cognitive Architecture Emerges in Large Language ModelsarXiv cs.AI · August 17, 2026
- No Universal Signal Predicts Sample-Level LLM Regression under Version UpdatesarXiv cs.AI · August 17, 2026
- Cross-Disciplinary Taxonomy and Modeling of Misunderstanding Generation, Amplification, and Detection, from Pragmatics to AI AgentsarXiv cs.AI · August 17, 2026
- Reward Machines for Signal Temporal LogicarXiv cs.AI · August 17, 2026