discourse particles

MalayPrag: Benchmarking LLM Handling of Discourse Particles in Colloquial Malay

This paper introduces MalayPrag, a benchmark for evaluating LLMs' ability to handle discourse particles in colloquial Malay, a low-resource Southeast Asian language. The authors define five linguistically grounded attributes for interpreting pragmatic functions of discourse particles and test ten off-the-shelf LLMs on three prediction tasks. Results show substantial challenges for current LLMs in connecting discourse particles to their pragmatic functions in Malay. Providing the five structured attributes as scaffolding significantly improves model performance, suggesting that explicit pragmatic frameworks can compensate for low-resource language deficits.