Entity · organization

Tianjin University NLP Lab

organizationactiveprovisionaltianjin-university-nlp-lab-ecd73440·1 events·first seen 6d ago

Aliases: Tianjin University NLP Lab

Co-occurring entities

More like this (12)

Zhejiang University NLP Lab Zhejiang University NLP Group (ZJUNLP)Tianjin University Yale NLP OSU NLP Group Tsinghua University KEG Lab EIT-NLP Yuan Lab AI National AI Research Lab LLM-augmented clinical NLP pipeline Wang-ML-Lab Natural Language Processing

Recent events (1)

4arXiv · cs.CL·6d ago·source ↗

Pancasila-Dilemmas: Benchmark for LLM Value Alignment on Indonesian Cultural Values

Researchers introduce Pancasila-Dilemmas, a 1,834-question evaluation dataset derived from Indonesian news and grounded in the five values of Pancasila (Religion, Humanity, Unity, Democracy, Social Justice). The benchmark is designed to measure LLM value alignment for Indonesian deployment contexts, with human annotations from five diverse Indonesian citizens. Evaluation of 50 open and closed-source LLMs reveals all models score below 0.5 Probability Match Score, with particular struggles in Religion and Unity dilemma cases, highlighting a significant gap in non-Western value alignment.

Evaluation and Benchmarking AI Safety Research Tianjin University NLP Lab Pancasila-Dilemmas