on-the-systematic-challenges-of-culturally-loaded-machine-translation-dream-of-the-red-chamber-as-the-cultural-lens-039b3c17·1 events·first seen Aliases: On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens
Researchers construct a 500-segment Chinese-Japanese bilingual dataset from Dream of the Red Chamber to probe LLM-based machine translation on culturally loaded content. The study identifies three failure modes: frontier LLMs show notable performance gaps on cultural expressions, human evaluators disagree substantially based on background, and standard automatic metrics fail to reliably assess quality in this domain. The work highlights a systematic blind spot in current MT evaluation and model capability.