Crowdsourcing Piedmontese to Test LLMs on Non-Standard Orthography

Key Points

• Machine translation and topic classification show improved performance using the Piedmontese dataset, highlighting its utility.
• The evaluation dataset includes diverse linguistic features, fostering better understanding of non-standard orthography outcomes.
• Analysis relies on advanced datasets like FLORES+ and SIB-200 to assess language models' capabilities across languages.
• Utilization of crowdsourcing enhances dataset richness, yet further exploration on non-standard forms is necessary.