Large Language Models (LLMs) perform well in high-resource languages like English, but struggle with low-resource languages like Greek due to limited training data.
Meltemi is a new open-source Greek LLM trained by the Institute for Language and Speech Processing of Athena Research Center. It is bilingual (English and Greek) and built upon Mistral-7B with continual pre-training on a massive Greek text corpus.
Two models are released: Meltemi-7B-v1 and Meltemi-Instruct-7B-v1 (instruction-tuned for chatbots). They show a 14.9% improvement over Mistral-7B on an LLM evaluation suite.
The training was done on an AWS infrastructure provided by GRNET.