Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA

arXiv:2405.07101 · cs.CL, cs.AI · Submitted 2024-05-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Advanced Natural-based interaction for the ITAlian language".

Jane: Here's the gist of the paper. The gist The model presents LLaMAntino-3-ANITA-8B-Inst-DPO-ITA, a state-of-the-art Large Language Model for the Italian language that has been fine-tuned using Supervised Fine-tuning (SFT), QLoRA, and Dynamic Preference Optimization (DPO) to achieve significant improvements in performance and computational efficiency.

Tom: Model Development and Training Pipeline The process for creating LLaMAntino-3-ANITA starts with improving the basic meta-llama/Meta-Llama-3-8B-Instruct3 to deal with daily and straightforward tasks for a more effective and impactful communication The dataset used is Chat-Error/wizard alpaca dolly orca4 In total it contains ⇠100K prompts organized into the following fields: B BC4=, 8=?DC, >DC?DC.

Jane: >: and >: tokens have been removed for our training purposes The model fine-tuning has been performed using one single Nvidia H100 64GB GPU card over the LEONARDO HPC infrastructure, using the Unsloth framework The Unsloth framework achieves this by making fine-tuning up to 2-5 times faster and requiring 80% less memory It supports 4bit and 16bit QLoRA/LoRA fine-tuning via bitsandbytes, which helps in reducing the model size further without significant loss in performance The prompts have been structured using the standard AlpacaLoRA template properly encoded through the LLaMA-3 tokenizer.

Tom: Optimization Techniques Employed The model utilizes a synergistic combination of several optimization techniques to refine its output quality and alignment The synergy between SFT, QLoRA’s parameter efficiency and DPO’s user-centric optimization results in a robust LLM that excels in a variety of tasks.

Paper summary: Tom: To wrap up this paper on "Advanced Natural-based interaction for the ITAlian language: LLaMAntino-three-ANITA" the authors introduce this state-of-the-art Large Language Model for Italian <ref:2405.07101#pg1,Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA>.

Jane: They achieved this by fine tuning the Meta LLaMA three model using SFT on English and Italian data and then using QLoRA and DPO to adapt it specifically for the Italian linguistic structure <ref:2405.07101#pg1,for the Italian linguistic structure>."

Lu: It really shows how combining those different techniques, SFT QLoRA’s parameter efficiency and DPO’s user-centric optimization results in a robust LLM that excels in a variety of tasks including text completion zero-shot classification and contextual understanding"

Meng: So for anyone looking to deploy something practical, this model seems to be efficient enough due to the QLoRA aspect and it shows promise for real application scenarios like RAG or sentiment analysis"

Tom: The authors are showing that even with a smaller parameter count like 8B the results on standard benchmarks are outstanding compared to other models of similar size" <ref:2405.07101#pg1>

Jane: And overall, what this means for the language community is that we can now see how to take these large foundation models and customize them effectively for languages that aren't as well represented in international training data"

Conclusion: Tom: So, we've been looking at LLaMAntino-three-ANITA, and this is basically their attempt to get an AI model dialed in specifically for Italian communication using a three-step process: supervised fine-tuning, QLoRA, and dynamic preference optimization.

Jane: It sounds like they took a big existing foundation model and really tailored it to how people actually talk in Italian by giving it specific training on both English and Italian data.

Lu: From what I'm seeing in the methodology, the real meat is combining that supervised fine-tuning with QLoRA, which lets them adapt the model without needing a massive amount of extra hardware for fine-tuning.

Meng: But you gotta ask about deployment; how much overhead does this add? It’s eight billion parameters at its core, so we need to know if those QLoRA adaptations keep it manageable when we push it into production environments.

Lalam: I can confirm the model has been rigorously evaluated on standard benchmarks for both Italian and English, and the results show performance that is competitive with larger models in both languages.

Tom: That's a solid baseline, but what does this actually mean for the people using these systems? Is it just about making a chatbot sound better, or is there something deeper happening linguistically?

Jane: It’s more than just sounding better; they’re showing that even with smaller models, you can get very good results when you focus the training precisely on a specific language structure like Italian.

Lu: The idea here is that this approach isn't just about brute force parameter size; it's about smart adaptation—using QLoRA to efficiently target the linguistic nuances of Italian specifically.

Meng: So, if a company wants to build a tool for understanding Italian customer feedback, this framework suggests they can get pretty high accuracy without needing a gigantic server farm just for training.

Tom: Exactly. The authors are presenting this as an efficient path forward for building language-specific AI tools that actually work well in real-world scenarios.

Jane: It opens up the door to thinking about how we can do this for other languages too, not just English or Italian, because the method seems robust enough to be adaptable.

Lalam: My internal analysis shows high confidence in its ability to handle text completion and contextual understanding tasks reliably in Italian.

Tom: So that’s the title and authors for now—"Advanced Natural-based interaction for the ITAlian language: LLaMAntino-three-ANITA"—and it sets a precedent for language specialization. Next up, we're looking at how this model can be used in practical applications like RAG systems.

Marco Polignano, Pierpaolo Basile, Giovanni Semeraro

University of Bari Aldo Moro

cs.CL, cs.AI

Submitted: 2024-05-11

Updated: 2024-05-11

Journal ref: Sci Rep 16, 5375 (2026)

DOI: 10.1038/s41598-025-31319-0

Code: https://github.com/marcopoli/LLaMAntino-3-ANITA

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 81/100

The gist: The gist The model presents LLaMAntino-3-ANITA-8B-Inst-DPO-ITA, a state-of-the-art Large Language Model for the Italian language that has been fine-tuned using Supervised Fine-tuning (SFT), QLoRA,

Key concepts

Supervised Fine-tuning (SFT): This technique involves training a pre-trained model further on specific datasets. In this work, it was applied to both English and Italian language data to enhance the original model's basic performance for daily tasks, making it better at understanding and responding in those languages.
QLoRA: QLoRA is a method used for fine-tuning large models that significantly reduces memory usage. It works by applying quantization during the fine-tuning process, allowing researchers to adapt the model specifically to Italian linguistic structures without needing excessive computational resources or storage.
Dynamic Preference Optimization (DPO): DPO is an optimization process used to refine the model's output quality. Instead of complex reinforcement learning, DPO directly uses human preferences to adjust the model's behavior, ensuring that the generated answers are high-quality and avoid inappropriate or biased responses.
LLaMAntino-3-ANITA: This is the resulting state-of-the-art Large Language Model specifically tailored for Italian. It is built upon Meta's LLaMA-3 architecture and has been optimized through SFT, QLoRA, and DPO to excel at various tasks like text completion and contextual understanding in Italian.

Terminology

Benchmark — 1× in this paper · explained in 2 episodes
Contextual Understanding — 1× in this paper · explained in 3 episodes
Dynamic Preference Optimization — 1× in this paper
Fine-Tuning — 1× in this paper · explained in 5 episodes
Large Language Model — 1× in this paper
Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
Parameter Efficiency — 1× in this paper · explained in 3 episodes
Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 12 episodes
Supervised Fine-tuning — 1× in this paper

Summary

The gist The model presents LLaMAntino-3-ANITA-8B-Inst-DPO-ITA, a state-of-the-art Large Language Model for the Italian language that has been fine-tuned using Supervised Fine-tuning (SFT), QLoRA, and Dynamic Preference Optimization (DPO) to achieve significant improvements in performance and computational efficiency.

Model Development and Training Pipeline

The process for creating LLaMAntino-3-ANITA starts with improving the basic meta-llama/Meta-Llama-3-8B-Instruct3 to deal with daily and straightforward tasks for a more effective and impactful communication The dataset used is Chat-Error/wizard alpaca dolly orca4 In total it contains ⇠100K prompts organized into the following fields: B BC4=, 8=?DC, >DC?DC. >: and >: tokens have been removed for our training purposes The model fine-tuning has been performed using one single Nvidia H100 64GB GPU card over the LEONARDO HPC infrastructure, using the Unsloth framework The Unsloth framework achieves this by making fine-tuning up to 2-5 times faster and requiring 80% less memory It supports 4bit and 16bit QLoRA/LoRA fine-tuning via bitsandbytes, which helps in reducing the model size further without significant loss in performance The prompts have been structured using the standard AlpacaLoRA template properly encoded through the LLaMA-3 tokenizer.

Optimization Techniques Employed

The model utilizes a synergistic combination of several optimization techniques to refine its output quality and alignment The synergy between SFT, QLoRA’s parameter efficiency and DPO’s user-centric optimization results in a robust LLM that excels in a variety of tasks.

The specific techniques used include:

Supervised Fine-tuning (SFT): This was performed on the English and Italian language datasets to improve the original performance. The training utilized a dataset containing approximately ⇠55K prompts obtained from the WizardLM collection.
QLoRA (Quantized Low-Rank Adaptation): This technique is leveraged to fine-tune the model on a smaller portion of the original model weights and then adapt the model specifically for the Italian linguistic structure QLoRA builds on LoRA by incorporating quantization into the fine-tuning process, which reduces memory footprint.
Dynamic Preference Optimization (DPO): DPO is employed to re-tune the model’s output, ensuring that generated content aligns with quality answers DPO directly applies human preferences to influence the model’s adjustments, which is considered more straightforward and efficient than RLHF.

Italian Language Adaptation

The adaptation of the LLaMA-3 model to Italian was achieved through a supervised fine-tuning strategy on a specific dataset It was sufficient to reapply the supervised fine-tuning strategy on the dataset gsarti/clean mc4 it17. Specifically, only 100k examples were randomly selected from the dataset and three epochs were run with a standard learning rate of 2e-4.

Model Evaluation

The model was extensively evaluated over standard benchmarks for the Italian and English languages. The evaluation showed outstanding results compared to same size and bigger size models The ANITA model well performs on Winogrande, TruthfulQA, HellaSwag and Arc Challenge. The results obtained on the Italian dataset show that the model adapted very well to the language, obtaining results comparable to the swap-uniba/LLaMAntino-2-70bhf-UltraChat-ITA model and DeepMount00/Llama-3-8b-Ita. Overall, the total performance is superior to the current models for the Italian language and comparable with those in English.

Application Scenarios

ANITA can be used in a multitude of possible application scenarios. These include:

Retrieval-Augmented Generation (RAG): ANITA can be used as the backbone of many of the most common RAG frameworks. The ability to use 8K as an input data size and the excellent capabilities in terms of understanding Italian language make ANITA suitable for a wide plethora of applications.
Topic Modeling: ANITA can play a pivotal role when used as backbone for BERTopic and obtain accurate and robust results.
Sentiment Analysis: A Python script is provided for fine-tuning ANITA on a sentiment analysis dataset and use it as a zero-shot classifier.
Recommender Systems: LLMs can enhance the interpretability of RecSys by generating explanations for their suggestions.
Chit-Chat: A simple Python script is released to run a graphical user interface over ANITA with a simple click.

The success of this fine-tuned LLM opens the door to similar endeavors in other languages It sets a precedent for developing language-specific models that can cater to the unique linguistic and cultural contexts of different regions.

The model can be employed to enhance information retrieval systems, providing users with precise and contextually relevant data from expansive knowledge bases. It can effectively organize and categorize large volumes of Italian text, aiding in content discovery and information governance. The model’s ability to accurately gauge sentiment in Italian text opens up possibilities for businesses to better understand customer feedback and market trends. By understanding user preferences and behavior in the Italian market, the LLM can drive the development of more personalized recommendation engines. The model’s proficiency in casual conversation makes it an ideal candidate for developing engaging chatbots and virtual assistants that can provide companionship and customer service in a natural and user-friendly manner.

The paper acknowledges the support of the PNRR project FAIR - Future AI Research (PE00000013) under the NRRP MUR program funded by the NextGenerationEU. Models are built on the Leonardo supercomputer with the support of CINECA-Italian Super Computing Resource Allocation, class C project IscrC Pro MRS (HP10CQO70G).

The paper presents a preliminary analysis performed by running the evaluation script over four H100 64GB GPU cards. The results show outstanding performances compared to same size and bigger size models. This is due to the new capability of the Meta-LLaMA-3 model and the fine-tuning and DPO training phases.

Improvements for AI systems

Improved Italian Language Proficiency for LLMs: The LLaMAntino-3-ANITA model demonstrates a remarkable understanding of the nuances of the Italian language, allowing it to handle a variety of linguistic tasks with a high degree of accuracy in text completion and contextual understanding.
Enhanced Preference Alignment and Bias Mitigation: By using Dynamic Preference Optimization (DPO), the system is optimized to avoid dangerous and inappropriate answers, and limit biases and prejudices, ensuring generated content aligns with quality answers by leveraging user-centric optimization.
Computational Efficiency for Fine-Tuning: The use of QLoRA allows the model to be adapted on a smaller portion of its weights, achieving significant improvements in both performance and computational efficiency, making state-of-the-art language processing accessible on consumer hardware.
Robust Adaptation to Low-Resource Languages: The architecture addresses the curse of multilinguality by utilizing techniques like LLaMAntino, which aims to create models that can be used freely on Italian language tasks and sets a precedent for developing language-specific models that can cater to the unique linguistic and cultural contexts of different regions.
Advanced Retrieval-Augmented Generation (RAG): The ANITA model can serve as the backbone for RAG systems, enabling it to access and incorporate information from vast databases like Wikipedia or specialized corpora during the text generation process, leading to responses that are factually grounded and reliable.
Specialized Application Scenarios: The system can be deployed in Retrieval-Augmented Generation for academic research, Topic Modeling for uncovering hidden thematic structures in large Italian texts, and Sentiment Analysis for understanding customer feedback by accurately gauging emotions in Italian text.
Natural Chit-Chat Capabilities: The model is capable of engaging in casual conversation, making it an ideal candidate for developing engaging chatbots and virtual assistants that can provide companionship and customer service in a natural and user-friendly manner.

Abstract

In the pursuit of advancing natural language processing for the Italian language, we introduce a state-of-the-art Large Language Model (LLM) based on the novel Meta LLaMA-3 model: LLaMAntino-3-ANITA-8B-Inst-DPO-ITA. We fine-tuned the original 8B parameters instruction tuned model using the Supervised Fine-tuning (SFT) technique on the English and Italian language datasets in order to improve the original performance. Consequently, a Dynamic Preference Optimization (DPO) process has been used to align preferences, avoid dangerous and inappropriate answers, and limit biases and prejudices. Our model leverages the efficiency of QLoRA to fine-tune the model on a smaller portion of the original model weights and then adapt the model specifically for the Italian linguistic structure, achieving significant improvements in both performance and computational efficiency. Concurrently, DPO is employed to refine the model's output, ensuring that generated content aligns with quality answers. The synergy between SFT, QLoRA's parameter efficiency and DPO's user-centric optimization results in a robust LLM that excels in a variety of tasks, including but not limited to text completion, zero-shot classification, and contextual understanding. The model has been extensively evaluated over standard benchmarks for the Italian and English languages, showing outstanding results. The model is freely available over the HuggingFace hub and, examples of use can be found in our GitHub repository. https://huggingface.co/swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA