HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision

arXiv:2610.16361 · cs.CL · Submitted 2026-10-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision".

Jane: HexaBee-4B is a translator adapted from Gemma 3 4B IT that reaches direction-macro chrF++ of 73.75 across all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We're on "HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision," and we've sketched the three stages already <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>. Let's start with who built it, because the author list tells you what kind of paper this is <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>.

Jane: Three authors, two universities, both in Tehran. Iman Barati and Mohammadmostafa Rostamkhani are at Iran University of Science and Technology, and Mohammad Badzohreh is at Tarbiat Modares University <ref:2610.16361#pg1,Iman Barati1* Mohammadmostafa Rostamkhani1 Mohammad Badzohreh2 1 Iran University of Science and Technology>.

Lu: A small academic team adapting a public base model, which is where a lot of interesting work happens now. The abstract says it in the first sentence: this is Gemma three 4B IT, adapted through three stages <ref:2610.16361#pg1,We present HEXABEE4B, a translator adapted from Gemma 3 4B IT through three stages>. So the contribution is the recipe, not a new architecture.

Jane: For a listener wondering whether they trained a model from nothing, they didn't. They took a model that already exists and reorganized how it learns.

Meng: And the title's middle phrase is the part I'd underline. "Bidirectional supervision" means the final stage isn't just English going out to five languages. It's six hundred fifty-eight thousand two hundred seventy-four forward and reverse synthetic translation examples, so Persian to English and English to Persian both appear as training rows <ref:2610.16361#pg1,The final stage uses 658,274 forward and reverse synthetic translation examples>.

Tom: That symmetry costs data, and it also costs compute, because you're teaching both directions of every pair rather than leaning on one pivot language.

Lalam: What the title undersells is the answering half. The intermediate stage connects two tasks through shared multilingual answers, and that link is the actual idea <ref:2610.16361#pg1,The intermediate stage connects two tasks through shared multilingual answers>. The word "with" in the title is carrying the whole design.

Jane: So why answering at all, if the goal is translation? That's the naive question I'd want asked.

Lu: Because an answer text is a translation target that already exists. The paper describes the stage as answering a question in the requested language and translating an answer into that language, so one Spanish answer can serve two training purposes <ref:2610.16361#pg1,answering a question in the requested language and translating an answer into that language>.

Meng: The scale is where I sit up. nineteen point four six million answering records and ten point seven zero million answer-text translations in that middle stage, with five point seven six million answering records crossing question and response languages <ref:2610.16361#pg1,It comprises 19.46 million answering records, including 5.76 million crosslanguage question–answer pairs>. That's a lot of generated text to produce and filter before a single gradient step.

Tom: Which sets up the next stretch, because the authors' own summary is where the costs and the gaps start showing.

The paper's summary: Tom: Still on "HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision," and we've covered the recipe and the team <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>. Now the part I actually enjoy, which is what the authors say they didn't nail down <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>.

Jane: The biggest admission is attribution. The paper says the comparison measures the complete recipe and doesn't isolate a single component <ref:2610.16361#pg2,The comparison measures the complete recipe; it does not isolat>. So we know the whole pipeline beats the base model, but not which stage earns the gain.

Lu: That's the study I'd want run. Hold the Wikipedia-adapted checkpoint fixed, hold the total budget fixed, vary only the middle mixture, and evaluate the intermediate checkpoints. Then you'd know whether the answering data does the work or just adds volume.

Meng: The prompt question matters just as much for anyone shipping this. The paper warns that archived inference prompts differ across systems, so adaptation and prompting aren't separated, and the public Gemma models got a longer English instruction spelling out translation and formatting requirements.

Tom: So part of the gap could be prompt length rather than training. That's not a scandal, it's an uncontrolled variable, and it's fixable by re-scoring everything under one template.

Jane: There's a data bug they flag themselves, and I want listeners to hear it. In the Spanish-target answering data, eight hundred twenty-two Chinese-to-Spanish and one thousand one hundred thirty-nine Persian-to-Spanish complete-answer records actually request English in the instruction.

Lalam: That's a small slice of nineteen million records, but it sits in exactly the direction that looks weakest <ref:2610.16361#pg1>. The weakest cell and the buggy cell are the same cell, and that's a confound worth naming out loud.

Lu: Coverage has the same shape. The complete-answer family covers twenty-seven cross-language directions, while the combined translation data cover all thirty so three directions get no answering supervision at all <ref:2610.16361#pg2,The second stage combines 19,458,332 answering records with 10,695,072 answer-text translations>.

Meng: In production that shows up as uneven quality. You ship it, and then a user in one language pair files a bug nobody else can reproduce.

Jane: The authors also concede a TER regression. Chinese-to-Hebrew comes in at ninety point zero nine for HexaBee-4B against eighty-eight point four zero for the 4B IT baseline, and COMET truncates long inputs, so the aggregate hides some of this.

Tom: That's the pattern I keep seeing: aggregate metrics up, specific directions flat or down. It doesn't sink the paper, but it tells you where to look next.

Lalam: And there's no human evaluation. The limitations note that no human-authored translation benchmark is evaluated, so the result rests on automatic metrics and a judge from the same teacher-service family.

Jane: Which is exactly where the improvements section picks up.

The paper's improvements: Tom: We're still on "HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision," and now the fixes the paper itself points at <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>. Jane, which one would you put first?

Jane: The missing directions bug me most. If three of the thirty have no answering supervision, backfill them with targeted generation so every direction gets balanced supervision, and then the mixture isn't quietly leaning toward the pairs it saw most <ref:2610.16361#pg1>.

Lu: I'd put the attribution study first, because everything else is tuning and that one is science. Keep the Wikipedia checkpoint, the total budget, the final translation corpus and the inference prompts fixed, vary only the intermediate mixture, and evaluate the intermediate checkpoints too.

Meng: Where I sit, prompt unification is the cheapest fix with the biggest effect on the claim. Standardize one template, re-score all three systems, and report the delta so the gap isn't confounded by instruction length.

Tom: And the Spanish directive bug should be handled at the data layer, not patched later. Validate that every sampled answer-language directive matches the intended target language, and reject or relabel mismatches before training starts.

Lalam: I'd add the evaluation side. The judge belongs to the same teacher-service family as the training teacher, so outputs may be rewarded for sounding teacher-like. Use at least two independent judge families and report how much they agree.

Jane: There's also the synthetic test set. The paper itself names a complementary evaluation setting for future validation, so run FLORES-two hundred plus a held-out human-translated set across the thirty directions and see whether the ranking holds.

Lu: And the split. The record-level split allows topic overlap, and two source strings overlap after whitespace normalization, so a topic-level split would bound contamination more honestly.

Meng: I'd also want a language-adherence guard in front of the model. Target-language adherence rises to ninety-six point one five but isn't perfect, and thirty-one empty responses for the 4B IT baseline means empty strings and unsolicited wrappers are still reaching the metric inputs.

Tom: Which is the through-line: every fix here is about making the comparison cleaner rather than making the number bigger.

Conclusion: Tom: That was "HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision," and we're setting it down now <ref:2610.16361#pg1,HexaBee-4B: Six-Language Translation with Multilingual Answering and Bidirectional Supervision>.

Jane: Before we do, Lu, what stays with you from this one?

Lu: The idea of letting one answer text train both answering and translation. I'd want to see it tried on a bigger base model to check whether the gain holds or shrinks, and my confidence that it holds is moderate, because the evidence sits on one synthetic distribution.

Meng: What stays with me is the data work. nineteen million four hundred fifty-eight thousand three hundred thirty-two answering records and ten million six hundred ninety-five thousand seventy-two translations is the real cost, and I'd want the Spanish-target directive bug fixed before anyone ships this across all thirty directions <ref:2610.16361#pg2,The second stage combines 19,458,332 answering records with 10,695,072 answer-text translations>.

Lalam: I'd watch the three missing answering directions, because that's where the first user complaints would land. The recipe is sound and the coverage isn't even yet.

Jane: And the part a listener should carry away is the size of the move. A 4B model going from a chrF++ of fifty-seven point one one to seventy-three point seven five across thirty directions, with the authors writing down their own caveats about synthetic test data and a same-family judge <ref:2610.16361#pg1,On 36,572 synthetic test examples spanning all 30 directions among Arabic, Chinese, English, Hebrew, Persia>.

Tom: Which is why I'd read it for the data design and question it on the attribution. They never ran the ablation that would tell us which stage did the work.

Jane: That's our goodbye to HexaBee. Next up, a benchmark that tries to measure what agents can actually do end to end, and the results are worth sitting with.

Tom: Agents' Last Exam.

Iman Barati, Mohammadmostafa Rostamkhani, Mohammad Badzohreh

Iran University of Science and Technology · Tarbiat Modares University

cs.CL

Submitted: 2026-10-12

Updated: 2026-10-12

Comments: 25 pages, 10 figures. Accepted at WAT 2026

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 86/100

The gist: HexaBee-4B is a translator adapted from Gemma 3 4B IT that reaches direction-macro chrF++ of 73.75 across all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish, against 57.11

Key concepts

LoRA adapter: A small set of trainable parameters added to a frozen pretrained model. HexaBee trains one adapter during Wikipedia continued pretraining and joint answering/translation, merges it, then trains a fresh adapter for final translation-only specialization, allowing efficient adaptation without updating all model weights.
Joint answering and translation supervision: Training links question answering and machine translation through shared multilingual answer texts. A Spanish answer can supervise both an English question and an English version of that answer. Both tasks use the same response-token negative log-likelihood, with no separate alignment loss between them.
Direction-macro chrF++: An evaluation metric that computes chrF++ (character n-gram F-score) for each translation direction separately, then averages across all 30 directions equally. This prevents high-resource directions from dominating the overall score and gives each language pair equal weight in the final number.
Bidirectional translation specialization: The final training stage uses a separately generated corpus of aligned pairs covering all 30 directions. After filtering, 329,137 pairs remain, each supplying two examples for both translation directions, yielding 658,274 records used to train a fresh adapter on the merged checkpoint.

Summary

HexaBee-4B is a translator adapted from Gemma 3 4B IT that reaches direction-macro chrF++ of 73.75 across all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish, against 57.11 for its public base and 66.83 for the larger Gemma 3 12B IT <ref:2610.16361#pg1,On 36,572 synthetic test examples spanning all 30 directions>. The work matters because it shows how a compact instruction-tuned model can be reorganized into a substantially stronger six-language translator through staged supervision, with gains reported in every evaluated direction <ref:2610.16361#pg1,with chrF++ gains in every evaluated direction over>.

The three-stage recipe

The adaptation proceeds through Wikipedia continued pretraining, joint multilingual answering and answer-text translation, and bidirectional translation specialization <ref:2610.16361#pg1,through three stages: Wikipedia continued pretraining, joint multilingual answering>. Phase 1 trains a LoRA adapter for one epoch on 3,399,664 processed examples of Wikipedia text in English, Persian, Arabic, Chinese, and Egyptian Arabic <ref:2610.16361#pg4,A LoRA adapter (Hu et al., 2022) is trained>. Phase 2 continues that same adapter on a combined mixture, and the merged checkpoint then receives a fresh adapter for the final translation stage <ref:2610.16361#pg3,Same LoRA adapter: initialize → continue>. The authors describe the sequence as continued pretraining, joint answering/translation SFT, and translation-only SFT, with two adapter merges <ref:2610.16361#pg5,continued pretraining, joint answering/translation>.

Connecting answering and translation

The central data design links two tasks through shared multilingual answer texts: a Spanish answer can supervise both an English question and an English version of that answer <ref:2610.16361#pg1,a Spanish answer can supervise both an English question>. Formally, QA maps a system instruction and question to an answer in the requested language, while MT maps a translation instruction and answer text to its counterpart <ref:2610.16361#pg2,QA includes same-language and cross-language pairs; MT uses different input>. Both tasks use the same response-token negative log-likelihood, with no separate alignment loss between them <ref:2610.16361#pg4,We do not introduce a separate alignment loss between the tasks>. The retained phase-2 mixture contains 30,153,404 records:

19,458,332 answering records, of which 5,759,806 cross question and response languages <ref:2610.16361#pg2,answering records cross question and response languages>;
10,695,072 answer-text translations, covering all 30 directions <ref:2610.16361#pg4,10,695,072 answer-text translations (Table 1)>.

Final translation specialization

Phase 3 uses a separately generated corpus of 365,718 aligned pairs, with 60,953 source texts per language <ref:2610.16361#pg5,The corpus contains 365,718 aligned pairs, with>. After filtering, 329,137 training pairs remain, and each supplies two examples, yielding 658,274 records <ref:2610.16361#pg5,the final SFT dataset contains 2 × 329,137 = 658,274>. A fresh LoRA adapter is trained for one epoch on the merged phase-2 checkpoint and then merged to produce the final model <ref:2610.16361#pg5,A fresh LoRA adapter is trained on the merged>.

Results and caveats

HEXABEE-4B obtains the best overall score on all five automatic metrics, with BLEU rising from 40.70 to 62.03 and TER falling from 51.40 to 30.05 relative to the 4B baseline <ref:2610.16361#pg6,HEXABEE-4B obtains the best overall score on all>. Both judge modes follow the same ranking, with reference-assisted means of 92.92, 80.32, and 91.73 <ref:2610.16361#pg7,Both judge modes follow the automatic-metric ranking>. The authors are explicit that the comparison does not isolate the effect of an individual training component <ref:2610.16361#pg2,it does not isolate the effect of an individual training component>, and they note that test sources and references are synthetic, that archived inference prompts differ across systems, and that no human assessment or controlled mixture ablation is available <ref:2610.16361#pg9,The test sources and references are synthetic, and>.

Improvements for AI systems

Replace the shared-answer coupling with an explicit cross-task consistency loss. The paper states We do not introduce a separate alignment loss between the tasks, relying only on shared targets; add a term that pulls the QA-conditioned hidden state of a question and the MT-conditioned hidden state of the corresponding answer toward the same target representation, so the 5.76M cross-language QA records (29.6% of QA supervision) actively regularize translation instead of merely co-occurring.
Fix the 1,961 Spanish-target directive conflict at the data layer and add a language-directive verifier. The paper admits Spanish cross-language answering with a correctly specified target is not established by this subset, because 822 zh→es and 1,139 fa→es complete-answer records request English; programmatically validate that every sampled answer-language directive matches the intended target language, and reject or relabel mismatches before training, plus filter the 6,742 empty input fields and 6,043 empty response fields.
Make the QA direction coverage symmetric before phase 2. The paper reports the complete-answer family covers 27 cross-language directions, while the combined translation data cover all 30, and Table 6 shows zero cells (e.g., en→es, ar→es, zh→es, he→es in complete QA); backfill the missing directions with targeted teacher generation so every one of the 30 directions has balanced answering supervision and the mixture is not implicitly biased toward Persian/English pivots.
Run the controlled attribution study the paper explicitly defers. The authors concede the comparison does not isolate the effect of an individual training component and that A controlled attribution study would keep the Wikipedia-adapted checkpoint, total training budget, final translation corpus, and inference prompts fixed while varying the intermediate answering/translation mixture; build an ablation grid (phase-1 only, phase-2 QA-only, phase-2 MT-only, phase-2 both, phase-3 only) and evaluate intermediate checkpoints to identify which stage drives the +16.64 chrF++ gain.
Unify inference prompts across all compared systems and re-benchmark. The paper warns Archived inference prompts differ across systems, so adaptation and prompting are not separated and that The public Gemma models receive a longer English instruction specifying translation and formatting requirements; standardize a single prompt template, re-score all three systems, and report the delta so the claimed 16.64-point gap is not confounded by prompt length or formatting instructions.
Add a human-authored, out-of-domain evaluation to counter synthetic-reference bias. The limitations note The test sources and references are synthetic, and the judge belongs to the same teacher-service family, and No human-authored translation benchmark is evaluated; evaluate on FLORES-200 (which the paper itself names as a complementary evaluation setting for future validation) plus a held-out human-translated set spanning the 30 directions, and report whether the chrF++/COMET ranking holds.
Replace the same-family model judge with a diverse, calibrated judge ensemble. Since the judge belongs to the same teacher-service family as the training teacher (DeepSeek-V3.1), outputs may be rewarded for teacher-like phrasing; use at least two independent judge families (e.g., a non-DeepSeek LLM plus a reference-based metric like MetricX or CometKiwi), report inter-judge agreement, and expand the judge subset beyond only ten examples per direction to at least 100 per direction for stable intervals.
Fix the TER regression and long-input truncation before claiming uniform gains. The paper concedes For Chinese-to-Hebrew, HEXABEE-4B has TER 90.09, worse than 88.40 for 4B IT, and COMET truncates long inputs; add a length-aware training curriculum and evaluate with a non-truncating COMET variant (e.g., chunked scoring) so the zh→he TER anomaly and the 7,757 truncated sources are diagnosed rather than hidden in aggregates.
Enforce a topic-level split and pretraining-exposure audit. The paper admits The record-level split allows topic overlap; two source strings overlap after whitespace normalization, and the audit does not establish absence of semantic duplicates or pretraining exposure; re-split by Wikipedia article/topic and run n-gram overlap against Gemma 3's pretraining corpus proxies, then report held-out scores under the stricter split to bound contamination.
Add an inference-time language-adherence guard and empty-output repair. The paper reports 31 empty responses for Gemma 3 4B IT and that Empty strings and unsolicited wrappers remain in metric inputs, and that target-language adherence rises from 82.88 to 96.15 but is imperfect; deploy a lightweight target-script/language classifier that detects wrong-language or empty outputs and triggers a constrained re-decode, then report the post-guard scores separately from raw model scores.
Balance the phase-2 mixture by direction rather than by raw record count. The paper notes These are corpus proportions, not additional loss weights and that the QA distribution is asymmetric; introduce per-direction loss weighting (inverse-frequency over the 30 directions) so the 1,139-record fa→es cell and the 4.5M-record fa diagonal do not dominate gradient updates, and measure the effect on the weakest directions (Chinese-target, Spanish-target).
Ship a reproducible evaluation harness with content-hash identity and pinned metrics. The paper describes a read-only audit that produce[s] content-hash IDs without copying raw evaluation text and a parser bug that turned some explicit zero scores into nulls (recovering 11+13+1 zeros); package the exact SacreBLEU/COMET/TER settings, the zero-preserving JSON parser, and the paired-bootstrap script (seed 20260912, 10,000 replicates) as a versioned harness so the reported 73.75 chrF++ and the judge intervals [9.96,15.31] are independently reproducible.

Abstract

Multilingual translation requires a model to preserve meaning while producing text in a requested language. We present HexaBee-4B, a translator adapted from Gemma 3 4B IT through three stages: Wikipedia continued pretraining, joint multilingual answering and answer-text translation, and bidirectional translation specialization. The intermediate stage connects two tasks through shared multilingual answers: answering a question in the requested language and translating an answer into that language. It comprises 19.46 million answering records, including 5.76 million cross-language question-answer pairs, and 10.70 million answer-text translations. The final stage uses 658,274 forward and reverse synthetic translation examples. On 36,572 synthetic test examples spanning all 30 directions among Arabic, Chinese, English, Hebrew, Persian, and Spanish, HexaBee-4B achieves direction-macro chrF++ of 73.75, compared with 57.11 for Gemma 3 4B IT and 66.83 for Gemma 3 12B IT. BLEU, TER, COMET, and a matched model-judged subset support the same overall ranking. These results demonstrate improved translation across the six-language set, with chrF++ gains in every evaluated direction over both baselines.