TranslationBench 1.0 · Evidence for editorial teams

Preserve the meaning. Across the whole manuscript.

ProsaBridge combines greater fidelity to the original with fluent, readable prose. In a blind comparison of a 67,127-word scholarly book, ProsaBridge reduced weighted error points by 59% compared with the same AI model prompted directly.

See plans and pricing See ProsaBridge’s results

Book comparison: Same AI model, with ProsaBridge and prompted directly
Manuscript length: 67,127 words · 61 sections
Evaluation: Independent and blind
Published evidence: Book: DE → EN · short texts: EN → DE · Chinese texts: ZH → EN, ZH → DE

A clear advantage over DeepL and Google Translate

Across 12 texts translated from English into German, ProsaBridge with GPT-6 Astra clearly outperformed the traditional translation services.

Fewer error points than DeepL: 95%; Weighted error points in the short-text comparison.
Fewer error points than Google Translate: 98%; Weighted error points in the short-text comparison.

DeepL’s error score was around 20 times as high as ProsaBridge’s; Google Translate’s was around 49 times as high. Every translation was evaluated independently and blind.

Download the short-text comparison data

The ProsaBridge advantage

Fewer weighted error points: 59%; Across the whole book, compared with the same AI model prompted directly.
Especially strong on meaning: 64%; fewer weighted error points for changes in meaning, incorrect terms, and omissions.

Download the book comparison data

Book study · September 16, 2026

A whole book, translated more faithfully with ProsaBridge

−59%

fewer weighted error points with ProsaBridge than when the same model was prompted directly. The accuracy error score fell by 64%, combining greater fidelity to the original with fluent, readable prose.

ProsaBridge brought greater fidelity to a 67,127-word scholarly book translated from German into English. We compared it with the same AI model prompted directly with the complete manuscript. Every section was checked twice against the original, independently and blind.

The method is explained directly below. The underlying scores are available in the book comparison dataset.

How we compared the translations Download the book comparison data

How we compared the translations

The comparison: The same book, the same AI model: GPT-6 Astra translated the complete manuscript from German into English, once with ProsaBridge and once when prompted directly.
The evaluation: Claude Fable 5.1 and GPT-6 Astra independently checked every section against the original. Neither knew which translation came from ProsaBridge. They assessed fidelity to the original and readability.
The calculation: The judges weight errors by severity. The percentages show how much ProsaBridge reduces weighted error points compared with prompting the model directly, both overall and for accuracy.

The dataset includes overall scores and each judge’s scores. The customer’s book remains confidential, and we do not publish the test instructions. You can check the calculation from the data.

Test setup and scope

Minor errors count 1 point, major errors 5, and critical errors 10. We pool error points and word counts across all sections and both judges, then normalize to 100 source words. To calculate the reduction, we subtract ProsaBridge’s score from the direct model’s score and divide by the direct model’s score.

Both translations used the very high reasoning setting. The model prompted directly received the complete manuscript and was instructed to make the translation read as though it had originally been written in English. It did not receive ProsaBridge’s glossary. The judges also had no access to the glossary or preparation notes.

GPT-6 Astra served as both the translation model and one of the two judges. Each judge’s scores are available in the dataset. The evaluation uses error categories from MQM (Multidimensional Quality Metrics). Accuracy covers changed meaning, incorrect terms, and omissions; readability covers issues with the language itself.

The comparison includes one translation per workflow on one book. It shows the benefit of the complete ProsaBridge workflow. Models and settings can change as ProsaBridge develops.

Error scores are weighted points, not counts of incorrect words or required corrections. The study did not measure editing time or financial savings. Editorial review is still needed before publication.

Official Chinese texts, translated into English and German

We applied the method of the short-text comparison to excerpts from 11 official Chinese texts that state bodies and state media published between 10 September and 23 October 2026: Politburo and State Council readouts, policy opinions, a regulation, a notice, a statistics release, a Commerce Ministry Q&A and Foreign Ministry press conferences, 4,194 words in total. Each text was translated into English and into German, and every translation was scored twice, independently and blind.

Fewer error points than DeepL into English: 98%; Weighted error points of the best ProsaBridge entry compared with DeepL.
Fewer error points than DeepL into German: 96%; Weighted error points of the best ProsaBridge entry compared with DeepL.

DeepL’s error score was about 45 times ProsaBridge’s into English and about 28 times into German. Google Translate’s was about 59 and 48 times.

Every text was published after the training data of the tested models had been collected. The models could not have seen the texts or any translations of them, and they had no internet access while translating. The results therefore show how well ProsaBridge translates texts that have only just been published.

Show both rankings

The score is weighted error points per 100 words of the Chinese original. Chinese is written without spaces between words, so a program determined the word boundaries. Compute costs are model usage charges expressed as multiples of the lowest measured cost in each table. They are not ProsaBridge prices.

Chinese → English

Click a column to sort · sorted by Score

Rank	Entry
1	GPT-6 Astra†With ProsaBridge · high/high/disabled	0.09	0.08	0.01	0.18	0.00	2.7×
2	GPT-6 Astra†With ProsaBridge · xhigh/xhigh/disabled	0.17	0.13	0.03	0.22	0.11	5.7×
3	Opus 5.5With ProsaBridge · high/high/disabled	0.69	0.36	0.32	0.52	0.85	1×
4	Opus 5.5With ProsaBridge · xhigh/xhigh/disabled	0.83	0.45	0.37	0.53	1.12	1.6×
5	DeepLTranslation service	3.97	2.73	1.24	3.86	4.09	–
6	GoogleTranslation service	5.22	4.25	0.97	4.33	6.10	–
7	AmazonTranslation service	14.56	11.69	2.87	12.91	16.21	–

† This entry used the same AI model as one of the judges. The published score combines both judges’ assessments.

Download the Chinese → English data

Chinese → German

Click a column to sort · sorted by Score

Rank	Entry
1	GPT-6 Astra†With ProsaBridge · xhigh/xhigh/disabled	0.19	0.14	0.05	0.24	0.15	5.7×
2	GPT-6 Astra†With ProsaBridge · high/high/disabled	0.19	0.13	0.07	0.35	0.04	3.1×
3	Opus 5.5With ProsaBridge · xhigh/xhigh/disabled	0.37	0.16	0.22	0.38	0.37	2×
4	Opus 5.5With ProsaBridge · high/high/disabled	0.63	0.29	0.34	0.67	0.60	1×
5	DeepLTranslation service	5.48	4.20	1.29	4.39	6.58	–
6	GoogleTranslation service	9.30	8.80	0.50	7.05	11.56	–

† This entry used the same AI model as one of the judges. The published score combines both judges’ assessments.

Download the Chinese → German data

What you get with ProsaBridge

Set the terminology and style before translation begins, carry the manuscript’s context into every section, and review the finished translation in Word, LibreOffice, or InDesign.

Your terminology. Your style.: Review terminology and style instructions, and resolve questions about the source text before translation begins.
Context for every section: Every section is translated with the manuscript summary, glossary, and style instructions, for consistent terminology and tone throughout.
A document ready for review: Work in Word or in your browser and receive a translated Word, LibreOffice, or InDesign file that retains the document’s structure and formatting. A PDF comes back as a Word file.

Keep your team in control, from the first terminology decisions to the final review.

See plans and pricing

Short-text comparison: results and method

Alongside the book comparison, we translated twelve short texts from English into German. Explore all 23 entries, the evaluation method, and the dataset below.

Download the 1.0 dataset

Complete short-text results

All 23 published entries from twelve English-to-German texts, with model settings, separate judge scores and relative compute costs. The results describe the configurations tested in this study. Compute costs are model usage charges expressed as multiples of the lowest measured cost; they exclude document handling and editorial work and are not ProsaBridge prices.

Click a column to sort · sorted by Score

Rank	Entry
1	GPT-6 Astra†With ProsaBridge · high/xhigh/xhigh	0.14	0.06	0.08	0.27	0.02	501.9×
2	GPT-5.6 SolWith ProsaBridge · high/xhigh/xhigh	0.25	0.07	0.18	0.34	0.16	158.9×
3	GPT-6 Astra†Model on its own · xhigh	0.25	0.14	0.11	0.31	0.20	184.2×
4	GPT-6 Astra†With ProsaBridge · medium/medium/medium	0.25	0.09	0.16	0.42	0.09	109.5×
5	GPT-5.6 SolModel on its own · xhigh	0.40	0.20	0.21	0.47	0.34	84.9×
6	GPT-6 Astra†Model on its own · medium	0.44	0.12	0.32	0.61	0.27	21.4×
7	Fable 5.1†Model on its own · xhigh	0.45	0.18	0.27	0.33	0.58	96.6×
8	Fable 5.1†With ProsaBridge · high/xhigh/xhigh	0.48	0.23	0.25	0.29	0.67	354.4×
9	Gemini 3.8 FlashWith ProsaBridge · high/xhigh/xhigh	0.51	0.19	0.32	0.54	0.47	47.5×
10	GPT-5.6 LunaWith ProsaBridge · medium/xhigh/xhigh	0.66	0.31	0.35	0.77	0.56	10.9×
11	Opus 5With ProsaBridge · high/xhigh/xhigh	0.70	0.29	0.41	0.48	0.92	174.6×
12	Fable 5.1†Model on its own · medium	0.81	0.28	0.53	0.77	0.85	45.1×
13	Gemini 3.8 FlashModel on its own · xhigh	0.82	0.41	0.41	0.78	0.85	17.6×
14	Fable 5.1†With ProsaBridge · medium/medium/medium	0.83	0.41	0.42	0.64	1.02	169.1×
15	GPT-5.6 LunaModel on its own · xhigh	1.00	0.53	0.47	0.92	1.08	4.2×
16	Grok 4.6With ProsaBridge · high/xhigh/xhigh	1.33	0.42	0.91	0.99	1.67	101×
17	GLM-5.3 FlashWith ProsaBridge · xhigh/xhigh/xhigh	1.46	0.60	0.85	1.59	1.32	9.5×
18	GLM-5.3 FlashModel on its own · xhigh	1.65	0.66	0.99	1.57	1.73	1×
19	Opus 5Model on its own · xhigh	1.83	1.07	0.76	1.68	1.99	17.6×
20	Grok 4.6Model on its own · xhigh	2.57	1.01	1.57	2.51	2.64	15.2×
21	DeepLTranslation service	2.88	1.90	0.98	2.60	3.16	–
22	GoogleTranslation service	7.13	5.98	1.15	6.30	7.96	–
23	AmazonTranslation service	15.93	12.60	3.33	14.42	17.44	–

† This entry used the same AI model as one of the judges. The published score combines both judges’ assessments.

How to read the short-text results

An entry is one AI model used in one of two ways, or one translation service. There are 23 entries.

With ProsaBridge: The AI model worked inside ProsaBridge. It read the whole text first, prepared a translation plan, and translated using that plan.
Model on its own: The same AI model received the text once and translated it in a single step.
Translation service: DeepL, Google Translate, or Amazon Translate, each translating the whole document through its own service.

Error score: Each judge assigns the errors they find to Multidimensional Quality Metrics (MQM) categories. In this benchmark, a minor error counts 1 point, a major error 5, and a critical error 10. The error score is the sum of penalty points per 100 words, averaged over both judges and all twelve texts. Lower is better.; The score reflects both the number and severity of errors: one major error in meaning carries the same weight as five minor errors. It is not a count of errors or a percentage of incorrect words, and it does not predict how often an editor will need to make a correction.
Accuracy: The translation departs from the original meaning: an incorrect fact, an inaccurate term, or an omitted sentence.
Readability: The translation conveys the meaning but reads poorly: awkward phrasing, grammatical mistakes, or inappropriate tone.

How the short-text score works

Two judges, Claude Fable 5.1 and GPT-6 Astra, read every translation next to its original. They were not told which entry produced it and did not see each other’s work. Each judge recorded errors with a category, a severity weight, and an explanation. The categories follow MQM (Multidimensional Quality Metrics), a framework used to assess translation quality.

A minor error counts 1 point, a major error 5, a critical error 10. The penalty points for a translation are summed, divided by its word count, and normalized to 100 words. An entry’s score for a text is the average of the two judges; its published score is the average over the twelve texts.

In the short-text study, each ProsaBridge configuration read the text, prepared a translation plan, and translated using that plan. The direct configuration received the text and required output format. The published data identifies the model and settings used for each entry. The results describe the configurations tested at the time of each study.

How we checked the ranking

The follow-up comparisons support much of the ranking: the three translation services stay at the bottom, the order from the last AI entry down to Amazon Translate is confirmed, and 10 of the 22 neighbouring pairs keep their published order. The top entries are close together, so read the first few places as a group.

After the ranking was fixed, both judges compared each entry with the one ranked directly below it, text by text, without knowing which was which. A text only counts toward one side when both judges agree.

1GPT-6 Astra, With ProsaBridgevs.2GPT-5.6 Sol, With ProsaBridge
1 texts where both judges preferred the higher-ranked entry, 4 texts where both judges saw no difference, 7 texts where the judges disagreed
Too close to call
2GPT-5.6 Sol, With ProsaBridgevs.3GPT-6 Astra, Model on its own
2 texts where both judges saw no difference, 3 texts where both judges preferred the lower-ranked entry, 7 texts where the judges disagreed
Too close to call
3GPT-6 Astra, Model on its ownvs.4GPT-6 Astra, With ProsaBridge
1 texts where both judges preferred the higher-ranked entry, 3 texts where both judges saw no difference, 8 texts where the judges disagreed
Too close to call
4GPT-6 Astra, With ProsaBridgevs.5GPT-5.6 Sol, Model on its own
5 texts where both judges preferred the higher-ranked entry, 2 texts where both judges saw no difference, 5 texts where the judges disagreed
Order confirmed
5GPT-5.6 Sol, Model on its ownvs.6GPT-6 Astra, Model on its own
2 texts where both judges preferred the higher-ranked entry, 1 texts where both judges saw no difference, 2 texts where both judges preferred the lower-ranked entry, 7 texts where the judges disagreed
Too close to call
6GPT-6 Astra, Model on its ownvs.7Fable 5.1, Model on its own
4 texts where both judges preferred the higher-ranked entry, 8 texts where the judges disagreed
Too close to call
7Fable 5.1, Model on its ownvs.8Fable 5.1, With ProsaBridge
3 texts where both judges preferred the higher-ranked entry, 2 texts where both judges saw no difference, 4 texts where both judges preferred the lower-ranked entry, 3 texts where the judges disagreed
Judges preferred the lower-ranked entry
8Fable 5.1, With ProsaBridgevs.9Gemini 3.8 Flash, With ProsaBridge
2 texts where both judges preferred the higher-ranked entry, 2 texts where both judges saw no difference, 1 texts where both judges preferred the lower-ranked entry, 7 texts where the judges disagreed
Too close to call
9Gemini 3.8 Flash, With ProsaBridgevs.10GPT-5.6 Luna, With ProsaBridge
2 texts where both judges preferred the higher-ranked entry, 1 texts where both judges preferred the lower-ranked entry, 9 texts where the judges disagreed
Too close to call
10GPT-5.6 Luna, With ProsaBridgevs.11Opus 5, With ProsaBridge
1 texts where both judges preferred the higher-ranked entry, 1 texts where both judges preferred the lower-ranked entry, 10 texts where the judges disagreed
Too close to call
11Opus 5, With ProsaBridgevs.12Fable 5.1, Model on its own
6 texts where both judges preferred the higher-ranked entry, 1 texts where both judges saw no difference, 1 texts where both judges preferred the lower-ranked entry, 4 texts where the judges disagreed
Order confirmed
12Fable 5.1, Model on its ownvs.13Gemini 3.8 Flash, Model on its own
1 texts where both judges preferred the higher-ranked entry, 2 texts where both judges saw no difference, 7 texts where both judges preferred the lower-ranked entry, 2 texts where the judges disagreed
Judges preferred the lower-ranked entry
13Gemini 3.8 Flash, Model on its ownvs.14Fable 5.1, With ProsaBridge
5 texts where both judges preferred the higher-ranked entry, 2 texts where both judges preferred the lower-ranked entry, 5 texts where the judges disagreed
Order confirmed
14Fable 5.1, With ProsaBridgevs.15GPT-5.6 Luna, Model on its own
4 texts where both judges preferred the higher-ranked entry, 1 texts where both judges saw no difference, 7 texts where the judges disagreed
Too close to call
15GPT-5.6 Luna, Model on its ownvs.16Grok 4.6, With ProsaBridge
9 texts where both judges preferred the higher-ranked entry, 3 texts where the judges disagreed
Order confirmed
16Grok 4.6, With ProsaBridgevs.17GLM-5.3 Flash, With ProsaBridge
4 texts where both judges preferred the higher-ranked entry, 3 texts where both judges preferred the lower-ranked entry, 5 texts where the judges disagreed
Order confirmed
17GLM-5.3 Flash, With ProsaBridgevs.18GLM-5.3 Flash, Model on its own
6 texts where both judges preferred the higher-ranked entry, 3 texts where both judges preferred the lower-ranked entry, 3 texts where the judges disagreed
Order confirmed
18GLM-5.3 Flash, Model on its ownvs.19Opus 5, Model on its own
5 texts where both judges preferred the higher-ranked entry, 5 texts where both judges preferred the lower-ranked entry, 2 texts where the judges disagreed
Even
19Opus 5, Model on its ownvs.20Grok 4.6, Model on its own
8 texts where both judges preferred the higher-ranked entry, 2 texts where both judges preferred the lower-ranked entry, 2 texts where the judges disagreed
Order confirmed
20Grok 4.6, Model on its ownvs.21DeepL, Translation service
5 texts where both judges preferred the higher-ranked entry, 4 texts where both judges preferred the lower-ranked entry, 3 texts where the judges disagreed
Order confirmed
21DeepL, Translation servicevs.22Google, Translation service
11 texts where both judges preferred the higher-ranked entry, 1 texts where the judges disagreed
Order confirmed
22Google, Translation servicevs.23Amazon, Translation service
11 texts where both judges preferred the higher-ranked entry, 1 texts where the judges disagreed
Order confirmed

texts where both judges preferred the higher-ranked entry
texts where both judges saw no difference
texts where both judges preferred the lower-ranked entry
texts where the judges disagreed

Grok 4.6 on its own and DeepL are too close to separate. Scores that lie a few hundredths apart may swap places on another day; the download has every pair.

About the short-text comparison

One test round. The numbers are not averages over repeated rounds. Differences of a few hundredths of a point may reflect chance variation.
Two of the models that took part are also the judges, so we show each judge’s scores separately. The winner is first when scored by either judge on its own. In the winning entry, ProsaBridge translated with GPT-6 Astra, which was also one of the judges. Scored only by the judge from the other provider, Claude Fable 5.1, ProsaBridge still had about 90% fewer error points than DeepL.

What we keep private

The twelve texts. They were written for this benchmark and have never been published. Publishing them could allow future models to encounter them during training, weakening later comparisons. Their titles, text types, lengths, and translation challenges are listed below.
The instructions ProsaBridge gives the models, and the instructions the judges follow.
What the test rounds cost. The results table shows only multiples of the cheapest entry.
The judges’ notes and the translations themselves, because they would reveal the texts.

The twelve texts

Seven short stories and five nonfiction pieces, each written to be hard to translate in a particular way.

	Text	Kind	Words	What makes it hard
1	The Lease	Fiction, dialogue	431	A shift from formal to informal address, dialogue, dry humor
2	Gravel	Fiction, inner voice	485	Thoughts in free indirect speech, colloquial asides, unmarked questions
3	The Orchard Ledger	Fiction, lyrical	463	One long metaphor that must hold up from start to finish
4	Southpaw	Fiction, scene	449	Short, hard sentences next to one long one
5	The Committee	Fiction, satire	446	Long nested sentences, asides, irony that depends on word order
6	Moving the Needle	Fiction, workplace	462	Many idioms and one pun the whole text depends on
7	The Visitor	Fiction, inner voice	458	A deliberately unclear “she” who must remain unclear
8	Remarks at the Dedication of the Cedar Run Flood Memorial	Nonfiction, speech	441	American civic tone, repetition for effect, names of institutions
9	Allision of the Ferry Marguerite Bay at Harbor Point Terminal	Nonfiction, accident report	478	Nautical terms, passive voice, exact times and measurements
10	The Sleep of Swifts	Nonfiction, popular science	493	Lyrical science writing that moves between data and imagery
11	Expanding District Heating in Mid-Sized Cities	Nonfiction, policy brief	436	“Should”, “must”, and “may” that have to keep their exact strength; false friends; consistent terms
12	Midterm Evaluation of the Aldercreek Adult Literacy Program	Nonfiction, evaluation report	437	Paired terms that must stay distinct, careful attribution, measured recommendations

Models we removed before publishing

These models took part during development and are missing from the published results. Each removal is recorded with its evidence.

Claude Sonnet 5: Removed after seven texts because of weak translation quality: 3.18 and 3.22 error points per 100 words, behind every other AI model at the time.
Kimi K2 Thinking: Removed after it hit the provider’s output limit on three of the seven texts without delivering an answer.
Mistral Large 3: Removed after it added document formatting of its own and returned translated text where the process requires fixed reference codes.
Gemini 3.1 Pro and GPT-5.6 Terra: Withdrawn before the published test round: for what they cost, their scores were too weak. Cheaper models scored better, and better models cost about the same. Their earlier results are archived.

Download the data

The benchmark round: each entry on each text as scored by each judge, error counts by severity and category, relative compute costs, and comparisons between neighbouring entries in the ranking.

Download the 1.0 dataset

Test round: translationbench-run-fa35fa02-3190-4f82-b379-8d05b7dde5d7
Version: 1.0 (7.2)
Completed: September 12, 2026