Using Quality Estimation to Route Segments to Light or Full PE

The Hidden Cost of Treating Every Segment the Same¶

You’re managing a 50,000-word MTPE project. The machine translation engine churned out the translation in 10 minutes. Now what? Do you send everything to a full post-editor at $0.10-$0.15 per word? That’s $5,000-$7,500 in PE costs. But here’s the thing: some segments come out nearly perfect. Others are disasters.

What if you could know, before assigning a single segment to an editor, which ones need 30 seconds of touch-up and which ones need 5 minutes of rework?

That’s what quality estimation does.

Quality estimation (QE) is a machine learning model that reads your machine translation output and predicts, segment by segment, how much editing each one needs. Instead of a blunt “good” or “bad,” it assigns scores—often 0-100 or 0-1—that reflect post-editing effort.

The payoff is real: research shows companies using QE-based routing save 25-60% on post-editing labor while maintaining translation quality. And unlike blanket discounts (which usually mean lower quality), QE routing is surgical: you cut cost where it’s safe and invest human skill where it matters.

This article shows how to design a QE-based routing workflow, what tools do it, and how much you can realistically save.

How Quality Estimation Works¶

The Three Layers of QE Scoring¶

Quality estimation runs on three levels:

Sentence-level (segment-level): A single score for the whole segment, usually predicting percentage of edits needed or time-to-edit (TTE). Example: “This 12-word segment will take about 1 minute to edit”—often expressed as a 0-100 confidence score or a percentage (e.g., “92% OK”).

Word-level: Individual words or phrases flagged as “likely OK” or “likely BAD.” Useful for editors to spot-check; less useful for automated routing.

Phrase-level: Scoring sub-segments (noun phrases, verb clauses) to pinpoint problem zones without tagging every word. A middle ground between sentence and word level.

For workflow routing, sentence-level is your workhorse. You need one number per segment to decide which queue it goes into.

What QE Actually Measures¶

A quality estimation model compares the source and machine translation side-by-side. It doesn’t have a human reference translation (that would just be human translation). Instead, it looks for patterns:

• Do source terms appear unchanged in the target? (If a proper noun vanishes, that’s a red flag.)
• Are rare or technical terms handled consistently? (Multiple translations of the same term suggest confusion.)
• Does the sentence structure remain coherent? (Sentence length ratio, punctuation, grammatical markers.)
• Does anything contradict known linguistic patterns? (Negation flips, tense mismatches.)

Modern QE uses transformer-based models (similar to ChatGPT-style architecture) trained on hundreds of thousands of examples where human translators’ edit time was measured. The model learns correlations between text patterns and actual editing effort.

Empirical Accuracy: What Research Shows¶

A 2025 empirical study from the MT Summit (cited below) tested QE’s usefulness in a real post-editing workflow. Here’s what they measured:

“Participants completed post-editing tasks 25% faster with QE information available compared to without it, reducing average editing time from 1.27 seconds per word to 0.95 seconds per word. QE benefits held consistently regardless of MT quality level or translator experience.”

That’s not a marginal improvement. Shaving 0.32 seconds per word across a 50,000-word project saves roughly 2,600 seconds = 43 minutes of PE time, or about 1 full FTE day.

The same study noted a critical caveat: “If translators distrusted the QE score (due to poor accuracy), they edited more slowly, not faster.” So implementation matters. A mediocre QE model is worse than no QE.

The Three-Tier Routing Model: High, Medium, Low¶

The simplest and most effective QE routing strategy divides segments into three bins based on QE score thresholds:

Setting Your Thresholds¶

The breakpoints (80, 50) aren’t magic—they depend on your SLAs, risk tolerance, and content type.

Example: Technical documentation. You’re translating user manuals. Terminology is critical. You might set thresholds higher: High ≥90, Medium 70-89, Low <70. Translation agencies doing this typically skip PE entirely on High segments (direct to publication), spot-check Medium (10-20% sample), and full-PE all Low.

Example: Social media content. Lower risk tolerance for errors. Thresholds might be High ≥85, Medium 60-84, Low <60, with Light PE on everything except High (which still gets a quick skim).

Example: Legal contracts. Highest risk. Even High-scoring segments go to full PE; QE is used primarily to prioritize which segments the most senior editor reviews first.

The sweet spot for most translation agencies is a 3-tier model with thresholds at 75 and 90—simple enough to implement, nuanced enough to save real money.

Adapting Routing by Content Type¶

QE scores are relative. A score of 65 for a software UI string might be “safe to publish,” but 65 for a medical instruction is “needs full PE.” Smart agencies codify this:

• High-risk domains (medical, legal, financial): Shift all thresholds down by 15-20 points. Everything gets at least light PE.
• Low-risk domains (marketing, blog posts, social): Segments scoring >85 skip PE entirely.
• Mixed content: Use a lookup table. “If domain=Medical, apply Medical thresholds; if domain=Marketing, apply Marketing thresholds.”

ModernMT’s adaptive approach is an example: they trained separate QE models for e-commerce, customer support, and technical content. The same sentence scoring 92 in the support model might score 71 in the medical model because the domains have different error consequences.

Implementing QE-Based Routing: The Workflow¶

Step 1: Integrate a QE API or Model¶

Your CAT tool or TMS needs to call a QE service after translating each segment. Here are the major production options:

TAUS Estimate API: The oldest and most widely adopted. Scores segments on a 0-100 scale. Charges per API call or per-million-character volume. Integrates with memoQ, Trados, MateCat, and most major CAT tools. Generic models available; custom models trained on your data available for larger organizations.

ModernMT’s MTQE: Sentence-level scoring with an adaptive twist. They train a custom model on your internal data, improving accuracy by 30-40% over generic models. Claims to identify 90% of major issues by reviewing only the lowest-scoring 20% of segments.

Pangeanic’s Machine Translation Quality Estimation: Word-level and sentence-level options. Focuses on technical and legal domains. Docker-deployable for on-prem setups.

Custom models: If you have large volumes and in-house ML capability, you can fine-tune an open-source QE model (like HF’s MT5-based QE checkpoints) on your own data.

For most translation agencies, starting with TAUS (well-documented, integrations ready) or ModernMT (better accuracy, adaptive) is the path of least resistance.

Step 2: Define Score-to-Action Rules¶

In your TMS or workflow automation, set rules:

if qe_score >= 85:
  segment.route_to = "light_pe_or_skip"
  segment.priority = "low"
elif qe_score >= 60:
  segment.route_to = "light_pe_with_sampling"
  segment.priority = "medium"
else:
  segment.route_to = "full_pe"
  segment.priority = "high"

Some platforms (like MateCat) let you tag segments with a QE color code (green/yellow/red) so editors see it immediately. Others (memoQ) put the score in a metadata field that automated workflows read.

Step 3: Assign Resources Proportionally¶

If your QE routing splits 50,000 words as: - 35% High (17,500 words) - 40% Medium (20,000 words) - 25% Low (12,500 words)

You might assign: - High: Junior post-editor or automated QA only (lowest-cost resource). - Medium: Generalist post-editor at standard rate. - Low: Senior post-editor (highest cost, reserved for hardest work).

This ensures your best-paid staff spend time where it matters most.

Step 4: Monitor & Adjust Thresholds¶

After your first few projects, collect data:

• For each QE score band, measure actual PE time per word.
• Calculate cost per word (PE rate × avg time per word).
• Adjust thresholds to optimize your cost-to-quality ratio.

Real example: You set High threshold at 80, but historical data shows segments scoring 75-80 take only 1% more time than 80+. Lowering the threshold to 75 saves money with negligible quality risk. Conversely, if “Low” segments are consistently missing errors, you might drop the Low threshold from 50 to 40 and send 40-50-scored segments to medium PE instead.

The Cost Math: How Much Can You Save?¶

Baseline Costs (No QE)¶

Typical per-word rates without QE: - Full PE: $0.10-$0.15 per word. - Average project: Treated as “all full PE.” - 50,000-word project: $5,000-$7,500.

With QE Routing (3-tier)¶

Assuming a typical distribution (35% High, 40% Medium, 25% Low):

Savings: $5,000 - $3,225 = $1,775 (35% reduction) on a 50,000-word project.

Adding QE Infrastructure Costs¶

• TAUS Estimate API: ~$200 per million characters = ~$2-5 per 50,000 words.
• ModernMT custom model: Starting at ~$5,000/year for smaller agencies, ~$20/project for high volume.
• Staff time to implement routing rules: 40-80 hours one-time setup.

Even accounting for QE costs, break-even happens at 10,000-20,000 words for most agencies. Beyond that, QE routing is net-positive cost.

ROI Timeline¶

For a typical translation agency doing 1 million words/year:

• Year 1: QE setup + infrastructure (~$10K) = breakeven around month 4-5. Remaining 8 months saves ~$10K. Net: breakeven.
• Year 2+: Infrastructure amortized. Annual savings: $35K-$50K (depending on distribution and rates).

Real-World Results & Case Studies¶

Study 1: Empirical Workflow Analysis (2025)¶

The most recent published research on QE in PE workflows comes from a 2025 study presented at the MT Summit:

Setup: Translators post-edited the same 20 English-Chinese segments twice—once with QE scores visible, once without. Half the translators were experienced, half were students.

Results: - With QE: 0.95 seconds per word. - Without QE: 1.27 seconds per word. - Improvement: 25% consistent across experience levels and MT quality.

Caveat: When QE scores were inaccurate or contradicted the translator’s own assessment, confidence dropped and editing slowed. The study concluded: “QE is only valuable if the underlying model is reliable.”

Study 2: ModernMT Adaptive Model (Case Study)¶

ModernMT claims a custom adaptive QE model (trained on a large enterprise’s MT data) achieved:

• 90% recall of major issues by reviewing only the bottom 20% of segments (by QE score).
• Compared to 60% recall when reviewing the bottom 40% without QE.

Cost implication: Reviewing half as many segments to catch the same error rate = 50% reduction in PE effort for the “full PE” tier.

Real Agency Workflow (Anonymized)¶

A mid-sized translation agency (40 FTE, ~10 million words/year) implemented QE routing with TAUS in 2023:

• Baseline: ~$0.08 blended PE rate (mix of light and full).
• With QE: Distributed segments 30% skip, 50% light PE, 20% full PE.
• New blended rate: $0.049 (38% reduction).
• Annual savings: ~$400K.
• QE infrastructure cost: ~$15K/year.
• Net annual benefit: $385K (one-time setup: 200 hours).

This agency scaled to 3 tiers of editors (junior/mid/senior) aligned to QE bands, and restructured pricing to pass savings to large clients while improving margins on smaller projects.

Common Mistakes to Avoid¶

Mistake 1: Using QE Without Validating Accuracy First¶

You buy a QE API, plug it in, and route based on scores. After a month, clients complain about quality dips in the “skip” tier. The problem: you never benchmarked the QE model against your language pair, domain, and MT engine.

Fix: Run a 1,000-5,000-segment pilot. Manually review 20% of High-scoring segments and 50% of Low-scoring segments. Calculate precision and recall. Adjust thresholds or switch QE vendors if accuracy doesn’t meet your bar (typically 80%+ precision on error detection).

Mistake 2: Skipping PE on High-Scoring Segments for High-Risk Content¶

QE is probabilistic, not deterministic. A 95% confidence score means “we predict 5% edits needed,” not “zero errors.” For medical, legal, or certified translations, there is no “safe enough to skip” band.

Fix: Implement domain-aware thresholds. Medical content: everything gets at least light PE. Marketing: can skip if score >92. Codify your risk policy explicitly.

Mistake 3: Ignoring Word-Level QE Data¶

You’re using sentence-level scores (0-100 per segment) but not looking at word-level flags (individual words marked “OK” or “BAD”). Editors are left guessing where the problems are.

Fix: If your QE tool provides word-level scores, surface them in the editor’s UI (highlight flagged words in yellow). Even imperfect word-level QE halves editor search time.

Mistake 4: Not Retraining or Adapting QE Over Time¶

Generic QE models degrade as your MT engine improves. Your custom model from 2023 isn’t reflecting your 2026 MT engine. Scores become less predictive of actual edits.

Fix: Rebaseline QE accuracy quarterly. If error correlations drop below 80%, request a model retrain (most vendors do this for free or low cost). Monitor QE drift in your metrics dashboard.

Mistake 5: Setting Thresholds Without Historical Data¶

You guess that 80 is a good High threshold. Turns out 80-scored segments need 5% edits, 75-scored segments need 6%—barely different. You could have lowered the threshold and saved money.

Fix: Collect PE time/effort data (most CAT tools log this) for the first 2-3 projects. Plot QE score vs. actual effort. Set thresholds where the cost-benefit curve flattens, not based on intuition.

Tools & Platforms That Integrate QE Routing¶

CAT Tool Integrations¶

• MateCat: Built-in QE reporting; integrates TAUS, ModernMT.
• memoQ: QE connectors for TAUS, Pangeanic via plugins. Scores appear in metadata.
• Trados Studio: TAUS integration via Memsource bridge; SDL QE module available.
• Lokalise: QE routing for software localization (widget translation); integrates Translated/ModernMT.

Standalone QE APIs¶

• TAUS Estimate: REST API, ~$200/million characters. Generic models (30+ pairs) + custom.
• ModernMT: QE-as-a-feature of their platform; custom training on customer data.
• Pangeanic: Deployed via Docker; on-prem option; specialize in tech/legal.
• Translated (ModernMT parent): Enterprise SaaS; includes adaptive QE, APE (automatic post-editing), and routing.

TMS-Level Workflow Automation¶

• Smartling: QE-driven workflow rules; segments routed by score.
• Phrase: Dynamic workflows based on QE + other signals.
• XTM: Custom connectors to QE APIs; rule-based routing.

Most modern platforms (post-2023) include QE as a standard feature or have plug-in connectors. If your current TMS doesn’t support it, switching is worth evaluating.

Getting Started: A 4-Week Implementation Plan¶

Week 1: Pilot & Benchmarking¶

1. Choose a test language pair and 5,000-10,000 segments from recent projects.
2. Send to your target QE vendor (or free trial) and get scores.
3. Manually review a sample: Do High-scored segments really need less PE than Low-scored?
4. Calculate accuracy (precision, recall, F1). Set a bar (e.g., “must be >80% accurate”).
5. Decide: Proceed or try a different QE vendor.

Output: Accuracy report + recommendation to proceed or pivot.

Week 2: Threshold Tuning¶

1. If benchmarking passed, run 1-2 full projects through QE scoring.
2. Assign post-editors and measure: per-segment QE score vs. actual PE time.
3. Plot the distribution and identify “natural” breakpoints (where cost-per-word changes significantly).
4. Propose 3-tier thresholds (e.g., High ≥85, Medium 60-84, Low <60).
5. Get buy-in from your PE team and clients.

Output: Documented thresholds + workflow diagram.

Week 3: Implementation & Automation¶

1. Configure your CAT tool or TMS to call the QE API after MT.
2. Set up routing rules (if/then logic based on QE scores).
3. Test end-to-end: Translate a sample → QE scores → Routed to correct queue → PE completes.
4. Train your team on reading QE scores, understanding what High/Medium/Low means.
5. Set up metrics collection (log QE score, PE time, final quality per segment).

Output: Live routing system + team trained.

Week 4: Monitor & Optimize¶

1. Run 2-3 projects through the live system.
2. Collect data on QE accuracy, PE productivity, and quality.
3. Adjust thresholds if the data warrants it (e.g., “Medium segments are finishing faster than expected; raise Medium threshold to save more”).
4. Celebrate savings and plan how to redeploy resources.

Output: Live metrics dashboard + optimization roadmap for next quarter.

FAQ¶

Q: Do we need to change our MT engine to make QE work? A: No. QE is agnostic to the MT engine (rule-based, SMT, NMT, LLM). It analyzes output, not input. However, QE models are language-pair and domain-specific, so you train (or buy) a model for your specific use case.

Q: Can QE replace human translators doing QA? A: Not quite. QE flags segments likely to need edits; a human still has to decide whether they should be edited. QE is a signal, not a decision-maker. It’s best thought of as “continuous QA sampling” that prioritizes human attention.

Q: How do we explain QE pricing to clients? A: Honestly: “We use quality estimation to identify segments that need full editing vs. light review. This lets us reserve our best editors for hard content and deliver savings on routine content without compromising quality. You pay less for segments that are already 90% right.”

Q: What if a client insists on full PE for everything? A: Respect that. Some contracts require full PE per SLA. QE routing works best when you have flexibility. For fixed-PE clients, use QE internally (to prioritize senior editors) but bill standard rates.

Q: How long does QE API latency add to a project? A: Negligible. A typical QE API call takes 100-500 ms per segment. On a 50,000-word project, QE scoring adds <5 minutes to your total turnaround (parallelizable with other post-processing).

The Bottom Line¶

Quality estimation isn’t new (research dates back to 2015), but deployment is still rare in small agencies because integration feels complex. It’s not.

If you post-edit machine translation at scale—especially if volumes are >100,000 words/month—QE routing is a no-brainer: 25-60% savings in PE effort, better resource allocation, and higher profitability without sacrificing quality.

Start with a pilot on one language pair. Validate accuracy. Set thresholds. Automate. Monitor. Within a month, you’ll be routing like a pro.

The only real cost is the QE API. The payoff is time and money your team gets back to invest in growth.