(Go: >> BACK << -|- >> HOME <<)

Page MenuHomePhabricator

CX2: Failure in MT eliminates the paragraph and breaks alignment
Closed, ResolvedPublic

Description

Testing CX2 in production by translating "Atazanavir" from English to Spanish (trying to reproduce T195665), I found a paragraph where MT fails, and as a result gets eliminated from the translation document, causing the alignment of sections to break.

This is illustrated in this video with more detail, and in a short gif below.

Jun-01-2018 08-55-11.gif (322×640 px, 3 MB)

As part of this ticket, it would be good to:

  • Make MT failures less severe by not removing the paragraph. We can show an empty paragraph as if the user had selected "Don't use machine translation", allowing the user to switch to a different option using the MT card.
  • Determine why the paragraph consistently fails to be translated automatically.

Event Timeline

Pginer-WMF triaged this task as Medium priority.Jun 1 2018, 7:05 AM
Pginer-WMF moved this task from Needs Triage to CX2 on the ContentTranslation board.
Pginer-WMF moved this task from Backlog to Priority backlog on the Language-2018-Apr-June board.
Pginer-WMF raised the priority of this task from Medium to High.Jun 6 2018, 10:42 AM

This issue happened also when translating 1936 Cansiglio earthquake from English to Italian in production. Note that in this case there is no MT service available, so the failure happens when the source version of the (5th) paragraph is added.

load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3860 jQuery.Deferred exception: Cannot read property 'getRange' of undefined TypeError: Cannot read property 'getRange' of undefined
    at https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.InternalList.js?2eff4:426:83
    at Array.sort (native)
    at VeDmInternalList.ve.dm.InternalList.sortGroupIndexes (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.InternalList.js?2eff4:425:19)
    at VeDmInternalList.ve.dm.InternalList.onTransact (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.InternalList.js?2eff4:377:9)
    at VeDmDocument.oo.EventEmitter.emit (https://dev.translatewiki.net/w/resources/lib/oojs/oojs.jquery.js?b1631:824:12)
    at VeDmDocument.ve.dm.Document.commit (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.Document.js?11616:330:7)
    at VeDmSurface.ve.dm.Surface.changeInternal (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.Surface.js?95c80:880:25)
    at VeDmSurface.ve.dm.Surface.change (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.Surface.js?95c80:844:7)
    at VeDmSurfaceFragment.ve.dm.SurfaceFragment.change (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.SurfaceFragment.js?7dada:129:15)
    at VeDmSurfaceFragment.ve.dm.SurfaceFragment.removeContent (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.SurfaceFragment.js?7dada:947:8) undefined
jQuery.Deferred.exceptionHook @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3860
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3655
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3681
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3583
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3671
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3681
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3583
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
done @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:9272
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:9514
load (async)
send @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:9533
ajax @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:9173
jQuery.ajax @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:10483
mw.cx.MachineTranslationService.fetchTranslation @ mw.cx.MachineTranslationService.js?5bda5:185
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3583
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
add @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3376
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3709
jQuery.Deferred @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:10866
then @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3694
mw.cx.MachineTranslationService.getCXServerToken @ mw.cx.MachineTranslationService.js?5bda5:143
mw.cx.MachineTranslationService.translate @ mw.cx.MachineTranslationService.js?5bda5:42
ve.init.mw.CXTarget.translateSection @ ve.init.mw.CXTarget.js?9d852:681
ve.init.mw.CXTarget.changeContentSource @ ve.init.mw.CXTarget.js?9d852:726
(anonymous) @ ve.init.mw.CXTarget.js?9d852:561
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3583
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3455
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
fireWith @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3447
mightThrow @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3642
process @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651
setTimeout (async)
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3689
fire @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3317
add @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3376
(anonymous) @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3709
jQuery.Deferred @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:10866
then @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3694
mw.cx.MachineTranslationService.getProviders @ mw.cx.MachineTranslationService.js?5bda5:52
mw.cx.MachineTranslationManager.getAvailableProviders @ mw.cx.MachineTranslationManager.js?0679e:73
mw.cx.MachineTranslationManager.getPreferredProvider @ mw.cx.MachineTranslationManager.js?0679e:55
ve.init.mw.CXTarget.onDocumentActivatePlaceholder @ ve.init.mw.CXTarget.js?9d852:560
oo.EventEmitter.emit @ oojs.jquery.js?b1631:824
ve.ce.CXPlaceholderNode.executeCommand @ ve.ce.CXPlaceholderNode.js?47b0f:61
ve.ce.CXPlaceholderNode.onFocusableMouseDown @ ve.ce.CXPlaceholderNode.js?47b0f:55
dispatch @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:5206
elemData.handle @ load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:5014
11:30:13.498 load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3860 jQuery.Deferred exception: Cannot read property 'shallowCloneFromRange' of null TypeError: Cannot read property 'shallowCloneFromRange' of null
    at VeDmConverter.ve.dm.Converter.getDomFromNode (https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.Converter.js?e7c96:1254:21)
    at VeInitMwCXTarget.ve.init.mw.CXTarget.changeContentSource (https://dev.translatewiki.net/w/extensions/ContentTranslation/modules/ve-cx/init/ve.init.mw.CXTarget.js?9d852:711:25)
    at VeInitMwCXTarget.<anonymous> (https://dev.translatewiki.net/w/extensions/ContentTranslation/modules/ve-cx/init/ve.init.mw.CXTarget.js?9d852:565:16)
    at mightThrow (https://dev.translatewiki.net/w/load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3583:29)
    at process (https://dev.translatewiki.net/w/load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3651:12) undefined

I am having troubles finding the root cause, but my current suspicion it is about the shared references in the first three sections.

From cxserver logs:

{"name":"cxserver","hostname":"jadekukka","pid":29466,"level":20,"levelPath":"debug","msg":"Could not build a source doc. May be plain text? Re-attempting with <div> wrapper","time":"2018-06-19T06:37:37.670Z","v":0}
{"name":"cxserver","hostname":"jadekukka","pid":29466,"level":20,"levelPath":"debug","msg":"Not-adapting a reference node without data-mw.body.html: cite_ref-AHFS2016_2-6","time":"2018-06-19T06:37:37.671Z","v":0}
{"name":"cxserver","hostname":"jadekukka","pid":29466,"level":20,"levelPath":"debug","msg":"Not-adapting a reference node without data-mw.body.html: cite_ref-AHFS2016_2-7","time":"2018-06-19T06:37:37.671Z","v":0}
{"name":"cxserver","hostname":"jadekukka","pid":29466,"level":20,"levelPath":"debug","msg":"Not-adapting a reference node without data-mw.body.html: cite_ref-AHFS2016_2-8","time":"2018-06-19T06:37:37.671Z","v":0}

So it's possibly a cxserver issue.

load.php?debug=true&lang=fi&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0ubiy92:3860 jQuery.Deferred exception: Cannot read property 'getRange' of undefined TypeError: Cannot read property 'getRange' of undefined
    at https://dev.translatewiki.net/w/extensions/VisualEditor/lib/ve/src/dm/ve.dm.InternalList.js?2eff4:426:83

This is T191011: ve.dm.InternalList#sortGroupIndexes throws javascript error

I assume this means we need to wait for that to get fixed.

I checked if there is any issue with MT in this case. I did not see any issue. The warning about references are expected if there are repeated references, they are harmless. The error message shown in screen is slightly misleading since it is triggered by the VE internal error, not directly connected to MT.

I checked if there is any issue with MT in this case. I did not see any issue. The warning about references are expected if there are repeated references, they are harmless. The error message shown in screen is slightly misleading since it is triggered by the VE internal error, not directly connected to MT.

Maybe we can use this situation (an internal component failing) to improve how to better deal with such cases (in addition to solve the specific issue for this case). Is there anything that can be done in that regard?

A user reported an issue that may be this same one (related also to T191011), when translating the "Steiner inellipse" article from English to Spanish.

Trying to reproduce the issue, I observed how the MT error shows and the paragraph gets deleted when reaching to the math formula (note in the screenshot below how the "+add paragraph" in the translation causes a different paragraph to highlight because the removed paragraphs got misaligned). The issue happens both when using an MT service and when using "copy from source".

Screen Shot 2018-06-29 at 13.41.21.png (658×860 px, 93 KB)

Vvjjkkii renamed this task from CX2: Failure in MT eliminates the paragraph and breaks alignment to 3ubaaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii removed Nikerabbit as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Arrbee renamed this task from 3ubaaaaaaa to CX2: Failure in MT eliminates the paragraph and breaks alignment.Jul 2 2018, 1:35 PM
Arrbee assigned this task to Nikerabbit.
Arrbee updated the task description. (Show Details)

I assume this means we need to wait for that to get fixed.

That (T191011), got marked as resolved. So I guess we can check if the issue described in the current ticket still happens and act accordingly (resolve the current ticket, move it out of the "blocked" column, or describe any new blocker if any).

Etonkovidova subscribed.

I checked all three articles mentioned in the ticket - "Atazanavir" (en to es), "1936 Cansiglio earthquake" (en to italiano was blocked by ongoing translaton) , and "Steiner inellipse" (en to es) - the alignment of paragraphs is not changed if one of machine translation (Yandex) options fails.