# Changes from M9 to M10

## Applications

- Rainbow

  - Changed the help system to use the wiki (the plan is to have a snapshot of the wiki also available as local help).
  - Added the “Tools” > “Plugins Manager” command.
  - Added the “Plugins Location” option in the “User Preferences” dialog box.
  - Fixed locale variables (`${TrgLoc}`, etc.) to get a consistant casing regardless of the casing of the value in the Languages and Encodings tab.
  - Fixed corrupted path when dropping file on output using `${TrgLoc}`.
  - Added the Code Finder Editor (in Tools menu) to edit code finder rules for filters using them but not having any UI to define them.
  - The root of a list is now automatically adjusted to the longest root possible when a document above the current root is added, the documents already listed have their relative path adjusted as well.
  - Create Translation package:
    - Refactored heavily the leveraging mechanism and the output (e.g. added `<alt-trans>` output in Generic XLIFF)
  - Re-organized the Utilities menu with sub-menus.
  - Added the “XML Analysis”, the “XML Characters Fixing”, and the “XML Validation” pre-defined pipelines.
  - Changed command-line processing to handle pre-defined pipelines in addition to utilities (and pipeline).

- CheckMate

  - Added option in the Term tab to match strings only when enclosed in codes.
  - Added the “Reset to Defaults” command in the Configuration dialog.
  - Added the “Accept all next documents with their defaults” option when adding documents to the session.
  - Implemented tab-delimited format for the report, and added choice between HTML or tab-delimited in Configuration dialog.
  - Added support for term list in CSV format.
  - Added warning when the target from RTF contains an hidden part.

- OmegaT plugin

  - A new component, the Okapi Filters for OmegaT plugin is now part of the distributions. It allows to use some of the Okapi filters directly in OmegaT. Currently the filters for the following formats are included: JSON, TTX, Qt-TS and IDML.

- Tikal

  - Added the `-xm` command to extract files to Moses InlineText format.
  - Added the `-lm` command to leverage files from their Moses InlineText corresponding files.
  - Updated handling of default for source and target languages (allows autodetection before using system defaults).
  - Added the `-s` command to segment files.

## Steps

- Added the RTF Conversion Step

  - and replaced the “RTF Conversion” utility by a pre-defined pipeline.

- Added the BOM Conversion Step

  - and replaced the “byte-Order-Mark Conversion” utility by a pre-defined pipeline.

- Added the Encoding Conversion Step

  - and replaced the “Encoding Conversion” utility by a pre-defined pipeline.

- Added the Create Target Step

  - allows you to create target from the source.

- Added the Scoping Report Step

  - generates word-count reports

- Added the XML Characters Fixing Step

  - replaces invalid XML characters by markers

- Added the XLIFF Joiner Step

  - allows to re-join together XLIFF documents created by the XLIFF Splitter step.

- Added the Moses InlineText Extraction Step

  - to extract entries to Moses text files

- Added the Moses InlineText Leveraging Step

  - to leverage translation from a Moses text file

- Added the XML Analysis Step

  - to generate the list of elements in a set of XML document and guess their localization-related properties), this also closes issue #153.

- Leveraging Step

  - Added the threshold option for the match to copy into the target.

- Segmentation Step

  - Added option to overwrite type of output segmentation in files such as XLIFF.
  - Added option “Overwrite existing segmentation”.

- Sentence Alignment Step

  - Improved handling of whitespaces.
  - Updated default SRX rules for the step.

- Search and Replace Step

  - Added option to replace on source and or target when using filter events.
  - Added support for `\uHHHH` notation on all modes
  - IMPORTANT: The behavior of this step has changed when no target is in the text unit (in filter event mode):
    - Before M10: A copy of the source was automatically copied as the target and the sreach and replaced performed on that text.
    - Starting with M10: text unit with empty target are simply not processed. You must now have a Create Target step before this step to copy the source into the target.

- XLIFF Splitter Step

  - Improved support for large documents. This fixes issues #146 and #147.

## Connectors

- Microsoft MT Connector

  - Updated the connector to use the v2 HTTP API instead of the v1 SOAP one (which is no longer accessible)

## Filters

- Added the Moses Text Filter

  - for processing Moses MT system data files.

- XML Stream Filter

  - Fixed issue #145 about PI being moved.
  - Fixed issue #150 about inline codes being incorrectly escaped.
  - Changed default so apostrophes are not escaped in output.
  - Implemented default extraction for `CDATA` sections when no sub-filter is defined.
  - Fixed issue null point with empty `CDATA` sections.
  - Fixed issue of conditions not being applied on `CDATA` sections.

- HTML Filter

  - Fixed escaping of inline codes detected using the codeFinder option.
  - Fixed issue #90 (`CDATA` section not extracted)

- IDML Filter

  - Refactored the filter completely. The filter is still beta, but has been improved significantly.

- TS Filter

  - Fixed issue with null character output on string with inline codes when using TS encoder.
  - Finished implementation of the `<byte>` element as inline code.

- Table Filter

  - Improved handling of qualifiers.
  - Added pre-defined filter configuration for Haiku catkeys file format.

- TTX Filter

  - Improved mapping of leveraged entries with a score to Okapi annotations.
  - Fixed filter-writer for pre-segmented RTF output.
  - Fixed handling of split opening tags (TTX tags with Type=“start” and leftEdge=“split”).
  - Improved handling of isolated `</df>` tags in un-segmented content.
  - Changed extraction to extract non-segmented parts of text entries (this closes issue #151 and #157)

- Trados-Tagged RTF Filter

  - Improved parsing for fldinst and xmlopen fields: their content is not included in the extracted text.
  - Added warning when part of the target segment is hidden.

- XLIFF Filter

  - Added the option to override the original target language.
  - Improved mapping of alt-trans attributes to Okapi annotations.
  - Implemented options to add possible new `<alt-trans>` elements in output files (in addition to the one in the original document), and to include or not extension information in the new `<alt-trans>`.
  - Added the output option: “Segment only if the entry is segmented and regardless how the input was”.

## Libraries

- Completely refactored the `IQuery`, `QueryResult`, `QueryManager` and related classes.
- Removed the `ScoreAnnotation` and `ScoreInfo` classes that were deprecated in M7. Use `AltTranslationsAnnotation` instead.
- Added capability to write extra data in header and phase-name in trans-unit for `XLIFFWriter` class.
- Upgraded the SWT libraries to 3.6.1.
- Fixed the handling of literal `&#13;` and `&#xD;` in most XML-based filters and encoders, so `\r` is not stripped out or converted to `\n`.
- Added `setBoolean`, `setString` and `setInteger` by name in `IParameters`.
- Upgraded ICU4J library from 4.0.1 to 4.6.
- Fixed issue with not re-balancing codes after inser/append in `TextFragment` (this solve several `bx`/`ex`-related issues)
- Changed `transferCodes()` method used in merging in Rainbow, Tikal and xliffkit, to fix merging issue with `<g>`-type XLIFF codes.