# Changes from 1.42.0 to 1.43.0

- Changes from 1.42.0 to 1.43.0
  - Core
  - Filters
  - Connectors
  - TMs
  - Steps
  - Libs
  - Applications
  - General

## Core

- Issue #1104: HTML character references encoded in attribute values with disregard of the specified quotation mode.
- Fix textunitmerger and copy code logic, PR #590
- Added CodeMatchStrategy to guide code alignment
- Remove deprecated setCopySource parameter in segmenter code. The only code that should attempt to overwrite a target is the IFilterWriter or ISkeletonWriter.
- Add method StringUtil.generateIntId() to make a sane code id in a JVM independent manner
- Create new IResource.copy() method for annotations and properties
- New addMissingCodes parameter for TextUnitMerger (off by default).
- CodeComparatorOnData now ignores Code.outerData on compare. This missed many xliff1.2 sub codes as id strings were embedded in the Code.outerData.
- TextPartComparator now does not allow null == null to be equal as there is not enough info to compare. Count as mismatch.
- Various fixes for xliff2 using private customer data for testing, PR #600
- Add x- prefix for xliff1.2 output so the content is valid. xliff1.2 output uses ITS (similar to xliff1.2 filter support).
- Detach uber jars (okapi/deployment/okapi-framework-sdk and okapi-ui/swt/deployment/okapi-lib-ui) from main build
- Simplify copy properties and copy annotations into IResource.copy
- Code inline annotations now treated like regular inline codes in merging and compare (not skipped as before).
- Remove deprecated IFilterable and any related code.

## Filters

- Markdown Filter

  - Issue #1099: New lines insertion in lists with HTML tags clarified.
  - Issue #928: Inline HTML tag pairs are now correctly extracted as paired codes.
  - HTML Entities are now unescaped on import. A configuration option has been added to re-escape selected entities on export.

- MIF Filter

  - Issue #1052: Reference formats extraction clarified.

- OpenXML Filter

  - Issue #1051: Worksheet name, table column name and pivotal column name translations synchronised among all places in the target document.
  - Issue #1096: Booleans, numbers, dates and formula calculated values extracted as metadata.
  - Issue #1108: Dispersed translations contextualised.
  - Add openxml aggressivecleanup config and test files to integration tests

- XLIFF Filter

  - Issue #1111: SDL properties for split segments are now processed correctly despite the ID differences in the `space/_x0020_`special case.
  - Fix textunitmerger and copy code logic, PR #590
  - Remove xliff parameter option setBalancedCodes. This is now false as we should never balance as the code id’s may change and become desynchronize with the target codes
  - PR #600
  - Add x- prefix for xliff1.2 output so the content is valid. xliff1.2 output uses ITS (similar to xliff1.2 filter support).
  - Remove remaining copySource parameter defaults in xliff writers. Now default to copy source is false. Must set to true in API to get old behavior. Normally you should never copy source to target unless you have a specific use case (e.g., early preview of partially translated file)
  - IMPORTANT: `net.sf.okapi.common.filterwriter.XliffWriter` now copies source to target only if target is empty. We no longer check `srcHasText && !tc.hasText(false)`. But only if the source copy option is set (hopefully never will be as it is dangerous - see above)
    Was: `(tc == null || tc.isEmpty() || (srcHasText && !tc.hasText(false)))`
    Now: `else if (tc == null || tc.isEmpty())`

- XLIFF2 Filter

  - Fix textunitmerger and copy code logic, PR #590
  - xliff2 option `mergeAsParagraph` is deprecated (will not be supported in 1.44.00)
  - Remove xliff option `setBalancedCodes`. This is now false as we should never balance as the code id’s may change and desynchronize the target codes
  - Various fixes for xliff2 using private customer data for testing, PR #600
  - Fixes for embedded icu message inline codes across source and target. They matched before, which was incorrect. These now do not match and produce the correct output.
  - Add true mrk tag support by using GenericAnnotations. Primarily tested xliff2 term and comment mrk elements. xliff1.2 output uses ITS (similar to xliff1.2 filter support).
  - Fixes for icu message with different inline codes between source and target. These now do not match.
  - Add new xliff 2 extensions (.xliff2, .xlf2 etc.) to filter
  - Add new xliff2 test files - some mocks of private data

## Connectors

- Removed `SimpleTMConnector`

## TMs

- Removed `SimpleTM`

## Steps

- Removed `GenerateSimpleTmStep`
- Removed `SimpleTM2TMXStep`

## Libs

- xliff2 lib now uses xliff 2.1 schemas to validate

## Applications

- Tikal

  - Issue #1119: The “Help” buttons in the tikal filter editors don't work

- Rainbow

  - Removed the “ID-Based Alignment” to eliminate any dependency on H2.
    It's quite old and we shouldn't have step-like code in Rainbow anyway, it should be a separate step. It can't be used in pipelines, etc.
  - Generating the locales (showing in the UI) from ICU instead of using a hardcoded list.

## General

- Make the CI builds artifacts available for download
- Greatly improved continuous integration times
- Greatly reduced the number of leaked temp files
- Removed all dependencies on H2