# Changes from M4 to M5

## Libraries

- Changed minimum requirement to Java 1.6 instead of Java 1.5.
- Removed distribution for Mac Carbon, added distribution for Mac Cocoa-64-bit.
- Updated to Lucene 3.0.0
- Refactored Pensieve TM engine, added new API.

## Applications

- Rainbow

  - Added the duration of the process in the log.
  - Updated the UI of the Pipeline Edit / Execute facility to make the panels of each step accessible without clicking.
  - Replaced the utility “Generate SimpleTM Dabase” by the pre-defined pipeline “Import Into Pensieve TM” (the previous utility's functionality is still available using a custom pipeline).
  - Replaced the utility “Export SimpleTM Database” by the pre-defined pipeline “Convert File Format” (the previous utility's functionality is still available using a custom pipeline).
  - Fixed issue with Text Rewriting and empty `<target>` for XLIFF input.
  - Replaced the utility “Translation Comparison” by the pre-defined pipeline “Translation Comparison”.
  - Added the pre-defined pipeline “Create Translations in Batch Mode”
  - Replaced the utility “XSL Transformation” by the pre-defined pipeline “XSL Transformation”.
  - Replaced the utility “Used Characters Listing” by the pre-defined pipeline “Used Characters Listing”.

- Ratel

  - Fixed selection bug in UI.
  - Updated the default segmentation rules.

## Steps

- Added Batch Translation Step

  - Tested with ProMT and Apertium.

- Added Codes Removal Step

- Added Leveraging Step

- Completed initial Tokenization and Word-Count Steps

- Added the Sentence Alignment Step

- Translation resources

  - Fixed issue with score > 100 in Pensieve TM.
  - Added NCR support for Apertium connector.

## Filters

- In the Properties Filter: Added pre-defined configuration for Skype's `.lang` format.
- In the RTF parser:
  - Fixed the issue with `\'HHc` being read as `\'HH\'HH` in some cases.
  - Added support for additional DBCS encodings.
- Added TTX Filter for Trados TagEditor documents (Beta).
- In the HTML Filter: Added pre-defined configuration for well-formed files, providing groups and extra meta-data.
- In the XML Filter: Changed the ITS extension `idPointer` to `idValue` and modified its behavior to allow ID values to be generated from the expression, not just from the content pointed by the expression. The values are backward compatible, but existing parameters file will have to rename any reference to `idPointer` by `idValue`.
- Added the Vignette Filter for Vignette export XML documents (Alpha)
- Added the Pensieve Filter for reading and writing Pensieve translation memories.