X-Ways update release with many notable improvements, v20.0

What’s new in v20.0?
(please note that most changes affect X-Ways Forensics only)

File System/Disk Support

• UFS support has been revised. Significantly more UFS variants are now understood.
• APFS: Supports new Catalog ID structure as created by Mac OS Catalina.
• Technical Details Report/evidence object properties now show details of MacOS X Installations on HFS+ or APFS volumes: Exact OS X version, timezone, the system’s network and display names.
• Support for much more deeply nested subdirectories in XFS volumes.
• Supports Ext4 volumes with version 2 of sparse superblocks.
• Slightly more complete output of Ext* file system timestamps.
• Ability to choose which copy of a FAT12/FAT16/FAT32 file allocation table to work with, in Options | Volume Snapshot. This can be either a user-designated copy or the one that is defined as active in the boot sector (in case of FAT32). If neither the user selects a copy nor the boot sector defines a single copy as active, the first copy will be used, labelled as “FAT 1”, like in earlier versions. The copy that was selected at the time when the volume snapshot was taken will be used for the whole lifetime of that volume snapshot, even if the settings are changed. It is displayed in the Info Pane. The Technical Details Report now informs which copy or copies are considered active in the file system.
• Identifies unpartitioned physical disks or disk images as such in some rare cases where it previously didn’t.
• General option to open volumes including the slack that doesn’t add to another cluster just like when opening an entire partition. The data in that area, aside from a potential NTFS backup boot sector, does not belong to that volume logically and was stored there before the volume was created. It is not needed to parse the file system or to mount the volume (though some tools may output an error message if it’s not included). Including such data in a volume image can be an IT security leak if only the regularly accessible part of the volume had been sanitized before usage.
• Identifies some new bus types of currently attached storage devices.
• Active sector superimposition is now remembered in an evidence object and automatically re-activated when the evidence object is opened next time, and you will be reminded of that.
• Generally improved handling of incomplete/corrupted .e01 evidence files, similar to storage media with unreadable areas (bad sectors). NTFS: A limited listing of system files is now presented based on $MFTMirr if in an such an incomplete image $MFT is not included, but $MFTMirr is.
• Ability to abort the potentially time-consuming preparation of a cluster allocation map for huge volumes and still proceed with taking the actual volume snapshot if desired (without reverse cluster allocation information).

Picture Support

• New version of the internal picture viewing library.
• WEBP pictures are now supported in Preview, Gallery, and for the View command.
• Ability to view pictures in some variants of the DICOM format.
• Metadata extraction from WEBP pictures revised. Output of processing states, similar to PNG files. File type identification/verification for DICOM and WEBP revised.
• All JPEG files are now presented with a processing state in Details mode. Two additional state values were introduced.
• The processing state now depends on the detected generator, where each generator is now assigned to one of three generator classes D (device), E (editor), or C (content management system). JPEG files produced by generator class D are absolute originals. The processing state is always “original”. JPEG files produced by the generator class E are relative originals. Their processing state is always “Edited normally”. Examples are photos published by news agencies like Reuters.
• The detected processing state of the third generator class (CMS like WordPress, Drupal, TYPO3, Joomla etc.) can assume different values. They are usually irregularly edited, i.e. their edited status is not officially indicated. The state can be deducted indirectly based on filename, generator signature, pixel dimension. The state “irregularly edited” can also result from picture manipulations.
• The new processing state “scaled” means that a picture was created with a content management system such as WordPress, TYPO3, Drupal. It can be said with a high probability that such pictures have been released to the public, which entails a reduced intelligence value. Practically such pictures cannot be regarded as documents. They were automatically and individually adapted to the respective output display in order to optimize the loading time of the web page.
• The state “EXIF stripped” refers to JPEG pictures, whose device origin was detected although no EXIF metadata is present. The device can potentially be detected based on generator signature, filename or a characteristic pixel dimension.
• The state “social media” is indicated separately because such pictures often have a higher intelligence value. Unlike news agency pictures they are rather semi-public in nature.
• The state “minimized” is also new and indicates that the JPEG quality was reduced or that the file size was reduced by optimized recompression (jpeg-recompress, JPEGMini).
• The state “undefined” means that the status cannot be determined. It’s a category for everything that remains. Such pictures are usually also the output of content management systems, those that do not identify themselves and whose format is not yet identified (which may change in future versions).
• The processing state and other values (size, bits per pixel, filename analysis) are now also output for PNG files. The same processing states as for JPEG are used, except “Irregularly edited” and “EXIF stripped” are not possible. The value “Original” is used only for screenshots, if they have passed a special test.
• The “size” reported for JPEG pictures in Details mode now always has 1 or 2 values. Sizes that are not standard sizes with a common name (such as “XGA”) are described as “thumbnail”, “medium”, “medium large”, “large” or “big” based on the terminology established by WordPress. If a generating device is identified, the field is named “sensor size” instead or – in the case of scanners – “paper size”.
• Reduced false positive rate when detecting scanned documents.
• JPEG screenshot identification now based on generating device recognition.
• Improved classification of pictures based on pixel dimensions.
• X-Ways Forensics now knows an additional 5000+ devices to better identify the origins of JPEG pictures and other files. The generator signature table and video signature table updated.
• Simplified output of “Quality” in the summary table for JPEG files. It can assume one of the values High, Medium, Low and Very low. It is based on the lossy compression percentage of the DQT segment.
• Additional test in the check for camera originals based on whether the EXIF tags are sorted or not.
• Output of GPS coordinates with up to 6 digits after the decimal point. This is useful because of the habit of newer Samsung device models to specify more decimals and indirectly express the precision of the value by that, contrary to the convention to use the GPS Error tag for that information, unlike Apple and older Samsung models.
• If the GPS format encountered is “unexpected” based on the assumed source of the JPEG file, that is brought to the user’s attention in Details mode. The GPS format will be shown as “unknown” if it is not used in camera original pictures (for example the format of the Geosetter application).
• Generally improved GPS format consistency tests.
• Some more “content created” timestamps are now extracted from pictures, in particular from XMP metadata.
• Output of Photoshop’s “Preserved file name” in the metadata.

File Archive Support

Support for split zip archives in PKZIP/WinZip and 7-Zip styles (a.k.a. spanned or segmented archives).

Extended timestamps from the extra field in zip records are now extracted and presented in the timestamp columns based on Apple specifications, which however is not always how these timestamps are meant. An alternative interpretation can be seen for each zip record in Details mode when selecting the zip archive. The latter interpretation shows these timestamps with the “UT” prefix and tries to recognize the actual format variant, for example that used in GrayKey collections, and from GrayKey collection also extracts an additional type of timestamp (a record change timestamp).

The alternative interpretation of extended timestamps can also be made available in the directory browser. This is an option in Options | Volume Snapshot. The alternative processing currently takes some more time.

In newly refined volume snapshots, the column “1st sector” is now populated properly for files in Zip archives with the sector that contains the local zip record of the respective file. Clicking a file in a zip archive now automatically jumps directly to its local zip record, which is followed by the (usually) compressed file data. Does not apply to files in nested zip archives.

Detection and avoidance of more variants of zip bombs.

The alternative TAR extraction method now estimates the size of the MBOX e-mail archive in a Google Takeout TGZ file if that size was erroneously stored as 0, which apparently happens in real life. Only that work-around allows to extract the MBOX e-mail archive file from such a takeout at all, and once that has happened of course the e-mail messages and attachments can usually be extracted from the e-mail archive.

Navigation within a file archive with directories is now possible without leaving File mode when touching a directory.

Archive subtypes in a section that is not selected for automatic inclusion in the volume snapshot are now still explored when manually double-clicked by the user.

Support for .ctx Chrome Extensions as file archives. That file type is now included in the “Special interest” section of archives in a fresh installation.

Improved ability to extract attachments in PDF files, in particular in so-called PDF portfolios (user-compiled collections of arbitrary files), with the original names and internal paths of the attached/embedded files, where the Description column identifies these files as attachments.

Spreadsheet Support

New ability of the logical Simultaneous Search to find numbers and dates not only if stored literally as text, but also if numbers or dates are stored in binary form in certain spreadsheet files (e.g. in OLE2 compound file format) or in some other encoded form (e.g. dates encoded as textual integer numbers in XML), if the “decode text” option is on. This works pretty well with numbers in Excel and LibreOffice Calc spreadsheets, but can be tricky occasionally with the format of dates if the original Excel user has selected a custom date format instead of one of the standard date formats and also because of some specialties with certain Calc files where it’s not 100% predictable that a date will be extractable in the expected format. This kind of search likely works with some other file types as well, e.g. older spreadsheet types like MS Works or Lotus 123. You can try and define the file types in Options | Viewer Programs if needed. To quickly see and double-check the extraction of numbers and dates from a particular file of interest, you select that file in the directory browser and switch from ordinary to raw preview mode with the Shift key pressed. Please feel encouraged to completely remove that new file mask there for faster text decoding if you do not need to search for numbers and dates in spreadsheets.

Some more details about number searches: Consider a cell in an MS Excel spreadsheet that contains the number 1234567. You can now find that number with the Simultaneous Search searching simply for “1234567” (without the quotation marks). Even if you just know part of the sequence of digits and search for “34567”, you will get a search hit (unless the “whole words only” option is on). If the cell has the “number” format (not “general”), with digit grouping enabled, you can optionally get the number with digit grouping when the file is searched/indexed/decoded in that volume snapshot for the first time, using the digit grouping symbol that is defined in X-Ways Forensics in Options | Notation, but that is not generally recommended because you would have to search for the same number both with and without the grouping symbol if you don’t know whether the original spreadsheet cells were formatted as “number” with or without digit grouping or as “general”. Anyway, to give you another example, if you enable that option for digit grouping in number cells in Options | Viewer Programs and you live in an English speaking country, using a comma as the digit grouping symbol, you would thus search for “1,234,567” to find that number in a number cell. You can also search for just “,567” to find the digit group “567” at the end or in the middle of any longer number in that notation.

If the number that you are looking for is a floating point number, the same rules apply, and you can optionally enter the number with as many decimals as you expect to be visible in the cell in the original application (or less), with the same decimal symbol as in your notation settings in X-Ways Forensics (either a point or comma). If a floating point number is stored for example as 9.876 and formatted to show 2 decimals, it will be shown rounded as 9.88 in the original application and will also be searchable like that in X-Ways Forensics. The same rules apply to currency amounts. You can append or prepend the currency symbol if you know for sure that it was shown in the original formatting, and how (e.g. with or without space between currency symbol and number), or you just omit symbol.

You can search for dates in pure date cells using the notation that is active in X-Ways Forensics as the so-called simple date format. If your simple date format is MM/dd/YY, you would search for 12/31/19 to find the date Dec 31, 2019. Partial date searches are also possible, and make sense especially if you do not use American date styles. For example in ISO notation “yyyy-MM-dd” you can search for “2019-07-“. Or in German notation “dd.MM.yy” you can search for “.07.19” to find any date in July 2019.

Pure time cell searches have also become possible (with partial or whole time expressions). Just make sure to use the separator that is active in X-Ways Forensics for the display of times. Searches for combined date and time values are supported, however, the delimiter between date and time that you can expect is not the delimiter defined in Options | Notation, but typically a single space, or an individual delimiter defined by the user of the spreadsheet.

If an Excel worksheet is embedded in a .docx, .pptx, or .odt file and the volume snapshot has been sufficiently refined, the worksheet will be processed and searched in the same way as if it was a separate file. If embedded in a .doc file, you would get a notification in the form of a report table association “Contains embedded document(s)”, which is often useful to check manually anyway.

The number search capabilities should prove very useful especially in forensic accounting, tax fraud investigations etc. Please note that the simple search function of the viewer component (Ctrl+F3) in ordinary (“pretty”) Preview mode or the View command cannot find numbers or dates in spreadsheets, no matter how you type them.

Preview mode and the View command now use the same digit grouping character, decimal character, date separator, time separator and date order as active elsewhere X-Ways Forensics, to format numbers and dates in spreadsheets.

E-mail Support

• Alternative extraction methods are now available for PST/OST/MBOX e-mail archives (still in a testing stage). These methods will be used if the main extraction method fails to extract e-mails or if preferred by the user. There is a new check box for that preference, not labelled but tooltipped. The alternative method for PST/OST does not work with password-protected e-mail archives and cannot find previously existing objects.
• When attaching a directory with external files to an e-mail archive (PST, OST or MBOX), the contents of that directory will be treated like the result of an e-mail extraction performed by the viewer component. That means for example that redundant empty top-level directories like “Top of Personal Folders”, “Root – Mailbox”, “IPM_SUBTREE” will be skipped and that the MSG files will automatically be split up into to EML files with e-mail headers and bodies plus separate attachment files. Such an extraction can be performed with the context menu commands “Extract Selected Files” and “Extract All Files” in the preview or view of those e-mail archives.
• E-mails that are extracted from PST/OST e-mail archives and that are attached to other e-mails are now described as extracted e-mails and attachments at the same time.
• Support for more code pages in e-mail extraction from MSG.
• The alternative .eml preview option now affects PDF representations of e-mails generated by the Recover/Copy command.

General File Format Support

• Revised and more thorough metadata extraction from HTML files. In particular, “Open Graph” metadata is now extracted.
• Support for certain copy-protected PDF documents used by X-Ways.
• Ability to import hash values from v2.0 of Project VIC JSON files.
• Can now find search terms in ISO-2022 code pages (Japanese, Korean, Chinese) that span an escape sequence in the original data. Can now find individual characters that require escape sequences in Korean and Chinese ISO-2022 code pages.
• Improved conversion from/to ISO-2022 code pages.
• UTF-16 text from the clipboard is now pasted without the null terminator.

User Interface

• WinHex and X-Ways Forensics now respect Windows settings for window text and background colors. We are referring to the settings that you were able to reach with a few mouse clicks in the Control Panel in Windows XP, which in Windows 7 you can still find via Personalization | Window Color | Advanced appearance settings, and which in Windows 10 can still be edited as raw RGB value with the Registry Editor in this key: HKEY_CURRENT_USER | Control Panel | Colors (followed by logging in and out).
• Black backgrounds for almost all parts of the user interface (main window, data window, Case Data window, …) in particular are now supported in X-Ways Forensics, which can be helpful when working in an environment with little ambient light, which generally benefits users who think they can work longer with a less bright screen, and which in general should reduce the disruption of melatonin production and the circadian rhythm among people who face screens emitting unnatural light. The viewer component already previously respected those settings for most document types (it does not or cannot respect them for PDF files for example).
• For the most complete dark screen experience you would change your entire Windows system to a dark theme. The easiest way to achieve that not only for “apps”, but also real desktop applications, is to activate the black high contrast theme. In Windows 10 you would go to PC Settings | Personalization | Settings for high contrast | Activate high contrast | Contrast black.
• A “forced” dark mode just within WinHex/X-Ways Forensics is now also readily available, even without any of the above procedures or settings, in Options | General, which you can activate when needed for night time or generally, for health reasons or to attract less attention during secretive work in a dark adversary environment. It is not 100% complete, as for example it does not affect user interface elements such as window captions, pop-up menus, scrollbars, standard file selection windows or date selection boxes. For those dark mode support from Windows is needed (see above).
• Various meaningful colors in the graphical user interface had to be adjusted in X-Ways Forensics’ own dark mode or when a black background color in Windows settings is detected and adopted, for example the color of file types depending on the type status. In the calendar, the grayscale coding of days with lots of activities is inversed if the background color is black. If you discover text that is unreadable in dark mode, please report back. Color preferences for block selections, tag marks, “already viewed”, modified bytes, and positions/search hits highlighting are now remembered separately for normal mode and dark mode.
• A new option useful in conjunction with dark mode is the ability to render pictures with the internal graphics viewing library as well as all thumbnails in the gallery darker. If that check box, which can be found next to the check box for dark mode in Options | General, is half checked, that means the pixels will be darkened a little less.
• Some more GUI adjustments for high DPI settings.
• The Windows username of the current user is now logged in each section of msglog.txt, in addition to the exact program release, which was previously logged already.
• The command line parameter for automated (unsupervised) imaging is now supported in X-Ways Imager just like in X-Ways Forensics.
• The filters for size and first sector now have a modulo option. With that option in the Size filter you can for example filter out files that are not a multiple of the sector size, when looking for raw disk images or TrueCrypt/VeraCrypt container files. With that option in the First Sector filter you can for example focus on files that are cluster-aligned or not.
• Settings of the Size filter, the Hash Value filter, and the Device Type filter are now stored in .settings files and in .xfc case files like the settings of other column-based filters.
• The Flex filters now have the option for a logical AND combination of all filter terms, so that for example you can filter for e-mails that at the same time are described as attachments.
• Improved option to filter for carved files with the Description column.
• The text filters for comments, metadata, and event descriptions now have an option for case sensitivity.

X-Tension API

• New X-Tension API function XWF_ManageSearchTerm().
• Ability of the X-Tension API XWF_Search() function to specify the alphabet(s) that define word boundaries.
• XWF_OpenItem now supports a new flag to open only the plain text of files, which X-Ways Forensics is able to extract from various file types.
• C++ function definitions and C++ sample projects updated on the X-Tension API web page.
• Fixed an error in the disk I/O X-Tension API.

Miscellaneous

• More efficient generation of thumbnails of non-pictures in the gallery.
• The generation of thumbnails of non-picture files for the report is now more consistent in the results it produces.
• Usage of internal keyboard hooks for enhanced keyboard shortcuts is now optional, cf. Options | Security.
• Some improvements in stability and error handling.
• SR-1: Fixed an exception error that could occur when extracting embedded data from PDF documents.
• Users who are cut off from their offices and/or have no access to their dongles due to a regional lockdown, quarantine measures, travel restrictions or mail service disruptions have this option since May 2020: As long as someone else has access to your dongle (a colleague), they can temporarily deactivate (mothball) the dongle in v20.0, which allows you to use X-Ways Forensics with other means instead, for the time being, at a nominal price. For details please see www.x-ways.net/dongle_protection2.html.
• Many minor improvements.
• User manual and program help updated for v20.0.

Changes of further service releases of v19.9

SR-8: Password detection using a dictionary did not work in certain encrypted archives. That was fixed.

SR-8: Big-endian interpretation of data as FILETIME timestamps in the Data Interpreter failed when interpretation as a big-endian floating point number was active and not successful (“NAN”). That was fixed.

SR-8: Fixed processing of Windows.edb and SRUDB.dat files in v19.9.

SR-9: Prevented a rare exception error that could occur when resolving symlinks.

SR-9: Prevented a very rare exception error that could occur when parsing Zone.Identifier ADS.

SR-9: A rare error that could occur when reading XFS directories has been fixed.

SR-9: Ability to process certain MBOX files with unusual line break characters between e-mails.

SR-9: Fixed inability to read from files in some GZ archives that occurred if these files were opened repeatedly and the evidence object was not closed in between.

SR-9: Fixed RunCount interpretation of certain Windows 10 Prefetch files.

SR-10: Fixed an internal recoding error for search terms that could occur when the simultaneous search was run as part of volume snapshot refinement.

SR-10: Prevented a crash that could occur when extracting metadata from certain MP3 files with a corrupt ID3 tag.

SR-10: Under certain circumstances, logical searches with multiple threads unnecessarily processed the same file more than once. That was fixed.

SR-10: The alternative TAR extraction method no longer omits files with a size of 0 bytes in TAR archives.

SR-10: X-Tension API: XWF_GetVSProp() with XWF_VSPROP_SET_HASHTYPE1* and XWF_SetHashValue() did not work in volume snapshots with no previous or simultaneous hash value computation. That was fixed.

Viewer Component

A new download of v8.5.4 of the viewer component was made available on July 16. Oracle security fixes from July have been applied. The below issues were addressed. Some of these bullet points are quoted verbatim, others have been rephrased for better general understanding where possible.

E-mail with background color set to white color will make the white body text disappear
Email header is in a dark background
Issue with text extraction from One Note with non-ASCII characters
Issue with text extraction from PDF with broken words with spaces
Issue with text extraction from PDF incorrect Hebrew texts from PDF file
Conversion from MS Word DOC to PDF could produce garbage character
Viewer hangs while rendering a certain PDF file with formulas
Conversion from MS Word DOC to PDF could print certain paragraph numbers twice
Enhancement for support of HWP files 5.0.4 and above
Outlook Appointment files show as corrupted when viewing/exporting.
Candidate Word file attachments received corrupted from Outside-In service
Selecting from both body and header of MSG document creates redaction more than
Viewer failed to display content of a particular MS Excel document properly
msg file converted with extra question mark like character
OutsideIn garbles Japanese and other multi-byte characters
Drawpage produces half size view for the tiff file
Exporting PDF file results in inverted text
Conversion to PDF skips some text and graphics elements
Crashes when viewing a particular MS Excel document