What is Advanced Reconciliation?

Advanced reconciliation is an enhanced reconciliation process that provides users the ability to perform additional capabilities. Advanced reconciliation can be achieved by sending data using the Recon API.

What are the Additional Capabilities in Advanced Reconciliation?

The following capabilities are included as part of advanced reconciliation:

• SFTP to Assist in Collating Source Logs

  • This allows you to upload source logs or manifest files from external sources.

• Flexibility to Connect with any Upstream Service

  • Recon functionality can be extended to any upstream Capture service or product.

  • Not limited to only Smarsh products.

• Enhance existing Enterprise Reconciliation Capabilities

  • Continue to leverage existing UI and role enforcement.

  • Automation and incident management capabilities will continue to be available.

What is the Procedure to Initiate the Reconciliation Workflow?

The reconciliation process is performed using Transcript ID as the identifier for communication ingested into the archive.

Transcript ID: Unique identifier assigned to the conversation as recorded in a Capture source such as Vantage and Socialite.

Perform the following steps to initiate advanced reconciliation:

1. Create a file in the expected name and content format and upload to the SFTP server. Check the Formatting sections for more details.

2. The file must be uploaded before 6:00 AM UTC.

   The reconciliation process runs at 6:00 AM UTC (generic recon task) and 9:00 AM UTC (agregation task)

3. Reconciliation failures are made available to the user by:

• Recon Dashboard

• Recon Search

• Feed Tracker Report uploaded to the SFTP server

  Sample Feed Tracker Report

  

What is the Expected File Name and Content Format?

The expected File Name Format is given in the table below:

• XML (HTTP) - This channel is used for classification of non-email journal messages.

• SMTP - This channel is used for classification of email journal messages.

The expected File Content Format is given in the table below:

Sample CSV File Format:

What are the Failure Scenarios leading to File Rejection?

To ensure successful file data upload, the specified parameters and their conditions must be met as outlined below:

How to Remediate Failure Scenarios leading to File Rejection?

If the file does not meet the specified conditions, then the whole file is rejected and is notified using the file tracker collection. A tracker report is sent to the customer via SFTP for the previous day. The customer can rectify the CSV file, which will be processed during the next iteration.

To avoid rejections due to size limit, multiple CSV files can be uploaded, as shown in the example below:

AWSNAM_SMTP_Exchange_Journal_20230115.csv
AWSNAM_1_SMTP_Exchange_Journal_20230115.csv
-
-
-
AWSNAM_100_SMTP_Exchange_Journal_20230115.csv

How to View Reconciliation Insights?

Users can view and access insights from the go-live date to within 3 days from the current date by using the following tabs:

• Recon Search: View item-level reconciliation data. For more information refer the Recon Search topic.

• Recon Dashboard: View consolidated data based on existing networks. For more information refer the Recon Dashboard topic.

What is the Advanced Reconciliation Workflow and the Various Processing States?

The advanced reconciliation workflow is depicted in the workflow diagram below:

The processing states listed in the diagram are:

• Capture Received - Documents which have been captured successfully.

• Capture Not Received - Documents which the respective channels might have sent, however have not been captured successfully. The customer CSV contains transcript IDs which have not been captured successfuly by Smarsh Capture products such as EGW.

• Capture Unreconciled - Documents which are captured successfully, however are not yet exported to the archive.

• Capture Exported - Documents which are exported successfully to the archive.

• Archive Not Received - Documents which are exported by capture, however not yet received by the archive. This state primarily arises in case of direct ingestion via API scenarios. The customer CSV contains transcript IDs which have not been received by the ingestion API.

• Archive Received - Documents have been received successfully by the archive.

  Important

  This indicates the ingestion process has returned a success code. The archive SLO clock initiates at this point.

• Archive Unreconciled - Documents which are received by the archive, however are still not discoverable.

• Archived - Documents which have been stored and indexed successfully.

What are the Reconciliation SLOs that are Followed?

The following SLOs are currently in place as part of the reconciliation offering:

What is the Incident Process for excess IDs that are Uploaded?

Message IDs that are uploaded by the customer however that are not recieved by the Smarsh Enterprise platform, will be identified as Archive Not Received messages. On further investigation if these messages will not be ingested into the platform, they can be removed from appearing in reconciliation search results and the dashboard. To remove these messages, contact Smarsh support alongwith the affected message ID details.

What is the Delta Consolidation Process?

• Unreconciled Items: The auto reconciliation process will reconcile any unreconciled items.

• Excess IDs Uploaded: Contact Smarsh Support while providing the details in CSV format.