Disposition Workflow
The Smarsh Enterprise Platform leverages an automated, two-step disposition process to manage the lifecycle of archived data:
Disposition Evaluation
The system continuously evaluates the policy map, identifying documents that have reached their retention period based on assigned policies. This evaluation occurs every hour to ensure timely processing of eligible documents.
These potentially expired documents are then validated against the Smarsh Enterprise Discovery application to confirm they are not subject to legal holds.
Upon satisfying both criteria (expired retention and no legal hold), these disposition events are then published to downstream applications and data stores to facilitate their deletion. Additionally, Conduct queues also create policies, and the behaviour is similar to policy expiry.
Deletion and Reporting
Each application and data store utilizes a dedicated deletion service, and these services ensure the removal of eligible documents.
A dedicated monitoring process gathers information from all deletion services. This comprehensive approach offers two critical functionalities:
Disposition Summary Reports: The monitoring process generates detailed disposition summary reports, providing a clear audit trail of the deletion activity. These reports ensure transparency and facilitate compliance efforts.
Failure Management: In the event of any deletion failures within individual services, the central monitoring process identifies the issue. It then initiates appropriate remediation steps to ensure the successful completion of the disposition process.
Tier 1 vs. Tier 2 Disposition Differences
Grace Period: Tier 1 offers a three-day grace period after policy expiry, allowing for configuration changes to prevent accidental deletion. Tier 2 has no grace period.
Data Volume: Tier 2 is designed for handling larger datasets and higher data volumes.
Data Deletion Timeline: Tier 1 triggers deletion within three days of eligibility. Tier 2 triggers deletion within 24 hours of eligibility.
Tier 1 Disposition Grace Period
The Smarsh Enterprise Platform offers a three-day grace period for documents deemed eligible for disposition based on policy expiration. This window allows users to address any potential errors or unforeseen circumstances before permanent deletion.
Document Retention: Upon becoming eligible for disposition, documents are retained within the system for an additional three days.
Grace Period Functionality: This grace period empowers users to:
Extend the disposition timeline for documents disposed of in error.
Rollback disposition actions, thereby restoring the document's previous retention status.
Example Scenario
Consider a document reaching its retention expiry on June 10th, 2024, is protected from deletion due to an existing legal hold expiring on July 15th, 2024. However, if this hold is accidentally released on June 14th, 2024, a configured grace period offers an additional safeguard. This 3-day grace period (until June 17th, 2024) allows users to identify and rectify the accidental hold release by reapplying the hold or extending the retention policy. If no corrective action is taken within this window, the document becomes eligible for deletion on June 18th, 2024.
Disposition Remediation
The Smarsh Enterprise Platform incorporates a disposition remediation process to address potential failures that may occur during the automated deletion of documents reaching their designated retention expiry. The system employs a built-in automated retry mechanism for handling initial deletion failures. This mechanism functions as follows:
Upon encountering a deletion failure during the disposition process, the document is automatically added to a dedicated queue to manage failures.
The system initiates a pre-defined number of retries (currently set to two) for the failed deletion.
If the automated retry mechanism exhausts all attempts (two retries by default), manual remediation will be required.
Disposition Remediation Reporting (In Progress)
The Smarsh development team is actively building a comprehensive report on disposition failures. This report will provide valuable insights into the nature and frequency of deletion failures, enabling further optimization of the remediation process.
Disposition Observability and Monitoring
Currently, all failures are reported to engineering via Slack alerts. There is also tenant-level tracking of the current state of disposition. Open failures are tracked manually. The current service level objective (SLO) for completing disposition processing on initiated items is up to 90 days.
The roadmap for disposition observability and monitoring includes:
Automated incident creation for open failures based on the SLO.
A centralized dashboard for tracking failures across all tenants.