Resilient storage and archiving
- Kiss, Mate
Resilient storage infrastructure
Storage resiliency is an important requirement in many deployments where the customers want to ensure that the data captured by the system is stored in a resilient and highly available fashion on the storage infrastructure. It is recommended to consult your storage administrator for best practices and available options. It is highly recommended to use the resiliency features of the storage infrastructure whenever it is available. The resiliency and high availability options may vary greatly depending on the underlying infrastructure, but most vendors offer sophisticated options. Here you can find some examples:
- On-premise solutions from Dell EMC, Netapp, Hitachi, etc. have mirroring or replication options
- Cloud storages have great variate of resiliency options with geo-redundant capabilities:
- Windows DFS Namespaces have a replication option: https://docs.microsoft.com/en-us/windows-server/storage/dfs-replication/dfsr-overview
These options seamlessly provide storage resiliency for the recording system. Depending on the capabilities, you can potentially achieve automatic failover as well.
Dual archiving
AVAILABLE IN 9.6.7 AND LATER
If for some reason, the storage platform cannot provide resiliency, the recording system has a feature called dual archiving which allows storing the recordings on 2 different storage targets.
Do not use dual archiving if you can achieve storage resiliency with the built-in capabilities of the storage infrastructure. The storage platforms have more robust and advanced capabilities to provide resiliency with e.g. automatic synchronization across the instances, which are not supported by the dual archiving feature.
The system can upload/copy 2 copies of the same file to 2 different storage targets when this option is enabled. The system will not keep the storage targets in sync. If for some reason, there is a data loss on one of the storage targets, the system will not able to detect that and will not attempt to synchronize the data between the 2 storage targets.
During playback, download, or export, the system will seamlessly attempt to access the files on the first storage target, and if for some reason that is not available, it will turn to the second storage target.
The following table summarizes the availability of the dual archiving feature in various storage-related capabilities:
Feature | Availability | Impact |
|---|---|---|
| Playback / Multi Playback | Supported | The process tries to access the media files on the first storage target. If it cannot access the media (storage target unavailable, media file unavailable), the process automatically tries the second storage target. |
| Download / Multi Download | Supported | The process tries to access the media files on the first storage target. If it cannot access the media (storage target unavailable, media file unavailable), the process automatically tries the second storage target. |
Export (Advanced Export, Policy-basedExport, Policy-based Direct Export) | Supported | The export process tries to access the media files on the first storage target. If it is able to access the media, it will move to the next call. If it cannot access the media (storage target unavailable, media file unavailable), the process automatically tries the second storage target. |
| Import | Supported | Once a call is imported, the system can upload/dual archive the call from the default media folder on the server. |
Delete | Supported | The process only attempts to delete a copy only if the retention period has expired (if set). The system can delete one or both copies at one time. If only one copy is deleted, the CDR is updated accordingly. The audit log contains which copy was deleted during the transaction. |
Upload | Supported | The upload process supports one or two storage targets. The upload action uploads the files as a single transaction to both locations. If one fails, it will continue trying to upload. Direct upload policies are also supported. |
Move Media | Supported | The move policy is extended to allow dual archiving of existing records that are already uploaded (archived) to a storage target. Once a record is archived (uploaded to 2 storage targets), the policy only supports moving files from one of the locations. Both copies cannot be moved with a single move policy. The policy has a new filter option to define which copy to move (First, Second). If not set, the policy will automatically use the first copy only. |
Copy | Supported | The copy policy is extended to allow dual archiving of existing records that are already uploaded (archived) to a storage target. Once a record is archived (uploaded to 2 storage targets), the policy only supports copying files from one of the locations. Both copies cannot be copied with a single copy policy. The policy has a filter option to define which copy to copy (First, Second). If not set, the policy will automatically use the first copy only. |
Archive in DB | Supported | The database record is moved to the archive table. It has no impact on the files. |
Archive in DB and Move Media | Supported | See Move and Archive in DB policies. |
Increase Retention Period | Supported | The policy will increase the retention period of one or both copies. The policy has a filter option that defines which copy to increase (first, second, both). If not set, the policy will automatically apply the change on the first copy only. |
| Legal Hold | Supported | Legal hold works seamlessly across both copies. Only one of the copies cannot be under legal hold. If any of the copies are stored on a storage target that supports the legal hold flag on the storage, the system sets the flags accordingly. |
Encryption, Signing | Supported | The system only supports encrypting and/or signing both copies with the same keys. There is no way to encrypt/sign only one of the copies. The system does not support post-upload signing/encryption (the policy automatically filters out dual archived records). The Verify Signature action on the web interface verifies only the first available copy. |
Transcription | Supported | The transcription process tries to access the media files on the first storage target. If it fails, it will attempt to access the files from the second storage target. Currently, transcription is only supported for local and network shares (SMB/DFS). After successfully transcribing a call, the transcription files are uploaded to both storage targets. |
Delete Phonetic Index | Not Applicable | - |
Create Phonetic Index | Not Applicable | - |
Voice Quality Check | Supported | The process tries to access the media files on the first storage target. If it is able to access the media, it will move to the next call. If it cannot access the media (storage target unavailable, media file unavailable), the process automatically tries the second storage target. The system only uses one of the copies to calculate the score, because the 2 copies are identical (when both exist). |
Transcode | Supported | The transcode process tries to access the media files on the “primary” location. If it cannot access the media (storage target unavailable, media file unavailable), the process automatically tries the “secondary” location. After successfully transcoding a call, the new media files are uploaded to both storage targets. |
Delete Communication Policy Events | Not Applicable | - |
File Verification | Supported | The file verification process runs on both copies. It will alert in case one or both copies are missing. |
Deduplicate Recordings | Supported | Deduplication works on both copies seamlessly. The process will automatically remove duplicated records on both storage targets after the Primary/Secondary copy was selected. Primary/Secondary should not be confused with First/Second. Primary/Secondary designates the 2 copies created during 2N recording. These copies are initially stored on the same storage location. While First/Second is related to dual archiving. Without deduplication, someone can have 4 copies if dual archiving is enabled. |