BT ITS recorder resiliency

Media recording redundancy

Voice LAN redundancy

There can be two independent Voice LANs configured for media delivery. If VLAN A fails, the media streams can automatically fail over to VLAN B within <~5 seconds and recording continues with a small gap in the media record. Please note, IPSI management LAN is still a single point of failure.

Media Recorder load-balancing and failover (N+1)

The Recording Servers have two roles which can be co-located on the same server or distributed across multiple servers:

  • Recording Director: integration point with telephony vendors, provides a unified layer/acts as a mediator for media establishment, CDR events to Media Recorder role
  • Media Recorder: controlled by Recording Director, it records media and stores CDR information in the database

If the Recording Director and the Media Recorders are separated on multiple servers and there are at least one Recording Director and two Media Recorders, the Recording Director can:

  • Distribute the media recording tasks between Media Recorders to provide load-balancing between them. Load-balancing takes into account the recorder utilization report (CPU, number of concurrent recording, available disk space etc...) sent by the Media Recorders on a periodic fashion. Due to the characteristics of BT IPSI recording interface, load-balancing for BT IPSI, unlike in case of other vendors, does not work on a per call basis. Load balancing is implemented on a per TTP (Trunk Termination Point) basis. Since there are persistent recording channels established via the TTPs, the Media Recorder for a TTP is selected at service startup time.
  • If a failed/offline Media Recorder is detected then all the TTPs recorded on the server can failover to other Media Recorders and recording continues from the point of failure.

Detecting Media Recorder failure and moving TTPs to other recorders might take 10-15 seconds.

TTP manager redundancy

The Recording Director is responsible of distributing and moving TTPs over in case of Media Recorder failure and establishing new TTPs and media records in case of new verticals are provisioned for recording. In order to provide redundancy for the TTP management functionality in the Recording Director, an active - standby Recording Director can be deployed. It means that only the active Recording Director is able to manage TTP allocation across the Media Recorders, while the standby Recording Director only monitors the TTP distribution. When the active Recording Directors fails, the standby takes over and takes over TTP management. Until there is no need to change the current distribution (there is no Media Recorder failure, no new verticals provisioned with new TTP channels), the same distribution persists as before the failover.  

If TTP Manager redundancy is configured (standby Recording Director), there must be a very stable network connection between the two hosts. NIC teaming or crossover cable connection is required. If the network is lost between the two servers but both have an active connection to the BT infrastructure and to the Media Recorder servers, it could lead to unexpected situations and possible media loss, because both Recording Directors will actively try to control TTPs.

2N recording

The system can be deployed in a 2N recording configuration, where:

  • 2 separate Recording Server groups are deployed, implementing 2 separate recorder groups/lanes (primary and secondary)
  • all TTPs are duplicated and recorded twice by assigning the TTPs to 2 separate Media Recorders in the different recorder groups
  • the TTP allocation for 2N recording requires special care, see BT ITS TTP numbering conventions for more information
  • 2 Recording Directors are configured to manage the primary and secondary TTP allocation in each of the recorder groups
  • the 2 Recording Directors are separately connected to the BT CTILink server, handling the CTI messages

CTI redundancy

CTI resiliency can only be supported by deploying 2 Recording Directors in a 2N configuration (even if media recording might be N+1 or 2N, related CTI will always be 2N). For one recorder group, maximum 2 Recording Directors, acting as CTI listeners, can be configured. Each Recording Director creates the copy of the same CDR record, one marking it primary and the other marking it secondary. A deduplication data management policy can be configured to correlate and keep only one of the related records.