IM conversation fields

The system by default stores all chat conversations in the database. However, we have a file format that can be stored on the disk. This file format is used for exporting into file-based storage targets, and there is an option on each recorder service to generate them.

Cisco Jabber IM:

Cisco Compliance Service → IM Recording → Create Transcript and Metadata XML Files

SfB/Lync IM:

SfB/Lync IM Recorder → General → Create Transcript and Metadata XML Files

Import Service:

Import → General → Create IM Transcript Files


The .IM file is generally a zipped XML file. However, the storage service can skip the zipping step when exporting based on configuration.

Storage Management → Export → Compress IM conversations


The XML files are UTF-8 encoded.

The following table provides a description of the available fields. Not all fields are presented in all cases, only fields marked as mandatory.

FieldSample valueDescriptionMandatory
cdr_id4a9d5ec0-7a40-11e4-8271-001cc092f276

Unique conversation identifier in the Verba system.

It is possible that some recorders write this field with name 'cdr'.

No
messages
The array of 'message' elements.Yes


The following table provides a description of the message attributes.

FieldSample valueDescriptionMandatory
id

18

ab5d60f8:acb7:4140:9c51:31a04beb73ce

41271025-c517-4b91-a873-252acb6b67f8

Unique message identifier. Some platforms give us these ids.

In other cases, we fill it with auto-incremented numeric values.

Yes
conference_id394WJ6JRPlatform-specific conversation conference identifier.No


The following table provides a description of the message fields.

FieldSample valueDescriptionMandatory
timestamp2021.11.04 11:36:48.771

Date and time of the message in UTC timezone in the following format:

YYYY.MM.DD HH:mm:ss.SSS

Yes
sourcenorbi@verbalabs.comThe number/address of the sender of the message.Yes
source_nameNorber KakasThe name of the sender of the message.No
destinationjsikorsky@verbalabs.com

The number/address of the receiver of the message.

Can be filled only for peer 2 peer conversation.

No
plain_message

Hello World!


The plain text formatted content of the message.No
raw_message<SPAN style="margin-bottom:0pt; line-height:normal; font-size:10pt; font-family:"Segoe UI",sans-serif; color:black;">Hello World!</SPAN>The raw message that was received from the platform.No
message_headerINVITE sip:toth@verbalabs.com;maddr=fepool.verbalabs.com SIP/2.0Platform-specific message header.No
subject
Platform-specific message subject.No
message_id
Platform-specific message identifier. Deprecated. Use the message id attribute.No
thread_id48b173dd-ea2b-11eb-9bde-84fdd16620a5For advanced IM data model, this id connects/links messages to a conversation. (correlation id)No
ts_original2021.11.04 11:36:48.771For advanced IM data model, this is the original message timestamp.No
update_delete0

For advanced IM data model, this means the message action. The following values are possible:

  • 0 = created (a new message)
  • 1 = updated (an already existing message)
  • 2 = deleted (an already existing message)
No
is_reply0

For advanced IM data model, this means if the message is a reply to another message. The following values are possible:

  • 0 = not a reply
  • 1 = is a reply
No


Example XML formatted chat records:

<?xml version="1.0"?>
<im>
	<cdr_id>4a9d5ec0-7a40-11e4-8271-001cc092f276</cdr_id>
	<messages>
		<message id="28" conference_id="">
			<timestamp>2016.01.19 14:04:30.793</timestamp>
			<source>balazs@verbalabs.com</source>
			<destination>toth@verbalabs.com</destination>
			<message_header>INVITE sip:toth@verbalabs.com;maddr=fepool.verbalabs.com SIP/2.0
content-length: 244
via: SIP/2.0/TLS 192.168.1.25:52878;ms-received-port=52878;ms-received-cid=2B7A00
ms-application-via: ms-icr-to;state=ToPrimary;ms-pool=sfbpool.verbalabs.com;ms-application=http%3A%2F%2Fwww.microsoft.com%2FLCS%2FInterClusterRouting;ms-server=FE2SFB.VERBALABS.COM
ms-application-via: SIP;ms-urc-target;ms-urc-rs-from;routed-to-role=registrar;target-hash=0FB1EE9E;ms-server=FE2SFB.VERBALABS.COM;ms-pool=sfbpool.verbalabs.com;ms-application=ad894dc3-55e0-44bf-a07e-3c073aaa4a57
from: &quot;Balázs Oláh&quot;&lt;sip:balazs@verbalabs.com&gt;;tag=6107b9355b;epid=88509262b4
ms-application-via: ms-udc.cdr%3D5aeeece7785e9fc5365e0e83e03ef634%3A1;ms-pool=sfbpool.verbalabs.com;ms-application=http%3A%2F%2Fwww.microsoft.com%2FLCS%2FUdcAgent;ms-server=FE2SFB.VERBALABS.COM
max-forwards: 70
to: &lt;sip:toth@verbalabs.com&gt;
call-id: 4a8a761f58a14a3c9c4a048d2d936095
cseq: 1 INVITE
contact: &lt;sip:balazs@verbalabs.com;opaque=user:epid:sNs9l4A1x1Gehv02bvFmGgAA;gruu&gt;
user-agent: UCCAPI/15.0.4779.1000 OC/15.0.4779.1001 (Skype for Business)
supported: ms-dialog-route-set-update
ms-text-format: text/plain; charset=UTF-8; ms-body=VGVzenQNCg==
ms-im-format: text/html; charset=UTF-8; ms-body=PFNQQU4gc3R5bGU9Im1hcmdpbi1ib3R0b206MHB0OyBsaW5lLWhlaWdodDpub3JtYWw7IGZvbnQtc2l6ZToxMHB0OyBmb250LWZhbWlseToiU2Vnb2UgVUkiLHNhbnMtc2VyaWY7IGNvbG9yOmJsYWNrOyI+VGVzenQ8L1NQQU4+
ms-im-format: text/rtf; charset=UTF-8; ms-body=e1xydGYxXGZiaWRpc1xhbnNpXGFuc2ljcGcxMjUwXGRlZmYwXG5vdWljb21wYXRcZGVmbGFuZzEwMzh7XGZvbnR0Ymx7XGYwXGZuaWxcZmNoYXJzZXQyMzggU2Vnb2UgVUk7fXtcZjFcZm5pbCBTZWdvZSBVSTt9fQ0Ke1xjb2xvcnRibCA7XHJlZDBcZ3JlZW4wXGJsdWUwO30NCntcKlxnZW5lcmF0b3IgUmljaGVkMjAgMTUuMC40NzM3fXtcKlxtbWF0aFByXG13cmFwSW5kZW50MTQ0MCB9XHZpZXdraW5kNFx1YzEgDQpccGFyZFxjZjFcZjBcZnMyMCBUZXN6dFxmMVxwYXINCntcKlxseW5jZmxhZ3M8cnRmPTE+fX0NCg==
supported: ms-embedded-first-message
supported: ms-delayed-accept
supported: ms-renders-isf
supported: ms-renders-gif
supported: ms-renders-mime-alternative
Ms-Conversation</message_header>
			<plain_message>Teszt</plain_message>
			<raw_message>&lt;SPAN style=&quot;margin-bottom:0pt; line-height:normal; font-size:10pt; font-family:&quot;Segoe UI&quot;,sans-serif; color:black;&quot;&gt;Teszt&lt;/SPAN&gt;</raw_message>
		</message>
		<message id="29" conference_id="">
			<timestamp>2016.01.19 14:04:55.967</timestamp>
			<source>balazs@verbalabs.com</source>
			<destination>toth@verbalabs.com</destination>
			<message_header>MESSAGE sip:toth@verbalabs.com;opaque=user:epid:1cTFqkG_-V26ZHP5x_TS9gAA;gruu SIP/2.0
via: SIP/2.0/TLS 192.168.1.25:52878;ms-received-port=52878;ms-received-cid=2B7A00
content-length: 295
max-forwards: 70
from: &lt;sip:balazs@verbalabs.com&gt;;tag=6107b9355b;epid=88509262b4
to: &lt;sip:toth@verbalabs.com&gt;;epid=747d7f1c7a;tag=62be4425f9
call-id: 4a8a761f58a14a3c9c4a048d2d936095
cseq: 2 MESSAGE
route: &lt;sip:fepool.verbalabs.com:5061;transport=tls;ms-fe=FE2.VERBALABS.COM;opaque=state:T;lr&gt;
user-agent: UCCAPI/15.0.4779.1000 OC/15.0.4779.1001 (Skype for Business)
supported: ms-dialog-route-set-update
supported: timer
content-type: text/rtf
ms-application-via: ms-udc.cdr%3D5aeeece7785e9fc5365e0e83e03ef634%3A1;ms-pool=sfbpool.verbalabs.com;ms-application=http%3A%2F%2Fwww.microsoft.com%2FLCS%2FUdcAgent;ms-server=FE2SFB.VERBALABS.COM
ms-routing-phase: from-uri-routing-done
			</message_header>
			<plain_message>IM test.</plain_message>
			<raw_message>{\rtf1\fbidis\ansi\ansicpg1250\deff0\nouicompat\deflang1038{\fonttbl{\f0\fnil\fcharset238 Segoe UI;}{\f1\fnil Segoe UI;}}
{\colortbl ;\red0\green0\blue0;}
{\*\generator Riched20 15.0.4737}{\*\mmathPr\mwrapIndent1440 }\viewkind4\uc1 
\pard\cf1\f0\fs20 IM test.\f1\par
{\*\lyncflags&lt;rtf=1&gt;}}
			</raw_message>
		</message>
	</messages>
</im>


<?xml version="1.0" encoding="utf-8"?>
<im>
  <cdr>ea643f69-9750-4d69-b914-ce18107bb0cb</cdr>
  <messages>
    <message id="41271025-c517-4b91-a873-252acb6b67f8">
      <timestamp>2021-07-22T15:38:19.8252288Z</timestamp>
      <source>kakas@verbalabs.com</source>
      <destination>columbo@verbalabs.com</destination>
      <plain_message>4864</plain_message>
      <raw_message>4864</raw_message>
      <message_header>INVITE sip:columbo@verbalabs.com SIP/2.0
content-length: 242
via: SIP/2.0/TLS 10.110.77.11:52952;branch=z9hG4bKDAC5B8C4.DE48301CFE2FF8FB;branched=FALSE;ms-received-port=52952;ms-received-cid=52ED00
via: SIP/2.0/TLS 10.95.71.38:1078;ms-received-port=1078;ms-received-cid=4B6800
ms-application-via: ms-udc.cdr%3D9013333e5089eac0bebfdf56c591306b%3A3%3Barch%3D9013333e5089eac0bebfdf56c591306b%3A3;ms-pool=devsfbpool.verbalabs.com;ms-application=http%3A%2F%2Fwww.microsoft.com%2FLCS%2FUdcAgent;ms-server=DEVFE2SFB.VERBALABS.COM
record-route: &lt;sip:devsfbpool.verbalabs.com:5061;transport=tls;ms-fe=DEVFE1SFB.VERBALABS.COM;opaque=state:T;lr&gt;;tag=3A98EE90947CB2BBC79A0580CE644BB9
max-forwards: 69
ms-application-via: SIP;ms-urc-target;ms-urc-rs-from;target-hash=5113A4A8;ms-server=DEVFE1SFB.VERBALABS.COM;ms-pool=devsfbpool.verbalabs.com;ms-application=ad894dc3-55e0-44bf-a07e-3c073aaa4a57
from: "Norbert Kakas"&lt;sip:kakas@verbalabs.com&gt;;tag=7a7bf9ca6b;epid=a0586a87b7;verbasession=verbaim%7c3;verbaimrecorder=verbaim%7cdev-rs1.verbalabs.com
ms-application-via: ms-udc.cdr%3D9013333e5089eac0bebfdf56c591306b%3A1%3Barch%3D9013333e5089eac0bebfdf56c591306b%3A1;ms-pool=devsfbpool.verbalabs.com;ms-application=http%3A%2F%2Fwww.microsoft.com%2FLCS%2FUdcAgent;ms-server=DEVFE1SFB.VERBALABS.COM
to: &lt;sip:columbo@verbalabs.com&gt;
call-id: 6fbab6557277490d9b0de0822d2bd5a9
cseq: 1 INVITE
contact: &lt;sip:kakas@verbalabs.com;opaque=user:epid:2CUGOjm8rlquPhxd21p37AAA;gruu&gt;
user-agent: UCCAPI/16.0.14131.20010 OC/16.0.14131.20320 (Skype for Business)
supported: ms-dialog-route-set-update
ms-text-format: text/plain; charset=UTF-8; ms-body=NDg2NA==
ms-im-format: text/html; charset=UTF-8; ms-body=PFNQQU4gc3R5bGU9J2ZvbnQtc2l6ZToxMHB0OyB3b3JkLXdyYXA6YnJlYWstd29yZDsgbWFyZ2luLWJvdHRvbTowY207IGZvbnQtZmFtaWx5OiJTZWdvZSBVSSIsc2Fucy1zZXJpZjsgY29sb3I6YmxhY2s7IGxpbmUtaGVpZ2h0Om5vcm1hbDsnPjQ4NjQ8L1NQQU4+
ms-im-format: text/rtf; charset=UTF-8; ms-body=e1xydGYxXGZiaWRpc1xhbnNpXGFuc2ljcGcxMjUyXGRlZmYwXG5vdWljb21wYXRcZGVmbGFuZzEwMzN7XGZvbnR0Ymx7XGYwXGZuaWxcZmNoYXJzZXQwIFNlZ29lIFVJO317XGYxXGZuaWwgU2Vnb2UgVUk7fX0NCntcY29sb3J0YmwgO1xyZWQwXGdyZWVuMFxibHVlMDt9DQp7XCpcZ2VuZXJhdG9yIFJpY2hlZDIwIDE2LjAuMTQxMzF9XHZpZXdraW5kNFx1YzEgDQpccGFyZFxjZjFcZjBcZnMyMCA0ODY0XGYxXHBhcg0Ke1wqXGx5bmNmbGFnczxydGY9MT59fQ0K
supported: ms-embedded-first-message
supported: ms-delayed-accept
supported: ms-renders-isf
supported: ms-renders-gif
supported: ms-renders-mime-alternative
Ms-Conversation-ID: Add/D5eRuGPd4+k4T8eoSj8NHHI0FA==
supported: timer
supported: histinfo
supported: ms-safe-transfer
supported: ms-sender
supported: ms-early-media
Roster-Manager: sip:kakas@verbalabs.com
EndPoints: &lt;sip:kakas@verbalabs.com&gt;, &lt;sip:columbo@verbalabs.com&gt;
supported: com.microsoft.rtc-multiparty
ms-keep-alive: UAC;hop-hop=yes
allow: INVITE, BYE, ACK, CANCEL, INFO, MESSAGE, UPDATE, REFER, NOTIFY, BENOTIFY
ms-subnet: 10.95.70.0
supported: ms-conf-invite
content-type: application/sdp
Verba-ImRecorder: dev-rs1.verbalabs.com:10220
ms-routing-phase: from-uri-routing-done
ms-user-data: ms-publiccloud=TRUE;ms-federation=TRUE
</message_header>
      <subject />
    </message>
  </messages>
</im>