Capturing network traffic for troubleshooting

There are certain situations when a problem can only be resolved if the support team can take a closer look at the actual network traffic seen by the applications. This can help identify network issues and bottlenecks, and analyze network and application level protocols.

We recommend using the following network capture tools:

  • Wireshark
  • Tshark
  • Verba Packet Capture

Installing Wireshark and Tshark on Verba servers

You can download and install Wireshark (and the Tshark command line tool with it) on the Verba servers. To learn more about Wireshark and download the installer, visit https://www.wireshark.org/.

Do not remove the Winpcap driver when installing Wireshark. Otherwise, the system components relying on the Wincap drive will not work anymore

Capturing on Recording Servers causes extra load on CPU and disk utilization and can interfere with the recording process which can lead to data loss under critical circumstances. The load of the packet capture on the Recording Servers should be always considered and if possible should only be used during non-busy hours.

Tshark

Tshark is a high performance packet capture application that is part of the Wireshark installation package. It is a command line tool for high performance continuous capturing. It is useful when network traffic is high and/or capturing with Wireshark becomes unstable, and when we need to leave tracing on for a longer period of time (many hours or days).Examples

Get help:

tshark -h


List interfaces:

tshark -D


Start capturing with capture file rotation:

tshark -i 3 -B 96 -b filesize:250000 -b files:100 -w c:\tmp\test.pcap -F pcap

Where:

  • -i specifies the interface with the ID retrieved by tshark –D
  • -B sets capture buffer size in Mbyte. Default is 2 Mbyte, if there is a large traffic, you should go up to 96 Mbyte
  • -b specifies the capture rotation: filesize:xxx max size of a single capture file in Kbytes, files:xxx the number of files after which the oldest one is overwritten. Using file rotation set based on available disk space we can make sure there will be enough space left for the other applications when we leave tracing on for a longer time.
  • -w output file
  • -F output format (could be pcap or pcapng. We prefer pcap, that can be directly processed by the passive recorder)
  • -f can specify capture filter in BPF syntax, see below. Using an appropriate capture filter highly reduces the load on CPU and Disk

BPF filter syntax

For the full syntax, see  https://biot.com/capstats/bpf.html or https://www.ibm.com/support/knowledgecenter/SS42VS_7.3.3/com.ibm.qradar.doc/c_forensics_bpf.html.

Important filter examples:

Filtering all (src/dst) IP traffic of given host:

host 10.110.77.200


Filtering inbound/outbound IP traffic of given host:

src host 10.110.77.200 / dst host 10.110.77.200


Filtering for specific subnet:

(src/dst) net 10.110.77.0/24


Filtering for specific TCP or UDP port:

tcp/udp src/dst 443


Logical combination: or/and and grouping with () supported


Filtering for SIP (non secure, 5060 default port):

traffic of 10.110.77.200 CUCM: (udp port 5060 or tcp port 5060) and host 10.110.77.200


Filtering for proxy-filter communication:

tcp port 10201


Filtering for proxy-recorder communication:

tcp port 11112


Filtering for recording director – media recorder communication:

tcp port 10500

Wireshark

Wireshark is a GUI based packet trace analyzer. It can parse the real-time traffic capture or read a network packet capture file. The application can identify encapsulations and interpret and visualizes protocol data at all layers. It uses a different syntax than the BPF capture filter (used by Tshark). Filtering can only be applied to displaying the packets (and not capture filtering). For more information on display filters, see https://wiki.wireshark.org/DisplayFilters.

Display filter examples

Searching for string "xxxx" in the whole packet:

frame contains "xxxx"


Filtering for SIP or Skinny or H.323 call control packets:

sip or sccp or h323


Filtering for HTTP messages:

http


Filtering for IP traffic of 10.110.77.200:

ip.dst=="10.110.77.200" or ip.src=="10.110.77.200"


Filtering for TCP traffic on 5060 port (works with udp as well):

tcp.dstport==5060 or tcp.srcport==5060 

Verba packet capture

The system comes with a built-in packet capture tool called Verba Packet Capture. This tool collects and stores network traffic without analyzing it or interfering with the recording progress, similar to Tshark. Verba Packet Capture creates standard PCAP files that can be opened by WireShark.

The tool is especially useful when troubleshooting proxy based recording issues because the tool can take into account the recorder settings and connect to the same proxies as a redundant/2N recorder pair of the recorder. That way it receives exactly the same traffic as the recorder service. The tool should be run on the servers where the Passive Recorder service runs.