As Wi-Fi becomes more business critical for all organizations, wireless troubleshooting solutions (skills & tools) like wireless packet captures have become popular.
However, to get to the root of the problem, it is essential to get access to the raw packet traces- basically seeing what exactly was sent and received through the air. From our experience, the truth always lives in the packets. Sadly, it can sometimes be a chore to find that truth. So, we’d like to share some thoughts on the traditional methods of capturing packets, its challenges, and ask the loaded question, “Is there something easier?”
Thoughts on Traditional Wireless Packet Capture
Traditionally, the practice of capturing Wi-Fi packets requires several things: a laptop, software for capturing, a supported Wi-Fi adapter, and, often a supported Wi-Fi driver. Many believe the MacOS offers a simpler (costs & bundle) solution for capturing packets than Windows based machines. Some of the more popular packet capturing tools are: Airtool, LiveAction OmniPeek, WireShark and the Ekahau Sidekick.
Aside from the above equipment, it is important to have the following information to address the issue:
- The person(s) experiencing the problems
- In case of IoT device, there’s no human operating the device. This, and the fact that IoT devices may only communicate to the network, say, once a day, poses its own challenges to manual packet capture.
- The problematic devices
- The problematic AP (Access Point)
- Time of the incident
- Type of error: constant, periodic, or occurring randomly
Mental note is that the less frequently that error occurs, the more painful it will be to reproduce the problem. As stated previously, an IoT device may only wake up once a day! Recreating the issue alone can be a challenge and typically leads to one of three results:
- “Great success!”, when the ability to capture the issue just as the client described it is reproduced. Not saying that this is rare or anything, but legends are told of this scenario actually having happened once in real life, sometime in the late 1990s.
- Several packet capture attempts are done, but the issue can’t be easily reproduced -the client device works perfectly with no issues. Murphy’s law at play here. This may lead to you capturing packets even throughout the course of several days.
- The engineer ends up down a rabbit hole, when several issues are found, but they are not at all related to the problem are discovered.
Yet, even with these complexities and challenges, the process of gathering wireless packet captures is one of the de-facto troubleshooting methods in the industry. This leads to extensive resource utilization of skilled network engineers: Every time your network faces a tough problem, that skilled engineer needs to fly on site. It also means only the selected few can perform advanced troubleshooting.
A case that I would like to share is one of our customers that had been hunting down a roaming problem for their wireless scanners in their warehouse for over a year. Unable to capture at the right place at the right time, they were facing productivity problems. Luckily, the customer deployed Mist after hearing about some of the benefits from the wireless subscription services. The roaming problem was captured automatically by the Mist system via the Dynamic Packet Capture the first day. The packet trace was then sent to the manufacturer of the wireless scanner, and the problem was fixed by the wireless scanner vendor within a week.
Mist Dynamic Packet Capture
The need for packet captures will never completely disappear. But what if we could automatically gather the information we need precisely when the client has the problem? Not after the event.
This is exactly what Mist does. Mist AP’s (access points) continuously store the packets going through the access point radio to a buffer. Most of the time, the contents of the buffer aren’t used until an error is detected. But when an error is detected, the data packets in the buffer – meaning the packets during the failure, as well as right before and right after, are sent to the Mist Cloud.
An important note here is that the system doesn’t capture data payload, just the radio tap headers, control and management frames, to meet data privacy best practices. So, user data isn’t going anywhere.
Again, any time a network problem is detected, whether it is an incorrect PSK key, DHCP issue, DNS problem, or any other issue from our wide and ever-growing list of issues, a packet capture of the event will be sent to the Mist Cloud. The Mist Dashboard shows the problems the client has had, a pre-processed problem description, as well as gives you the opportunity for a single-click packet capture download. Now, this is a great example of IT automation.
The best thing is, no additional hardware, no additional licenses, no additional set-up is needed to make all this happen. This works with all Mist access points, with the basic licensing, zero boxes needed besides the APs.
More importantly, Mist doesn’t address all packet captures equally. Some systems do not capture the packets with the same radio that’s involved in the transaction. This means some of the information will be missing. Mist utilizes the radio serving the client for capturing the packets for the dynamic packet capture. This means that everything will be caught properly, regardless of the technology or the circumstances – OFDMA, MIMO, you name it. All the packets, all the information will be there. This is not “let’s get what we can from another radio”, and it is not “let’s parse the logs to reverse-engineer a half-baked capture-looking thing”.
And it’s not “enable the feature and wait and hope for the best” either: There’s zero pre-configuration needed – this feature is fully enabled the second you take the first access point out of the box and connect it. And it’s enabled 24/7.
For those deep-packet die-hards, I’d like to tell you that Mist does also support capturing packets on demand. For example, you could capture all the packets of a given client device for 10 minutes – regardless of which AP the client connects to. Even if the client roams, it doesn’t matter. Or, you could capture using a certain AP radio constantly.
To summarize: Mist solves or helps you solve most of the of the Wi-Fi problems through automation and a super-intuitive dashboard before there’s a need for analyzing a packet capture. Yet, when the inspection of data packets is needed, it’s nice to know that they are available for download.
Less truck rolls. More problems solved.
If you would like to hear more about this in a fireside chat, feel free to view the recorded webinar with some of the packet capture diehards here.
Dynamic Packet Capture
On-Demand Capture
