Friday, March 31, 2017

Scripting Against The Web Interface, Part 2

Different Ways To Get Devices

These examples should work with any language, but for now we'll look at using the Python class provided with Kismet; no particular reason, but I needed Python for some other tasks, it's used in GNURadio, and it's a just generally a popular language.

(If you'd like to implement a similar utility class in Ruby or any other language, please do, and consider sending it my way!)

JSON vs Msgpack

Nearly all the interfaces in Kismet support multiple types of output serialization - currently JSON and Msgpack, which is a binary packed format which can represent the same style of data.

The Python API favors the msgpack variants because pymsgpack is easy to use and quickly returns complete Python objects; there's no reason not to use JSON however.  Both will return identical data in identical structures, and can be turned into native language objects.

All of the example URLs should work on either - substitute ".json" as the extension instead of ".msgpack" if you're not using the Python KismetRest interface code.

Listing devices

There are several ways to get a list of devices from Kismet.  In general they all allow selecting specific fields, groups of fields, or simple matching.

The simplest method for fetching devices is to simply ask for a list of all of them; this will enumerate every device Kismet knows about and return a list:


for instance will return all devices in the system, formatted as JSON, with full objects for all devices.

The downside to the simple method is that it enumerates all devices in the system.  This works fine for dozens or hundreds of devices, but if you're querying a system with thousands or tens of thousands of devices, it can cause significant lag and visible issues - dropped packets, an unresponsive server, etc.  (Internally, this happens because Kismet needs to lock the device list while it processes all the devices, which means new packets can't update devices as they arrive).  In the worst case this can cause Kismet to abort if it takes more than 5 seconds to process the list; on a slow system with tens of thousands of devices it can take some time.

The simplest query

Using the KismetRest client, you can get a list of devices with the device_summary() function:

import KismetRest
import time

kr = KismetRest.KismetConnector("http://localhost:2501")
devices = kr.device_summary()
while 1:
    print devices

This will print all the devices Kismet knows about (as python objects, not the prettiest output in the world) once a second.

This method will ONLY work well for very small numbers of devices - tens to hundreds.  If your Kismet server can see thousands of devices or more, this will likely have a noticeable performance hit.

Making it better: Query by time

But how do we make this better?  The first, and easiest, way to reduce the impact is to reduce the number of devices we look at.  We can accomplish this with the device_summary_since(ts) function, which calls /devices/last-time/[ts]/devices.msgpack (or devices.json of course) endpoint with a timestamp; Kismet will only return devices which have been created or modified after the timestamp provided.

import KismetRest

kr = KismetRest.KismetConnector("http://localhost:2501")

# Initialize our timestamp to 0
ts = 0

while 1:
    summary = kr.device_summary_since(ts)

    # Get our timestamp from the response!
    ts = summary['kismet.devicelist.timestamp']

    print summary['kismet.device.list']


By starting with a timestamp of 0 we request every device Kismet knows about (since all have been modified since the start of time).  After that first request, we save the current server timestamp (from the 'kismet.devicelist.timestamp' field) and use it to make our queries in the future; from then on we'll only get devices which have changed since the last time we looked, no matter how long we sleep in between.

This approach still has some problems; the initial query can take a LOT of processing power if there are a huge number of devices, because it still has to look up every single device.

Make it better: Simplify what we're asking for

The next way to reduce the impact on the server is to apply a simplification of the fields: This returns only the fields specified, instead of every field (and sub-field) in the device object, which can reduce the load of serialization by a factor of hundreds.

If you're familiar with SQL, this is the difference between doing a select * and doing a select field1, field2, ... ; By excluding data you don't care about you can greatly reduce the impact.

Fields can be specified by name or by path:

  • "" would select the channel field from the device object
  • "kismet.device.base.signal" would select the entire signal object from the device object and include it.
  • "kismet.device.base.signal/kismet.common.signal.last_signal_dbm" would select the last signal record from the signal object.  Fields selected by path are always placed in the result as the final field in the path, so our object would contain kismet.common.signal.last_signal_dbm
Fields can also be specified as pairs, where a field is renamed - if you need two fields in different paths, of the same name, they can be specified as an array of 2 elements: ["kismet.device.base.signa/kismet.common.signal.last_signal_dbm", "last_base_signal"] would rename the last_signal_dbm field.

Field simplification is done by POSTing to /devices/summary/devices.json or to /devices/last-time/[ts]/devices.json (or, of course, .msgpack).

In the Kismet Rest API, you can use the smart_summary_since(...) function:

fields = [
        "", # Standard channel field
        "unknown_field", # Requesting an unknown field will return a field containing integer 0
        "dot11.device/dot11.device.last_beaconed_ssid", # Complex path specification
        ["kismet.device.base.key", "dev.key"], # Fields may be renamed
        "kismet.device.base.packets.rrd" # Fields can be complex objects

while 1:

    # Get summary of devices
    devices = kr.smart_summary_since(sincets, fields)

Installing the Kismet Rest code

The Python utility class used by all of the python examples is in rest_examples/KismetRest and operates like a normal Python module - to install it, install like you would any other module:

$ cd rest_examples/KismetRest
$ python2 ./ build
$ sudo python2 ./ install

While development is ongoing it's probably a good idea to keep an eye on the changelog for the Python KismetRest interface in git and upgrade it; I'll try to make any API-breaking changes pretty explicit in the commit logs.

Next time...

That should be enough to get started with some simple scripting interfaces; the next post will talk about how to filter results with regular expressions, get info about the general state of the system, and some other fun tricks.

A huge thanks to all who support Kismet on Patreon - if you'd like to help, you can become a patron here!

No comments:

Post a Comment