DOCSIS FBC Impairment Algorithms - Curious Filter
Filtering is an extremely useful method in DOCSIS cable networks. There are many use cases - for instance one could disable data channels to alleviate other impairments and thus make the connection stable or even just to match the level of service the customer is paying for. One could also disable analog channels completely, and so on. A list of filters by PPC can be found here. This pdf has a list of little devices as the following:
Neat little things!
* Source: PPC
Of course, those filters can also be detected in FBC (Full Band Capture) downstream spectrum scans. While filters are not impairments, they are still useful to detect. For instance, one could keep track of statistics (e.g. total number of filters) across the whole network to see whether the usage of filters increases. In such case, the number of impairments in the network is possibly increasing unnoticed!
Usually, the operator implements a filter on purpose, to simply disable part of the spectrum. Example: Maybe there is a known impairment in the neighborhood that occurs whenever the cable is used close to its full capacity. Filtering part of the spectrum decreases the maximum possible bandwidth, but fixes the issue.
We focus on a certain type of filter, a data-only filter and try to detect it. Since the shape is very pronounced, it is also not too difficult to achieve. Let's take a look.
In the following the algorithm for detecting filters is presented. It is shown that using a tanh is appropriate for representing a filter. First, a definition of tanh is needed. Since tanh is one of the trigonometric functions, the same transformation rules apply.
Now, using scipy's curve_fit and supplying a few carefully crafted initial parameters for a, b, c and d (simply done by fitting a curve on a few scans with known filter impairments and using the average as the initial guess), a tanh can be fitted on a given sample. Then, the resulting tanh (its parameters) and the fitting error can be compared to a hard coded constant. In this case, this would be the initial guess extended by a range; e.g. the parameter c could then be in the range 80 to 150 to be a tanh that represents a filter impairment.
In this example, the following parameters and ranges make sense:
To validate this algorithm, a clustering on 200000 scans was done using sklearn's MiniBatchKmeans and partially fitting 10000 samples at once. This results in roughly 2GB RAM usage compared to over 64GB when using vanilla Kmeans.
Assuming 12 clusters (more ideas were tried – 12 seems roughly the best) the following cluster center for filters can be observed:
A small dataset of 208 samples was drawn from scans that the given MiniBatchKmeans model predicts to be in the above cluster. This dataset was used for adapting the algorithm (to support steeper filters).
Drilling down, it can be seen that most filters have this form, which can be represented by a tanh.
Unfortunately it seems the above cluster still contains a few false positives – a few samples are not filters at all. To make this exact, the 12 clusters would need adjusting. This could be future work. However, for the purpose of creating a small dataset of filters, this is irrelevant.
This wraps up the post. You have hopefully gained some intuition into filtering in DOCSIS cable networks. You have seen how those little devices look like, why we would want to detect them being used and how we go about doing so. You have also observed a Machine Learning method of confirming that our algorithm actually works - K-means clustering. Stay tuned for more technical posts on FBC impairments!
* The figures do not represent a product by Axiros, but merely show the algorithm.
From this blog series:
DOCSIS FBC (Full Band Capture) Impairment Algorithms - Wave
DOCSIS FBC Impairment Algorithms - Adjacency