REDtalks.live #38 – Grafana Dashboard for BigStats

Having good data is only part of a solution. There’s loads of data on my laptop, for example, that goes unused almost everyday. However, the practice of putting data to work via meaningful representation is how we plan effectively and make informed decisions!

For those who haven’t been following the development of BigStats, its an open source project for ‘exporting’ telemetry off of F5 BIG-IP devices to various destinations and databases. Now, with the release of the ‘Grafana Dashboard for BigStats’, you can start realizing the value of that exported data.

dashboard_overview1

Requires BigStats v0.4 or newer (get that here: https://github.com/f5devcentral/BigStats) installed on your BIG-IP.

Take it for a spin: https://grafana.com/dashboards/7958

Would be great to hear your thoughts!

Twitter: @PearceNathan
https://redtalks.live/BigStats/

 

REDtalks.live #37 – BigStats v0.4 released

BigStats v0.4 has been released! Yay!!! BigStats-300dpi

Way back in REDtalks.live #32 – Stats & Dashboards (May 25th) I demonstrated a prototype solution for getting useful data out of a BIG-IP and into various telemetry pipelines… Well, that was version 0.1, and now we’re up to version 0.4! So, what’s happened since?

Highlights from the RELEASE_NOTES.md file:

  1. We now have a RELEASE_NOTES.md file to track all the awesome!
  2. Support for Apache Kafka message brokers
  3. Support for both AS3 declarations AND traditional configurations (it originally only supported the AS3 declarative interface).
  4. Support for output sizing: Small (VIP Stats only), Medium (VIP + Pool stats), Large (working on this….)
  5. Support for Device Stats (RAM and CPU)
  6. Apache Kafka ‘topic’ config:
    1. All data in one topic, or;
    2. Separate topics per BIG-IP Tenant
  7. Enforced minimum polling interval of 10 seconds (play safe, kids)
  8. Added “config.enabled: true|false” to the config because, sometimes you want some peace and quiet.
  9. Provided a BigStatsSettings object schema (schema validator might be coming soon..)

Here’s an sample stats object built from crawling the running config of a BIG-IP every ‘config.interval: n‘ seconds:

{
        "ip-172-31-1-20-us-west-1-compute-internal": {
                "services": {
                        "Tenant_01/App1": {
                                "/Tenant_01/App3/172.31.4.11:80": {
                                        "clientside_curConns": 0,
                                        "clientside_maxConns": 0,
                                        "clientside_bitsIn": 0,
                                        "clientside_bitsOut": 0,
                                        "clientside_pktsIn": 0,
                                        "clientside_pktsOut": 0,
                                        "/Tenant_01/App1/web_pool1": [
                                                {
                                                        "172.31.10.112:80": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                },
                                                {
                                                        "172.31.10.111:80": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                },
                                                {
                                                        "172.31.10.113:80": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                },
                                                {
                                                        "172.31.10.114:80": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                }
                                        ]
                                }
                        },
                        "Common": {
                                "/Common/172.31.4.200:80": {
                                        "clientside_curConns": 0,
                                        "clientside_maxConns": 0,
                                        "clientside_bitsIn": 0,
                                        "clientside_bitsOut": 0,
                                        "clientside_pktsIn": 0,
                                        "clientside_pktsOut": 0,
                                        "/Common/noAS3_POOL": [
                                                {
                                                        "172.31.10.200:8080": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                },
                                                {
                                                        "172.31.10.201:8080": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                },
                                                {
                                                        "172.31.10.202:8080": {
                                                                "serverside_curConns": 0,
                                                                "serverside_maxConns": 0,
                                                                "serverside_bitsIn": 0,
                                                                "serverside_bitsOut": 0,
                                                                "serverside_pktsIn": 0,
                                                                "serverside_pktsOut": 0,
                                                                "monitorStatus": "down"
                                                        }
                                                }
                                        ]
                                }
                        }
                },
                "device": {
                        "memory": {
                                "memoryTotal": 7574732800,
                                "memoryUsed": 1525312880
                        },
                        "cpu0": {
                                "cpuIdle": 161495459,
                                "cpuIowait": 169763,
                                "cpuSystem": 292088,
                                "cpuUser": 973939
                        },
                        "cpu1": {
                                "cpuIdle": 160343033,
                                "cpuIowait": 68690,
                                "cpuSystem": 426881,
                                "cpuUser": 992052
                        }
                }
        }
}

Here’s the code/docs: https://github.com/f5devcentral/BigStats

In addition to all this awesome, I’ve also shared my lab environment setup details! So, if you want to build a Graphite/Grafana demo like in episode #32, or maybe you want to test against a single node Kafka Broker but don’t know how to do this, well, look no further. Here are all my lab setup instructions: https://gist.github.com/npearce/

Thanks for listening!