Getting into it Again
It was not the most auspicious of starts. Two introductory blog posts, a couple of draft posts in the pipeline then nothing… a six year hiatus from the fun. The draft posts no longer apply so have been deleted.
I’ve got a little more spare time now for my own projects so am back again. I have plans for a small number of projects that involve a mixture of software, electronics, programmable interface controllers and micro-computers. I have added a permanently running Linux server to my household and will be using that as well as my Windows PC. I’ve also installed a Power-Over-Ethernet (POE) switch so I can run Ethernet-connected, USB-powered devices anywhere in the house.
The first hardware projects I want to work on are likely to be devices that collect data from sensors. When this data is collected, over time, I want to be able to present it. This very first project is a software solution that can be used to present data over time.
First Project – Presenting Data Over Time
Many years ago, a colleague of mine setup a simple web page that presented the quantity of network traffic going to and from the internet. My colleague used a tool called the Multi-Router Traffic Grapher (MRTG) that created static image files of graphs. One graph presented the data for the last 24 hours, a second for the last month and the third covered the last year. When I asked that same colleague, recently about it, he suggested RRDtool or Cacti.
The requirements I set myself were to create something similar. My goal is to capture a value at regular intervals and present the data in 3 graphs showing 3 amounts of history. The most recent 24 hours, most recent month and most recent year.
The data that I am going to capture is the up-time of the PC. The decimal number of days that it has been turned on and running. It is an easily accessible, changing value and is more useful than presenting a generated random number. The data-point will be captured every 5 minutes. I’ve also decided to make this cross-platform so it runs on the Linux server and my work PC (Windows 10) at the office.
To implement this, I broke it down into a number of steps:
- Examine RRDtool to see if it does what I want. Explore what would be required to achieve it.
- Determine how to extract the up-time as a decimal number of days from a Windows 10 PC.
- Determine how to run this every 5 minutes to load an RRDtool database.
- Determine how to get RRDtool to write out the graphs I want to see.
- Implement an html file to present the graphs.
- Repeat steps 2 to 5 but on the Linux server.
I will not go into much detail of these individual steps as there are plenty of information resources covering the topics in depth. I’m just going to state what each stage involved and note any problems I encountered. The code files for the result are available on GitHub some detail about how to install the various elements of the solution are detailed in the project’s README.md file.
1. RRDtool
I’ve chosen RRDtool rather than Cacti as it is at the lower level of the two. Cacti provides tools and services that extend RRDtool and sit on top of it. I’ll look into Cacti at a future time.
RRDtool does what I want. I’ve followed a few online resources and I found the Getting Started with RRDtool at cuddletech.com to be concise and easy to follow.
2. Determine Up-time of the PC
The same software is being used on Windows and Linux so I needed to extract the up-time in the same format. I chose the decimal number of days because it makes sense on the y-axis of the graphs I want to see. On Linux, getting the up-time involved piping 3 commands:
-
echo “($(date +%s) – $(date -d “$(uptime -s)” +%s))/(60*60*24)”
This writes out the calculation for the up-time in days as a string. It generates a string something like: “(1557700803 – 1557657626)/(60*60*24)”. -
bc -l
This takes the previous string and executes the calculation. -l makes it use the standard maths library. This yields the value “.49973379629629629629” for the above string. - awk ‘{printf “%.15f\n”, $0}’
This formats the above number to 15 decimal places and includes a leading zero. It converts the above to “0.499733796296296”.
On Windows, the following commands are used in Powershell:
- $lb=Get-WmiObject Win32_OperatingSystem | Select-Object LastBootUpTime
This gets the last boot-up time of the PC. When “Fast-startup” turned on, the PC will retain the same last boot-up time between shutdowns using the power button (see here). Turning Fast-startup off will fix this issue. - $lbdt = [Management.ManagementDateTimeConverter]::ToDateTime($lb.LastBootUpTime)
This simply converts the above $lb value to a DateTime value. - $ut = (get-date) – $lbdt
This calculates the difference between now and the last boot-up time. The resulting value is a TimeSpan object. - $strString = “{0}” -f $ut.TotalDays
This sets $string to the decimal number of days of up time to 15 decimal places.
The calculated up-time value is then passed to RRDtool to be stored in the N time-point (meaning the current time) in the database.
3. Scheduled Execution
Windows and Linux both have excellent scheduled job running tools. I’ve used these (cron on Linux and Task Scheduler on Windows) to run the uptime-capture script every 5 minutes when the PC is running.
4. Creating the Graphs
The “rrdtool graph” commands is used to generate the graphs. The only difference between Windows and Linux calls is that Windows does not carry the font file that rrdtool uses by default. A font file (including location) has to be explicitly stated for the Title, Axis, Legend and Units of the graph.
5. Implement HTML to Present the Graphs
I’ve created a simple html file and css file to present the graphs.
As previously mentioned, the code and installation instructions are available on GitHub.
Software Development & Electronics 