Speedtest and graph my internet uplink :-)

Yeah, got it working, testing my internet link now, every 15 minutes 😉

There are several howto’s(link is external) around on how to install and use speedtest-cli(link is external), the commandline flavor of Ookla -based test site(link is external).

I wanted something more, not much but being curious on what happens around in my small systems, I stapled together this pile of spaghetti…

It uses speedtest-cli(link is external) as initial data input, RRDtool(link is external) as visualizer, bash and misc programs related to it, and whatever (link is external)webserver for publishing. I’m not guru on bash, mentioning this ’cause I do enjoy good omg-w4tDidUDo -spam from the ones that are…

1. Install speedtest-cli and make sure it works:

wget https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py

sudo mv speedtest_cli.py /usr/local/bin/speedtest-cli

sudo chmod a+rx /usr/local/bin/speedtest-cli

speedtest-cli

(in case you didn’t figure above out, just head to the website(link is external) and don’t bother reading more)

After final command you should get results on your ping, upload and download. Let’s move forward.

2. Write code that figures out your hometown server and uses it to test speed and gives you a nice pic to share the results. (using vi is considered ultrageeky, reason why I nano)

mkdir -p /usr/src/scripts/speedtest

nano /usr/src/scripts/speedtest/get_my_speedtest_status.sh

#!/bin/bash
# This is ‘what speedtest-cli –server XXXXX –share’ typically gives:
#
# Retrieving speedtest.net configuration…
# Retrieving speedtest.net server list…
# Testing from Masternet Inc (185.9.12.1)…
# Hosted by Masternet Inc (Eastwesttown) [1.41 km]: 15.286 ms
# Testing download speed………………………………….
# Download: 98.56 Mbit/s
# Testing upload speed…………………………………………..
# Upload: 1.91 Mbit/s
# Share results: https://www.speedtest.net/result/076719.png

cd /usr/src/scripts/speedtest
DATE=`/bin/date “+%Y.%m.%d-%H%M%S”`

#this a regexp to figure out numbers most of the kind

re=’^-?[0-9]+([.][0-9]+)?$’
MYPING=0
MYUPLOAD=0
MYDOWNLOAD=0

# Replace this with our known city

MYTOWN=”Eastwesttown”

MYSERVER=`/usr/local/bin/speedtest-cli –list | grep -i $MYTOWN|awk ‘{print $1}’|sed -e “s/)//g”`
#echo “server=$MYSERVER”
if [[ $MYSERVER =~ $re ]] ;
then
MYRETURN=`/usr/local/bin/speedtest-cli –server $MYSERVER –share`
#       echo “return=$MYRETURN”
echo “$MYRETURN”>tmp
#       cat tmp
MYPING=`cat tmp|grep -i “^Hosted by”|awk ‘{print $(NF-1)}’`
#       echo “ping=$MYPING”
MYUPLOAD=`cat tmp|grep -i “^Download”|awk ‘{print $(NF-1)}’`
#       echo “uplink=$MYUPLOAD”
MYDOWNLOAD=`cat tmp|grep -i “^Upload”|awk ‘{print $(NF-1)}’`
#       echo “downlink=$MYDOWNLOAD”
MYLINK=`cat tmp|grep -i “^Share results”|awk ‘{print $(NF)}’`
echo $MYLINK>/var/www/speedtest/lastlink2pic
#       echo “piclink=$MYLINK”
fi

if [[ $MYPING =~ $re ]] && [[ $MYUPLOAD =~ $re ]] && [[ $MYDOWNLOAD =~ $re ]] ;
then
#       echo “Got values:”
echo “$MYPING”
echo “$MYUPLOAD”
echo “$MYDOWNLOAD”
else
#       echo “NO values:”
echo “0”
echo “0”
echo “0”
fi

And then save.

3. Make another code that uses above to dump the data into RRD:

nano /usr/src/scripts/speedtest/update_my_speedtest_rrd
#!/bin/bash
DATE=`/bin/date “+%Y.%m.%d %H:%M:%S”`
MYMONTH=`/bin/date “+%Y-%m-%d”`
MYPATH=”/var/www/speedtest/speedtest.rrd”
/home/root/scripts/speedtest/get_speedtest_status.sh>/var/www/speedtest/raw.txt
MYDATA=`cat /var/www/speedtest/raw.txt|tail -n20|gawk ‘BEGIN{ORS=”:”;i=0;}{print;i=i+1;}END{if(i<3) for(;i<3;i++)print “-400”;}’|sed -e ‘s/:$//g’`
echo -e “$MYDATA\r\n”
echo -e “$DATE:$MYDATA”>>/var/www/speedtest/log-$MYMONTH.txt
rrdtool update $MYPATH N:$MYDATA

and save,

4. Now create initial RRD database, you might want to read few faq’s around the net on rrdtool create.

nano /usr/src/scripts/speedtest/create_my_speedtest_rrd

#!/bin/bash
DATE=`/bin/date “+%Y.%m.%d-%H%M%S”`

MYPATH=”/var/www/speedtest/speedtest.rrd”

rrdtool create “$MYPATH” \
–start N –step 3600 \
DS:MY-PING:GAUGE:7200:0:6000 \
DS:MY-DOWNLINK:GAUGE:7200:0:3000 \
DS:MY-UPLINK:GAUGE:7200:0:3000 \

RRA:MIN:0.5:60:8785 \
RRA:MAX:0.5:60:8785 \
RRA:AVERAGE:0.5:1:8785

 

save and run:

mkdir -p /var/www/speedtest

bash /usr/src/scripts/speedtest/create_my_speedtest_rrd

5. now create code for updating the graphs:

 nano /usr/src/scripts/speedtest/graph_speedtest_rrd
#!/bin/bash

DATE=`/bin/date “+%Y.%m.%d-%H%M%S”`
MYRRD=”/var/www/speedtest/speedtest.rrd”
MYPATH=”/var/www/speedtest/”
cd $MYPATH
# last 12hours: –end now –start end-43200s
# last 24hours: –end now –start end-86400s
# last week: –end now –start end-604800s
# last month: –end now –start end-2419200s
# last year: –end now –start end-31536000s

PARAMS_ARRAY=(“–” “-s e-12h” “-s e-24h” “-s e-1w” “-s e-1m” “-s e-1y”)
TITLE_ARRAY=(“” “Last 12h” “Last 24h” “Last week” “Last month” “Last year”)
FILENAME_ARRAY=(“a” “12h” “24h” “week” “month” “year”)

#DS:MY-PING:GAUGE:120:0:6000 \
#DS:MY-DOWNLINK:GAUGE:120:0:3000 \
#DS:MY-UPLINK:GAUGE:120:0:3000 \

for N in {0..5}
do
rrdtool graphv -J “$MYPATH”my-ping-“${FILENAME_ARRAY[$N]}”.png -a PNG –title=”Ping ${TITLE_ARRAY[$N])}” \
–vertical-label “ms” \
“${PARAMS_ARRAY[$N]}” \
“DEF:T=speedtest.rrd:MY-PING:AVERAGE” \
“DEF:TMIN=speedtest.rrd:MY-PING:MIN” \
“DEF:TMAX=speedtest.rrd:MY-PING:MAX” \
‘AREA:TMIN#00ff00:Ping min’ \
‘AREA:TMIN#ffb6c1:Ping max’ \
‘LINE1:T#ff0000:Ping’ \
‘GPRINT:T:LAST:Last Ping\: %2.1lf ms’ \
‘GPRINT:TMAX:MAX:Max Ping\: %2.1lf ms’ \
‘GPRINT:TMIN:MIN:Min Ping\: %2.1lf ms’

rrdtool graphv -J “$MYPATH”my-downlink-“${FILENAME_ARRAY[$N]}”.png -a PNG –title=”Downlink speed ${TITLE_ARRAY[$N])}” \
–vertical-label “Mbit/s” \
“${PARAMS_ARRAY[$N]}” \
“DEF:T=speedtest.rrd:MY-DOWNLINK:AVERAGE” \
“DEF:TMIN=speedtest.rrd:MY-DOWNLINK:MIN” \
“DEF:TMAX=speedtest.rrd:MY-DOWNLINK:MAX” \
‘AREA:TMIN#00ff00:Down min’ \
‘AREA:TMIN#ffb6c1:Down max’ \
‘LINE1:T#ff0000:Downlink’ \
‘GPRINT:T:LAST:Last downlink\: %2.1lf Mbit/s’ \
‘GPRINT:TMAX:MAX:Max downlink\: %2.1lf Mbit/s’ \
‘GPRINT:TMIN:MIN:Min donwlink\: %2.1lf Mbit/s’

rrdtool graphv -J “$MYPATH”my-uplink-“${FILENAME_ARRAY[$N]}”.png -a PNG –title=”Uplink speed ${TITLE_ARRAY[$N])}” \
–vertical-label “Mbit/s” \
“${PARAMS_ARRAY[$N]}” \
“DEF:T=speedtest.rrd:MY-UPLINK:AVERAGE” \
“DEF:TMIN=speedtest.rrd:MY-UPLINK:MIN” \
“DEF:TMAX=speedtest.rrd:MY-UPLINK:MAX” \
‘AREA:TMIN#00ff00:Up min’ \
‘AREA:TMIN#ffb6c1:Up max’ \
‘LINE1:T#ff0000:Uplink’ \
‘GPRINT:T:LAST:Last uplink\: %2.1lf Mbit/s’ \
‘GPRINT:TMAX:MAX:Max uplink\: %2.1lf Mbit/s’ \
‘GPRINT:TMIN:MIN:Min uplink\: %2.1lf Mbit/s’

done

6. Easy, eh? Now prepare minimalistic index.php:

nano  /var/www/speedtest/index.php
<html>
<table>
<tr>
<td></td>
<td align=”center”>
<img src=”<?php require_once(“lastlink2pic”);?>”>
</td>
<td></td>
</tr>
<?php
$PARAMS_ARRAY=array(” “,” –end now –start end-3600s “,” –end now –start end-28800s “,” –end now –start end-86400s “,” –end now –start end-604800s “,” –end now –start end-2419200s “,” –end now –start end-3153600s “);
$TITLE_ARRAY=array(” “,”Last 12″,”Last 24h”,”Last week”,”Last month”,”Last year”);
$FILENAME_ARRAY=array(“12h”,”24h”,”week”,”month”,”year”);

for ($jj = 0; $jj<=4; $jj++) {
echo “<tr>\n”;
echo “\t<td><img src=\”my-ping-“.$FILENAME_ARRAY[$jj].”.png\”></td>\n”;
echo “\t<td><img src=\”my-uplink-“.$FILENAME_ARRAY[$jj].”.png\”></td>\n”;
echo “\t<td><img src=\”my-downlink-“.$FILENAME_ARRAY[$jj].”.png\”></td>\n”;
echo “</tr>\n”;
}
?>
</html>

7. almost done, set things up in crontab:

sudo crontab -e

1 * * * * nice -n 19 bash /usr/src/scripts/speedtest/update_my_speedtest_rrd>/dev/null 2>&1
3 * * * * nice -n 19 bash /usr/src/scripts/speedtest/graph_speedtest_rrd>/dev/null 2>&1

8. Done. Now wait and later check the site, it should look a bit like this:

Thanks to http://jyge.fi/node/229 of Kirjautuminen

Monitoring time capsule bandwidth via MRTG? | Apple Support Communities

Source: Monitoring time capsule bandwidth via MRTG? | Apple Support Communities

I just had to figure this out on a Time Capsule, and since there is a lack of information on the Internet about the new models I did a bunch of trial and error.

Here’s what I found:

  1. They report the wrong interface speeds via SNMP, which is why cfgmaker is confused about the speeds.
  2. They do SNMPv2 just fine.
  3. cfgmaker is otherwise accurate.

I believe the interfaces are:

mgi0   – LAN
mgi1  – WAN
bwl0  – 2.4 Ghz Radio
bwl1  – 5.0 Ghz Radio
lo0  –  loopback
wlan0  – 2.4 Ghz Network
wlan1  – 5.0 Ghz Network
pppoe0  – PPP Over Ethernet Client
bridge0  – Internal bridge between wired an wireless
gif0  – 6in4 Tunnel
stf0  – 6to4 Tunneling

The architecture seems to be that mgi0 is attached to the 3-port switch for the LAN ports, so they all aggregate under that one interface.  bwl0/wlan0 always report the same bits for 2.4G clients, bwl1/wlan1 the same bits for 5G clients.  bridge0 bridges between mgi0, bwl0/wlan0, and bwl1/wlan1.  I don’t use pppoe, so I can’t confirm what that shows, but I assume it’s the traffic _inside_ the PPPOE session on the WAN, if used.  gif0 is 6in4 tunneling, and stf0 is 6to4 tunneling.

The only thing I’m unsure about is the WiFi interfaces are set to 300Mbps/max in this config.  I am unclear if this box can do 450Mbps throughput with the right clients.

Time+Capsule+Architecture

mgi0 is the WAN.

 

mgi1 is connected to an internal 4 port GigE switch that appears to not have any individual pollable ports via SNMP.  The other three ports are exposed on the back of the unit as the LAN ports.

 

wlan0/wlan1 are the data connections for the 2.4Ghz and 5Ghz radios, respectively.

 

bwl0/bwl1 appear to be some sort of managmenet interface for the radios.  They report outbound traffic only, never inbound.  Traffic on these interfaces increases as traffic increases on the wlan0/wlan1 interfaces, but at a much lower rate.

 

bridge0 is a software bridge that sits between wlan0, wlan1, and mgi1 briding the WiFi’s and LAN ports together.  Traffic between any two of these three will pass the bridge interface.

 

pppoe0 is for tunneling PPP over Ethernet, if you use that on the WAN port, basically in that case mgi0 would be with PPPoE overhead, and pppoe0 would be without.

 

gif0 is 6in4 tunneling, if you have for instance a tunnel broker tunnel.  Again, this would be traffic with no overhead, then as the packets go out mgi0 they would count there with the tunnel overhead.

 

sth0 is 6to4 (automatic) tunneling.  Again, this would be traffic with no overhead, then as the packets go out mgi0 they would count there with the tunnel overhead.

 

Lastly, there’s a SATA port to the disk in a Time Capsule.

Change the changes

Handig, een overzichtje van alles mbt change! (BRON: http://aardbron.nl/change-the-changes/)

Een praatplaat uit 2000 over:

  • de cirkel Bewust Bekwaam → Onbewust Bekwaam → Onbewust Onbekwaam → Bewust Onbekwaam → Bewust Bekwaam → …; in het Engels You Know That You Know → You Don’t Know That You Know → You Don’t Know That You Don’t Know → You Know That You Don’t Know → You Know That You Know → …
  • leiderschap, team building en kampioenen;
  • de fundamentele—en voor velen confronterende—vraag die je elk ander teamlid kan stellen:
    Wil jij mij helpen beter te worden dan dat jij bent?

Wat mij triggerde om deze plaat weer boven water te halen was Dan Roam’s artikel Seeing What We Don’t Know.

You are here

If you don’t know where you are going, you might end up somewhere else.

—Yogi Berra
If you don’t know where you are going, any path will take you there.

—Alice in Wonderland

Vision, Strategy, Execution

  • Change the Changes
  • State of Flow
  • Explore & Innovate
  • Increasing Returns
    • Positive Feedback
    • Self-reinforcement
    • Attracts Staff and Business and Wealth
    • Captures & Cultivates Collective Knowledge & Intelligence

Vision

  • Architectural Principles (5-10 year view)
  • Take Risks
  • Lead the Pack
  • Show Direction

Strategy

  • Informatiebeleid (2-5 year view)
  • Mergers & Acquisitions

Execution

  • Planning (1-2 year view)
  • Process
  • Goals & Milestones
  • Short, iterative processes; driving a car

Flow Spiral

Goal is to spiral yourself to ever higher states of consciousness, to self-unfold, to realize your full potential—as an individual, as an organization, and as a society—by taking many small steps rather than a few big ones. Small steps are less painful.

YDKTYK: State of Flow

  • You Don’t Know That You Know
  • Process feels “effortless”
  • On the edge between Chaos and Order (Chaordic State)
  • Best place to be
  • Optimal balance between Effort & Comfort
  • Danger of “switching” to YDKTYDK Comfort State

YDKTYDK: Extreme Discomfort & Death

  • You Don’t Know That You Don’t Know
  • Comfortably Numb
  • Cruise Control
  • Danger Zone!!!
  • Self-affirmative; more secure; less risk
  • Do the things you’ve so often done already
  • Over-structured (bureaucracy)
  • Further strengthen Comfortable Numb State: Equilibrium or Complete Order: R.I.P.
  • Outsmarted or Outpaced
  • Wake-up call by competition
  • Glide into total Chaos & Panic (“brandjes blussen”: R.I.P.
  • Discomfort to the Extreme
  • Best way out: Transition to YKTYDK State

YKTYDK: Therapy, Rehabilitation and Revalidation

  • You Know That You Don’t Know
  • Train, Coach, Facilitate [often by External Consultants]
  • Effort Required! Investment in Time & Material
  • How? Many small steps
    • Avoid getting swamped, panic; c.f. your first driving lesson
    • Too large a change will increase resistance; Try to avoid by taking many small steps rather than one big step
    • Loosing weight: not 10 kg in a year, but 1 kg every month: short result-driven iterations
    • Focus on Genot Gemak Gewin
    • Be aware of Affect, Behavior, Cognition (Gevoel, Gedrag, Gedachten)
      • Identify typical patterns of ABC
      • Understand how these unwanted patterns emerged from the past
      • Describe the complete ABC chain (this is time intensive)
      • Create a break-through strategy to address the issue
      • Practice new ABC over and over again (therapy)
      • Changing Affect is hardest because of emotional resistance
    • Focus on positive feedback, increasing returns
  • Transition to YKTYK State

YKTYK: Repeat, over and over again

  • You Know That You Know
  • Effort still required
  • Exercise, Execute, over and over again, until you get into a State of Flow again

State of Flow: Flow = Team + Process

  • Charter: have a common direction and still remain individual distinct
    • Avoids Not Invented Here Syndrome
  • Objective
    • Change (better still: Change the Changes)
    • Do New Things
  • Threat: internal competition and envy
  • Key Strengths
    • Diversity: Total is much much more than the sum of its parts!
    • Symbiosis: State of Living Together or Cooperative Relationship
    • Fully Interlocking
  • Killing Teams:
    • Envy
    • Reject others
    • Feel victimized by others
    • Feel dependent on others
  • Killer Teams:
    • Cultivate Diversity
      • Therefore advance faster
      • Are Constantly Getting Better in Getting Better
    • Values the others
    • Admires the others
    • Compliments the others (be/ask about specifics)
    • Emphasize the Importance of Contribution of Others to Your Performance
    • Survive Disasters
    • Fully Respect other Team Members
    • Are Committed
    • Give Good Feedback
    • Are Involved
    • Take many small steps, rather than a few big ones
  • Topper
    • Wins the Golden Medals
    • Creates Positive Attention for the Whole Team
    • Feels Responsible for His Effect on Whole Team
  • Individual Team Members
    • Search Teams
    • Want Team to Thrive & Survive
    • Make others competent
    • Ask: “Will you help me become better than you are?”
    • Train in Skills they don’t already have
    • Excels in some Specific Areas
    • Know that they’ll die if the team dies
  • Process
    • Process is most interesting, not the goal itself
    • Explore!!!
    • The Pleasure is in the Climbing, You’re Only at the Top for a Short Time
    • Software Process Improvement is a process in itself
    • Results (goals & milestones met) are the byproducts of process

All this is recursive, fractal

  • Remember that Process Improvement is a Process in Itself

Kijken Kijken kijken!

Hoe simpel kan het zijn: inspireren, dat is pas groot leiderschap!

Je mag daar even tekenen…

IMG_0007

Hij is binnen! BICT mag ik nu voeren; Bachelor of ICT 🙂

Al het harde werk, gevloek, uitstelgedrag, soggen, etc. etc. ten spijt: afgestudeerd met goede cijfers en een diploma om op te hangen. Heeel blij zeg maar!!!

En wat doet dat met deze (toch al niet echt diepzinnige) blog? Doel was om mee te schrijven met mijn studie. Dat is denk ik maar een heel klein beetje gelukt, nee een schrijver ben ik niet.

Voorlopig hier vooral links en kladjes over van alles en nog wat. Niet zo serieus zeg maar.

Jeroen

Er is licht aan het eind van de studietunnel!

Studenten, werkende studenten, blijf volhouden, maak het af, niets is zo zonde als het weggooien van al je inspanningen!

In september 2010 ben ik op deze blog (heel af en toe) mijn studie voortgang gaan verslaan. Het afgelopen half jaar (grofweg) ben ik bezig geweest met de laatste etappe: het schrijven van mijn eindscriptie. En ja, dat process is er een van veel vallen en opstaan, schrijven en herschrijven, die bron tegenkomen die veel mooier, duidelijker schrijft en oh wat een mooie opbouw….

Focus op je eigen verhaal. Ga niet teveel veranderen. Bijt je erin vast. en zorg voor een goede begeleider met eigen deadlines: echt waar als je het tot je scriptie hebt geschopt kun je het!

Ik ben nu aan het wachten op de uitnodiging voor mijn verdediging. Als ik daar dan sta rond ik 5 jaar studeren naast een drukke baan af. Een studie waarin ik veel mensen heb leren kennen, veel geleerd heb van docenten, maar ook vooral van de soms heel persoonlijke praktijk ervaringen van mijn mede studenten.

Iedereen die hier een deel van is geweest, dank je wel en jullie ook gefeliciteerd (of misschien door het lezen van dit open verhaaltje, veel succes met afmaken)!!!