Introduction

Polestar is a general purpose Internet of Things sensor monitor, data logger, controller and hub. It is designed to be fully extensible on the fly with all configuration or scripting editable through the web interface it provides.

Polestar implements the following main concepts:

• Sensors may be physical or logical. Their values are updated by customisable scripts. Their values are stored in a database periodically.
• Scripts define everything from what sensors are in the system to how those sensors are updated and control scripts acting on sensor values. Scripts can be automatically triggered based on state changes in the system or exposed as "webhooks" so external events can call them.
• Capture all sensor values are captured every 5 minutes (soon to be configurable.) Scripts can use this historical data for analysis or visualisations.
• Logging of important events from scripts.

Installation

Prerequisites - NetKernel and MongoDB.

Install NetKernel

Download NetKernel from 1060research.com. Polestar supports both NetKernel Standard Edition and Enterprise Edition. Follow the appropriate installation procedure.

Go to the apposite tool on management console: http://localhost:1060/tools/apposite/ and apply all available updates.

Also select and install the following packages:

• hds-g2
• email-core

Install MongoDB

Install using the instructions provided on the Mongo DB Site. MongoDB can be run as a service or on the commandline. For deployment of the system as a server it is better to install as a service. Polestar expects to find MongoDB on it's default port 27017 on localhost.

Install Polestar

Clone the git repository

Use gradle and the NetKernel Gradle Plugin to build Polestar. (I hope to get pre-built builds up on github soon.)

Deploy the Polestar jar on NetKernel. Copy the jar to a suitable location then edit [netkernel]/etc/modules.xml. Add a reference to the Polestar jar.

Change passwords

Polestar should now be running. Connect to it on http://[hostname]:8080/polestar/ where [hostname] is the hostname of the server where you have installed it.

You should be prompted for a username and password. There are two usernames hardcoded. One is "admin", the other is "guest". Both have the password "password". You should log into admin and change them both. The guest account has readonly access and can only execute scripts marked as public or guest. It can only view scripts not marked as secret. The admin account has full access.

Change the password by appending altering the URL in your address bar, i.e. http://[hostname]:8080/polestar/changePassword You will be prompted to enter your new password twice. Don't forget it. If you do the only way to reset it is to delete the database and start again. (Actually not quite true, you can just drop the collection named "authentication".)

Configuration script

The first script you need to know about is the the configuration script. Create it by click on Scripts in the navigation bar and the clicking the +New button. It must be named "Configuration".

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;
m=HDSFactory.newDocument()
	.addNode("title","Tony's Deathstar")
	.addNode("subtitle","deathstar automation!")
	.addNode("icon","/polestar/pub/icon/deathstar.png")
	.addNode("show-script-triggers",false)
	.addNode("logRetentionPeriod",1000L*60*60*24*7)
	.pushNode("sensor-ticker")
	.addNode("period",1000L*60*60*24*1)
	.addNode("samplePeriod",1000L*60*60);
context.createResponseFrom(m.toDocument(false));

Optional title, subtitle and icon fields in the configuration are used to customise the navigation bar branding. Sensor-ticker period and samplePeriod are used to customise the ticker shown in the sensor view.

Configuring sensors

Sensors are defined by the sensor script. This is a script named "SensorList". Here is a template with a single sensor:

import org.netkernel.mod.hds.*;
m=HDSFactory.newDocument()
	.pushNode("sensors")
	.pushNode("sensor")
	.addNode("id","urn:random1")
	.addNode("name","Random 1")
	.addNode("units","")
	.addNode("format","%.2f")
	.addNode("icon","/polestar/pub/icon/sun.png")
	.addNode("keywords","test")
	.addNode("errorIfNoReadingsFor",800L);
context.createResponseFrom(m.toDocument(false));

There are seven fields that can be defined for each sensor, let us go through them:

• id mandatory Each sensor must have a unique URN (universal resource name)
• name mandatory This is the human readable display name of the sensor.
• units optional This specifies the display units of the sensors values.
• format optional Format is used to format the raw sensor values for display. Boolean values can use "on" for on/off or "open" for open/close. Formats containing the percent character are formatted according to the specifications on this page.
• icon optional Specifies a custom icon URL. There is a library of icons within Polestar. (See below.)
• chart-period optional Specifies default displayed period of chart in sensor detail view. Can be hour|day|week|month|year.
• chart-type optional Specifies type of rendering of chart in sensor detail view. Can be analog|discrete|stacked|count. Count causes value to be rendered as incremental change. Discrete causes both numeric and maps to be rendered as bar chart. Stacked causes map values to rendered in bar stack.
• keywords optional Comma separated list of keywords that are display in sensor list and can be used to filter sensors.
• errorIfNoReadingsFor optional An integer number of seconds after which an error will be reported in a sensor if it's value is not updated (regardless of if it's value changed) by a script.
• errorIfNotModifiedFor optional An integer number of seconds after which an error will be reported in a sensor if it's value has not been modified.
• errorOnlyAfter optional An integer number of seconds to wait after an error is raised on a sensor before reporting it. This is useful to avoid constant reporting of error transients that quickly resolve themselves.
• errorIfGreaterThan optional A numeric value that is compared against the sensor value. If sensor value is numeric and it's value greater then this sensor is in error. To add hysteresis to this error you can an additional field of errorClearIfLessThan. The sensor will then remain in error status until the value falls below the value specified in this field.
• errorIfLessThan optional A numeric value that is compared against the sensor value. If sensor value is numeric and it's value lesser then this sensor is in error. To add hysteresis to this error you can an additional field of errorClearIfGreaterThan. The sensor will then remain in error status until the value rises above falls the value specified in this field.
• errorIfEquals optional A numeric or boolean value that is compared against the sensor value. If sensor value is equal then this sensor is in error.
• errorIfNotEquals optional A numeric or boolean value that is compared against the sensor value. If sensor value is not equal then this sensor is in error.

Icons

The following icons are built in. Use a URL of the form /polestar/pub/icon/bell.png. You can also use external images or create a Netkernel module with some icons exposed. Ideally icons should be 48x48 pixels.

bell.png

circle-dashed.png

co2.png

deathstar.png

door.png

electricity.png

fan.png

fire.png

gate.png

humidity.png

lightbulb.png

motion.png

network.png

person.png

polestar.png

pressure.png

radiator.png

rain.png

security.png

shower.png

snowflake.png

socket.png

sun.png

switch.png

temperature.png

wind.png

sound.png

car.png

river.png

server.png

letterbox.png

wateringcan.png

toxic.png

dust.png

workload.png

battery.png

oil.png

radiation.png

paw.png

random.png

atom.png

probe.png

plant.png

wireless.png

disk.png

Working with scripts

Polestar is completely configured and customised through the use of scripts. Scripts are code snippets in the Groovy programming language. Scripts are used, as we have seen, to configure sensors, but also for updating the state of those sensors, driving actions based on the sensor states, creating visualizations of the current and historical sensor state.

Each script has the following fields:

• Id each script is assigned a unique id when it is created, you cannot change this.
• Name the display name of the script.
• Triggers the triggers field is primarily a comma separated list of sensor ids. When any of the listed sensors change the script will be executed automatically. In addition to sensor ids the following values are supported:
  • error execute script when a new error is reported by any sensor.
  • startup execute script when Polestar is started.
  • shutdown execute script before Polestar is stopped.
• Keywords the keywords field is a comma separated list of keywords that are displayed in the script list and can be used to filter scripts.
• Periodicity selecting a value other than none here will cause the script to be automatically called periodically.
• Target a sensor to update with response. See on-demand sensors below.
• Access by default a script is private and can only be executed internally, either triggered or from another script, or manually by admin. Change to guest to allow guest user to execute too. Change to public to make this script executable without authentication. (For this to work you need to change the "execute" in the path to "webhook".) This is useful for providing "webhooks" for updating of sensors by external systems. If a script is marked as secret then the guest user can neither execute or view the script.
• Code the code field contains the source code for the script written in the Groovy programming language.

Script context

Every script gets passed a variable called context. The context gives access to all the polestar and NetKernel functionality for more details of the API see the Javadoc.

Script connectivity

There are a number of services made available for the use of scripts. These will be documented in the given examples. In addition you can issue web requests for data and create additional NetKernel modules with services that connect to external hardware devices.

Issuing a web request for data

import org.netkernel.mod.hds.*;
url="http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=INORTHYA1";
doc=context.source(url, IHDSDocument.class).getReader();
outdoor=Float.parseFloat((String)doc.getFirstValue("/current_observation/temp_c"));

req=context.createRequest("active:httpGet");
req.addArgument("url","https://selfsigned.io/polestar/scripts/execute/A476535511DE78B7");
req.addArgument("connectionManager", "active:polestarSelfSignedConnectionManager");
representation=context.issueRequest(req);	
		

Adding hardware connectivity through NetKernel modules

Polestar will import any spaces which make exports available with the type "Polestar" using the dynamic import hook mechanism. Once the spaces are imported you can issue requests to the services within from your scripts.

Sensor update script

Sensors are typically updated on a periodic basis using a script by either specifying a periodicity or by making a script public and allowing an external system to execute the script passing in data. This is called a webhook. Let us have a look at examples of these two approaches. Although these examples show only one sensor multiple sensors can be updated by adding more sensor updates into the update document.

First we need to create a sensor to work with so edit your "SensorList" script and add the following sensor definition:

m.pushNode("sensor").addNode("id","urn:exampleSensor").addNode("name","Example Sensor").popNode();

Polled sensor script

Create a new script and set a periodicity of 30 seconds. Add the following code:

import io.polestar.api.*;
value=Math.random();
context.setSensorValue("urn:exampleSensor",value);

Webhook sensor script

Create a new script, leave the trigger field blank but check the "public" checkbox. Add the following code:

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

queryParam=context.source("httpRequest:/param/value",String.class);
if (queryParam!=null)
{	value=Double.parseDouble(queryParam);
	context.createResponseFrom("sensor updated").setExpiry(INKFResponse.EXPIRY_ALWAYS);
	context.setSensorValue("urn:exampleSensor",value);
}

Registering errors on sensors

Sensors can report an error automatically if you specify an "errorIfNoReadingsFor" value in the sensor definition however sometimes it is desirable to report a specific error directly if one is known. Here is an example based on the polled random number example above which will report an error if the random number is above 0.95.

import org.netkernel.layer0.nkf.*;
value=Math.random();
context.setSensorValue("urn:exampleSensor",value);
if (value>0.95)
	context.setSensorError("urn:exampleSensor","Value is too large");
else
	context.setSensorError("urn:exampleSensor",null);

Control scripts

Control scripts are used to make decisions and take actions based upon the state of sensors. For example turning on/off lights, sending messages or sounding alarms. Control scripts can be triggered periodically or, for more immediate action, when sensors that effect it's outcome change.

Control scripts usual want to read the current state of sensors, detect changes and them perform actions or update further "logical" sensors.

Here is an example of reading sensor values:

import org.netkernel.layer0.nkf.*;

//example of using the value and last modified time of a sensor
atHome=context.getSensorValue("urn:atHome");
lastAtHome=context.getSensorLastModified("urn:atHome");
now=System.currentTimeMillis();
if (atHome && now-lastAtHome>24*60*60*1000)
{	context.logRaw(INKFLocale.WARNING,"Did you know you really need to get out more?");
}

booleanChangeDetect

The problem with the above script is that after 24 hours of being at home you will be warned everytime the script is run. Usually you determine a condition and then act when that condition becomes true or stops being true. To do this we need to remember state to know the state of the condition last time the script ran.

import org.netkernel.layer0.nkf.*;

//example of using the value and last modified time of a sensor
atHome=context.getSensorValue("urn:atHome");
lastAtHome=context.getSensorLastModified("urn:atHome");
now=System.currentTimeMillis();
atHomeToLong=atHome && now-lastAtHome>24*60*60*1000;
status=context.booleanChangeDetect(atHomeToLong,0,0,"atHomeToLong");
changed=status[1];
if (changed && atHomeToLong)	
{	context.logRaw(INKFLocale.LEVEL_INFO,"Did you know you really need to get out more?");
}

analogueChangeDetect

There is also the analogue equivalent of booleanChangeDetect() called analogueChangeDetect(). It works in a similar way but looks for a numeric value to be above a trigger level for the condition to become true.

import org.netkernel.layer0.nkf.*;

light=context.getSensorValue("urn:daylight:db");
status=context.analogueChangeDetect(light, -55.0D, -50.0D, "light");
changed=status[1];
isDay=status[0];
if (changed)
{	if (isDay)
	{	context.logRaw(INKFLocale.LEVEL_INFO,"Dawn");
	}
	else
	{	context.logRaw(INKFLocale.LEVEL_INFO,"Dusk");
	}
}

atMostEvery

Sometimes it is useful to stop a particular action from happening too often. For example my kids love to press the door bell about 10 times but if I'm not at home I only want to be send a message if the door bell rings for the first time in 5 minutes.

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

boolean requireQuiet=true; // timer resets everytime bell is rung
newEvent=context.atMostEvery(period: 300000L, requireQuiet, "bell");

if (newEvent)
{	context.logRaw(INKFLocale.LEVEL_INFO,"This might actually be a new person ringing");
}

Script persistent state

In these examples above atMostEvery, analogueChangeDetect, and booleanChangeDetect all require state to be stored between invocations in order to function. Each script maintains a state resource automatically between invocations.

You can access values from that state easily using the API:

default=0L;
value=context.getScriptState("myVar",default)
value+=1;
context.setScriptState("myVar",value);

You can also access the scripts persistent state directly as a HDS document:

import org.netkernel.layer0.nkf.*;
import org.netkernel.mod.hds.*;

state=context.source("arg:state",IHDSDocument.class).getMutableClone();
count=state.getFirstValueOrNull("/state/count");
if (count==null)
	count=0;
else
	count++;
state.createIfNotExists("state/count").setValue(count);
context.sink("arg:state",state.toDocument(true));
resp=context.createResponseFrom(state.toString());
resp.setExpiry(INKFResponse.EXPIRY_ALWAYS);

Calling other scripts

One script can call another script. This is useful for generic services such as sending a message. You can pass arbitrary arguments between scripts:

result=context.callScriptById("47CCD102175BC395",
	"argument1","abc"
);

result=context.callScriptByName("Send Message",
	"message","something happened"
);

argument1=context.source("arg:argument1");

Logging

Scripts can write to the log:

import org.netkernel.layer0.nkf.*;
context.logRaw(INKFLocale.LEVEL_INFO,"Something happened");
context.logRaw(INKFLocale.LEVEL_WARNING,"Something bad happened");

Homepage script

To customize the homepage create a script which returns XHTML with the name "Homepage". You can use this page to put visualizations or status icons.

import org.netkernel.layer0.nkf.*;
s="""<div class='container'>
<iframe src="/polestar/scripts/execute/287CB3EAD99CA3A6" frameborder="0" width="280" height="160" scrolling="no">_</iframe>
</div>""";

resp=context.createResponseFrom(s.toString());
resp.setHeader("polestar_wrap",true);
	

On Demand Sensors

On demand sensors are sensors which are updated by scripts when they are queried. Maybe the term sensor is being stretched a bit far. These sensors represent a single value over time and it's value is usually defined from data available in other sensors. For example we could define an on-demand sensor for maximum temperature in day. This sensor will update at midnight each day with a value of the maximum temperature over that day. So typical these sensors provide aggregated higher level data.

To define an on demand sensor define a sensor in the Sensors script as usual (see above.) Then create a script which will update the sensor. This script should have a periodicity of how often the sensor should update and target of the on-demand sensor. The script will receive a timestamp argument of a Long millisecond timestamp when it is called. It should return a single sensor value as it's response.

Backup/Restore

You can backup and restore all the scripts through the and buttons on the headerbar of the script list (only available on desktop not mobile.)

Inhibit Polling

I've noticed that restarting polestar after significant hardware changes can lead to a lockup because the polling continues to happen and with delays such as connecting to hardward or networking timouts the requests get backlogged to a degree that recovery and editing of scripts is impossible. To workaround this there is a java system property that can be set at boot time which inhibits all polling scripts from automatically executing. Add the following to [install]/bin/jvmsettings.cnf