Introduction

Polestar is a general purpose Internet of Things sensor monitor, data logger, controller and hub. It is designed to be fully extensible on the fly with all configuration or scripting editable through the web interface it provides.

Polestar implements the following main concepts:

Installation

Prerequisites - NetKernel and MongoDB.

Install NetKernel

Download NetKernel from 1060research.com. Polestar supports both NetKernel Standard Edition and Enterprise Edition. Follow the appropriate installation procedure.

Go to the apposite tool on management console: http://localhost:1060/tools/apposite/ and apply all available updates.

Also select and install the following packages:

optional
Configure NetKernel to run as a service.

Install MongoDB

Install using the instructions provided on the Mongo DB Site. MongoDB can be run as a service or on the commandline. For deployment of the system as a server it is better to install as a service. Polestar expects to find MongoDB on it's default port 27017 on localhost.

Install Polestar

Clone the git repository

Use gradle and the NetKernel Gradle Plugin to build Polestar. (I hope to get pre-built builds up on github soon.)

Deploy the Polestar jar on NetKernel. Copy the jar to a suitable location then edit [netkernel]/etc/modules.xml. Add a reference to the Polestar jar.

Change passwords

Polestar should now be running. Connect to it on http://[hostname]:8080/polestar/ where [hostname] is the hostname of the server where you have installed it.

You should be prompted for a username and password. There are two usernames hardcoded. One is "admin", the other is "guest". Both have the password "password". You should log into admin and change them both. The guest account has readonly access and can only execute scripts marked as public or guest. It can only view scripts not marked as secret. The admin account has full access.

Change the password by appending altering the URL in your address bar, i.e. http://[hostname]:8080/polestar/changePassword You will be prompted to enter your new password twice. Don't forget it. If you do the only way to reset it is to delete the database and start again. (Actually not quite true, you can just drop the collection named "authentication".)

Configuration script

Everything in Polestar is driven by scripts.

We hope to have a set of standard scripts initialised when Polestar is installed. At the moment you can manually create scripts from the information in this guide.

Alternatively there are set of scripts to download and install from the Github repository.

We hope to have a set of standard scripts either installed by default or available in the repo. At the moment you must manually create them to seed Polestar.

All the scripts are coded using the Groovy programming language.

The first script you need to know about is the the configuration script. Create it by click on Scripts in the navigation bar and the clicking the +New button. It must be named "Configuration".

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;
m=HDSFactory.newDocument()
	.addNode("title","Tony's Deathstar")
	.addNode("subtitle","deathstar automation!")
	.addNode("icon","/polestar/pub/icon/deathstar.png")
	.addNode("show-script-triggers",false)
	.pushNode("sensor-ticker")
	.addNode("period",1000L*60*60*24*1)
	.addNode("merge",6);
context.createResponseFrom(m.toDocument(false));

Optional title, subtitle and icon fields in the configuration are used to customise the navigation bar branding. Sensor-ticker period and merge are used to customise the ticker shown in the sensor view.

Configuring sensors

Sensors are defined by the sensor script. This is a script named "SensorList". Here is a template with a single sensor:

import org.netkernel.mod.hds.*;
m=HDSFactory.newDocument()
	.pushNode("sensors")
	.pushNode("sensor")
	.addNode("id","urn:random1")
	.addNode("name","Random 1")
	.addNode("units","")
	.addNode("format","%.2f")
	.addNode("icon","/polestar/pub/icon/sun.png")
	.addNode("keywords","test")
	.addNode("errorIfNoReadingsFor",800L);
context.createResponseFrom(m.toDocument(false));

There are seven fields that can be defined for each sensor, let us go through them:

Icons

The following icons are built in. Use a URL of the form /polestar/pub/icon/bell.png. You can also use external images or create a Netkernel module with some icons exposed. Ideally icons should be 48x48 pixels.

bell.png
circle-dashed.png
co2.png
deathstar.png
door.png
electricity.png
fan.png
fire.png
gate.png
humidity.png
lightbulb.png
motion.png
network.png
person.png
polestar.png
pressure.png
radiator.png
rain.png
security.png
shower.png
snowflake.png
socket.png
sun.png
switch.png
temperature.png
wind.png
sound.png
car.png
river.png
server.png
letterbox.png
wateringcan.png
toxic.png
dust.png
workload.png
battery.png
oil.png
radiation.png
paw.png

Working with scripts

Polestar is completely configured and customised through the use of scripts. Scripts are code snippets in the Groovy programming language. Scripts are used, as we have seen, to configure sensors, but also for updating the state of those sensors, driving actions based on the sensor states, creating visualizations of the current and historical sensor state.

Each script has the following fields:

Script connectivity

There are a number of services made available for the use of scripts. These will be documented in the given examples. In addition you can issue web requests for data and create additional NetKernel modules with services that connect to external hardware devices.

Issuing a web request for data

import org.netkernel.mod.hds.*;
url="http://api.wunderground.com/weatherstation/WXCurrentObXML.asp?ID=INORTHYA1";
doc=context.source(url, IHDSDocument.class).getReader();
outdoor=Float.parseFloat((String)doc.getFirstValue("/current_observation/temp_c"));

Connecting to self signed HTTPS servers

Self signed servers are very useful because you can avoid the complexity of setting up real certificates for your servers and yet still have an encryped connection. However by default Java doesn't allow you to connect to self signed servers. Here is a simple solution, Polestar has an alternate connection manager to allow this:

req=context.createRequest("active:httpGet");
req.addArgument("url","https://selfsigned.io/polestar/scripts/execute/A476535511DE78B7");
req.addArgument("connectionManager", "active:polestarSelfSignedConnectionManager");
representation=context.issueRequest(req);	
		

Adding hardware connectivity through NetKernel modules

Polestar will import any spaces which make exports available with the type "Polestar" using the dynamic import hook mechanism. Once the spaces are imported you can issue requests to the services within from your scripts.

Sensor update script

Sensors are typically updated on a periodic basis using a script with either a 1s, 30s or 5m trigger or by making a script public and allowing an external system to execute the script passing in data. This is called a webhook. Let us have a look at examples of these two approaches. Although these examples show only one sensor multiple sensors can be updated by adding more sensor updates into the update document.

First we need to create a sensor to work with so edit your "SensorList" script and add the following sensor definition:

m.pushNode("sensor").addNode("id","urn:exampleSensor").addNode("name","Example Sensor").popNode();

Polled sensor script

Create a new script and enter a trigger of 30s. Add the following code:

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

m=HDSFactory.newDocument()
	.pushNode("sensors")
	.pushNode("sensor")
	.addNode("id","urn:exampleSensor")
	.addNode("value", Math.random());

req=context.createRequest("active:polestarSensorUpdate");
req.addArgumentByValue("state",m.toDocument(false));
context.issueRequest(req);
This script will run every 30 seconds and update this single sensor with a random number between 0 and 1.

Webhook sensor script

Create a new script, leave the trigger field blank but check the "public" checkbox. Add the following code:

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

queryParam=context.source("httpRequest:/param/value",String.class);
if (queryParam!=null)
{	value=Double.parseDouble(queryParam);
	context.sink("arg:state",queryParam);
	context.createResponseFrom("sensor updated").setExpiry(INKFResponse.EXPIRY_ALWAYS);

	m=HDSFactory.newDocument()
		.pushNode("sensors")
		.pushNode("sensor")
		.addNode("id","urn:exampleSensor")
		.addNode("value", value );
	req=context.createRequest("active:polestarSensorUpdate");
	req.addArgumentByValue("state",m.toDocument(false));
	context.issueRequest(req);
}
To update the value we now need to issue a web request to the script. So for example if our script has an id of EAC75E961241D889 then we can issue the following request in our web browser addressbar http://localhost:8080/polestar/scripts/execute/EAC75E961241D889?value=1.0.

Registering errors on sensors

Sensors can report an error automatically if you specify an "errorIfNoReadingsFor" value in the sensor definition however sometimes it is desirable to report a specific error directly if one is known. Here is an example based on the polled random number example above which will report an error if the random number is above 0.95.

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;
value=Math.random();
m=HDSFactory.newDocument()
	.pushNode("sensors")
	.pushNode("sensor")
	.addNode("id","urn:exampleSensor")
	.addNode("value",value);
if (value>0.95)
	m.addNode("error","Value is too large");	

req=context.createRequest("active:polestarSensorUpdate");
req.addArgumentByValue("state",m.toDocument(false));
context.issueRequest(req);

Control scripts

Control scripts are used to make decisions and take actions based upon the state of sensors. For example turning on/off lights, sending messages or sounding alarms. Control scripts can be triggered periodically or, for more immediate action, when sensors that effect it's outcome change.

Control scripts usual want to read the current state of sensors, detect changes and them perform actions or update further "logical" sensors.

Here is an example of reading sensor values:

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

//example of using the value and last modified time of a sensor
sensors=context.source("active:polestarSensorState",IHDSDocument.class).getReader();
atHome=sensors.getFirstValue("key('byId','urn:atHome')/value");
lastAtHome=now-sensors.getFirstValue("key('byId','urn:atHome')/lastModified");
now=System.currentTimeMillis();
if (atHome && now-lastAtHome>24*60*60*1000)
{	context.logRaw(INKFLocale.WARNING,"Did you know you really need to get out more?");
}

booleanChangeDetect

The problem with the above script is that after 24 hours of being at home you will be warned everytime the script is run. Usually you determine a condition and then act when that condition becomes true or stops being true. To do this we need to remember state to know the state of the condition last time the script ran.

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;
import io.polestar.ScriptUtils;

//example of using the value and last modified time of a sensor
sensors=context.source("active:polestarSensorState",IHDSDocument.class).getReader();
atHome=sensors.getFirstValue("key('byId','urn:atHome')/value");
lastAtHome=now-sensors.getFirstValue("key('byId','urn:atHome')/lastModified");
now=System.currentTimeMillis();

atHomeToLong=atHome && now-lastAtHome>24*60*60*1000;
conditionChanged=ScriptUtils.booleanChangeDetect(
	value:atHomeToLong,
	context:context );
if (changed && atHomeToLong)
{	context.logRaw(INKFLocale.LEVEL_INFO,"Did you know you really need to get out more?");
}

The booleanChangeDetect() method is contained within the MonitorUtils class. It stores state between invocations of the script so as to detect when the condition changes. It also supports the following optional arguments:

analogueLevelChangeDetect

ScriptUtils also contain the analogue equivalent of booleanChangeDetect() called analogueLevelChangeDetect(). It works in a similar way but looks for a numeric value to be above a trigger level for the condition to become true.

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

sensors=context.source("active:polestarSensorState",IHDSDocument.class).getReader();
light=(double)sensors.getFirstValue("key('byId','urn:daylight:db')/value");
changed=io.polestar.ScriptUtils.analogueLevelChangeDetect(
	value:light,
	trueThreshold: -50.0D,
	falseThreshold: -55.0D,
	context:context );
day=light>-50.0D;
if (changed)
{	if (day)
	{	context.logRaw(INKFLocale.LEVEL_INFO,"Dawn");
	}
	else
	{	context.logRaw(INKFLocale.LEVEL_INFO,"Dusk");
	}
}

It also supports the following optional arguments:

atMostEvery

Sometimes it is useful to stop a particular action from happening too often. For example my kids love to press the door bell about 10 times but if I'm not at home I only want to be send a message if the door bell rings for the first time in 5 minutes.

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;

newEvent=io.polestar.ScriptUtils.atMostEvery(
	period: 300000L,
	requireQuiet: true,
	context:context );

if (newEvent)
{	context.logRaw(INKFLocale.LEVEL_INFO,"This might actually be a new person ringing");
}

It supports the following optional arguments:

Script persistent state

You can access the scripts persistent state directly:

import org.netkernel.layer0.nkf.*;
import org.netkernel.mod.hds.*;

state=context.source("arg:state",IHDSDocument.class).getMutableClone();
count=state.getFirstValueOrNull("/state/count");
if (count==null)
	count=0;
else
	count++;
state.createIfNotExists("state/count").setValue(count);
context.sink("arg:state",state.toDocument(true));
resp=context.createResponseFrom(state.toString());
resp.setExpiry(INKFResponse.EXPIRY_ALWAYS);

Calling other scripts

One script can call another script. This is useful for generic services such as sending a message. You can pass arbitrary arguments between scripts:

req=context.createRequest("active:polestarExecuteScript");
req.addArgument("script","47CCD102175BC395");
req.addArgumentByValue("argument1","abc");
rep=context.issueRequest(req);
You can also reference scripts by their name as well as their id:
req=context.createRequest("active:polestarExecuteScript");
req.addArgument("name","Send Message");
req.addArgumentByValue("message","something happened");
rep=context.issueRequest(req);
The called script can access the argument:
argument1=context.source("arg:argument1");

Logging

Scripts can write to the log:

import org.netkernel.layer0.nkf.*;
context.logRaw(INKFLocale.LEVEL_INFO,"Something happened");
context.logRaw(INKFLocale.LEVEL_WARNING,"Something bad happened");

Visualization scripts

You can define scripts which generate visualizations of historical data captured by Polestar.

You can read, aggregate and apply operations such as averaging on sensor values using the active:polestarHistoricalQuery data service:

import org.netkernel.mod.hds.*;
import org.netkernel.layer0.nkf.*;
import java.text.*;

m=HDSFactory.newDocument();
m.pushNode("query");
m.addNode("start",-1000L*60*60*24*4); // start 4 days ago
//m.addNode("end",0); // end now
m.addNode("merge",8); // merge 8 records into one
m.addNode("json",true);
m.pushNode("sensors"); //list of sensors that we want to get data for
m.pushNode("sensor").addNode("id","urn:co2").addNode("mergeAction","average").popNode();
m.pushNode("sensor").addNode("id","urn:home").addNode("mergeAction","sample").popNode();
m.pushNode("sensor").addNode("id","urn:daylight:db").addNode("mergeAction","average").popNode();
req=context.createRequest("active:polestarHistoricalQuery");
req.addArgumentByValue("operator",m.toDocument(false));
rep=context.issueRequest(req)

The merge actions available are:

Once you have the resultset in JSON format it is quite easy to generate charts and visualizations from it using protovis (built in) or any other clientside Javascript library. Because the code sample to do this can be quite large take a look at the examples on GitHub or on the Live Demo.

Homepage script

To customize the homepage create a script which returns XHTML with the name "Homepage". You can use this page to put visualizations or status icons.

import org.netkernel.layer0.nkf.*;
s="""<div class='container'>
<iframe src="/polestar/scripts/execute/287CB3EAD99CA3A6" frameborder="0" width="280" height="160" scrolling="no">_</iframe>
</div>""";

resp=context.createResponseFrom(s.toString());
resp.setHeader("polestar_wrap",true);
	

On Demand Sensors

On demand sensors are sensors which are updated by scripts when they are queried. Maybe the term sensor is being stretched a bit far. These sensors represent a single value over time and it's value is usually defined from data available in other sensors. For example we could define an on-demand sensor for maximum temperature in day. This sensor will update at midnight each day with a value of the maximum temperature over that day. So typical these sensors provide aggregated higher level data.

To define an on demand sensor define a sensor in the Sensors script as usual (see above.) Then create a script which will update the sensor. This script should have a periodicity of how often the sensor should update and target of the on-demand sensor. The script will receive a timestamp argument of a Long millisecond timestamp when it is called. It should return a single sensor value as it's response.

Backup/Restore

You can backup and restore all the scripts through the and buttons on the headerbar of the script list (only available on desktop not mobile.)

Inhibit Polling

I've noticed that restarting polestar after significant hardware changes can lead to a lockup because the polling continues to happen and with delays such as connecting to hardward or networking timouts the requests get backlogged to a degree that recovery and editing of scripts is impossible. To workaround this there is a java system property that can be set at boot time which inhibits all polling scripts from automatically executing. Add the following to [install]/bin/jvmsettings.cnf

-Dpolestar.inhibitPolling=true