3 years in and processing 10 million messages: a retrospective

Our company is built around and dedicated to enabling customers to access data they haven’t previously been able to using sensors.  Three years into this journey finds us now processing over 10 million sensor messages per day from more than 100 companies in Europe and North America!  

What have we learnt so far?
– Multi disciplinary teams; In IoT we believe the strength of a team is in building experitise in project management, hardware, networks, software, data and open APIs.   

– Minimising sensor installation headaches are key; Our latest sensor deployment in London last week saw us reduce installation time to only 20 seconds per sensor.  We believe sensors should be easy to place and simple to move which is why we also have training schemes available to our clients so that they can manage the physical repositioning of their own sensors post deployment should they choose to. Flexibility and control are as important to us as they are to you.

We are always investigating and building partnerships to find the best solution for our clients and this has focused us on standardising around open network protocols such as LoRa.  We now have a large number of happy customers using the following types of LoRa sensors.

  • Office utilisation monitoring: desks, meeting rooms and breakout areas
  • Air quality
  • Environmental sensing
  • Parking sensors
  • Footfall monitors
  • Asset management sensors such as vibration, temperature monitoring

Recent clients include; Zaha Hadid Architects, 360 Workgroup, Fourfront group, TripAdvisor and many others.

As much as possible we have tried to share tangible learnings and experience reports with the wider community. Some examples of our publications and webcasts that have proven hugely popular include:

  • An IoT university – with over 1,500 registered students

https://university.opensensors.io/university/

  • Webcasts with Multi Tech about LoRa and the pros and cons

https://www.youtube.com/watch?v=M1jMheYnRq4&t=326s

Thank you to all our customers, partners, advisors and the wider IoT community.  We look forward to the next 3 years of learning, growing and being part of this incredible ecosystem.

WELL building standard and indoor air quality

What is the WELL building standard?

WELLBritish employees will tell you how hard they work and they are right; these days we are spending more time in the office than any other EU country except Austria and Germany.  How can the time spent indoors be made more comfortable and agreeable? It’s hard to isolate one sole influence that affects productivity when in fact there are many to consider. Studies have shown that despite an average working day of 8 hours you’re probably only productive for around 3 of those. The WELL Building Standard is a set of best practices and guidelines focused on human health and wellness. The culmination of seven years of evidence-based medical research in partnership with leading scientists, doctors, architects and wellness thought leaders was pioneered by Delos.  WELL is based on medical research on the health and wellness impacts of the buildings we spend most of our time in.  

WELL certification concentrates on performance and requires a ‘pass’ score in these seven categories: air, water, nourishment, light, fitness, comfort and mind. With WELL certification awarded at one of three levels: Silver, Gold, and Platinum, it is now much easier to gauge the gaps between comfort, well being and employee work output.

What variables are likely to impact productivity?

There is much research showing the effects of lighting, noise, temperature, and CO2 on our productivity.

Poor lighting makes us sleepy

That lull in energy you’re feeling might not be a sugar crash after all. Don’t blame it all on cake. There is a marked connection between light and productivity amongst other key factors. Daylight and “blue-enriched light bulbs” help employees stay alert by lowering melatonin, the hormone that makes us sleepy. But that’s not the only factor at play here influencing our alertness.

Breath of fresh air

We are less conscientious of air quality, but poor indoor air quality also cuts productivity. Insufficient ventilation concentrates pollutants, such as volatile organic compounds (VOCs) and carbon dioxide (CO2). High CO2 levels have been shown to reduce concentration, attention span, and memory in classrooms.

The adverse effects of poor air quality can be dangerous (see Sick Building Syndrome).

To address these issues companies are recycling indoor air to maintain healthy CO2 levels.

Screen Shot 2017-08-31 at 14.52.29

Noise reduces concentration

Clacking keyboards, co-workers yakking away and phones ringing are some of the common gripes employees give for the reduction in their concentration. World GBC’s 2016 report estimates that productivity dropped by 66% in performance and concentration due to ambient noise distractions.

http://www.worldgbc.org/news-media/building-business-case-health-wellbeing-and-productivity-green-offices

Adjust your thermostat

It’s probably no surprise that with high temperatures (over 32C), productivity declines but the same is true when the temperature decreases below 15C, making people less focused on work and thus less productive. A 2004 study links fewer typing errors and higher productivity when work spaces are warm enough. Let’s not forget about humidity either as that affects perceived temperature and comfort levels so keeping a good level of it is key to maintaining a healthy and productive office environment.

Sensors improve workspace

Our clients use sensors for light, noise, temperature, and CO2, as well as measuring air quality (i.e. particulates) and various gasses including potentially harmful ones (e.g. VOC and CO) to monitor the workplace environment and help create healthier surroundings. Pollution in large cities is increasingly problematic and it is vital that HVAC systems successfully filter out pollutants and gases from the outside world so employees can go about their days confident they are not compromising their well being just by going to work. OpenSensors aggregates data from a variety of sensors for the next generation of smart Building Management Systems and with experience in helping companies combine data from new workplace sensors, even interoperating it seamlessly with existing systems is easily done. We also operate the world’s largest repository of air quality data and process over 10 million sensor messages per day – that’s the equivalent of one message each for the entire population of Portugal!  

Conclusion

It has never been easier to measure environmental factors within buildings and analyse the data to give a fully comprehensive overview. Companies can optimise employee well being and efficiency using data from light, noise, temperature, and CO2 sensors with unprecedented ease.  A win win for everyone.

 

Monitoring for Earthquakes With Node-red

Develop your own earthquake-triggered workflows. Let’s shake it.

OpenSensors now capture seismic data from the Euro-Med Seismic Centre (EMSC) and the United States Geological Survey (USGS). Every ten minutes we are polling the latest information of major and minor earthquakes around the globe and make this information available via our programming interface (API) or as MQTT feed. In this short tutorial, we’re showing you how to use OpenSensors together with Node-RED to receive email alerts whenever there’s a major incident in a region of interest. You can use this guide as starting point for further experiments with Node-RED and develop your own earthquake-triggered workflows. Let’s shake it.

On OpenSensors

  • First, you need to login to your account on OpenSensor or sign up for one if you haven’t done so already at https://OpenSensors.io.
  • Next, it’s good practice to have a new ‘device’ for this application, i.e. a dedicated set of credentials you’re going to use to log in to OpenSensors for this particular set of MQTT feeds.
    • In the panel on the left, click My Devices in the Devices menu.
    • Click the yellow Create New Device button at the top of the page.
    • Optional: Add some optional descriptions and press the disk icon to save your new device.
    • Take a note of your ‘Client id’ and ‘Password’ as you’re going to need them in your Node-RED workflow.

For Node-RED

Install node.js and Node-RED on your system. There’s a very good guide for this on the Node-RED website. Follow the instructions, including the separate section on Running Node-RED.

Once you’re ready, open a web browser and direct it to localhost:1880, the default address and port of the Node-RED GUI on your system.

(A very basic description of the Node-RED vocabulary can also be found at SlideShare.)

Developing a workflow

  • From the input panel of your nodes library on the left side, drag and drop a pink mqtt input node into the work area named Sheet 1.
  • Double-click the mqtt node. A window with configuration details opens.
    • Click the pen symbol next to ‘Add new mqtt-broker…’. Your Broker is opensensors.io, your Client ID and Password those you generated in the previous step on the OpenSensors website, and User is your OpenSensor user name.
  • Once the Broker is defined, enter /orgs/EMSC/+ into the Topic field. This is going to instruct Node-RED to subscribe to all MQTT topics generated by the EMSC.
  • Optional: Set the Name of this node to ‘EMSC’.
  • Drag and drop a second mqtt input node. When you double-click the node, you will realise that the Broker settings default to the ones you previously entered.
    • Enter /orgs/USGS/+ in the Topics field and ‘USGS’ as optional Name.
  • Drag and drop a dark green debug node from the output panel on the left. While debugging has the connotation of fixing a problem, in Node-RED it’s the default way of directly communicating messages to the user.
  • Draw connection lines (“pipes”) from both mqtt nodes to the debug node.
  • Press the red Deploy button in the upper right corner. This starts your Node-RED flow. If everything worked, you should see ‘connected’ underneath the mqtt nodes and your debug panel (on the right) should soon produce the following JSON-formatted output if there’s an event (which may take a while!):

While it is pleasing to be informed about every time the earth shakes, it soon becomes tedious staring at the debug panel in expectation of an earthquake. Also, you may not be interested in events in remote areas of the world, or exactly in those – whatever interests you.

We are going to extend our flow with some decision making:

First, we need to parse the information from the EMSC and USGS. For this example, we’re going to be particularly interested in the fields region and magnitude. There are plenty more fields in their records, and you may want to adjust this flow to your needs.

  • Drag and drop a pale orange function node from the functions panel into your flow. Connect both mqtt nodes to the input side (the left side) of your function node. Function nodes allow you directly interact with your data using JavaScript.
  • Enter the following code (or download the OpenSensors workflow).

Here be a JavaScript course… :–) In a nutshell, this code takes data from the ‘payload’ of the incoming message (read up on the topic and payload concept of Node-RED in the SlideShare article suggested earlier). The payload is then parsed for the region and magnitude fields using standard regular expressions. If we can successfully extract information (in this case: the region containing ‘ia’ somewhere in it’s name), we’re going to set the outgoing message’s payload to the magnitude, its topic to ‘EVENT in ‘ plus the name of the region and pass it on (‘return msg’) to the next node.

  • Drag and drop a lime green switch node from the function panel into your workflow. Connect the output of the function node to the input of the switch node. Configure (by double-clicking) the switch node to assert if the payload (being the magnitude of the earthquake) is greater than 2. Only then the message is going to be passed on
  • Last, we’re going to drag and drop a light green e-mail output node from the social panel and configure it like an e-mail client, but with a default recipient: here in this case, ohmygodithappend@gmail.com.
  • Connect the output of the switch node to our debug node, as well as to the outgoing e-mail node.
  • We can then deploy the new workflow and should see something like this after a while:

In this case, an event was detected ‘off the coast of Northern California’ with a magnitude of 4.4 and at the same time, you should receive an e-mail with the region as subject and the magnitude in the body of the e-mail.

We hope that this flow is getting you started! Remember that Node-RED is superbly suited to interact with hardware… …imagine LEDs and buzzers indicating an earthquake.

The flow JSON: [{“id”:“e9024ae0.16fdb8”,“type”:“mqtt-broker”,“broker”:“opensensors.io”,“port”:“1883”,“clientid”:“1646”},{“id”:“2952b879.d6ad48”,“type”:“mqtt in”,“name”:“EMSC”,“topic”:“/orgs/EMSC/+”,“broker”:“e9024ae0.16fdb8”,“x”:127,“y”:104,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“163677af.e9c988”]]},{“id”:“54239d6.fabdc64”,“type”:“mqtt in”,“name”:“USGS”,“topic”:“/orgs/USGS/+”,“broker”:“e9024ae0.16fdb8”,“x”:128,“y”:159,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“163677af.e9c988”]]},{“id”:“490a140f.b6f5ec”,“type”:“debug”,“name”:“”,“active”:true,“console”:“false”,“complete”:“false”,“x”:538,“y”:86,“z”:“82a1c632.7d5e38”,“wires”:[]},{“id”:“163677af.e9c988”,“type”:“function”,“name”:“parse”,“func”:“// uppercase the payload (different centres report in mixed formats)nmsg.payload = msg.payload.toUpperCase();nn// extracting interesting fields with regular expressions,n// instead of using JSON.parse which fails with null fieldsnvar places_with_ia_regex = new RegExp(“REGION”:“(.IA.)”,“UPDATED”);nvar result1 = places_with_ia_regex.exec(msg.payload);nnvar magnitude_regex = new RegExp(“MAGNITUDE”:([0-9].[0-9]+)“);nvar result2 = magnitude_regex.exec(msg.payload);nn// if successful, sets topic to the region and payload to the magnitudenif (result1 && result2) {n msg.topic = ‘EVENT in ’+result1[1];n msg.payload = result2[1];n return msg;n}”,“outputs”:1,“noerr”:0,“x”:296,“y”:251,“z”:“82a1c632.7d5e38”,“wires”:[[“64f4f2ea.9b0b0c”]]},{“id”:“64f4f2ea.9b0b0c”,“type”:“switch”,“name”:“at least magnitude 2”,“property”:“payload”,“rules”:[{“t”:“gte”,“v”:“2”}],“checkall”:“true”,“outputs”:1,“x”:428,“y”:179,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“f7bcc59c.084338”]]},{“id”:“f7bcc59c.084338”,“type”:“e-mail”,“server”:“smtp.gmail.com”,“port”:“465”,“name”:“ohmygodithappened@gmail.com”,“dname”:“”,“x”:581,“y”:256,“z”:“82a1c632.7d5e38”,“wires”:[]}]

First ‘Things’ First

What’s needed to make smart city data exchange a reality?

I was pleased to see the recent post by the ODI on the open-shared-closed data spectrum since it resonates with the challenges faced at OpenSensors. To date most of our commercial projects have been at the private end of the spectrum; they are challenging, they are innovative, but they are often not ingesting open data or publishing data as an exhaust.

Are we worried about private IoT messaging? Not too much. Most of our private clients choose to get their own house in order first, after all typically there’s a lot of opportunity to juice existing sensors. First ‘things’ first as they say.

The good news is these deployments are sowing the seeds of sharing behaviours by distributing content internally, releasing data that used to terminate and die. They are unlocking data and distributing for access via API for dashboards, data science and decision support, which is the first step on a journey to openness.

So as a tech company how do we lead our clients and help them deliver open data strategy? We provide the tools to allow organisations to manage data entitlements pushing themselves up the data spectrum to become open. Each of our clients will make their own journey to open up their content, our job is to deliver infrastructure allowing them to manage data at a privacy that works for them.

This is important stuff. IoT tech companies are developing the smart city data network, and we don’t want it to be private. We want pain free navigation from edge to edge of our urban data grid, whilst feeling secure and confident about the data we consume. Our platforms must secure data whilst facilitating its exchange and entitlement control, so what’s needed to make smart city data exchange a reality? A couple of things spring to mind, we need to …

Evolve Topics and Communities – Expect faster adoption of sharing behaviours within trusted communities. By curating communities with shared interests expect adoption of localised data exchange, say amongst tenants of a commercial property. Communities sharing data should ease the path to universal open data.

Evolve Exchange Mechanisms – Transparent pain free data exchange is key to delivering a functionally rich lean IoT data infrastructure, the alternative could be akin to a ‘european data mountain’ of needless and costly sensor deployments.

Building the tech stack for these needs is plenty of work, so as we define the business and technical models for IoT we need to act responsibly. Deploying and decommissioning software is cheap, just a couple of mouse clicks away. IoT deployments are very real, they consume natural resource, risk cluttering our environment and can loiter well past their usefulness.

Encouraging sharing behaviour within IoT through lean shared infrastructure will prevent waste. The alternative would be a legacy of urban junk, we made a mess of space by not decommissioning hardware, lets not do the same with our urban environment and keep it open and centred on communities.