3 years in and processing 10 million messages: a retrospective

Our company is built around and dedicated to enabling customers to access data they haven’t previously been able to using sensors.  Three years into this journey finds us now processing over 10 million sensor messages per day from more than 100 companies in Europe and North America!  

What have we learnt so far?
– Multi disciplinary teams; In IoT we believe the strength of a team is in building experitise in project management, hardware, networks, software, data and open APIs.   

– Minimising sensor installation headaches are key; Our latest sensor deployment in London last week saw us reduce installation time to only 20 seconds per sensor.  We believe sensors should be easy to place and simple to move which is why we also have training schemes available to our clients so that they can manage the physical repositioning of their own sensors post deployment should they choose to. Flexibility and control are as important to us as they are to you.

We are always investigating and building partnerships to find the best solution for our clients and this has focused us on standardising around open network protocols such as LoRa.  We now have a large number of happy customers using the following types of LoRa sensors.

  • Office utilisation monitoring: desks, meeting rooms and breakout areas
  • Air quality
  • Environmental sensing
  • Parking sensors
  • Footfall monitors
  • Asset management sensors such as vibration, temperature monitoring

Recent clients include; Zaha Hadid Architects, 360 Workgroup, Fourfront group, TripAdvisor and many others.

As much as possible we have tried to share tangible learnings and experience reports with the wider community. Some examples of our publications and webcasts that have proven hugely popular include:

  • An IoT university – with over 1,500 registered students

https://university.opensensors.io/university/

  • Webcasts with Multi Tech about LoRa and the pros and cons

https://www.youtube.com/watch?v=M1jMheYnRq4&t=326s

Thank you to all our customers, partners, advisors and the wider IoT community.  We look forward to the next 3 years of learning, growing and being part of this incredible ecosystem.

Lessons Learned From First Generation IoT Installations

The significant drawbacks of Wi-Fi

At first glance, Wi-Fi-based sensors seems like a good choice for a non consumer facing sensor network, however, we have discovered that Wi-Fi has some significant drawbacks.

Access

One of the biggest drawbacks to Wi-Fi enabled sensors in a corporate environment at many of the companies is gaining access. Corporate IT often has valid security concerns of hundreds if not thousands of sensors joining the network and have deployed corporate firewalls that block any access. Often this means that we are not allowed to spin up our own Wi-Fi network in order to have a gateway for a customer’s IoT sensor network. If IT has already deployed a Wi-Fi network they are rarely willing to provide the passwords to allow the IoT network devices and gateways to take advantage of it. Relying on corporate Wi-Fi can make on-site installations and maintenance extremely complex and painful. The whole project becomes dependent on the goodwill of a network administrator for access every time maintenance needs to be performed.

Power

Wi-Fi has good transmission range but that comes with a cost of high power usage. With a short battery life, maintenance costs for Wi-Fi sensors are higher than low-power alternatives. One wireless protocol that is we see in many successful deployments is LoRa because it offers long transmission range at a much lower battery usage than Wi-Fi.

Moving to LoRa and other long range protocols

If you follow our blog and publications, you will notice we have been talking a lot about network technologies, this isn’t a coincidence. We have spent a long time evaluating and piloting these stacks with our community.

Network access and battery constraints are driving the move to long range networks and off WiFi for many IoT installations. LoRa is working well for us so far for a number of use cases most of our customer spin up a private network. The ecosystem of providers is maturing and we are finding a lot of companies who are adopting existing sensors for their networks Gateway providers such as Multi Tech provide good support for the long tail of small scale (> 250 sensor installs) hardware providers to thrive.

LoRa is a wireless protocol that promises between two and five kilometers transmission range between sensors and gateway, if you haven’t already done so please read our introduction to what it is. With a separate LoRa network, facilities and/or operations can install and manage the whole operation without the access and security issues of using the corporate Wi-Fi network. A typical network will have hundreds of sensor devices sending messages to a gateway. The LoRa gateway is a self contained system, we can have the LoRa network sit completely outside of the corporate firewall (GSM) and minimize IT security concerns.

One LoRA gateway can normally cover an entire real estate. This can significantly reduce infrastructure, deployment, and administration costs compared to other shorter range wireless options like Zigbee or Bluetooth that requires complex installs. Our aim is to have a system that non technical engineers can roll out and support, more on how to do this on later blog posts, but in most cases the OpenSensors team is the equivalent of ‘2nd line support’ to the onsite team who have intergrated our apis to their helpdesk ticketing systems etc.

LoRa networks can be public or private. An example of a public network is The Things Network, we continue to work with and support that community. Most current commercial projects are running private networks at this time but will be interesting to see how that evolves over time.

To conclude, LoRa is working well for us at the moment but we will keep researching other networks to enable us to understand the pros and cons of all the network providers. Sigfox is a very interesting offering that we will properly test over the next few months, for example.

European Parliament Approves eCall Technology

The Internet of Things threatens to revolutionise everyday life

European Parliament approves eCall connected car platform

The Internet of Things threatens to revolutionise everyday life, embedding and imbuing everyday objects and the world around us with sensors, software and electronics. Through machine-to-machine communication, automation and advanced analytics, we are able to understand and scrutinise our environment and the processes which surround us in ways never conceived. From high level analysis allowing automated condition monitoring of critical engine parts, giving engineers the tools to reduce costly operational downtime to embedding real-time sensors in bridges to predict stresses and flooding. Beyond the Cloud, the Internet of Things brings the internet to the everyday, and there are clear use cases for such technologies in the realm of road safety.

This is where eCall comes in. eCall is a European Commission initiative coming into force on 31 March 2018, making mandatory the deployment of internet-connected sensors into cars that enable emergency services to be immediately contacted and requested automatically after a serious road incident within the European Union. EC VP for Digital, Neelie Kroes, argues “EU-wide eCall is a big step forward for road safety. When you need emergency support it’s much better to be connected than to be alone.” eCall will drastically cut European emergency service response times, even in cases where passengers are unable to speak through injury, by sending a Minimum Set of Data (MSD), including the exact location of the crash site.

The deployment of eCall is one of most ambitious EU-wide programs since the 2007 enlargement, rolling out implementation of the eCall platform to some 230 million cars and 33 million trucks in the European Union. Implementation of eCall at a European level (including Norway, Switzerland etc) however benefits consumers and industry through reducing costs due to economies of scale, reducing the installation cost to as little as €100. The basic pan-European eCall service will be free at the point of use for equipped vehicles. It is likely that the eCall technology platform (i.e., positioning, processing and communication modules) will be exploited commercially too, for stolen vehicle tracking, dynamic insurance schemes, eTolling and emerging forms of vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) road safety systems. eCall will be based upon a standardised platform, one system for the entirety of Europe, aimed at enabling both car and telecoms industries a quick roll out and to avoid crippling OEM versioning and patching issues.

In terms of privacy, the basic eCall system has been given the green light by the European Commission on the express condition that firm data protection safeguards are in place and that the sensor-equipped vehicles will not push data to external servers except in the case of a crash, or by the actions of the driver, in order to contact the PSAP (Public Safety Answering Point) and will lie dormant until that point. The data transmitted to the emergency services, described as MSD, Minimum Set of Data, are those strictly needed by the emergency services to handle the emergency situation. While in normal operation mode the system is not registered to any telecoms network and no mediating parties have access to the MSD that is transmitted to the PSAPs.

Today the European Parliament’s Internal Market and Consumer Protection Committee MEPs voted on and approved eCall pushing forward a life-saving Internet of Things technology that will significantly improve European road safety. The UK Government however, has not followed suit, whilst welcoming the implementation in other member states, feels that “it is not cost-effective … given the increasing responsiveness of our road network, we feel that smart motorways do the same thing,” remarked Minister Perry on behalf of the Department of Transport. Whilst it can be argued that ‘Smart Motorways’ are far from a worthy substitute to connected cars & V2V/V2I systems, the UK’s criticism belies a certain caution with regards to green-lighting large and costly IT projects. Only time will tell whether the UK Govt’s decision has left those drivers not on Britain’s Smart Motorways in the lurch.

Monitoring for Earthquakes With Node-red

Develop your own earthquake-triggered workflows. Let’s shake it.

OpenSensors now capture seismic data from the Euro-Med Seismic Centre (EMSC) and the United States Geological Survey (USGS). Every ten minutes we are polling the latest information of major and minor earthquakes around the globe and make this information available via our programming interface (API) or as MQTT feed. In this short tutorial, we’re showing you how to use OpenSensors together with Node-RED to receive email alerts whenever there’s a major incident in a region of interest. You can use this guide as starting point for further experiments with Node-RED and develop your own earthquake-triggered workflows. Let’s shake it.

On OpenSensors

  • First, you need to login to your account on OpenSensor or sign up for one if you haven’t done so already at https://OpenSensors.io.
  • Next, it’s good practice to have a new ‘device’ for this application, i.e. a dedicated set of credentials you’re going to use to log in to OpenSensors for this particular set of MQTT feeds.
    • In the panel on the left, click My Devices in the Devices menu.
    • Click the yellow Create New Device button at the top of the page.
    • Optional: Add some optional descriptions and press the disk icon to save your new device.
    • Take a note of your ‘Client id’ and ‘Password’ as you’re going to need them in your Node-RED workflow.

For Node-RED

Install node.js and Node-RED on your system. There’s a very good guide for this on the Node-RED website. Follow the instructions, including the separate section on Running Node-RED.

Once you’re ready, open a web browser and direct it to localhost:1880, the default address and port of the Node-RED GUI on your system.

(A very basic description of the Node-RED vocabulary can also be found at SlideShare.)

Developing a workflow

  • From the input panel of your nodes library on the left side, drag and drop a pink mqtt input node into the work area named Sheet 1.
  • Double-click the mqtt node. A window with configuration details opens.
    • Click the pen symbol next to ‘Add new mqtt-broker…’. Your Broker is opensensors.io, your Client ID and Password those you generated in the previous step on the OpenSensors website, and User is your OpenSensor user name.
  • Once the Broker is defined, enter /orgs/EMSC/+ into the Topic field. This is going to instruct Node-RED to subscribe to all MQTT topics generated by the EMSC.
  • Optional: Set the Name of this node to ‘EMSC’.
  • Drag and drop a second mqtt input node. When you double-click the node, you will realise that the Broker settings default to the ones you previously entered.
    • Enter /orgs/USGS/+ in the Topics field and ‘USGS’ as optional Name.
  • Drag and drop a dark green debug node from the output panel on the left. While debugging has the connotation of fixing a problem, in Node-RED it’s the default way of directly communicating messages to the user.
  • Draw connection lines (“pipes”) from both mqtt nodes to the debug node.
  • Press the red Deploy button in the upper right corner. This starts your Node-RED flow. If everything worked, you should see ‘connected’ underneath the mqtt nodes and your debug panel (on the right) should soon produce the following JSON-formatted output if there’s an event (which may take a while!):

While it is pleasing to be informed about every time the earth shakes, it soon becomes tedious staring at the debug panel in expectation of an earthquake. Also, you may not be interested in events in remote areas of the world, or exactly in those – whatever interests you.

We are going to extend our flow with some decision making:

First, we need to parse the information from the EMSC and USGS. For this example, we’re going to be particularly interested in the fields region and magnitude. There are plenty more fields in their records, and you may want to adjust this flow to your needs.

  • Drag and drop a pale orange function node from the functions panel into your flow. Connect both mqtt nodes to the input side (the left side) of your function node. Function nodes allow you directly interact with your data using JavaScript.
  • Enter the following code (or download the OpenSensors workflow).

Here be a JavaScript course… :–) In a nutshell, this code takes data from the ‘payload’ of the incoming message (read up on the topic and payload concept of Node-RED in the SlideShare article suggested earlier). The payload is then parsed for the region and magnitude fields using standard regular expressions. If we can successfully extract information (in this case: the region containing ‘ia’ somewhere in it’s name), we’re going to set the outgoing message’s payload to the magnitude, its topic to ‘EVENT in ‘ plus the name of the region and pass it on (‘return msg’) to the next node.

  • Drag and drop a lime green switch node from the function panel into your workflow. Connect the output of the function node to the input of the switch node. Configure (by double-clicking) the switch node to assert if the payload (being the magnitude of the earthquake) is greater than 2. Only then the message is going to be passed on
  • Last, we’re going to drag and drop a light green e-mail output node from the social panel and configure it like an e-mail client, but with a default recipient: here in this case, ohmygodithappend@gmail.com.
  • Connect the output of the switch node to our debug node, as well as to the outgoing e-mail node.
  • We can then deploy the new workflow and should see something like this after a while:

In this case, an event was detected ‘off the coast of Northern California’ with a magnitude of 4.4 and at the same time, you should receive an e-mail with the region as subject and the magnitude in the body of the e-mail.

We hope that this flow is getting you started! Remember that Node-RED is superbly suited to interact with hardware… …imagine LEDs and buzzers indicating an earthquake.

The flow JSON: [{“id”:“e9024ae0.16fdb8”,“type”:“mqtt-broker”,“broker”:“opensensors.io”,“port”:“1883”,“clientid”:“1646”},{“id”:“2952b879.d6ad48”,“type”:“mqtt in”,“name”:“EMSC”,“topic”:“/orgs/EMSC/+”,“broker”:“e9024ae0.16fdb8”,“x”:127,“y”:104,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“163677af.e9c988”]]},{“id”:“54239d6.fabdc64”,“type”:“mqtt in”,“name”:“USGS”,“topic”:“/orgs/USGS/+”,“broker”:“e9024ae0.16fdb8”,“x”:128,“y”:159,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“163677af.e9c988”]]},{“id”:“490a140f.b6f5ec”,“type”:“debug”,“name”:“”,“active”:true,“console”:“false”,“complete”:“false”,“x”:538,“y”:86,“z”:“82a1c632.7d5e38”,“wires”:[]},{“id”:“163677af.e9c988”,“type”:“function”,“name”:“parse”,“func”:“// uppercase the payload (different centres report in mixed formats)nmsg.payload = msg.payload.toUpperCase();nn// extracting interesting fields with regular expressions,n// instead of using JSON.parse which fails with null fieldsnvar places_with_ia_regex = new RegExp(“REGION”:“(.IA.)”,“UPDATED”);nvar result1 = places_with_ia_regex.exec(msg.payload);nnvar magnitude_regex = new RegExp(“MAGNITUDE”:([0-9].[0-9]+)“);nvar result2 = magnitude_regex.exec(msg.payload);nn// if successful, sets topic to the region and payload to the magnitudenif (result1 && result2) {n msg.topic = ‘EVENT in ’+result1[1];n msg.payload = result2[1];n return msg;n}”,“outputs”:1,“noerr”:0,“x”:296,“y”:251,“z”:“82a1c632.7d5e38”,“wires”:[[“64f4f2ea.9b0b0c”]]},{“id”:“64f4f2ea.9b0b0c”,“type”:“switch”,“name”:“at least magnitude 2”,“property”:“payload”,“rules”:[{“t”:“gte”,“v”:“2”}],“checkall”:“true”,“outputs”:1,“x”:428,“y”:179,“z”:“82a1c632.7d5e38”,“wires”:[[“490a140f.b6f5ec”,“f7bcc59c.084338”]]},{“id”:“f7bcc59c.084338”,“type”:“e-mail”,“server”:“smtp.gmail.com”,“port”:“465”,“name”:“ohmygodithappened@gmail.com”,“dname”:“”,“x”:581,“y”:256,“z”:“82a1c632.7d5e38”,“wires”:[]}]