Zach Shelby has spent most of the last decade and a half on the front line of the Internet of Things (IoT). His company Sensinode, which was acquired by Arm in 2013, provided enterprise wireless sensor networks to system integrators and product providers. Shelby did lots of interesting work on embedded systems, and incorporating standards such as Bluetooth Low Energy (BLE). But he wanted to go a step further.
Enter Edge Impulse. The company, with Shelby as co-founder and CEO – Jan Jongboom, a colleague at Arm, as co-founder and CTO – is looking to enable developers to create next-gen applications with embedded machine learning (ML).
“We had already done IoT – we’ve been there, we scaled it, matured it, and we were looking for the next technology that really moves things for developers,” Shelby (left) tells Edge Computing News. “What we stumbled upon was that Moore’s Law had caught up with embedded in a big way.
“We’re using IoT to measure temperature once every hour, or day even, and that’s not even making a nick on the amount of math capabilities that these little microcontrollers have,” he adds.
Shelby and Jongboom had to defer to expertise to make the concept a reality. The two men were, in Shelby’s words, ‘not machine learning people’ – they were embedded engineers who like to solve problems for people who like to build systems. Yet at Arm, they had GPU experts among their developer team, and they learned how techniques from the world of cloud ML can be utilised at a much smaller scale for microcontrollers.
The moment ‘where the light bulb turned on’ for Shelby came with the realisation of efficiency. “You’re able to make use of those techniques to solve problems in a data-driven way, rather than a coding way,” he notes.
“Almost all algorithms today are hand-coded – trial and error,” Shelby adds. “They’re using data as a kind of testing facility, but not as a way to really drive what’s being designed. What machine learning does is flip the entire equation on its head, and now you have the ability to use data to drive design. That’s powerful.”
But how can you get developers to make the most of this opportunity? The tools at their disposal was less than stellar; ‘1990s development difficult’, as Shelby puts it. “We were kind of horrified – oh my God, nobody’s ever going to be able to do this,” he adds. “So that’s the point where we said we’ve got to solve this problem in a very different way, and we’re going to have to do it as an innovative, independent software startup. That’s how Edge Impulse was formed.”
For those who remember when low-code platforms first appeared on the market, the call to arms for learning code, and the empowerment of citizen developers, was keen. Developers who had studied and worked for years on their craft, however, felt taken aback.
Shelby notes it is not the same in this instance. “We do have a role for data scientists and ML engineers – the people who create these very complex new algorithms,” he says. “The people who are going to contribute new algorithms that other people will use; that’s deep code. We have very sophisticated tools and ways for them to contribute those things, writing Python and C++, and doing math optimisation. But that [requires] a very special person.”
Another problem the company wished to alleviate was around efficiency and overheads. In September, the new EON (Edge Optimized Neural) Compiler was launched. The goal is to help developers run neural networks with up to 55% less RAM, and 35% less flash.
“One of the problems we were seeing from our user base was engineers wanted to put ML on more of their existing hardware targets,” says Shelby. “But they don’t want to do an entire revamp of the whole product. So they came to us many, many times asking – could you squeeze a little bit more in?
“So we went and then did the work to go and create some technology which gets rid of all the extra overhead that we typically see from machine learning frameworks that are being used to build neural network models, and then deployed on embedded targets,” Shelby adds. “Those typically have a bunch of unnecessary overhead.”
Edge Impulse’s platform is available free to developers who want to build, but the monetisation comes through the enterprise interest. Shelby says the company is working with around 10,000 enterprises, with ‘well over’ 1,000 companies actively using the product. A typical use case revolves around wearable health devices – a particular boon amid Covid-19.
“Some of these customers are seeing huge benefits – not only from new functionality, but just from enabling them to apply their data,” says Shelby. “These companies are sitting on a kind of goldmine of data – and I don’t mean IoT data where you just build up lots of it, but building up very high quality data.
“We’ve helped those companies really open up this data, make sure it’s organised and marked up in a way that can be used for ML, and made sure that their entire organisations are able to make use of it.”
Shelby is speaking at the Edge Computing Expo this week and he is keen to clarify a couple of things. Not just the benefits of embedded machine learning, but understanding the position of edge in the wider ecosystem.
“We throw around a lot of really interesting terms like deep neural networks, and machine learning, and AI, but none of that actually means much – apart from the fact it’s smart and shiny,” says Shelby. “I’d like to explain to people what we’re really seeing in practice in these industries – this machine learning on the edge is really about working with real-time sensor data.
“This isn’t the edge versus cloud, which I think a lot of people misunderstand,” he adds. “This isn’t ‘the edge is going to kill the cloud’, or there won’t be any ML in the cloud any longer. ML when doing real time sensor and pattern matching is a very different thing than cloud ML algorithms.
“So the fact that edge machine learning and cloud machine learning are going to work together – we do need both the edge and the cloud to do their jobs, to build these whole systems.”
Want to learn more about topics like this from thought leaders in the space?Find out more about the Edge Computing Expo, a brand new, innovative event and conference exploring the edge computing ecosystem.