Next Step in Enterprise IoT

Next Step in Enterprise IoT

Virtualize Edge Gateways

Introduction

The Internet of Things (IoT) is here and companies are starting to invest heavily in it to be the first in their industry to drive digital transformation. IoT-enabled solutions cover a wide array of business applications and use cases, powered by the ability to connect millions of devices to the Internet and take autonomous actions based on the information they generate. With this enormous potential, it is no wonder organizations are embracing IoT to bridge their physical and digital worlds.

Market Size

IoT gateways are at a very critical point within IoT systems. According to Gartner, “90% of IoT data will go through gateways”. More specific forecasting comes from Research and Markets: 

• IoT gateway revenue will grows to $12.64B by 2022;

• The annual market size increase on IoT gateway will be around 14.7% in average till 2021.

Challenges with the Convergence of OT & IT on the Gateway

While the potential of IoT can only be realized by connecting the physical and digital worlds, it also mandates the convergence of information technology (IT) and operational technology (OT).

Security is a Big Concern

Many IoT devices, sensors and gateways are placed on open fields, fenced or non-fenced from the public. There could be some cases where these devices are put in house, but they are definitely not well physically protected as servers and storage in data centers or clouds. On the other hand, quite a big part of IoT devices are connected directly or indirectly via gateways backward to data centers or clouds in a wireless fashion, e.g. WiFi and mobile networks. So physically, there are more attacking interfaces exposed to the air.

Specifically, on gateways, communication interfaces upward to the Internet and downward to IoT devices are generally reachable for applications at the same time. That means malwares cracking from one side could possibly reach the other side quite easily.

Due to historical reasons, most industrial devices and communication protocols created before the Internet era with very limited security mindset in design and implementation. Now when they are connecting to the Internet, security issues could show up as a major risk concerning enterprise customers.

In fact, multiple severe security accidents happened yearly on different things, including but not limited to cars, cities, factories, trains, grids etc.

The Whole Stack is quite Fragmented

On either business or technology, specifically hardware, OS, application architectures, it is very difficult to create collaboration and integration based on fragmented stacks on the Edge.

On hardware platform level, multiple CPU architectures are used widely: mostly ARM based CPUs on low and mid ends; even some MCU chips (e.g. Atmel) are used in very low power scenarios. In some traditional cases of industrial automation, MIPS or special SoC chips are common too. At the high end, x86 based CPUs are used in gateways (e.g. Intel Atom) and some industrial PCs (e.g. Intel Xeon D).

On OS level, mostly various distributions of Linux or real-time OS are used, QNX, Windows CE, Windows IoT Core, VxWorks etc.

On Application level, all kinds of coding languages and run time are leveraged, Java, Python, JavaScript, C/C++, .Net, Assembly etc. 

Fragmentation status on the business side puts flexibility to IoT developers definitely, while it surely adds more complexity on building up a compatible OS image for that application and framework. Considering how to manage the life cycle of applications on IoT gateways, different App plus OS bundles definitely put more pressure on heterogeneous physical gateways for different purposes.

In fact, most IoT systems in enterprise customer side are silos, or vertical systems on things and edges (devices, sensors and gateways), although they could well connect backward to data centers and clouds. It’s not easy to migrate edge computing workloads within the edge layer in such a fragmented situation.

Consolidation and Multiple Tenancy is Impractical

Generally, just native OS or container on bare metal is used on the gateway, with no or very limited consolidation and isolation.

With long-term lifecycle of IoT devices, it is natural to see that gradually old and new IoT devices co-exist in same venues and they demand for different gateways to pass through data and instructions in the same way. In some enterprise IoT cases, like factories, grids, cities and vehicles, spaces are limited and there comes stronger expectation to save spaces, CAPEX/OPEX for similar functions of gateways.

For example, in case of smart factories, there could be industrial robots from 8-10 vendors and everyone leverage their own gateways to connect robots back to data centers or clouds. For factories, they want to save spaces but have to ensure data security and isolation of applications from different vendors. If the factories have their own custom code on gateways, it is also necessary to isolate the custom code from any vendor code. 

Another case is in connected vehicles, there are multiple ECUs (Electronic Control Units, like small-size computers) controlling all components in vehicles. Generally Head Unit (or Transmission Control Unit, TCU) is most vulnerable to be hacked because that is the external interface for vehicles to provide media entertainment (called infotainment) and connect outbound to the Internet and other vehicles or infrastructure. Customers want to isolate the Head Unit from other ECUs for control subsystems safely, while consolidate more physical ECUs into a smaller number and spare more spaces for passengers.

Overlapped Scope & Lack of Trust

Moreover, an obvious overlap on the Edge from both OT and IT organizations and a serious lack of trust for each other, only exacerbates these problems. Both sides want to control the gateway in the Edge, but neither has enough domain knowledge and skillset to achieve completely and independently. The impact of these two organizations operating separately not only poses a serious security risk but also slows down innovation, risking an enterprise to lose its competitive edge.

VMware’s Solution to Virtualize Edge Gateways

VMware proposes a new way to manage enterprise IoT gateways with virtualization. Our mission is to build a unified platform on gateways in IoT systems, to provide software defined edge computing with strong isolation, enhanced security and multiple tenancy.

Project Asteroid is targeting to virtualize IoT gateways and extend VMware Cloud Foundation (i.e. Software Defined Data Center, mostly vSphere/ESXi and NSX data plane/Edge) stack from clouds in data centers to IoT edge layer, and integrate with Pulse IoT Center solution and third party application frameworks. We call this vision Software Defined Edge Computing (SDEC). With virtualized IoT gateways and SDEC further, we could provide enhanced security, multiple tenancy with VM level and networking isolation, and consolidate spaces for physical gateways in certain circumstances.

In Project Asteroid, we virtualize physical IoT gateways with ESXi, and put NSX data plane daemon and Edge VM atop. We put Liota agent into VMs, and register them back to Pulse IoT Center. Above that, customers could put any IoT application frameworks, e.g. EdgeX Foundry and Eurotech ESF. This is just computing virtualization. Networking virtualization has a little bit variant and we will put that into several reference designs for different user scenarios, with either vSS, vDS or NSX.

If customers demand a very secure gateway with multiple tenancy, we put NSX in place. We separate tenant traffic from management traffic via different physical connections, separate tenant networks on virtual wires via overlay networks, and put a NSX Edge VM in place shared among tenant apps. Device/sensor I/O are pass-through directly to specific tenant app VMs and TPM is enabled by default.

Security Enhanced Stack

It is well known that in IoT world, people concerns a lot on security, but more often, the discussion is around OS and application level topics, like access control, firewall, PKI and encryption. After we virtualize the gateway, we expect to enhance the security situation than physical more, with some additional protection layers.

Below is a high-level security stack with additional value points contributed:

• Secure hardware platform with TPM or Secure Boot to resist tamper.

• Hypervisor with ESXi to separate VMs.

• Network isolation with vSS/NSX to protect intrusion outside

• OS access control with enhanced Photon OS to enable authorization policy

• Application sandboxing with runtime to provide app level isolation

 

Driver Models for IoT Interfaces

There are hundreds of IoT interfaces and protocols in the industry and it is impossible and unreasonable for ESXi to support all these drivers natively. So we need a smart strategy to handle them well.

Pass Through Mode

We generally prefer pass through mode driver for majority pure IoT interfaces and protocols, considering quite a few of them are not IP based or even not Ethernet, and they are not in the business consideration of a hypervisor. We pass through certain drivers/devices directly to certain app VMs, and that naturally isolate one app end to end from another. Examples are CANBus, ModBus or other RS485 based protocols.

Native Drive

For some other protocols which are used on both north and south bounds, like Wifi and mobile broadband, we provide native drivers.

If in a long term we need a native driver, but it’s too complex to do in a short term, we have an option to put it in pass though firstly as mitigation.

OT/IT Collaboration on Gateways

After IoT gateways are virtualized, the overlapping and trust issue could be resolved as done in data centers. OT organization manages IoT applications and data in gateways, similar to developers in Line of Businesses; and IT team manages infrastructure (computing, storage, networking and security) resources in gateways, as in data centers.

Why Intel CPU Based IoT Gateways is the Best Platform for Virtualization Technology from VMware

Intel is a recognized world leader in personal computer and commercial server markets. To strengthen enterprise capabilities on industrial IoT and edge computing, Intel provides multiple series of CPUs and NIC chips to fulfill this great mission. With the great performance and virtualization features provided by Intel chips, like 64bit, VT-x, VT-d, EPT, TPM support in new Atom and Xeon D series, we could consolidate low-end ARM based gateways into high-end Intel CPU based gateways with security hardening and isolation as in data centers.

VMware is the global leader on enterprise IT infrastructure and mobile device management. With its core expertise in Device Management, Operational Analytics, Security, and Cloud Management, VMware is working with strategic IoT partners to help meet the needs of IoT across things, edge, and platforms. On the gateways, VMware is working with multiple hardware vendors in Intel IoT Solutions Alliance to build an integrated platform in the Edge. As enterprises prepare for the onslaught of upcoming IoT use cases, VMware IoT solutions ensure your business is ready to support them.

Conclusion

Enterprise IoT is extremely hard to scale. Customers have to cobble together different heterogeneous offerings to make their IoT use cases work. 

Project Asteroid brings market proven and mature virtualization technology into the Edge layer of enterprise IoT solutions. We expect to work closely with enterprise IT teams together to build a unified platform on virtualized gateways, and provide multiple tenancy, isolation and enhanced security on edge computing for enterprise IoT customers.

When looking at ways to improve your business with IoT scenarios, some of the most important decisions you’ll make are the vendor and partners to work with. While there are many IoT vendors with conflicting messages, VMware has a clear strategy, a long history of success and trust in your current environment, and a global ecosystem of partners who are experts at managing the “Content plane.” Working together, we can help your business unlock the potential of the IoT so you get a full, end-to-end IoT solution that works for you.