Exploration and Practices of Edge Computing: Chapter 2 Build and Installation of Virtualization Equipment
Exploration and Practices of Edge Computing
Chapter 2 Build Image and Install Virtualized Devices
The previous article outlined the direction of cloud-based collaboration and device management. This article will describe how to use VMware's free software to build image and install virtualized devices.
Introduction to vSphere Hypervisor
VMware is the global leading provider of virtualization software, hybrid cloud computing and enterprise user computing solutions, and vSphere is its core flagship product. In April 2020, VMware released vSphere 7.0 to the world, providing more new features, especially the revolutionary Tanzu platform that fully supports Kubernetes.
vSphere is a widely used virtualization platform. Not only in data centers, the core functions required to run on devices have been included in the free product vSphere Hypervisor.
Verified device specifications
Here is the technical specifications of free vSphere Hypervisor to ru on devices, NOT the Hardware Compatibility List (HCL) required to provide commercial product support.
About 20 models of devices from multiple manufacturers that meet at least the following hardware specifications have been verfied technically to run vSphere Hypervisor.
- CPU: x86-64 2+ core (1GHz +), support VT-x / AMD-V
- Memory: 4GB (ECC or non-ECC)
- Storage: SSD / HDD / Flash 16GB
- Network: 1Gbps wired NIC
Download related software
Mandetory software
Note: You need to register an account at VMware.com before download the free vSphere Hypervisor and get the serial number. An ISO image is necessary. If you want to customize the image, you must download the offline package too.
- vSphere Hypervisor
Optional drivers
Note: Because some devices carry Realtek or other NIC chips unsupported officially by vSphere 6.x / 7.0, you need to download them from the community website. Such as:
- Realtek NIC driver: Realtek PCIe NIC
- USB NIC driver: USB NIC (including onboard, non physical USB NIC)
Optional tools
Note: You may choose either CLI or GUI approach to customize ESXi image. If you want to thoroughly customize the image, it is recommended to leverage vCenter Server.
- vCenter Server 7.0: GUI approach. After registering an account, you can download and try it for free for 60 days, or use an installed vCenter instance.
- VMware PowerCLI 12.0: CLI approach
- OVF Tools 4.4: Tools for deploying VM images via CLI
- ESXCLI 7.0: CLI for remote management of ESXi
Customize ESXi image for devices
CLI approach
The steps to install VMware PowerCLI are skipped.
The following is an example to add a USB NIC driver into ESXi 7.0 image.
- Add vSphere 7.0 depot
- Add USB NIC driver depot
- Obtain the image profile name in ESXi 7.0
- Copy a new image profile ESXi7-USB from the image profile ESXi-7.0.0-15843807-no-tools without vmtools
- Add USB NIC driver vmkusb-nic-fling to the newly created image profile ESXi7-USB
- Set AcceptanceLevel to VMwareCertified for the new image profile ESXi7-USBNIC (this step is optional)
- Export the new image profile ESXi7-USBNIC as ISO image ESXi7-USBNIC.iso
This exported ISO image ESXi7-USBNIC.iso can be used to install ESXi 7.0 on devices after burned to a USB thumb disk.
Note: The Realtek PCIe NIC driver provided by the community does not yet support ESXi 7.0, and you can only customize ESXi 6.7 image with it. The steps are similar to the above example.
GUI approach
You can also leverage Auto Deploy in vCenter Server to customize ESXi image. More benefits are easily choosing to add or remove other integration packages (VIB), further reducing the size of the image, or adding certain Kickstart scripts.
Here is another example to customize ESXi 7.0 image.
- Add vSphere 7.0 software depot
- Add USBNIC software depot
- Create new software depot for customized ESXi 7.0 image
- Create a new image profile, add USB NIC driver vmkusb-nic-fling, delete vmtools, and set Acceptance Level to VMware Certified (optional) .
Note: In this step, you can delete more integrated packages (VIBs) to compress the size of the image further, such as unused NIC and HBA drivers on the device. If you add PCIe Realtek NIC driver to ESXi 6.7 image, remember to select Community Support for Acceptance Level.
- Export new image
This exported ISO image can be used to install ESXi 7.0 after burned to a USB thumb disk.
Install ESXi on devices
Connect hardware
Before installing the ESXi image on devices, make sure that the device has sufficient hardware resources, especially the memory listed above (at least 4GB), hard disk (non-eMMC, at least 16GB) and network (wired NIC on 1Gbps).
Connect the keyboard, monitor, USB thumb disk, network cable, etc. according to the user manual of the device. If that is a headless device, you need to connect the appropriate serial port to a display in debug mode according to the device manual.
BIOS settings
In some devices, even if 4GB memory is configured, the video memory is shared within the main memory. In this case, it is recommended to set the shared video memory to the minimum (such as 32MB) in the BIOS before the official installation.
In addition, make sure that the hardware virtualization instructions in the BIOS, either Intel Virtualization Technology or AMD Secure Virtual Machine, is set to Enabled, and Secure Boot or TPM is set to disabled.
Start the installation
Boot from the “burned” USB flash drive according to the BIOS settings. When you see the following ESXi loading screen, quickly press Shift + O keys and add the kernel parameter ignoreHeadless = TRUE. This is to support Headless mode.
After that, ESXi should be able to be installed normally as on the server. After the installation is complete, you will be prompted to restart.
Restart settings
After restarting, remember to quickly press Shift + O keys again when you see the loading screen above, and also add the kernel parameter ignoreHeadless = TRUE. This is because the kernel parameter has not been saved yet, but will be saved in the next step. After this restart, in addition to setting the NIC address (if not DHCP), it is recommended to open the SSH and ESXi Shell services for remote debugging.
After logging into the device with ssh root@192.168.0.10 for example, change and save the kernel parameters as follows.
esxcfg-advcfg --set-kernel "TRUE" ignoreHeadless
esxcfg-advcfg --- get-kernel ignoreHeadless
Verify successful installation
Restart the device again, this time there is no need to type Shift + O keys to adjust the kernel parameter. You can now log in on ESXi web UI too.
- GUI: https://192.168.0.10/
After logging in, you can apply the serial number which you obtained when you downloaded free vSphere Hypervisor before.
Create datastore manually (optional)
For large computers such as servers, there is generally enough storage space. After installing vSphere Hypervisor or ESXi, usually at least one available datastore is automatically generated. Only with datastore can virtual machines be created and actual edge applications can be deployed. However, for a device with only a small amount of storage space (such as a USB flash disk less than 32GB), a default datastore may be be created automatically.
For example: on ESXi 6.7, the remaining space after installing the system will not be partitioned; on ESXi 7.0, in the left space after installing the base system a VMFSL partition will be created as follows. This VMFSL partition does not actually store any data, nor can it be used as a normal VMFS partition. On virtualized devices, this VMFSL partition will not be used in most cases. We will delete it and free up space to create a VMFS partition manually.
- Log in to the virtualized device
- Get the target storage partition table
The key point following is to replace AA31E02A400F11DB9590000C2911D1B8 with the VMFS UUID AA31E02A400F11DB9590000C2911D1B8 in the 7th partition. Do not modify anything else.
"1 64 204863 C12A7328F81F11D2BA4B00A0C93EC93B 128" \
"5 208896 2306047 EBD0A0A2B9E5443387C068B6B72699C7 0" \
"6 2308096 4405247 EBD0A0A2B9E5443387C068B6B72699C7 0" \
"7 4407296 20971486 AA31E02A400F11DB9590000C2911D1B8 0"
- Get coredump partition status
- Unlock coredump partition
- Refresh the partition table of the target storage
"1 64 204863 C12A7328F81F11D2BA4B00A0C93EC93B 128" \
"5 208896 2306047 EBD0A0A2B9E5443387C068B6B72699C7 0" \
"6 2308096 4405247 EBD0A0A2B9E5443387C068B6B72699C7 0" \
"7 4407296 20971486 AA31E02A400F11DB9590000C2911D1B8 0"
- Create a datastore on the new VMFS partition
Now you can find that a new datastore has been created, and you can rename it to "datastore1".
Deploy edge applications
Next, virtualized devices can be used to deploy edge applications. For example, the EdgeX Foundry project under LF Edge can be successfully deployed to virtualized devices with a Ubuntu virtual machine image. In the workshop on EdgeX Foundry China Day in December 2019, we have shown examples of EdgeX Foundry deployed on virtualized devices, working properly. More than 10 container instances in EdgeX Foundry microservice architecture are all running in the same Ubuntu virtual machine instance on virtualized devices.
If the same virtualized device has considerable left resources after deploying an EdgeX Foundry instance, you can deploy several other edge applications supported by any other OSes. For example, Azure IoT Edge based on Windows IoT Core, “real-time” control software based on Zephyr, Wind River or QNX, legacy systems based on DOS, signage applications based on Android x86, modern edge applications based on Raspbian or Ubuntu Core, etc.
There are two common ways to deploy virtual machine images: GUI approach via ESXi Web UI or CLI approach via OVF Tools. Because it is the same as the deployment process of other virtual machine images, the detailed steps are not repeated here.
Passthrough PCIe and USB devices
When deploying edge applications on virtualized devices, you may need to set up pass-through for certain PCIe or USB devices, especially to connect wireless sensors in the field, accelerators in the device, Wifi or LTE/5G NICs to the cloud. These settings are not different from the methods in the data center and will not be repeated here.
Next
In the manner described above, the entire process of installing a virtualized device and deploying edge applications on it can be completed. This is acceptable for developers to conduct development, testing, and small-scale experiments.
But it's far from desireable if considering the deployment of edge computing in a production environment, such as large-scale, unattended, remote monitoring, automatic update, etc. The next article will introduce how to manage large-scale virtualized devices in the cloud.
Comments