Sunday 19 May 2024 - 01:52:06 UTC 
RSS Feed
SARPi Project - Slacking on a Raspberry Pi
Install a PCIe M.2 NVMe SSD on a Raspberry Pi 5
This SARPi mini-project will enable you to install and configure a PCIe M.2 NVMe SSD on a Raspberry Pi 5 using the FPC connector. The flexible printed circuit (FPC) connector is located on top of the board, next to the power button and above the microSD card slot, marked "PCIe". This connector breaks out PCIe Gen 2.0 x1 interface for fast peripherals and is exposed in a 16-pin 0.5mm pitch configuration.
Raspberry Pi 5 PCIe interface FPC connector
More detailed information and specifications for third-party developers is available on the Raspberry Pi Connector for PCIe Datasheet.
The Raspberry Pi 5 PCIe Gen 2.0 x1 interface for fast peripherals supports more than just M.2 NVMe SSDs. Graphics cards, wireless cards, TPU accelerators, M.2 HATs, and other adapters, are being tested and used by hobbyists and hardware DIY enthusiasts. As time progresses more hardware will become available that makes use of the Raspberry Pi 5 PCIe interface which will greatly expand the functionality and scope of the device.
•
What you will need for this project
• A Raspberry Pi 5.
• A PCIe M.2 NVMe adapter board, such as the NVMe Base by Pimoroni, Geekworm X1002 or a HatDrive! by Pineberry.
• A M.2 NVMe SSD that's compatible with the PCIe adapter board you're using.
• A microSD card with the Raspberry Pi OS installed on it.
The PCIe M.2 NVMe adapter board that's used is entirely your choice. There are a few available at the time of writing this guide [24 January 2024] but bear in mind that the one you use may not be the same as the one featured in this SARPi mini-project. Whether it's a top or bottom adapter board they generally work in a similar way. So adjustments will need to be made from any instructions given here, as applicable.
Quick reference links
For quick reference within this PCIe M.2 NVMe SSD guide:
• Installing a PCIe M.2 NVMe adapter board on the Raspberry Pi 5
• Configuring the Raspberry Pi 5 for the NVMe Base adapter board
• Detecting and configuring the PCIe NVMe SSD on the operating system
• Update the Raspberry Pi 5 EEPROM to the latest version
• Raspberry Pi 5 EEPROM BOOT_ORDER boot modes settings
• Installing Slackware AArch64 on a PCIe NVMe SSD on the Raspberry Pi 5
•
Installing a PCIe M.2 NVMe adapter board on the Raspberry Pi 5
Specific details on how to install and use PCIe M.2 NVMe adapter boards on the Raspberry Pi 5 are usually included with them, and/or made available by other means. Sometimes manufacturers and suppliers offer instructions online, and/or as a download, or by means of a video (e.g. on YouTube, etc.). End users and enthusiasts also create their own blogs and videos to assist in installing new hardware.
NVMe Base by Pimoroni
The PCIe M.2 NVMe adapter board we are using is the NVMe Base by Pimoroni. In our deliberations over which PCIe M.2 NVMe adapter board to purchase we investigated quite a few suppliers, and many user reviews and videos, to determine which would best suit our specific requirements. The SARPi Project chose the NVMe Base by Pimoroni over any other alternatives.
NVMe Base adapter board (full kit contents) by Pimoroni
The NVMe Base by Pimoroni seemed like the best option for its design, functionality, accommodating a range of 2230-2242-2260-2280 M.2 SSDs, no additional power supply required (it uses the Rasbperry Pi 5 PSU), and with a retail price of approx. half that of its nearest competitors. NVMe Base follows the new PIP design guidelines by Raspberry Pi Ltd. ensuring full compatibility, ease of use, and long term support.
Which M.2 NVMe SSD to use?
Almost any M-key NVMe SSD can be used. It really doesn't matter if it's PCIe Gen5, PCIe Gen4, or PCIe Gen3, (etc.), as nearly all of these drives are technically both forwards and backwards compatible. Some SSDs will perform better than others and be less problematic. It's prudent to do some investigating beforehand into which SSDs may be most suitable/reliable based on your choice of NVMe adapter.
The M.2 NVMe SSD that we've chosen is a Kioxia Exceria G2 1TB M.2 PCIe NVMe 3D TLC SSD. It was selected because the Kioxia Exceria G2 SSD was successfully tested and benchmarked by Pimoroni and is recommended by them as one of the compatible drives to use with the NVMe Base for the Raspberry Pi 5.
Kioxia Exceria G2 SSD
Additionally, and by comparison, the cost of the Kioxia Exceria G2 1TB M.2 PCIe NVMe 3D TLC SSD was just a few dollars more than a Samsung 980 250GB PCIe 3.0 NVMe M.2 SSD. So, with 4x the storage capacity within the same price range, it made a lot of sense to select this Kioxia Exceria G2 1TB SSD. It may not be the fastest SSD available in its class but is more than adequate for our needs and purposes on the Raspberry Pi 5.
Assembly
Pimoroni have intuitively created a step by step Getting Started with NVMe Base for Raspberry Pi 5 guide, and configuration instructions for the NVMe Base on a Raspberry Pi 5, on their website. They've also made available a video guide for assembling and installing the NVMe base on a Raspberry Pi 5 (on YouTube) as additional visual guidance. We took full advantage of these resources which enabled us to easily mount and install the Kioxia Exceria G2 1TB M.2 PCIe NVMe 3D TLC SSD on the NVMe Base adapter board. The FPC cable was fitted and installed between the NVMe Base adapter board and the Raspberry Pi 5. The NVMe Base adapter board was subsequently mounted underneath the Raspberry Pi 5 using the stand-offs and screws supplied in the kit. It was relatively simple to assemble and quicker than anticipated. All thanks to the extensive instructions and end-user support from Pimoroni.
Raspberry Pi 5 with NVMe Base adapter board installed underneath (credit: Pimoroni)
•
Configuring the Raspberry Pi 5 for the NVMe Base adapter board
For setting up and configuring the Raspberry Pi 5 to use the PCIe interface and M.2 NVMe SSD we'll need to use the Raspberry Pi OS. This is what the microSD card with the Raspberry Pi OS installed will be used for. Seeing as we're using the Raspberry Pi OS to configure the Raspberry Pi 5 to use the PCIe interface, and NVMe SSD we've just assembled and installed, we might as well do everything that's required at the same time, to save time.
"Why aren't you using Slackware for this?" you may wonder. Well, at the time of writing this mini-project [24 January 2024], we had initially intended to use Slackware AArch64 but came across one small snag. This related to editing the Raspberry Pi 5 EEPROM bootloader configuration, primarily for setting the BOOT_ORDER boot modes sequence.
Consequently, we decided that the easiest and most logical solution to this conundrum was to use the Raspberry Pi OS. Unfortunately, there seemed to be no relatively easy way to achieve this with Slackware AArch64 running on the Raspberry Pi 5 at the time.
The SARPi Project fully expects that it will be possible to update the Raspberry Pi 5 EEPROM, and all other elements, using Slackware AArch64 once this device becomes officially supported by Slackware developers. Which will hopefully be very soon™.
Updating the Raspberry Pi OS
We plugged in the microSD card containing the Raspberry Pi OS into the Raspberry Pi 5 and booted it.
The first thing we want to do after logging into the Raspberry Pi OS is evelate to 'root' user.
root@raspberrypi:~#
Then we are going to fully update the system using the following commands:
root@raspberrypi:~# apt full-upgrade
Enabling the PCIe interface on the Raspberry Pi 5
Once the system has been fully updated we're going to add a few lines to the /boot/config.txt file. This can be done with whichever text editor is preferred. We'll use 'nano' for this:
The following lines that we'll be adding will appear at the end (i.e. bottom) of the /boot/config.txt file:
# Enable the PCIe external connector
#dtparam=pciex1
dtparam=nvme
When it's been done, it should look similar to the following screenshot:
NB: We're adding these lines to the /boot/config.txt file in order to initialise (activate) the PCIe interface. Without this parameter being loaded, the interface will not be enabled on the system and the NVMe SSD will not be useable as a storage device. Although we've added two device-tree parameters, one is commented out and we'll only be using one of them.
• The 'dtparam=pciex1' and 'dtparam=nvme' device-tree parameters both do the same thing. 'dtparam=nvme' is just an alias of 'dtparam=pciex1' so either can be used for this purpose.
• By default the Raspberry Pi 5 PCIe interface operates in PCIe Gen 2.0 mode. To enable the experimental PCIe Gen 3.0 mode, add the following line to the end of the /boot/config.txt file:
dtparam=pciex1_gen=3
• The PCIe interface can also be downgraded to PCIe Gen 1.0 mode by adding the following line to the end of the /boot/config.txt file:
dtparam=pciex1_gen=1
Once these lines have been added to the /boot/config.txt file the system needs to be restarted:
•
Detecting and configuring the PCIe NVMe SSD on the operating system
After rebooting the system, we'll elevate to 'root' user once again and check that the PCIe NVMe SSD has been detected and is seen as a storage device.
The 'lsblk' and 'fdisk' commands will be used to confirm this. The following screenshot shows the output of running these commands on our system:
What this output tells us:
• The 'lsblk' output shows that our microSD card and PCIe NVMe SSD is detected by the system, but only the microSD card contains partitions and data. It also shows us that the device ID of our NVMe SSD storage device is 'nvme0n1' which we then used with the 'fdisk' command to get more information about it.
• The 'fdisk -l /dev/nvme0n1' output shows that the KIOXIA-EXCERIA G2 SSD has been detected and is a blank storage device containing no partitions or data.
These are exactly the results we were expecting. Which means we can now go ahead with adding partitions to our PCIe NVMe SSD storage device. This will be done while we're still using the Raspberry Pi OS but with installing Slackware AArch64 on the PCIe NVMe SSD in mind, as this is our intended objective and ultimate goal.
Adding partitions to the PCIe NVMe SSD storage device
We're going to create a /boot partition on our NVMe SSD and format it with a VFAT filesystem. We'll also create /swap and / (root) partitions at the same time, but we won't format these as this can be done during the Slackware AArch64 installation process.
We already know the device ID of our PCIe NVMe SSD storage device and we'll use this information specifically to target this device in order to create partitions on it.
Adding partitions to a storage device can be achieved using the 'cdgisk' command TUI (text user interface):
Creating a partition for /boot
In the 'cgdisk' TUI we navigate with the cursor keys to select [ New ] and press the key to create a partition. Pressing for the default starting block. We input 200M (megabytes) of size and press the key. When asked to enter the type (Hex code or GUID) we type 0700 and press the key - because we want this to be a VFAT filesystem so that the Raspberry Pi 5 will boot from it. When asked for a partition name we just hit the key.
Creating a partition for /swap
Once we're back at the main 'cgdisk' TUI we'll use the cursor keys to highlight the free space below the partition we've just created. Then we select [ New ] once again and press the key to create an additional partition. This will be for our /swap partition. Again, we press the for the default starting block. Because our Raspberry Pi 5 has 8GB RAM and we have a 1TB NVMe SSD available, we are going to go completely OTT here and input 16G (gigabytes) of size and press the key. When asked to enter the type (Hex code or GUID) we type 8200 and press the key - because we want this to be a recognised swap partition. When asked for a partition name we just hit the key.
Creating a partition for / (root partition)
Once more, when we're back at the main 'cgdisk' TUI we'll highlight the free space below the newly created partitions. Then we select [ New ] again and press the key to create an additional partition. This will be for our / root partition. Again, we press the for the default starting block. When asked for a size we'll just press the key to use up any available free space on the NVMe SSD. When asked to enter the type (Hex code or GUID) we type 8300 (if that's not already offered as the default) and press the key - because we want this to be a recognised Linux filesystem partition on which we'll install our operating system. When asked for a partition name we just hit the key.
Once we're back at the main 'cgdisk' TUI and can see the list of our partitions and are happy with them, we can select [ Write ] and press the key. When we're asked, "Are you sure you want to write the partition table to disk?" we type yes and press to save our partition table.
NB: Ignore the 1007.0 KiB free space at the top of the list - and any similar unallocated space at the start of a storage drive for a Linux operating system. A certain amount (approx. 1-2MB) of unallocated space is standard in this location and very much expected. Just leave it alone. Mess around with it at your own peril! 
• Unlike 'fdisk' and 'cfdisk', 'gdisk' and 'cgdisk' has no ability to assign a bootable flag to any partition table entries. This is because 'gdisk' and 'cgdisk' creates partitions in the Globally Unique Identifier (GUID) partition table (GPT) format and does not require any bootable flag to know where a boot partition is located. The bootable flag is only relevant to MBR partition table schema, where any record that's been tagged as bootable indicates where the boot loader resides.
Now we are able to review what our new GPT partition table contains:
1 200MiB (Microsoft basic data) /boot partition
2 16.0 GiB (Linux swap) /swap partition
3 915.3 GiB (Linux filesystem) / partition (root partition)
This is looking very good indeed. If we weren't happy with any partitions that have been created, or made any mistakes in their creation, we can easily delete and re-create them in the 'cgdisk' TUI. Always remember that any changes are not saved until [ Write ] is carried out successfully.
Once we're done with creating partitions we can select [ Quit ] to exit the 'cgdisk' TUI and go back to the command prompt.
Formatting the /boot partition
Now we need to format the /boot partition. So we'll use the 'fdisk' command once more to check the IDs of the partitions we've just created:
The output will show all the information we need to identify the /boot partition:
From this output we now know that the /boot partition (i.e. 200M Microsoft basic data) is identified as '/dev/nvme0n1p1'. So, to format this partition with a VFAT filesystem we will use this command:
It only takes a few seconds and it's all done.
To check and make sure everything went according to plan, we can use the 'lsblk -f' or 'lsblk -o FSTYPE' command options to verify the filesystem on the /boot partition:
Which confirms exactly what we expected to see. Now we're finished with creating and formatting partitions on our new PCIe NVMe SSD storage device.
•
Update the Raspberry Pi 5 EEPROM to the latest version
The next thing that needs to be done is to update the Raspberry Pi 5 EEPROM to the 'latest' release version. To change from a 'stable' to the 'latest' release version the 'raspi-config' TUI dialog needs to be used.
Before doing that, we can check the status of our Raspberry Pi 5 EEPROM bootloader by using the 'rpi-eeprom-update' command:
The output of the 'rpi-eeprom-update' command will display status of the current EEPROM bootloader and if there are any updates available.
Note the "Use raspi-config to change the release" message. If yours is the 'stable' release version then that should be changed to the 'latest' release version. To do this run the 'raspi-config' command:
Navigate with the cursor keys down to "6 Advanced Options" and press the key. Then navigate down to "A5 Bootloader Version"and press the key.
Make sure "E1 Latest Use the latest version boot ROM software" is highlighted and press the key.
You'll be asked to confirm this by selecting "Yes" and pressing the key. Once this has been done select <Finish> at the bottom of the 'raspi-config' TUI dialog to exit back to the command prompt.
•
Raspberry Pi 5 EEPROM BOOT_ORDER boot modes settings
For the Raspberry Pi 5 to boot from a PCIe NVMe SSD it needs to be specified in the EEPROM boot loader configuration. By default, booting from a PCIe NVMe SSD is not an option and we need to edit the Raspberry Pi 5 EEPROM BOOT_ORDER boot modes settings in order to make that possible. This will be done by using the the 'rpi-eeprom-config' command.
DON'T use the 'rpi-eeprom-config' command just yet, but first read the following IMPORTANT information:
!!! IMPORTANT !!! The Raspberry Pi 5 EEPROM contains essential system configuration and boot code that is responsible for initializing the hardware during startup. Things can, and do, go wrong at the best of times. So, make sure that you fully understand what is required, and what needs to be done in order to succeed. Also make sure to check, double-check, and triple-check, any and all modifications to configuration settings before saving and/or exiting the 'rpi-eeprom-config' editor.
NB: It's always prudent, if not essential, to have a good read through the Raspberry Pi documentation BOOT_ORDER configuration settings and understand them before entering the rpi-eeprom-config editor and/or making any changes.
Just in case you missed it above, it's always prudent (if not essential) to have a thorough read of the Raspberry Pi documentation BOOT_ORDER configuration settings and fully understand them before entering the rpi-eeprom-config editor and/or making any changes.
From the BOOT_ORDER boot modes options available, we'll always want our Raspberry Pi 5 to boot from a microSD card first and foremost, if one is available, before any other devices. This is for our own needs and personal requirements. For example, if we ever get to a stage where our SSD storage device will not boot to a command prompt, or into the operating system, then we want to be able to boot from a microSD card that contains a rescue disk environment in order to address and fix any problems. So, this is precisely why we want this device to come first in our Raspberry Pi 5 BOOT_ORDER sequence. This way, nothing else will interefere with a rescue disk environment being loaded successfully.
Our second BOOT_ORDER option will be the PCIe NVMe SSD storage device. This being second in the BOOT_ORDER sequence (a bootable microSD card being first in line) means the system installed on our PCIe NVMe SSD will load successfully when a bootable microSD card is not present.
We're also going to specify a third BOOT_ORDER sequence option, which will be a bootable USB storage device. Such as a USB memory stick or a USB-SATA adapter connected to a SSD storage device.
For our own needs and personal requirements the three BOOT_ORDER options we've just highlighted are all we'll ever use. Yours may not be the same so bear this in mind when considering your own personal requirements, moving forward.
While referring to the Raspberry Pi documentation BOOT_ORDER configuration settings, they are represented as a 32-bit unsigned integer where each nibble represents a boot-mode. The boot modes are attempted in lowest significant nibble to highest significant nibble order. The BOOT_ORDER property defines the sequence for the different boot modes. These are read sequentially, from right to left, and up to 8 digits may be defined. The BOOT_ORDER configuration examples in the Raspberry Pi documentation are a very good guide of what is required to succeed.
Seriously, if you haven't already read through the Raspberry Pi documentation BOOT_ORDER configuration settings and understood them, then do it now before entering the 'rpi-eeprom-config' editor and/or making any changes to the Raspberry Pi 5 EEPROM bootloader configuration.
Updating the Raspberry Pi 5 EEPROM using rpi-eeprom-config
So, we've decided that we want our Raspberry Pi 5 to boot storage devices specifically in the following order:
◦ 1. microSD card
◦ 2. PCIe NVMe SSD
◦ 3. USB storage device
Now we'll run the 'rpi-eeprom-config' command and make the necessary modifications to effectuate this BOOT_ORDER sequence.
We are only going to modify the BOOT_ORDER integer values (i.e. the whole numbers on the end of this line) to enable the Raspberry Pi 5 to boot in the predefined boot modes sequence according to what we've decided previously.
NB: If you are using a PCIe M.2 NVMe SSD adapter board with the Raspberry Pi 5 that doesn't follow the PIP design guidelines by Raspberry Pi Ltd. (or doesn't support the auto-detection mechanism) you'll need to add the following additional line in the EEPROM config below the BOOT_ORDER line:
PCIE_PROBE=1
We've edited the BOOT_ORDER line to reflect our choice of sequential boot modes in a fixed order. From the screenshot above this is:
These values have been assigned according to the Raspberry Pi documentation BOOT_ORDER configuration settings instructions and will be read sequentially, from right to left:
1 = microSD card
6 = PCIe NVMe SSD
4 = USB storage device
So in our chosen BOOT_ORDER sequence; (1) microSD card will be attempted first, (6) PCIe NVMe SSD will be attempted second, (4) USB storage device will be attempted third.
A device that is detected and initiates the BOOT_ORDER process will take precedence over any others. So, using our settings for example, if a microSD card is present but not bootable then the BOOT_ORDER will attempt to boot from the PCIe NVMe SSD instead, and then attempt to boot from a USB storage device after that if the PCIe NVMe SSD is not bootable. If a bootable microSD card is present then the BOOT_ORDER will boot from it and not attempt to try any other devices. Basically, the BOOT_ORDER sequence that's been determined within the EEPROM bootloader will attempt to boot each boot modes device and the first bootable device it successfully finds will be the one that the system boots with.
If we were unsure, or think we may have made mistakes, or not fully understand what we're trying to do, there is a way to go back, revise, and start over with the 'rpi-eeprom-config' editor. Just exit out of the 'rpi-eeprom-config' editor by pressing the and keys together. As a result we'll be back at the command prompt. Then, if the message is not already displayed on the screen, we can type the following command to cancel any pending changes:
This will result in the following output shown in the screenshot below:
From here we can start over by running the 'rpi-eeprom-config --edit' command once more. We can practically do this repeatedly. Or at least until we are certain about keeping any changes that have been made.
When we have made absolutely sure what we want our BOOT_ORDER sequence to be, we can save and exit the 'rpi-eeprom-config' editor by pressing the and keys together.
Once we're back at the command prompt there's a message requesting a reboot of the Raspberry Pi 5 to apply any pending updates. So we'll do that now.
However, after logging in again we then want to power down the system so that we can swap microSD cards and install Slackware AArch64 onto our new PCIe NVMe SSD on the Raspberry Pi 5.
•
Installing Slackware AArch64 on a PCIe NVMe SSD on the Raspberry Pi 5
When the system has shut down completely, the Raspberry Pi OS microSD card is removed and replaced with another microSD card that contains the Slackware AArch64 installer.
From here we can boot into the Slackware AArch64 installer and run 'setup' to go through the familiar Slackware installation process as usual, remembering to install the entire operating system onto our PCIe NVMe SSD storage device and not the microSD card.
•
Slackware AArch64 on a Raspberry Pi 5 PCIe M.2 NVMe SSD [Updated: 25 January 2024]
Slackware AArch64 took approx. 30 minutes for a full installation onto the Kioxia Exceria G2 1TB M.2 PCIe NVMe 3D TLC SSD using the NVMe Base on the Raspberry Pi 5 Model B (8GB RAM). The system was much more responsive from the outset, much faster to install, and completed without any issues or problems whatsoever.
The SARPi Project is extremely pleased and surprised by the addition of the NVMe Base by Pimoroni and Kioxia Exceria G2 1TB M.2 PCIe NVMe 3D TLC SSD to our Raspberry Pi 5. It has increased data throughput speeds, decimated data transfer times, and greatly decreased waiting and response times. Plus it's much more convenient than the SATA-USB3 SSD adapter that was used previously.
•
Thanks & Credits!
Thank you for reading and taking part in this SARPi mini-project. We hope you found it interesting, educational, and great fun like we did.
Credit and a huge thank you to Pimoroni for their NVMe Base for the Raspberry Pi 5. As always, our perpetual gratitude and sincere thanks to Patrick Volkerding, and the entire Slackware Team, for producing a truly wonderful OS. Without you, and the work you do, the SARPi Project would not exist.
If you have any questions or need help, visit the Slackware Linux Forum on Linux Questions. Or get in touch on the #SARPi IRC channel on irc.libera.chat.
•
