.png){kind=logo}
Anyone remotely familiar with PCIe technology will recognize the terms “x1”, “x4”, “x8” and “x16”. They are typically part of a PCIe device’s name or description. The “x” value represents the device’s lane count. In many cases, this numbers represent the PCIe cards performance capabilities (electrical lanes or bandwidth). However, it is best not to take the number at face value. In some cases, “x#” can reflect the card’s physical size or PCIe slot requirement (known as PCIe length or mechanical lanes). This value may or may not correspond with its actual performance capability.
Determining a PCIe device’s true electrical and mechanical lane rating is of critical importance when evaluating a high-performance PCIe card, especially an NVMe device, as it (in part) determines how well the SSDs will be able to perform. “x16” is paramount, but you will need to make sure this number isn’t just about the card’s physical requirement, and provided x16 lanes of bandwidth is available, that the card in question can make the most of it.
This article attempts to shine a spotlight on the terminology associated with PCIe lanes, and examine the differences between electrical and mechanical lanes.
Deciphering PCIe Terminology: Examining the difference between Electrical and Mechanical Lanes
How do you determine the card’s true PCIe bandwidth capability and lane speed? As mentioned previously, “x#” of lanes doesn’t always translate directly into the card’s throughput. This article examines “x” lanes terminology; Electrical lanes (the actual PCIe host bandwidth), Mechanical Lanes (the physical size requirement of the AIC), and how the lane count influences storage performance.
What is meant by Mechanical and Electrical PCIe lanes?
First, let’s examine the following product title:
4x M.2 NVMe SSD to PCIe 3.0 x8 / x16 Adapter Card
The first part, “4x M.2 NVMe SSD” suggests the card supports up to four M.2 NVMe SSDs – pretty self-explanatory. The second part of the description “PCIe 3.0 x8 /x16” is a bit more complicated – this refers to the card’s PCIe lanes. In the above example, “PCIe 3.0 x8 / x16” can have two meanings:
1) Mechanical Lane requirement: the type of PCIe slot required by the card (physical connection, to the computer’s motherboard). “x8 / x16” suggests the card can be physically installed into a PCIe slot with x8 or x16 mechanical lanes. This means the card has a mechanical lane rating of x8.
How did you determine this, you may ask? To explain, the general rule of thumb for a PCIe connector is that they are upwards compatible; that is, a PCIe card can be physically installed into any PCIe slot with the same “x#” rating, or higher. For example:
· a card with x4 mechanical lanes can be physically installed into a x4, x8 or x16 slot
· a card rated for x8 mechanical lanes can be installed into a x8 or x16 slot
· however, a card with a mechanical lane rating of 16 can only be installed into PCIe slot with x16 mechanical lanes.
There are exceptions, of course. Some computers/devices have PCIe slots that are classified as “open-end”, “slotted” or “notched” – this means there is a physical indentation in the slot that enables cards with a high x# mechanical lane rating to be installed.
This PCIe x4 slot is “open-ended”. The slot, or cut on the right-hand side of the slot enables it to accept larger PCI cards.
2) Electrical Lanes (PCIe bandwidth): the card’s performance level. This corresponds with the PCIe card’s “Upstream & Downstream Bandwidth” capabilities. In this particular example, we can deduce that the card is rated at x8 electrically.
Why you ask? Recall our rule for the Mechanical Lane requirement; x8 is the maximum the card can possibly deliver. In order to provide x16 lanes of bandwidth, the card would have to be equipped with a mechanical x16 connector. And, a mechanical x16 connector is simply too large to insert into a standard slot rated at x8 mechanically.
As a general rule, a PCIe device can only provide x# of electrical lanes equal to or lesser than its “x#” mechanical lane rating. For example, a PCIe card rated at x16 mechanically, could potentially provide x16, x8, x4 or even x1 lanes of electrical bandwidth. However, a card with an x4 mechanical rating could only provide x4 or x1 lanes of electrical bandwidth.
How do PCIe lanes influence NVMe storage performance?
PCIe lanes, or PCIe bandwidth, is really referring to the AIC’s electrical lanes (the actual PCIe lane speed). As a general rule, the higher this value (from x1 to x16) the better – “more lanes” essentially means “more performance”. However, the reasons go far beyond an increasing number.
Transfer Speed (Throughput): The most obvious advantage that more lanes provide is the larger performance threshold. A PCIe device rated at x16 provides 16-times the transfer bandwidth as one with an x1 rating. Naturally, x16 is optimal; it provides 16GB/s of bandwidth for PCIe Gen3, and 32 for Gen4. This translates into a real world 14,000MB/s and 28,000MB/s, respectively. This is obviously useful for large SSD configurations, as a single Gen3 SSD can deliver approximately 3,500MB/s, while a Gen4 doubles this to 7,000MB/s. The x16 threshold enables up to four NVMe SSDs to operate at full speed, concurrently.
Concurrent I/O (Parallelism): The higher the threshold, the more simultaneous I/O is possible. NVMe SSDs can execute a huge number of concurrent tasks, as their Queue depth (the number of I/O requests that storage device can handle at one time) is far superior to that of an SAS/SATA SSD (tens of thousands, compared to tens or hundreds). The advantage NVMe has over convention storage is eye opening; 64K commands with a depth of 64K vs. 32 commands and a depth of 256. Unsurprisingly, a larger performance threshold (PCIe lane bandwidth) streamlines this process, as a larger number of simultaneous data streams can be sustained.
Minimizes Latency: More bandwidth lowers latency. In other words, a wider lane speeds up the follow of traffic, and helps eliminate the risk of a bottleneck. Minimizing latency improves performance on multiple levels. It shortens response times, loads software applications faster, and streamlines file transfer (whether it be read or write).
Avoid or Eliminate Performance Bottlenecks: The more lanes that are available to the NVMe storage media, the faster and more efficiently it can operate. NVMe media performs at its best when each SSD has access to x4 lanes. As such, reducing lane count or PCIe generation can seriously degrade performance. An NVMe AIC with insufficient bandwidth will be unable to allocate x4 lanes to each SSD; they will be forced to operate at lower speeds (x2 or even x1).
Scalability: The benefits of a higher lane count isn’t exclusive to the AIC in question. A motherboard or computing platform with a larger number of PCIe lanes (both electrical and mechanical) will be able to support faster and/or more NVMe AICs, and enable each hosted SSD perform at optimal speeds. Platforms with healthy lane counts provide more flexibility to an administrator to expand or upgrade NVMe storage to keep pace with critical applications.
Conclusion: x16 is ideal, but make sure that bandwidth isn’t going to waste.
By now it should be clear how Electrical and Mechanical lanes are related, but different. An electrical lane rating of x16 is what you want to shoot for when evaluating PCIe NVMe AICs; it provides the maximum transfer bandwidth possible for a single PCIe slot, and will help maximize the performance potential of any NVMe configuration. And of course, an x16 bandwidth requires a card slotted for an x16 mechanical slot. However, it’s important to remember that the raw numbers don’t tell the whole story. You will want an NVMe AIC that can allocate the maximum number of lanes to each hosted SSD. HighPoint NVMe AICs and PCIe AIC Drives do exactly that – x4 lanes per SSD to ensure optimal performance.
Want to how this is done? Learn More
Exploring the Powerhouse: A Deep Dive into PCIe Switch Chipsets for PCIe Gen3-M.2 NVMe Cards