Tables of Contents:

Introduction

This document should be read together with the following articles:

And the references used here are, among the documents above, the following docs:

How to read this document ?

You should read this!
This is deeper information, you may ignore.
This is a CPU vulnerability!
This is a mitigation technique.
This is a CPU flag to advertise Kernel and/or Hypervisor.

Notes:

1. Sometimes the mitigation name is the acronym for the CPU flag, example: Indirect Branch Prediction Barrier - ibpb. And sometimes some CPU flag might mitigate more than 1 vulnerability.

2. Downloading the HTML from the wiki (through the proper Download link at the top) and opening it locally will make it better for reading (no CSS themes). You can also print it to PDF if you'd like.

Also, do check articles dates and look for newer ones not yet updated in this document. Newer side channel attack techniques might have been discovered by the time this page was created. If you feel this page should be updated, please send us an e-mail at: ubuntu-server@lists.ubuntu.com mentioning what should be updated and why.

Related CPU Vulnerabilities (CVEs)

Side Channel Attacks - Spectre and Meltdown

1. CVE-2017-5753 - Bounds Check Bypass (Variant 1 / Spectre)

2. CVE-2017-5715 - Branch Target Injection (Variant 2 / Spectre)

3. CVE-2017-5754 - Rogue Data Cache Load (Variant 3 / Meltdown)

Side Channel Attacks - Others

1. CVE-2018-3665 - Lazy FP Save/Restore (LazyFP)

2. CVE-2018-3693 - Bounds Check Bypass Store (Variant (or Spectre) 1.1 and 1.2 / BCBS)

3. CVE-2018-3640 - Rogue System Register Read (RSRE / Variant 3a)

4. CVE-2018-3639 - Speculative Store Bypass (SSB / Variant 4 / Spectre-NG)

L1 Terminal Fault (L1TF)

1. CVE-2018-3615 - Intel SGX (Software Guard Extensions) (Foreshadow / L1TF)

2. CVE-2018-3620 - Operating Systems and System Management Mode (Fault-OS / SMM) (L1TF)

3. CVE-2018-3646 - Virtualization Extensions (L1TF)

Microarchitectural Data Sampling (MDS)

1. CVE-2018-12126 - Microarchitectural Store Buffer Data Sampling (MSBDS / Fallout)

2. CVE-2018-12127 - Microarchitectural Load Port Data Sampling (MLPDS / RIDL)

3. CVE-2018-12130 - Microarchitectural Fill Buffer Data Sampling (MFBDS / ZombieLoad)

4. CVE-2019-11091 - Microarchitectural Data Sampling Uncacheable Memory (MDSUM)

This CVE topic structure is used in the same order in all subsequent sections of this document.
The idea is to allow reader to map:

New CPUs & Firmwares | Kernel and OS Mitigations

Side Channel Attacks - Spectre and Meltdown

1. CVE-2017-5753 - Bounds Check Bypass (Variant 1 / Spectre)

   _user pointer sanitization

   
   To mitigate this exploit technique in the Linux kernel, we first identify instruction sequences that can be tricked into exploitable behavior. Since a sequence must fit a specific pattern and operate on untrusted data before any potential exploit, not all conditional branches are exploitable. Techniques such as static analysis or manual inspection can identify these sequences.

2. CVE-2017-5715 - Branch Target Injection (Variant 2 / Spectre)

   
   The branch target injection exploit targets a processor’s 'indirect branch predictor'. Indirect branches are used very differently than the conditional branches that the first exploit may target. In Linux, indirect branches are used relatively rarely compared to conditional branches, but they are used in critical locations. In addition, the compiler may insert indirect branches without the programmer ever being aware.
   
   Since the compiler generates these branches, mitigation against this exploit is the most straightforward when the compiler can simply avoid generating vulnerable branch sequences. A software construct called retpoline can be deployed to help ensure that a given indirect branch is resistant to exploitation. The compiler, automatically, or the programmer, manually, can insert the retpoline logic in binary code. Retpoline deliberately steer the processor’s branch prediction logic to a trusted location, preventing a potential exploit from steering them elsewhere.
   

   OS and Kernel Mitigation: Retpoline

   
   Retpoline needs code (kernel and/or userland) to be recompiled by a compiler with this feature enabled. If not using retpoline, firmware microcode might ALSO mitigate against Spectre Variant 2 on several different platforms.
   
   

   Firmware Mitigations

   
   for Spectre Variant 2 are also provided by different CPU microcodes. Taking amd64 by example, there are MORE THAN 1 mitigation included in more recent microcodes AND about to (or being, or recently done) be included in newer CPUs internal logic.
   

   Enhanced Indirect Branch Restricted Speculation (IBRS): Restricts speculation of indirect branches

   
   With enhanced IBRS, the predicted targets of indirect branches executed cannot be controlled by software that was executed in a less privileged predictor mode or on another logical processor. As a result, software operating on a processor with enhanced IBRS need not use WRMSR to set IA32_SPEC_CTRL.IBRS after every transition to a more privileged predictor mode. Software can isolate predictor modes effectively simply by setting the bit once.
   
   

   Single Thread Indirect Branch Predictors (STIBP): Prevents indirect branch predictions from being controlled by the sibling HW thread

   
   STIBP is an indirect branch control mechanism that restricts the sharing of branch prediction between logical processors on a core. ... as the logical processors sharing a core may share indirect branch predictors, allowing one logical processor to control the predicted targets of indirect branches by another logical processor of the same core.
   
   Enabling IBRS prevents software operating on one logical processor from controlling the predicted targets of indirect branches executed on another logical processor. For that reason, it is not necessary to enable STIBP when IBRS is enabled.
   
   

   Indirect Branch Predictor Barrier (IBPB): Ensures earlier code's behavior does not control later indirect branch predictions. Complements Enhanced IBRS

   
   IBPB is an indirect branch control mechanism that establishes a barrier, preventing software that executed before the barrier from controlling the predicted targets of indirect branches executed after the barrier on the same logical processor. A processor supports IBPB if it enumerates CPUID.(EAX=7H,ECX=0):EDX[26] as 1.
   
   IBPB does not define a new mode of processor operation that controls the branch predictors, unlike Indirect Branch Restricted Speculation (IBRS) and Single Thread Indirect Branch Predictors (STIBP). As a result, it is not enabled by setting a bit in the IA32_SPEC_CTRL MSR. Instead, IBPB is a “command” that software executes when necessary.
   

3. CVE-2017-5754 - Rogue Data Cache Load (Variant 3 / Meltdown)

   
   The rogue data cache load exploit targets a processor’s speculative data loading mechanisms. Even though the processor’s access control may protect a piece of data, it may still be read speculatively before an illegal access is determined to exist. The first two exploits require manipulating the kernel to do something for an attacker, while this exploit occurs entirely within code which is under the control of an attacker. This means we cannot modify the kernel to mitigate this exploit, we must fundamentally change where kernel data is available.

   Kernel Mitigation: Kernel Page Table Isolation

   
   The mitigation for this is conceptually very simple: instead of relying on a processor’s access-control mechanisms to protect data, simply remove the data instead.
   
   In Linux, this mitigation is referred to as Kernel Page Table Isolation (PTI). This mitigation removes the data from the reach of exploits by having the kernel maintain two independent copies of the hardware page tables. One copy contains the minimal set of data and code needed to run an application and enter or exit the kernel, but it does not contain valuable kernel data. The other set of page tables, active only while the kernel is running, contains everything needed for the kernel to function normally, including its private data.
   

Side Channel Attacks - Others

1. CVE-2018-3665 - Lazy FP Save/Restore (LazyFP)

   
   Lazy FPU state leak, or Lazy FP State Restore, or LazyFP, affects only Intel Core CPUs. It is a combination of flaws in the speculative execution technology presented in some CPUs, and how certain the OS handle context switching on the floating point unit. Basically a local process can leak the content of the FPU register to another process.

   Kernel Mitigation

   
   Intel recommends OS to use Eager FP state restore in lieu of Lazy FP state restore. Linux Kernel does that since kernel 4.5. Older kernels running on processors that support the xsaveopt instruction are also not affected. You can verify if your system has support for xsaveopt by locating the "xsaveopt" feature listed in the flags section of the /proc/cpuinfo file. No particular mitigation for QEMU/KVM.

2. CVE-2018-3693 - Bounds Check Bypass Store (Variant (or Spectre) 1.1 and 1.2 / BCBS)

   
   Many Intel processors use memory disambiguation predictors that allows loads to be executed speculatively before it is known whether the load’s address overlaps with a preceding store’s address. This may happen if a store’s address is unknown when the load is ready to execute. If the processor predicts that the load address will not overlap with the unknown store address, the load may execute speculatively. However, if there was indeed an overlap, then the load may consume stale data. When this occurs, the processor will re-execute the load to ensure a correct result.
   
   Through the memory disambiguation predictors, an attacker can cause certain instructions to be executed speculatively and then use the effects for side channel analysis.

   Software Based Mitigations

   Process Isolation

   
   To run untrusted code in different processes, so the address space is not shared among trusted and untrusted code.
   
   

   LFENCE (requires new firmware/microcode)

   
   Software can insert a LFENCE between a store and a subsequent load to prevent the load from executing before the previous store's address is known. This should be applied only in places with realistic risk, as it incurs in performance penalty.
   

   Firmware Mitigation

   
   Speculative Store Bypass Disable (SSBD)

3. CVE-2018-3640 - Rogue System Register Read (RSRE / Variant 3a)

   
   The rogue system register read method, uses both speculative execution and side channel cache methods to infer the value of some processor system register state which is not architecturally accessible by the attacker. This method uses speculative execution of instructions that read system register state while the processor is operating at a mode or privilege level that does not architecturally allow the reading of that state. The set of system registers that can have their value inferred by this method is implementation-specific.
   

   
   Intel’s analysis is that the majority of state exposed by the Variant 3a method is not secret or sensitive, nor directly enables attack or exposure of user data. The use of the Variant 3a method by an attacker may result in the exposure of the physical addresses for some data structures and may also expose the linear addresses of some kernel software entry points.
   

   
   Knowledge of these physical and linear addresses may enable attackers to determine the addresses of other kernel data and code elements, which may impact the efficacy of the Kernel Address Space Layout Randomization (KASLR) technique.
   
   

Firmware Mitigation


Speculative Store Bypass Disable (SSBD)

4. CVE-2018-3639 - Speculative Store Bypass (SSB / Variant 4 / Spectre-NG)

   
   The speculative store bypass method takes advantage of a performance feature present in many high performance processors that allows loads to speculatively execute even if the address of preceding potentially overlapping store is unknown. In such a case, this may allow a load to speculatively read a stale data value. The processor will eventually correct such cases, but an attacker may be able to discover “confused deputy” code which may allow them to use speculative execution to reveal the value of memory that is not normally accessible to them.
   
   

OS and Firmware Mitigation: Speculative Store Bypass Disable (SSBD)


SSBD prevents a load from executing speculatively until the address of all older stores are known. In Ubuntu, SSBD is OFF by default because it is not needed by most programs and carries a notable performance impact. A prctl() has been added (PR_SPEC_STORE_BYPASS) that enables developers to opt into the mitigation on a per process basis.

Applications using seccomp filters will be implicitly opted into the mitigations (snaps, processes inside LXD containers, sandboxed firefox and chromium processes, will have SSBD mitigation enabled out of the box).

L1 Terminal Fault (L1TF)

1. CVE-2018-3615 - Intel SGX (Software Guard Extensions) (Foreshadow / L1TF)

2. CVE-2018-3620 - Operating Systems and System Management Mode (Fault-OS / SMM) (L1TF)

3. CVE-2018-3646 - Virtualization Extensions (L1TF)

Microarchitectural Data Sampling (MDS)

1. CVE-2018-12126 - Microarchitectural Store Buffer Data Sampling (MSBDS / Fallout)

2. CVE-2018-12127 - Microarchitectural Load Port Data Sampling (MLPDS / RIDL)

3. CVE-2018-12130 - Microarchitectural Fill Buffer Data Sampling (MFBDS / ZombieLoad)

4. CVE-2019-11091 - Microarchitectural Data Sampling Uncacheable Memory (MDSUM)

QEMU/KVM ONLY - Mitigations and CPU Flags

Spectre and Meltdown mitigation detection tool

• 
  There are different tools that can be used to detect these vulnerabilities in an environment (Architecture, CPU, Firmware Levels, OS Kernel, Hypervisor Types and Versions) AND to show you if all of them are mitigated or not.
  

  
  A very good one that can be used is: https://github.com/speed47/spectre-meltdown-checker

  
  Bellow you will find this tools output from a Cascade Lake CPU Server. Note that this document tried to explain all its acronyms for CPU Vulnerabilities and/or Mitigation Techniques above, so do search this document for an acronym found at the tool output if you need further explanation.
  

  
  TIP: I'd run the tool in the Hypervisor Host AND inside the VM OS, to make sure that both, the Host and the Guest have all mitigations in place. With the information provided in previous section of this document (QEMU/KVM Guests HW Mitigations) you will be able to make sure you're correctly advertising your REAL CPU Mitigation Flags into your Guests (so their OS can choose better technique for each of the vulnerabilities).
  
  

  Spectre and Meltdown mitigation detection tool v0.42-1-g91d0699

  Checking for vulnerabilities on current system Kernel is Linux
  4.18.0-23-generic #24~18.04.1-Ubuntu SMP Thu Jun 13 17:08:52 UTC 2019 x86_64
  CPU is Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz

  Hardware check

  * Hardware support (CPU microcode) for mitigation techniques
    * Indirect Branch Restricted Speculation (IBRS)
      * SPEC_CTRL MSR is available: YES
      * CPU indicates IBRS capability: YES (SPEC_CTRL feature bit)
    * Indirect Branch Prediction Barrier (IBPB)
      * PRED_CMD MSR is available: YES
      * CPU indicates IBPB capability: YES (SPEC_CTRL feature bit)
    * Single Thread Indirect Branch Predictors (STIBP)
      * SPEC_CTRL MSR is available: YES
      * CPU indicates STIBP capability: YES (Intel STIBP feature bit)
    * Speculative Store Bypass Disable (SSBD)
      * CPU indicates SSBD capability: YES (Intel SSBD)
    * L1 data cache invalidation
      * FLUSH_CMD MSR is available: YES
      * CPU indicates L1D flush capability: YES (L1D flush feature bit)
    * Microarchitecture Data Sampling
      * VERW instruction is available: YES (MD_CLEAR feature bit)
    * Enhanced IBRS (IBRS_ALL)
      * CPU indicates ARCH_CAPABILITIES MSR availability: YES
      * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: YES
    * CPU explicitly indicates not being vulnerable to Meltdown/L1TF (RDCL_NO): YES
    * CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO): NO
    * CPU/Hypervisor indicates L1D flushing is not necessary on this system: YES
    * Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA): NO
    * CPU explicitly indicates not being vulnerable to Microarchitectural Data Sampling (MDS_NO): YES
    * CPU supports Software Guard Extensions (SGX): NO
    * CPU microcode is known to cause stability problems: NO
      (model 0x55 family 0x6 stepping 0x6 ucode 0x4000021 cpuid 0x50656)
    * CPU microcode is the latest known available version: NO
      (latest version is 0x4000024 dated 2019/04/07 according to builtin MCExtractor DB v112 - 2019/05/22)

  CPU vulnerability to the speculative execution attack variants

    * Vulnerable to CVE-2017-5753 (Spectre Variant 1, bounds check bypass): YES
    * Vulnerable to CVE-2017-5715 (Spectre Variant 2, branch target injection): YES
    * Vulnerable to CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load): NO
    * Vulnerable to CVE-2018-3640 (Variant 3a, rogue system register read): YES
    * Vulnerable to CVE-2018-3639 (Variant 4, speculative store bypass): YES
    * Vulnerable to CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault): NO
    * Vulnerable to CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault): NO
    * Vulnerable to CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault): NO
    * Vulnerable to CVE-2018-12126 (Fallout, microarchitectural store buffer data sampling (MSBDS)): NO
    * Vulnerable to CVE-2018-12130 (ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)): NO
    * Vulnerable to CVE-2018-12127 (RIDL, microarchitectural load port data sampling (MLPDS)): NO
    * Vulnerable to CVE-2019-11091 (RIDL, microarchitectural data sampling uncacheable memory (MDSUM)): NO

  CVE-2017-5753 aka Spectre Variant 1, bounds check bypass

  * Mitigated according to the /sys interface: YES (Mitigation: __user pointer sanitization)
  * Kernel has array_index_mask_nospec: YES (1 occurrence found of x86 64 bits array_index_mask_nospec())
  * Kernel has the Red Hat/Ubuntu patch: NO
  * Kernel has mask_nospec64 (arm64): NO
  
  > STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)

  CVE-2017-5715 aka Spectre Variant 2, branch target injection

  * Mitigated according to the /sys interface: YES
   (Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling)
  
  * Mitigation 1
    * Kernel is compiled with IBRS support: YES
      * IBRS enabled and active: YES
    * Kernel is compiled with IBPB support: YES
      * IBPB enabled and active: YES
  
  * Mitigation 2
    * Kernel has branch predictor hardening (arm): NO
    * Kernel compiled with retpoline option: YES
    * Kernel supports RSB filling: YES
  
  > STATUS: NOT VULNERABLE (IBRS + IBPB are mitigating the vulnerability)

  CVE-2017-5754 aka Variant 3, Meltdown, rogue data cache load

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports Page Table Isolation (PTI): YES
    * PTI enabled and active: NO
    * Reduced performance impact of PTI: YES 
      (CPU supports INVPCID, performance impact of PTI will be greatly reduced)
  * Running as a Xen PV DomU: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-3640 aka Variant 3a, rogue system register read

  * CPU microcode mitigates the vulnerability: YES
  
  > STATUS: NOT VULNERABLE (your CPU microcode mitigates the vulnerability)

  CVE-2018-3639 aka Variant 4, speculative store bypass

  * Mitigated according to the /sys interface: YES
    (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)
  * Kernel supports disabling speculative store bypass (SSB): YES (found in /proc/self/status)
  * SSB mitigation is enabled and active: YES (per-thread through prctl)
  * SSB mitigation currently active for selected processes: YES
    (systemd-journald systemd-logind systemd-networkd systemd-resolved systemd-timesyncd systemd-udevd)
  
  > STATUS: NOT VULNERABLE (Mitigation: Speculative Store Bypass disabled via prctl and seccomp)

  CVE-2018-3615 aka Foreshadow (SGX), L1 terminal fault

  * CPU microcode mitigates the vulnerability: N/A
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-3620 aka Foreshadow-NG (OS), L1 terminal fault

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports PTE inversion: YES (found in kernel image)
  * PTE inversion enabled and active: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-3646 aka Foreshadow-NG (VMM), L1 terminal fault

  * Information from the /sys interface: Not affected
  * This system is a host running a hypervisor: YES
  * Mitigation 1 (KVM)
    * EPT is disabled: NO
  * Mitigation 2
    * L1D flush is supported by kernel: YES (found flush_l1d in /proc/cpuinfo)
    * L1D flush enabled: NO
    * Hardware-backed L1D flush supported: YES (performance impact of the mitigation will be greatly reduced)
    * Hyper-Threading (SMT) is enabled: YES
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-12126 aka Fallout, microarchitectural store buffer data sampling (MSBDS)

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
  * Kernel mitigation is enabled and active: NO
  * SMT is either mitigated or disabled: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-12130 aka ZombieLoad, microarchitectural fill buffer data sampling (MFBDS)

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
  * Kernel mitigation is enabled and active: NO
  * SMT is either mitigated or disabled: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2018-12127 aka RIDL, microarchitectural load port data sampling (MLPDS)

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
  * Kernel mitigation is enabled and active: NO
  * SMT is either mitigated or disabled: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

  CVE-2019-11091 aka RIDL, microarchitectural data sampling uncacheable memory (MDSUM)

  * Mitigated according to the /sys interface: YES (Not affected)
  * Kernel supports using MD_CLEAR mitigation: YES (md_clear found in /proc/cpuinfo)
  * Kernel mitigation is enabled and active: NO
  * SMT is either mitigated or disabled: NO
  
  > STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)