Game on RAID0

TomCatT · February 25, 2023, 9:11am

Hello,

Not sure what's different, but I would like to know if anyone has managed to boot off of a 4 RAID0 system. I can only manage to have three drives to work with a raid.

I'm not all sure what changed between the two.

Almost forgot:

System:
Kernel: 6.1.12-zen1-1-zen arch: x86_64 bits: 64 compiler: gcc v: 12.2.1
parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
root=UUID=539e8cf3-d4eb-4b5e-b185-25a850911191 rw rootflags=subvol=@
quiet quiet splash rd.udev.log_priority=3 vt.global_cursor_default=0
loglevel=3 dm-mod=1
Desktop: KDE Plasma v: 5.27.1 tk: Qt v: 5.15.8 wm: kwin_x11 vt: 1 dm: SDDM
Distro: Garuda Linux base: Arch Linux
Machine:
Type: Desktop System: Gigabyte product: X570 AORUS XTREME v: -CF serial: N/A
Mobo: Gigabyte model: X570 AORUS XTREME serial: N/A UEFI: American
Megatrends LLC. v: F36 date: 12/27/2022
CPU:
Info: model: AMD Ryzen 9 5950X socket: AM4 bits: 64 type: MT MCP
arch: Zen 3+ gen: 4 level: v3 note: check built: 2022 process: TSMC n6 (7nm)
family: 0x19 (25) model-id: 0x21 (33) stepping: 2 microcode: 0xA20120A
Topology: cpus: 1x cores: 16 tpc: 2 threads: 32 smt: enabled cache:
L1: 1024 KiB desc: d-16x32 KiB; i-16x32 KiB L2: 8 MiB desc: 16x512 KiB
L3: 64 MiB desc: 2x32 MiB
Speed (MHz): avg: 3400 min/max: 2200/5083 boost: enabled
base/boost: 3400/5050 scaling: driver: acpi-cpufreq governor: performance
volts: 1.1 V ext-clock: 100 MHz cores: 1: 3400 2: 3400 3: 3400 4: 3400
5: 3400 6: 3400 7: 3400 8: 3400 9: 3400 10: 3400 11: 3400 12: 3400
13: 3400 14: 3400 15: 3400 16: 3400 17: 3400 18: 3400 19: 3400 20: 3400
21: 3400 22: 3400 23: 3400 24: 3400 25: 3400 26: 3400 27: 3400 28: 3400
29: 3400 30: 3400 31: 3400 32: 3400 bogomips: 217617
Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm
Vulnerabilities: <filter>
Graphics:
Device-1: AMD Navi 31 [Radeon RX 7900 XT/7900 XTX] driver: amdgpu v: kernel
arch: RDNA-3 code: Navi-3x process: TSMC n5 (5nm) built: 2022+ pcie: gen: 4
speed: 16 GT/s lanes: 16 ports: active: DP-1 off: DP-2
empty: DP-3,HDMI-A-1 bus-ID: 0e:00.0 chip-ID: 1002:744c class-ID: 0300
Device-2: Valve 3D Camera type: USB driver: uvcvideo bus-ID: 2-3.1:3
chip-ID: 28de:2400 class-ID: 0e02 serial: <filter>
Display: x11 server: X.Org v: 21.1.7 with: Xwayland v: 22.1.8
compositor: kwin_x11 driver: X: loaded: modesetting dri: radeonsi
gpu: amdgpu display-ID: :0 screens: 1
Screen-1: 0 s-res: 1920x1080 s-dpi: 96 s-size: 508x285mm (20.00x11.22")
s-diag: 582mm (22.93")
Monitor-1: DP-1 res: 1920x1080 hz: 60 dpi: 90
size: 543x302mm (21.38x11.89") diag: 621mm (24.46") modes: N/A
API: OpenGL v: 4.6 Mesa 22.3.6 renderer: AMD Radeon RX 7900 XTX (gfx1100
LLVM 15.0.7 DRM 3.49 6.1.12-zen1-1-zen) direct-render: Yes
Audio:
Device-1: AMD driver: snd_hda_intel v: kernel pcie: bus-ID: 1-3.3.2:8 gen: 4
chip-ID: 28de:2102 class-ID: 0a00 speed: 16 GT/s lanes: 16 serial: <filter>
bus-ID: 0e:00.1 chip-ID: 1002:ab30 class-ID: 0403
Device-2: AMD Starship/Matisse HD Audio vendor: Gigabyte
driver: snd_hda_intel v: kernel pcie: gen: 4 speed: 16 GT/s lanes: 16
bus-ID: 10:00.4 chip-ID: 1022:1487 class-ID: 0403
Device-3: Valve VR Radio type: USB driver: cdc_acm,hid-generic,usbhid
Device-4: Valve VR Radio & HMD Mic type: USB
driver: hid-generic,snd-usb-audio,usbhid bus-ID: 1-3.3.3:10
chip-ID: 28de:2102 class-ID: 0102 serial: <filter>
Device-5: Plantronics RIG 800HX type: USB
driver: plantronics,snd-usb-audio,usbhid bus-ID: 1-4:3 chip-ID: 047f:c045
class-ID: 0300
Device-6: WOER type: USB driver: snd-usb-audio bus-ID: 7-4.2:4
chip-ID: 8888:1234 class-ID: 0300 serial: <filter>
Sound API: ALSA v: k6.1.12-zen1-1-zen running: yes
Sound Server-1: PulseAudio v: 16.1 running: no
Sound Server-2: PipeWire v: 0.3.66 running: yes
Network:
Device-1: Intel I211 Gigabit Network vendor: Gigabyte driver: igb v: kernel
pcie: gen: 1 speed: 2.5 GT/s lanes: 1 port: f000 bus-ID: 07:00.0
chip-ID: 8086:1539 class-ID: 0200
IF: enp7s0 state: up speed: 1000 Mbps duplex: full mac: <filter>
Device-2: Intel Wi-Fi 6 AX200 driver: iwlwifi v: kernel pcie: gen: 2
speed: 5 GT/s lanes: 1 bus-ID: 08:00.0 chip-ID: 8086:2723 class-ID: 0280
IF: wlp8s0 state: down mac: <filter>
Bluetooth:
Device-1: Intel AX200 Bluetooth type: USB driver: btusb v: 0.8 bus-ID: 1-5:5
chip-ID: 8087:0029 class-ID: e001
Report: bt-adapter ID: hci0 rfk-id: 0 state: up address: <filter>
RAID:
Supported mdraid levels: raid0
Device-1: md0 maj-min: 9:0 type: mdraid level: raid-0 status: active
state: clean size: 2.79 TiB
Info: report: N/A blocks: 3000216576 chunk-size: 512k super-blocks: 1.2
Components: Online:
0: sdb maj-min: 8:16 size: 953.87 GiB state: active sync
1: sdc maj-min: 8:32 size: 953.87 GiB state: active sync
2: sdd maj-min: 8:48 size: 953.87 GiB state: active sync
Drives:
Local Storage: total: raw: 10.14 TiB usable: 10.14 TiB used: 8.86 GiB (0.1%)
ID-1: /dev/nvme0n1 maj-min: 259:4 vendor: TeamGroup model: TM8FP6001T
size: 953.87 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 type: SSD serial: <filter> rev: T0709A3 temp: 39.9 C scheme: MBR
SMART: yes health: PASSED on: 304d 6h cycles: 873
read-units: 17,568,138 [8.99 TB] written-units: 60,023,020 [30.7 TB]
ID-2: /dev/nvme1n1 maj-min: 259:0 vendor: Crucial model: CT1000P1SSD8
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 type: SSD serial: <filter> rev: P3CR010 temp: 35.9 C scheme: MBR
SMART: yes health: PASSED on: 1y 156d 4h cycles: 2,241
read-units: 33,421,684 [17.1 TB] written-units: 33,141,533 [16.9 TB]
ID-3: /dev/nvme2n1 maj-min: 259:2 vendor: Crucial model: CT1000P1SSD8
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 type: SSD serial: <filter> rev: P3CR013 temp: 38.9 C scheme: MBR
SMART: yes health: PASSED on: 1y 88d 19h cycles: 1,786
read-units: 21,905,987 [11.2 TB] written-units: 23,976,727 [12.2 TB]
ID-4: /dev/nvme3n1 maj-min: 259:6 vendor: Sabrent model: Rocket Q4
size: 3.64 TiB block-size: physical: 512 B logical: 512 B speed: 63.2 Gb/s
lanes: 4 type: SSD serial: <filter> rev: RKT40Q.2 temp: 36.9 C scheme: GPT
SMART: yes health: PASSED on: 83d 22h cycles: 247
read-units: 2,143,259 [1.09 TB] written-units: 6,744,553 [3.45 TB]
ID-5: /dev/sda maj-min: 8:0 model: SSD-DCXGCC-1TB size: 953.87 GiB
block-size: physical: 512 B logical: 512 B sata: 3.2 speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 8A0 temp: 42 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 78d 17h cycles: 1000
read: 55.3 MiB written: 159.3 MiB
ID-6: /dev/sdb maj-min: 8:16 model: SSD-DCXGCC-1TB size: 953.87 GiB
block-size: physical: 512 B logical: 512 B sata: 3.2 speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 8A0 temp: 43 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 75d 17h cycles: 982
read: 47.6 MiB written: 71.5 MiB
ID-7: /dev/sdc maj-min: 8:32 model: SSD-DCXGCC-1TB size: 953.87 GiB
block-size: physical: 512 B logical: 512 B sata: 3.2 speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 8A0 temp: 40 C
SMART: yes state: enabled health: PASSED on: 76d 1h cycles: 987
read: 60.9 MiB written: 146.6 MiB
ID-8: /dev/sdd maj-min: 8:48 model: SSD-DCXGCC-1TB size: 953.87 GiB
block-size: physical: 512 B logical: 512 B sata: 3.2 speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 8A0 temp: 40 C
SMART: yes state: enabled health: PASSED on: 75d 12h cycles: 985
read: 54.2 MiB written: 64.9 MiB
ID-9: /dev/sde maj-min: 8:64 type: USB vendor: SanDisk model: USB 3.2Gen1
size: 28.65 GiB block-size: physical: 512 B logical: 512 B type: N/A
serial: <filter> rev: 1.00 scheme: MBR
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
ID-1: / raw-size: 953.37 GiB size: 953.37 GiB (100.00%)
used: 8.86 GiB (0.9%) fs: btrfs block-size: 4096 B dev: /dev/sda2
maj-min: 8:2
ID-2: /boot/efi raw-size: 511 MiB size: 511 MiB (100.00%)
used: 608 KiB (0.1%) fs: vfat block-size: 512 B dev: /dev/sda1 maj-min: 8:1
ID-3: /home raw-size: 953.37 GiB size: 953.37 GiB (100.00%)
used: 8.86 GiB (0.9%) fs: btrfs block-size: 4096 B dev: /dev/sda2
maj-min: 8:2
ID-4: /var/log raw-size: 953.37 GiB size: 953.37 GiB (100.00%)
used: 8.86 GiB (0.9%) fs: btrfs block-size: 4096 B dev: /dev/sda2
maj-min: 8:2
ID-5: /var/tmp raw-size: 953.37 GiB size: 953.37 GiB (100.00%)
used: 8.86 GiB (0.9%) fs: btrfs block-size: 4096 B dev: /dev/sda2
maj-min: 8:2
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default)
ID-1: swap-1 type: zram size: 31.27 GiB used: 1024 KiB (0.0%)
priority: 100 dev: /dev/zram0
Sensors:
System Temperatures: cpu: 44.1 C mobo: N/A gpu: amdgpu temp: 39.0 C
mem: 63.0 C
Fan Speeds (RPM): N/A gpu: amdgpu fan: 37
Info:
Processes: 542 Uptime: 2h 53m wakeups: 0 Memory: 31.27 GiB
used: 3.47 GiB (11.1%) Init: systemd v: 253 default: graphical
tool: systemctl Compilers: gcc: 12.2.1 Packages: pm: pacman pkgs: 1837
libs: 516 tools: octopi,paru Shell: Bash v: 5.1.16 running-in: konsole
inxi: 3.3.25
Garuda (2.6.15-1):
System install date:     2023-02-25
Last full system update: 2023-02-25 ↻
Is partially upgraded:   No
Relevant software:       snapper NetworkManager dracut
Windows dual boot:       No/Undetected
Failed units:

BluishHumility · February 26, 2023, 10:51am

What was your method for configuring the RAID?

TomCatT · February 27, 2023, 12:29pm

sgdisk all four drives

sda1 will be for /boot/efi

sd[b-d]1 are blank space holders

sd[a-d]2 will be used for /

Build the RAIDs

sgdisk the raids (just to be sure) /dev/md0

Preformat all drives now because the installer can't do it.

Run the installer and manually tell it where to put everything.

Once done, close the installer, and find where the installer mounts the / directory. That way, I don't have to use the subvol=@ option.

arch-chroot there

Save the raid0 data to mdadm.conf

Edit the mdadm.conf to include the gaming rig's hostname.
Ensure the modprobe has the raid drivers installed again, then dracut the bootloader.

Write to the grub to ensure that data get updated too.

Exit chroot, then reboot.

These same steps will work for a simple 3-drive raid0 setup.

This data is what I put together after trolling the interwebs for a few days now. I wanted to ask here because I'm sure I could use one or two extra steps. With the four-drive step, some dm_mdadm isn't getting loaded, but with three-drives, it is...?

BluishHumility · February 27, 2023, 1:23pm

I think you may be over-complicating the setup a little bit; setting up RAID on Btrfs is more simple than all of that, especially RAID0.

You can leave the installer out of it altogether–don’t worry about the disks at that stage, just pretend they aren’t even there.

Once you get the installation up, you can simultaneously format the drives as Btrfs and also identify them as RAID devices with mkfs.btrfs. See here: Using Btrfs with Multiple Devices - btrfs Wiki

# Stripe the data without mirroring, metadata are mirrored
mkfs.btrfs -d raid0 /dev/sdb /dev/sdc

That command is just an example, be sure to read through the rest of the available options to determine which specific flags you want to use (see another example below).

That’s basically it, you don’t even need to change fstab or anything because you can mount a Btrfs RAID using any disk used in creating the RAID. Grub is definitely not part of it, I’m not sure where you are getting that from.

This looks like a pretty good write-up if you want a (rather short) walk-through for the needed steps: How to Set Up Btrfs RAID

Setting Up RAID-0

In this section, you will learn how to set up a Btrfs RAID in the RAID-0 configuration using four HDDs (sdb, sdc, sdd, and sde). The HDDs are 20 GB in size.

$ sudo lsblk -e7

To create a Btrfs RAID in the RAID-0 configuration using four HDDs (sdb, sdc, sdd, and sde) run the following command:

$ sudo mkfs.btrfs -L data -d raid0 -m raid0 -f /dev/sdb /dev/sdc /dev/sdd /dev/sde

Here,

The –L option is used to set the filesystem label data.

The –d option is used to set the RAID profile raid0 for the filesystem data.

The –m option is used to set the RAID profile raid0 for the filesystem metadata.

The –f option is used to force the creation of the Btrfs filesystem, even if any of the HDDs have an existing filesystem.

The Btrfs filesystem data in the RAID-0 configuration should now be created, as you can see in the screenshot below.

You can mount the Btrfs RAID using any HDD/SSD you used to create the RAID.

The article continues with a few steps for confirming the RAID is properly configured and for checking filesystem usage information

TomCatT · March 1, 2023, 11:17am

You know, I played with that setup, but I didn't try to install it. I had a question about it and would seek it out later. This was also when I had to give myself a crash course in BTRFS too. I'm amazed at what it can do.

It shows me that I don't have to worry about drivers or the fstab. I have enough RAM. The installer will for sure take care of it. It's for gaming, anyways.

I've got the system built and don't mind starting over. But thank you for showing that method. I'll start again this weekend.

The only change I see is it needs to be on partition 2. sd[a-d]1 is for the UEFI. The 2s will be the root partition. Meaning sda1 is for only UEFI. The others 3 are just space holders.

TomCatT · March 2, 2023, 1:55pm

It worked like a champ on my MBP83. Garuda booted up in under 10 seconds. Maybe 8.

So the answer was in my face the whole time. I'm going to have to learn more about BTRFS.

I made the swap a RAID0 with mdadm. Thank you.

system · March 4, 2023, 1:55pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.