Mysteriously inconsistent RAM and swap use

Issues & Assistance
1

Hello,

I’ve noticed lately a strange behavior on my install due to my recent heavier use of RAM.

My system has 32GB of RAM, it is detected by the System Monitor, /proc/meminfo, and everything that seem to handle RAM.

The thing is that from time to time, I’ve noticed my computer freezing and slowing down, which made me curious about what was happening. Every time it slows down, the swap usage goes up, and it happens always around the use of 16GB of RAM, which is half what I’ve got.

Finding that strange, I’ve tried lowering the swappiness of the system, which changed nothing, and disabling completely the swap, which lead to a complete system freeze around 16GB use of RAM, kind of expected.

I’ve reproduced this by firing my firefox instance which has several hundreds of tabs (I know, but you’ll never know when you’ll need these tabs), and 2 or 3 instances of Jetbrain’s IDEs which are based on java (no docker or anything that could have “invisible” use of RAM).

This tends to push the overall RAM use to above 16GB.

When this happens, or near this happening (so around the use of 16GB RAM), I get a notice from some installed by default app called nohang that tells me that not much RAM is left and that I should close applications, even though I have almost 16GB free RAM to use.

Strangest thing is that it doesn’t happen when allocating more than 16GB block of memory through nohang’s own check (nohang -m).
Doing that, I can use my 32GB of ram without any swap happening.

I checked nohang’s config without seeing anything strange, and I cannot identify anything out of the ordinary overall that would explain this kind of behavior, so if anybody has a clue …

–
garuda-inxi:

System:
  Kernel: 6.5.3-zen1-1-zen arch: x86_64 bits: 64 compiler: gcc v: 13.2.1
    clocksource: tsc available: hpet,acpi_pm
    parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
    root=UUID=2a2936a6-cd5b-4072-b26c-35c719f86da7 rw rootflags=subvol=@
    quiet
    cryptdevice=UUID=47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c:luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
    root=/dev/mapper/luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
    rd.udev.log_priority=3 vt.global_cursor_default=0
    systemd.unified_cgroup_hierarchy=1 loglevel=3 ibt=off
  Desktop: KDE Plasma v: 5.27.8 tk: Qt v: 5.15.10 info: latte-dock
    wm: kwin_wayland vt: 1 dm: SDDM Distro: Garuda Linux base: Arch Linux
Machine:
  Type: Desktop System: CSL- GmbH & KG product: 5930 v: N/A
    serial: <superuser required>
  Mobo: ASUSTeK model: ROG STRIX X570-E GAMING WIFI II v: Rev X.0x
    serial: <superuser required> UEFI: American Megatrends v: 4204
    date: 02/24/2022
CPU:
  Info: model: AMD Ryzen 9 5900X bits: 64 type: MT MCP arch: Zen 3+ gen: 4
    level: v3 note: check built: 2022 process: TSMC n6 (7nm) family: 0x19 (25)
    model-id: 0x21 (33) stepping: 0 microcode: 0xA201016
  Topology: cpus: 1x cores: 12 tpc: 2 threads: 24 smt: enabled cache:
    L1: 768 KiB desc: d-12x32 KiB; i-12x32 KiB L2: 6 MiB desc: 12x512 KiB
    L3: 64 MiB desc: 2x32 MiB
  Speed (MHz): avg: 3578 high: 4563 min/max: 2200/4950 boost: enabled
    scaling: driver: acpi-cpufreq governor: schedutil cores: 1: 3639 2: 3623
    3: 3616 4: 3653 5: 4563 6: 4491 7: 3593 8: 3652 9: 3590 10: 4491 11: 3617
    12: 2200 13: 3700 14: 3595 15: 3139 16: 2200 17: 3590 18: 2869 19: 3593
    20: 3611 21: 3593 22: 4072 23: 3609 24: 3580 bogomips: 177252
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm
  Vulnerabilities: <filter>
Graphics:
  Device-1: AMD Navi 21 [Radeon RX 6900 XT] driver: amdgpu v: kernel
    arch: RDNA-2 code: Navi-2x process: TSMC n7 (7nm) built: 2020-22 pcie:
    gen: 4 speed: 16 GT/s lanes: 16 ports: active: DP-1,DP-2
    empty: DP-3,HDMI-A-1 bus-ID: 0d:00.0 chip-ID: 1002:73af class-ID: 0300
  Display: wayland server: X.org v: 1.21.1.8 with: Xwayland v: 23.2.1
    compositor: kwin_wayland driver: X: loaded: amdgpu
    unloaded: modesetting,radeon alternate: fbdev,vesa dri: radeonsi
    gpu: amdgpu d-rect: 5120x1440 display-ID: 0
  Monitor-1: DP-1 pos: primary,left res: 2560x1440 size: N/A modes: N/A
  Monitor-2: DP-2 pos: right res: 2560x1440 size: N/A modes: N/A
  API: OpenGL v: 4.6 Mesa 23.1.7-arch1.1 renderer: AMD Radeon RX 6900 XT
    (navi21 LLVM 16.0.6 DRM 3.54 6.5.3-zen1-1-zen) direct-render: Yes
Audio:
  Device-1: AMD Navi 21/23 HDMI/DP Audio driver: snd_hda_intel v: kernel pcie:
    gen: 4 speed: 16 GT/s lanes: 16 bus-ID: 0d:00.1 chip-ID: 1002:ab28
    class-ID: 0403
  Device-2: AMD Starship/Matisse HD Audio vendor: ASUSTeK
    driver: snd_hda_intel v: kernel pcie: gen: 4 speed: 16 GT/s lanes: 16
    bus-ID: 0f:00.4 chip-ID: 1022:1487 class-ID: 0403
  Device-3: FiiO K7 driver: snd-usb-audio type: USB rev: 2.0 speed: 480 Mb/s
    lanes: 1 mode: 2.0 bus-ID: 7-3:4 chip-ID: 2972:0047 class-ID: fe01
  Device-4: Elgato Systems GmbH Wave:3 driver: snd-usb-audio type: USB
    rev: 2.0 speed: 12 Mb/s lanes: 1 mode: 1.1 bus-ID: 7-4:7 chip-ID: 0fd9:0070
    class-ID: fe01 serial: <filter>
  API: ALSA v: k6.5.3-zen1-1-zen status: kernel-api with: aoss
    type: oss-emulator tools: N/A
  Server-1: PipeWire v: 0.3.80 status: active with: 1: pipewire-pulse
    status: active 2: wireplumber status: active 3: pipewire-alsa type: plugin
    4: pw-jack type: plugin tools: pactl,pw-cat,pw-cli,wpctl
Network:
  Device-1: MEDIATEK MT7921K Wi-Fi 6E 80MHz driver: mt7921e v: kernel pcie:
    gen: 2 speed: 5 GT/s lanes: 1 bus-ID: 05:00.0 chip-ID: 14c3:0608
    class-ID: 0280
  IF: wlp5s0 state: down mac: <filter>
  Device-2: Realtek RTL8125 2.5GbE vendor: ASUSTeK driver: r8169 v: kernel
    pcie: gen: 2 speed: 5 GT/s lanes: 1 port: e000 bus-ID: 06:00.0
    chip-ID: 10ec:8125 class-ID: 0200
  IF: enp6s0 state: up speed: 1000 Mbps duplex: full mac: <filter>
  Device-3: Intel I211 Gigabit Network vendor: ASUSTeK driver: igb v: kernel
    pcie: gen: 1 speed: 2.5 GT/s lanes: 1 port: d000 bus-ID: 07:00.0
    chip-ID: 8086:1539 class-ID: 0200
  IF: enp7s0 state: down mac: <filter>
  IF-ID-1: pan1 state: down mac: <filter>
Bluetooth:
  Device-1: MediaTek Wireless_Device driver: btusb v: 0.8 type: USB rev: 2.1
    speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 1-6:3 chip-ID: 0e8d:0608
    class-ID: e001 serial: <filter>
  Report: btmgmt ID: hci0 rfk-id: 0 state: up address: <filter> bt-v: 5.2
    lmp-v: 11 status: discoverable: no pairing: no class-ID: 7e0104
Drives:
  Local Storage: total: 1.36 TiB used: 355.35 GiB (25.4%)
  SMART Message: Unable to run smartctl. Root privileges required.
  ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Samsung model: SSD 980 1TB
    size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
    lanes: 4 tech: SSD serial: <filter> fw-rev: 1B4QFXO7 temp: 30.9 C
    scheme: GPT
  ID-2: /dev/nvme1n1 maj-min: 259:3 vendor: Samsung model: SSD 980 500GB
    size: 465.76 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
    lanes: 4 tech: SSD serial: <filter> fw-rev: 1B4QFXO7 temp: 52.9 C
    scheme: GPT
Partition:
  ID-1: / raw-size: 431.01 GiB size: 431.01 GiB (100.00%)
    used: 353.34 GiB (82.0%) fs: btrfs dev: /dev/dm-0 maj-min: 254:0
    mapped: luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
  ID-2: /boot/efi raw-size: 300 MiB size: 299.4 MiB (99.80%)
    used: 26.1 MiB (8.7%) fs: vfat dev: /dev/nvme1n1p1 maj-min: 259:4
  ID-3: /home raw-size: 431.01 GiB size: 431.01 GiB (100.00%)
    used: 353.34 GiB (82.0%) fs: btrfs dev: /dev/dm-0 maj-min: 254:0
    mapped: luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
  ID-4: /var/log raw-size: 431.01 GiB size: 431.01 GiB (100.00%)
    used: 353.34 GiB (82.0%) fs: btrfs dev: /dev/dm-0 maj-min: 254:0
    mapped: luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
  ID-5: /var/tmp raw-size: 431.01 GiB size: 431.01 GiB (100.00%)
    used: 353.34 GiB (82.0%) fs: btrfs dev: /dev/dm-0 maj-min: 254:0
    mapped: luks-47f3b1d0-c3ce-4a6e-bca6-cfe8ae775c2c
Swap:
  Kernel: swappiness: 10 (default 60) cache-pressure: 100 (default) zswap: no
  ID-1: swap-1 type: partition size: 34.45 GiB used: 1.99 GiB (5.8%)
    priority: -2 dev: /dev/dm-1 maj-min: 254:1
    mapped: luks-37248bd4-9b18-4bc6-9e4f-f90d911370b1
Sensors:
  System Temperatures: cpu: 48.0 C mobo: 46.0 C gpu: amdgpu temp: 62.0 C
    mem: 62.0 C
  Fan Speeds (rpm): N/A gpu: amdgpu fan: 538
Info:
  Processes: 643 Uptime: 11h 58m wakeups: 0 Memory: total: 32 GiB
  available: 31.25 GiB used: 15.67 GiB (50.1%) Init: systemd v: 254
  default: graphical tool: systemctl Compilers: gcc: 13.2.1 clang: 16.0.6
  Packages: pm: pacman pkgs: 2639 libs: 542 tools: octopi,paru pm: rpm pkgs: 0
  Shell: fish v: 3.6.1 running-in: konsole inxi: 3.3.29
Garuda (2.6.16-1):
  System install date:     2021-12-31
  Last full system update: 2023-09-23 ↻
  Is partially upgraded:   No
  Relevant software:       snapper NetworkManager mkinitcpio
  Windows dual boot:       Probably (Run as root to verify)
  Failed units:
2

Just post your garuda-inxi, please, it include all we need.

3

I’ve updated the post with the information.

1 Like
4

Check

jctl

and use something like htop to see what is havily in action.

BTW, always restart your system, it’s KDE :slight_smile:

5

I reboot my PC everyday, rarely put it in sleep :cry:

jctl doesn’t show much it seems.

jctl:

sept. 23 11:58:29 user-5930 systemd-modules-load[479]: Failed to find module 'vboxpci'
sept. 23 11:58:31 user-5930 kernel: ucsi_ccg 0-0008: failed to get FW build information
sept. 23 11:58:34 user-5930 bluetoothd[894]: src/adapter.c:reset_adv_monitors_complete() Failed to reset Adv Monitors: Failed (0x03)
sept. 23 11:58:34 user-5930 bluetoothd[894]: Failed to clear UUIDs: Failed (0x03)
sept. 23 11:58:34 user-5930 bluetoothd[894]: Failed to add UUID: Failed (0x03)
sept. 23 11:58:34 user-5930 bluetoothd[894]: Failed to add UUID: Failed (0x03)
sept. 23 11:58:34 user-5930 bluetoothd[894]: Failed to add UUID: Failed (0x03)
sept. 23 11:58:34 user-5930 bluetoothd[894]: src/service.c:btd_service_connect() a2dp-sink profile connect failed for 24:29:34:A9:F0:1F: Protocol not available
sept. 23 11:58:46 user-5930 systemd[1543]: Failed to start BTRFS Assistant Snapper check.
░░ Subject: A start job for unit UNIT has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit UNIT has finished with a failure.
░░
░░ The job identifier is 96 and the job result is failed.
sept. 23 11:58:47 user-5930 bluetoothd[894]: src/adv_monitor.c:btd_adv_monitor_power_down() Unexpected NULL btd_adv_monitor_manager object upon power down
sept. 23 11:58:47 user-5930 bluetoothd[894]: Failed to add UUID: Failed (0x03)
sept. 23 11:58:48 user-5930 bluetoothd[894]: src/service.c:btd_service_connect() a2dp-sink profile connect failed for 24:29:34:A9:F0:1F: Protocol not available
sept. 23 11:58:59 user-5930 bluetoothd[894]: src/profile.c:record_cb() Unable to get Hands-Free Voice gateway SDP record: Host is down
sept. 23 17:23:18 user-5930 systemd[1543]: Failed to start Burp Suite Free Edition.
░░ Subject: A start job for unit UNIT has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit UNIT has finished with a failure.
░░
░░ The job identifier is 532 and the job result is failed.
sept. 23 17:35:15 user-5930 kernel: sched: RT throttling activated
6

The swoosh indicates you need to reboot after your last update. Please do so.

  1. Does this happen only with the Zen kernel? Have you tried others?
  2. Are there more current BIOS updates other than your last @ 02/24/2022?
  3. What has your online research shown? What have you read? What have you tried? What were the results (other than nohang’s)?
2 Likes
7

The swoosh indicates you need to reboot after your last update. Please do so.

Doesn’t change anything, this is happening for a while, still happening today after the daily reboot.

Does this happen only with the Zen kernel? Have you tried others?

I’ve tried the “normal” one LTS with same issue a while ago, not sure if there are more specific kernels to try.

Are there more current BIOS updates other than your last @ 02/24/2022?

Seems like it. Nothing in the changelogs indicate a fix for a specific issue but I’ll apply it next time I’m on windows anyway (gotta love windows only BIOS updaters)

Edit: I badmouthed, it’s only their additional tool that is Windows only. Flashed the BIOS to latest version and changed nothing.

What has your online research shown?

Not much except the potential swappiness setting that did nothing. I didn’t find anything about this specific issue, it’s usually RAM that is unrecognized completely indicating hardware issue, or a BIOS settings which are fine here from what I can gather, and do not lead to the same symptoms…

What have you read? What have you tried? What were the results (other than nohang’s)?

I’ve tried nohang’s specific things indeed, but since I cannot really isolate the issue, it’s kind of difficult to test anything else, hence this thread.

The RAM is both recognized and unrecognized at the same time, sometimes it can be allocated, sometime it cannot, there’s no error anywhere I’ve checked.

I can replicate the issue in a scenario, but in a similar scenario I can fully use the RAM and I’m kind of at a loss.

8

Please check out.

1 Like
9

Swappiness is probably not what you think it is. Changing the swappiness is unlikely to have any noticeable effect on this issue. If you would like to learn more about swappiness and what it is, check this out:

If by this you mean “in use and available at the same time”, I assure you that is completely normal and has to do with how Linux caches memory. Here is another article that I think explains it very well.

https://www.linuxatemyram.com/

That’s good because there isn’t enough information in the thread to see what is going on. Go ahead and reproduce the issue, then let’s see some terminal output:

free -h

cat /proc/meminfo

htop

When you run htop, sort the processes by memory usage and take a screenshot. The other ones please paste into the thread as text.

3 Likes
10

I’m not sure I described my problem clearly.

When I use more than 16gb of RAM over my 32gb, it acts just like if I was out of RAM by swapping until the swap is full.

The expected behavior would be to do that once the 32gb are used, not at 50% of use. This is why I reduced swappiness, which unfortunately doesn’t change much in that case.

11

2 free -m taken with 10 seconds between them while reproducing:

               total        used        free      shared  buff/cache   available
Mem:           31997       23769        2040         743        7392        8228
Swap:          35273        2178       33095


               total        used        free      shared  buff/cache   available
Mem:           31997       24372        2315         472        6241        7625
Swap:          35273        2642       32631

First meminfo:

/bin/cat /proc/meminfo
MemTotal:       32765832 kB
MemFree:         4761020 kB
MemAvailable:    9905552 kB
Buffers:              16 kB
Cached:          5591160 kB
SwapCached:        75608 kB
Active:         14154604 kB
Inactive:       11841572 kB
Active(anon):   11434268 kB
Inactive(anon):  9406936 kB
Active(file):    2720336 kB
Inactive(file):  2434636 kB
Unevictable:       21748 kB
Mlocked:           21748 kB
SwapTotal:      36120332 kB
SwapFree:       31884556 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:            122560 kB
Writeback:          8276 kB
AnonPages:      20111332 kB
Mapped:          2545140 kB
Shmem:            429416 kB
KReclaimable:     453612 kB
Slab:             881988 kB
SReclaimable:     453612 kB
SUnreclaim:       428376 kB
KernelStack:       68304 kB
PageTables:       175828 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    52503248 kB
Committed_AS:   49666600 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      191024 kB
VmallocChunk:          0 kB
Percpu:            38144 kB
HardwareCorrupted:     0 kB
AnonHugePages:   3977216 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:    176128 kB
FilePmdMapped:    176128 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     1667164 kB
DirectMap2M:    31793152 kB
DirectMap1G:           0 kB

Second one:

/bin/cat /proc/meminfo
MemTotal:       32765832 kB
MemFree:         4647584 kB
MemAvailable:   11521156 kB
Buffers:              24 kB
Cached:          7294852 kB
SwapCached:        68216 kB
Active:         15312288 kB
Inactive:       10681332 kB
Active(anon):   12070856 kB
Inactive(anon):  7100884 kB
Active(file):    3241432 kB
Inactive(file):  3580448 kB
Unevictable:       21748 kB
Mlocked:           21748 kB
SwapTotal:      36120332 kB
SwapFree:       31688972 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:              1036 kB
Writeback:             0 kB
AnonPages:      18395512 kB
Mapped:          2455380 kB
Shmem:            466268 kB
KReclaimable:     515744 kB
Slab:             937736 kB
SReclaimable:     515744 kB
SUnreclaim:       421992 kB
KernelStack:       62640 kB
PageTables:       168108 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    52503248 kB
Committed_AS:   48252740 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      185600 kB
VmallocChunk:          0 kB
Percpu:            38144 kB
HardwareCorrupted:     0 kB
AnonHugePages:   3676160 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:    176128 kB
FilePmdMapped:    161792 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     1755228 kB
DirectMap2M:    31705088 kB
DirectMap1G:           0 kB

I can provide a htop screenshot if you want, but this is mostly hundreds of firefox threads appearing due to how it’s managed.

Edit:

Second RAM “load test” with the linux-clear kernel just to see if anything changes with a kernel that aims to be optimized (didn’t change the strange behavior).

Test 1

                total        used        free      shared  buff/cache   available
Mem:            31Gi        24Gi       490Mi       2,0Gi       9,2Gi       7,0Gi
Swap:           34Gi       1,9Gi        32Gi

Test 2

                total        used        free      shared  buff/cache   available
Mem:            31Gi        24Gi       530Mi       2,0Gi       9,2Gi       7,0Gi
Swap:           34Gi       2,5Gi        31Gi

Test 3

                total        used        free      shared  buff/cache   available
Mem:            31Gi        24Gi       525Mi       1,9Gi       9,2Gi       7,0Gi
Swap:           34Gi       2,9Gi        31Gi

Test 4

                total        used        free      shared  buff/cache   available
Mem:            31Gi        24Gi       534Mi       1,9Gi       9,1Gi       7,0Gi
Swap:           34Gi       3,2Gi        31Gi

While the upper limit of RAM use seem to have shifted a bit which might be due to me restarting the applications a lot of time to use more RAM, we can still see a lot of free RAM not being used and swap being filled up instead, making my computer almost unuseable during the test.

12

Not to be cruel, but it all sounds like a case of having your cake but wanting to eat it, too. In this case, keeping your RAM free but using it in swap, too. It appears to me that your RAM and the installed operating system is doing its job, as designed. It’s just not doing it the way you think/want it to do.

Another platitude is it’s just a tempest in a teapot.

13

I don’t care about keeping the RAM free, it’s the opposite.

I want my system to use my RAM instead of the swap when I have ample RAM available.

For some reason it doesn’t most of the time and keeps RAM free when it should use it.

14

Swappiness doesn’t change the threshold at which your system starts or stops swapping, it changes how aggressively swapping occurs. Did you read the article I posted above?

If you have been a Linux user or enthusiast for a considerable amount of time, then the term swap or swap memory should not be news to you. But, unfortunately, many Linux users tend to confuse the concept of swap memory with swappiness. The most common misconception is that a swappiness value indicates the maximum usable RAM before the actual swapping process begins.

My guess is one of the applications you are using is responsible for this odd memory usage pattern. Swap is not just for when you run out of RAM, it also gets used in certain other situations. Sometimes, for example, applications will map memory directly to swap even when RAM is available.

I think these applications are a good place to start investigating. Can you reproduce the issue without Firefox running? Can you reproduce the issue without Jetbrains running? If you can narrow it down to a specific application then you can more specifically direct your troubleshooting from there.

If Firefox is the culprit, see if you can take the curse off by moving the disk cache to RAM as described here: https://wiki.archlinux.org/title/Firefox/Tweaks#Move_disk_cache_to_RAM

If you determine Jetbrains is causing the issue, try increasing the memory heap as described here: Increase the memory heap of the IDE | IntelliJ IDEA Documentation

By the way, it looks like when you completely disabled swap in your testing, you took down your zram:

It probably came back up on its own after a reboot, but check to be sure:

inxi -j

You want zram because it lets your device use RAM as a swap device, so you can have your cake and eat it too. :yum: :cake:

2 Likes
15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.