I got a 16GB swap partition. Htop indicates that it's filled about 1/4.
cacule kernel doesn't change much, it has similar performance and still random freeze upon starting the browser.
btw running both zram and swap together seemed even more unstable.
Have you applied some of the fixes to VapourSynth as suggested on there site i,m not sure they will work as we are a newer version of python
This could be it right there. If Windows holds more in swap, then it makes more ram available for the task. Here Linux uses only 1/4th of 16GB swap, about 4GB. I’m pretty sure Windows uses a lot more under the same circumstance.
Is there a way to change the Linux swap usage strategy?
Yes, increase "swappiness":
https://wiki.archlinux.org/title/Swap#Swappiness
I played with swappiness, zram and processing threads combinations.
For processing 5K video, higher swappiness helps a little bit, but it remains at a deadly 0.2 to 0.1 fps. With just 6 processing threads on 12 cores. And that's with x264 instead of x265. It's not even falling back to swap, mostly just using ram. Swap usage goes up after a while for cache I think. In Windows, it would immediately start at a higher fps and then drop to ~.7-.5fps. Another observation, I ran 2.7K clip encoding with 6 threads and it used 95% CPU (x264 with preset slow and auto threads). With 5K clip, CPU usage goes at about 40-50% ... it's struggling somewhere.
I'm unable to process 5K clips under Linux... running out of ideas. Luckily, it's something I had to do in the past, but not something I have to do now. Normally I can get somewhere between 0.7fps and 0.3fps at the lowest.
What happens if you limit the max cache size ?
Does memory usage lower?
or limit the frames it pre-fetches?
As what you are encoding is cpu bound not memory bound
Systemd also has many tunables.
Understanding systemd scheduling-related options
https://wiki.archlinux.org/title/systemd
These topics are very complex as systemd has a bottomless pit of options you can test.
Actually it’s bound to the memory, GPU, CPU and CPU cache. Also bound to the ventilation system.
If I want to encode 5K videos, I might have to re-install Windows as dual-boot to compare usage statistics… (or if I’m lucky I can get useful data running less-intense clips in a VM with GPU passthrough and CPU passthrough)
Damn KDenlive is in no way faster for encoding!! Gotta find something better.
I've done some more tests with a smaller script on 5K video that is not limited by memory.
Swap is not in use. Running only parts of the script, I get less than 0.3fps, and it's not even encoding with x264 or x265.
I ran the test in both Garuda Linux and in Win10 VM with GPU passthrough and CPU pinning. I see the same under-performance in both cases.
To do further tests, I'd have to re-install Windows 10 in dual-boot to test there... but I did encode a lot of videos before at more than 0.3fps.
Considering that memory and swap aren't the issue, and that VM with CPU passthrough & CPU pinning has the same performance drop. What does that point to?
Any non-optimized filter in the chain could be the culprit (I tested and MvTools denoising was the bottleneck, running alongside KNLMeansCL on the GPU. Looks normal), or it could be the way Linux manages things.
This discards the possibility of being a difference with the way NVidia driver handles memory for GPU operations.
Another freeze while opening the browser, running an Adobe Premiere encoding in a VM with only 13GB assigned out of 16GB; and with swap available. This is getting slightly annoying. Is this normal Linux behavior?
From what I'm seeing, Linux never really gets to use the swap much at all -- it becomes unstable before it starts really using it. I wouldn't expect that kind of instability though.
The Vm froze or the host?
The host froze while opening FireDragon. The VM shouldn't be affecting the host, particularly if it leaves 3GB of untouched RAM.
It always freezes while opening FireDragon, somehow. Once that's open, I can watch video or do anything without problem.
btw for those who were asking about denoising comparison, I just released a video showing denoising comparison
What does the denoising process do? Is the difference supposed to be visually obvious?
On my monitor the before and after look exactly the same, although I also have an untrained eye.
There's considerable noise in the grass, it's supposed to remove that grainy noise without losing details, and without making it look like plastic or distorted. Also a lot of the noise is temporal, so it's not just comparing static frames, but also have to see how it looks in motion. The difference in the last Webcam video is really obvious. As for the water & cliff video, the difference is very subtle; but the point is that it doesn't deteriorate the image. The problem I had with that video is that one part is dark and extremely noisy, so I need to process the clip; but didn't want it to degrade the rest.
Also keep in mind; you're looking at a 5K clip on a 1K display properly. It's VERY HARD to remove spacial noise, and particularly large-radius noise, on a 5K clip. Especially without losing fine details. I struggled a lot with large-radius noise such as the car dashboard. Also the video got encoded twice after processing, losing some details such as noise. You'll get a more accurate image if you select 4K quality in YouTube.
It combines motion-analysis with spacial analysis and temporal analysis.
Found a readme file I left with the settings I was using before; different from what I remembered. Even using those same settings, it's unusable.
What I may have to do is re-install Windows on dual-boot and run benchmarks on all the filters one-by-one. Then I may have some interesting discoveries. Either filters that don't perform well under Linux for some reason; or insights about what kind of code runs well or not under Linux.
Could there be some inline assembly code that compiles on Windows but not on Linux?
That would be time-consuming though.
oh I tried with kernel mitigations=off, didn't help much either.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.