Kmail + DavMail + office 365 - cannot sync big inbox

Issues & Assistance Unsupported Software (AUR & Other)
1

Hello,

I'm making an attempt to set up Kmail and the whole Kontakt suite to work with office 365 - instead of evoluiton. The problem i'm experiencing is that Kmail is having problems syncing big folders. I see the following behavior:

  1. It starts the process of sync Inbox or sent items
  2. The process moves very slowly, sometimes advancing to 10% (sent folder did), sometimes not really.
    Davmail shows in the logs that is connected and starts loading items
  3. The Kmail sync process is getting killed by Kmail
  4. DavMail reports client disconnect and stops the session

So far i was able to partially sync some folders that are not having many items in them, but inbox cannot be synced up at all - sent items is stuck on 500 count
It feels like Kmail is just doesn't wait long enough for batch to get ready or something like that. I was trying to find if there is any setting that would increase IMAP timeout, but to no avail.
Currently i'm trying to cleanup my inbox using outlook on web, and possibly create mbox file via evolution to then migrate the bulk of the items to kmail in hope that this will unblock it so that it can start receiving updates.

I would really appreciate some thoughts on this. Thank you!

╰─λ garuda-inxi
System:
Kernel: 5.17.1-zen1-1-zen arch: x86_64 bits: 64 compiler: gcc v: 11.2.0
parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
root=UUID=cfa490e8-fc51-4b99-b39e-9e21021d7ed1 rw rootflags=subvol=@
quiet quiet splash rd.udev.log_priority=3 vt.global_cursor_default=0
loglevel=3
Desktop: KDE Plasma v: 5.24.4 tk: Qt v: 5.15.3 info: latte-dock
wm: kwin_x11 vt: 1 dm: SDDM Distro: Garuda Linux base: Arch Linux
Machine:
Type: Desktop Mobo: ASUSTeK model: ROG CROSSHAIR VIII FORMULA v: Rev X.0x
serial: <superuser required> UEFI: American Megatrends v: 0702
date: 07/04/2019
CPU:
Info: model: AMD Ryzen 9 3950X bits: 64 type: MT MCP arch: Zen 2
family: 0x17 (23) model-id: 0x71 (113) stepping: 0 microcode: 0x8701013
Topology: cpus: 1x cores: 16 tpc: 2 threads: 32 smt: enabled cache:
L1: 1024 KiB desc: d-16x32 KiB; i-16x32 KiB L2: 8 MiB desc: 16x512 KiB
L3: 64 MiB desc: 4x16 MiB
Speed (MHz): avg: 4049 high: 4215 min/max: 2200/4761 boost: enabled
scaling: driver: acpi-cpufreq governor: performance cores: 1: 4186 2: 4090
3: 4173 4: 3364 5: 4203 6: 4176 7: 4192 8: 4185 9: 4192 10: 4207 11: 4205
12: 3374 13: 4187 14: 3373 15: 3860 16: 3826 17: 4198 18: 4211 19: 4195
20: 3372 21: 4179 22: 4215 23: 4188 24: 4212 25: 4189 26: 4207 27: 4193
28: 4190 29: 4186 30: 4215 31: 3793 32: 4042 bogomips: 223585
Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm
Vulnerabilities:
Type: itlb_multihit status: Not affected
Type: l1tf status: Not affected
Type: mds status: Not affected
Type: meltdown status: Not affected
Type: spec_store_bypass
mitigation: Speculative Store Bypass disabled via prctl
Type: spectre_v1
mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2
mitigation: Retpolines, IBPB: conditional, STIBP: always-on, RSB filling
Type: srbds status: Not affected
Type: tsx_async_abort status: Not affected
Graphics:
Device-1: AMD Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT]
vendor: Micro-Star MSI driver: amdgpu v: kernel pcie: gen: 4 speed: 16 GT/s
lanes: 16 ports: active: DP-2,DP-3 empty: DP-1,HDMI-A-1 bus-ID: 0e:00.0
chip-ID: 1002:731f class-ID: 0300
Device-2: Afatech NarviCapture U3 HD60 4K Device type: USB
driver: snd-usb-audio,uvcvideo bus-ID: 3-1:2 chip-ID: 1b80:e104
class-ID: 0102 serial: <filter>
Display: x11 server: X.Org v: 1.21.1.3 compositor: kwin_x11 driver: X:
loaded: amdgpu unloaded: modesetting,radeon alternate: fbdev,vesa
gpu: amdgpu display-ID: :0 screens: 1
Screen-1: 0 s-res: 5120x2880 s-dpi: 96 s-size: 1354x762mm (53.31x30.00")
s-diag: 1554mm (61.17")
Monitor-1: DP-2 mapped: DisplayPort-1 pos: primary,bottom-l
model: Samsung C49RG9x serial: <filter> built: 2018 res: 5120x1440 hz: 120
dpi: 109 gamma: 1.2 size: 1190x340mm (46.85x13.39") diag: 1239mm (48.8")
modes: max: 5120x1440 min: 720x400
Monitor-2: DP-3 mapped: DisplayPort-2 pos: top-right
model: LG (GoldStar) ULTRAWIDE serial: <filter> built: 2015 res: 3440x1440
hz: 60 dpi: 109 gamma: 1.2 size: 800x335mm (31.5x13.19")
diag: 867mm (34.1") modes: max: 3440x1440 min: 640x480
OpenGL: renderer: AMD Radeon RX 5700 XT (navi10 LLVM 13.0.1 DRM 3.44
5.17.1-zen1-1-zen)
v: 4.6 Mesa 22.0.0 direct render: Yes
Audio:
Device-1: AMD Navi 10 HDMI Audio driver: snd_hda_intel v: kernel pcie:
gen: 4 speed: 16 GT/s lanes: 16 bus-ID: 0e:00.1 chip-ID: 1002:ab38
class-ID: 0403
Device-2: AMD Starship/Matisse HD Audio vendor: ASUSTeK
driver: snd_hda_intel v: kernel pcie: gen: 4 speed: 16 GT/s lanes: 16
bus-ID: 10:00.4 chip-ID: 1022:1487 class-ID: 0403
Device-3: Afatech NarviCapture U3 HD60 4K Device type: USB
driver: snd-usb-audio,uvcvideo bus-ID: 3-1:2 chip-ID: 1b80:e104
class-ID: 0102 serial: <filter>
Sound Server-1: ALSA v: k5.17.1-zen1-1-zen running: yes
Sound Server-2: PulseAudio v: 15.0 running: yes
Sound Server-3: PipeWire v: 0.3.49 running: yes
Network:
Device-1: Aquantia AQC111 NBase-T/IEEE 802.3bz Ethernet [AQtion]
vendor: ASUSTeK driver: atlantic v: kernel pcie: gen: 3 speed: 8 GT/s
lanes: 1 link-max: lanes: 4 port: N/A bus-ID: 06:00.0 chip-ID: 1d6a:11b1
class-ID: 0200
IF: enp6s0 state: down mac: <filter>
Device-2: Intel I211 Gigabit Network vendor: ASUSTeK driver: igb
v: kernel pcie: gen: 1 speed: 2.5 GT/s lanes: 1 port: f000 bus-ID: 07:00.0
chip-ID: 8086:1539 class-ID: 0200
IF: enp7s0 state: down mac: <filter>
Device-3: Intel Wi-Fi 6 AX200 driver: iwlwifi v: kernel pcie: gen: 2
speed: 5 GT/s lanes: 1 bus-ID: 08:00.0 chip-ID: 8086:2723 class-ID: 0280
IF: wlp8s0 state: up mac: <filter>
Bluetooth:
Device-1: Intel AX200 Bluetooth type: USB driver: btusb v: 0.8
bus-ID: 1-6:4 chip-ID: 8087:0029 class-ID: e001
Report: bt-adapter ID: hci0 rfk-id: 0 state: up address: <filter>
Drives:
Local Storage: total: 4.58 TiB used: 1.63 TiB (35.7%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Samsung model: SSD 970 EVO 1TB
size: 931.51 GiB block-size: physical: 512 B logical: 512 B
speed: 31.6 Gb/s lanes: 4 type: SSD serial: <filter> rev: 2B2QEXE7
temp: 58.9 C scheme: GPT
ID-2: /dev/sda maj-min: 8:0 vendor: Samsung model: SSD 860 EVO 4TB
size: 3.64 TiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 4B6Q scheme: GPT
ID-3: /dev/sdf maj-min: 8:80 type: USB vendor: Generic
model: STORAGE DEVICE size: 31.25 GiB block-size: physical: 512 B
logical: 512 B type: N/A serial: <filter> rev: 9744 scheme: MBR
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
ID-1: / raw-size: 97.17 GiB size: 97.17 GiB (100.00%)
used: 27.99 GiB (28.8%) fs: btrfs dev: /dev/nvme0n1p5 maj-min: 259:5
ID-2: /boot/efi raw-size: 500.7 MiB size: 499.7 MiB (99.80%)
used: 576 KiB (0.1%) fs: vfat dev: /dev/nvme0n1p6 maj-min: 259:6
ID-3: /home raw-size: 97.17 GiB size: 97.17 GiB (100.00%)
used: 27.99 GiB (28.8%) fs: btrfs dev: /dev/nvme0n1p5 maj-min: 259:5
ID-4: /var/log raw-size: 97.17 GiB size: 97.17 GiB (100.00%)
used: 27.99 GiB (28.8%) fs: btrfs dev: /dev/nvme0n1p5 maj-min: 259:5
ID-5: /var/tmp raw-size: 97.17 GiB size: 97.17 GiB (100.00%)
used: 27.99 GiB (28.8%) fs: btrfs dev: /dev/nvme0n1p5 maj-min: 259:5
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default)
ID-1: swap-1 type: zram size: 62.78 GiB used: 2.2 MiB (0.0%)
priority: 100 dev: /dev/zram0
Sensors:
System Temperatures: cpu: 58.0 C mobo: 39.0 C gpu: amdgpu temp: 65.0 C
mem: 64.0 C
Fan Speeds (RPM): cpu: 1203 gpu: amdgpu fan: 0
Info:
Processes: 616 Uptime: 51m wakeups: 0 Memory: 62.78 GiB
used: 9.11 GiB (14.5%) Init: systemd v: 250 tool: systemctl Compilers:
gcc: 11.2.0 Packages: pacman: 1545 lib: 369 Shell: fish v: 3.3.1
default: Bash v: 5.1.16 running-in: konsole inxi: 3.3.14
Garuda (2.5.6-2):
System install date:     2022-03-17
Last full system update: 2022-04-04 ↻
Is partially upgraded:   No
Relevant software:       NetworkManager
Windows dual boot:       Probably (Run as root to verify)
Snapshots:               Snapper
Failed units:
2

Also keep in mind that Office365 could be limiting the transfers, so there's that variable to consider.

One thing to try would be to sync in smaller amounts, e.g. sync the last 30 days, then 90 days, etc. of email and see if that will help it complete.

1 Like
3

it would be an option if this option would exist. I can’t find anything like that anywhere in Kmail:(

As for transfers - evolution didn’t have any issues syncing stuff - so the server is most probably not at fault here…

1 Like
4

I've tried numerous times to stay with just KDE apps for my plasma install but always end up just going back to Evolution because of the polish and feature set.

2 Likes
5

Same. It is always a "heated" opinion, but mine is that the Akonadi database is....:poop:. The whole Kmail/Akonadi system, especially the davmail/caldav/cardav system is utter rubbish. I have had a bug report that has been confirmed on there for AGES...and they never fix it. Tear it down, start over....rant ended.

I would recommend staying with Evolution (although it has issues as well, but at least works) or Thunderbird which now has internal davmail, etc. plugins incorporated.

2 Likes
6

@RodneyCK @khaneliman It's a shame though - one of garuda's "selling" points is beautiful KDE dragonized UI and Kontakt suite is looking way nicer and works with dark themes way nicer than Evoluiton.
But i do agree that the combination with DavMail + office365 is not performing on par with Evolution so far.
I'm inclined to give Thunderbird a go - but worried about Mozilla dropping it from support. Is it still actively worked on - any insights?

7

I don’t think they are going to drop it. A lot of it was performed by outside add-ons, but now it works out of the box. I only use caldav/cardav from my mail client and it works perfectly, no more plugins needed. I have not used it with MS Exchange, but I hear it works.

If you do have issues, you might also want to check out BlueMail. I have never used it and its coding is proprietary, but then again, you are connecting to a proprietary server (Microsoft.) It is in the chaotic-aur and AUR repos.

8

Wow! at a first glance bluemail is spectacular! painless setup, quick sync - got to work right away, Calendar, contacts - everything is up and running with exchange ews.

2 Likes
9

Sadly that one seems even worse for my use case... I need something that lets me override user agent string to actually log into my company EWS server. So far, I've only gotten that to work on Thunderbird because I can set the user agent string of Thunderbird itself (But I've only found how to add the account using third party extension that's paywalled). Kmail lets me set it during the account setup, but it isn't respected during the login window that they use for Oauth2 so it's irrelevant... Evolution I don't have a way to set it at all.

1 Like
10

Is DavMail not able to help with that? I.e. eliminate the need to provide user agent string, because it can connect to EWS by itself?

11

Alright, hadn't tried using that before because the app looked unpolished and a pain to deal with... it definitely was and had some horrible bugs that I didn't know were causing issues with trying to change authentication settings... but after changing settings and relaunching a few times it finally worked... so, yes, I guess that allows me to bypass using the user agent string. :stuck_out_tongue: I dont know how much confidence I have that it wont give me a headache with things just randomly stopping working though after the setup process.

12

Glad to hear you got it to work. For what it's worth: the Owl for Exchange addon works wonders for Office365 + ThunderBird.

13

hmm, let me try that.

As for Blue mail - unfortunately it failed on calendar front “spectacularly”. It started to load all meetings that i ever had sent to me that were recurring and push them to although they were cancelled months ago… and the problem is - it doesn’t allow me to delete those, becasue they were not created by me…

14

So, to summarize:

  1. My work exchange admin doesn't allow direct connection using OAuth from tools like Kmail and Thunderbird and so on. SO for those DavMail is needed to act as proxy, because it can connect "natively". Evolution-ews can connect natively as well.
  2. Kmail and Kontakt suite coudn't sync up via Davmail. Uninstalled the whole thing
  3. Thunderbird can't sync up calendar via Davmail. Sync is very slow. Most probably goes out the window too
  4. bluemail very promising from the start, bu then calendar is not workable. So, no go too
  5. Evolution is the best soltuion - it just works. BUT it sucks on UI front and especially to when working in dark mode and has bugs in HTML messages presentation.

That's the state of affairs for now.

1 Like
15

After some more testing - evolution doesn't solve it all :frowning:
It actually fails to sync contacts from outlook, so I can't compose emails and be able to "inline" search people. It also has problems consistently pulling addresses from emails - refresh does nothing - the only workaround i found was to parse an mbox file generated from evolution, pull all unique addresses and store them in local contacts. Not straight forward at all.

So , the only option that works reliably is to open up outlook web interface in Chrome and then create a shortcut, so it becomes a Chrome app, then use that as a replacement for email client. Outlook even knows how to open mailto links while working in that mode...

YMMV

1 Like
16

Have you tried Prospect Mail Client?

prospect-mail
prospect-mail1335Ă—928 66.7 KB

I haven’t used it, but it is a standalone new Outlook Interface that would probably be better than a Chrome app type situation. It is available in Chaotic-AUR and AUR as prospect-mail-bin. More about it here.

17

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.