Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
  • v3.5.0-beta3
  • v3.5.0-beta2
  • v3.5.0-beta1
  • v3.4.4
  • v3.4.4-rc1
  • v3.4.3
  • v2.2.9
  • v3.4.2
  • v3.4.1
  • v2.2.8
  • v3.4.0
  • v3.4.0-rc.3
  • v3.4.0-rc.2
  • v3.4.0-rc.1
  • v3.4.0-beta.4
  • v3.3.1
  • v3.3.0
  • v3.2.7
  • v3.3.0-rc.4
  • v3.4.0-beta.3
21 results
Search by author
  • Any Author
  • Laura Kolijn Laura Kolijn lkolijn
1 author
  1. Sep 01, 2020
  3. Aug 24, 2020
  5. Aug 11, 2020
  7. Aug 10, 2020
  9. Jul 13, 2020
  11. Jul 10, 2020
  13. Jul 08, 2020
  15. Jul 03, 2020
  17. May 28, 2020
  19. May 27, 2020
    • Christoph Wurst's avatar
      Fix sync'ing with many known UIDs · b5dafce5
      Christoph Wurst authored 
      
When looking for changed and vanished UIDs we typically have to send the
list of the known UIDs to the IMAP server to know what went missing
meanwhile. With the number of UIDs sent, the size of the IMAP command
grows. At some point the server will just refuse the command due to its
size.
To circumvent this scenario, this patch splits the sync process into
new, changed and vanished. New is cheap in terms of data sent, so it
stays untouched. Changed and vanished will now split the known UIDs list
into chunks and retrieve partial sync results that way.

Signed-off-by: default avatarChristoph Wurst <christoph@winzerhof-wurst.at>
      b5dafce5
  21. May 19, 2020
  23. Apr 28, 2020
    • Christoph Wurst's avatar
      Fix endless initial sync due to empty partial page · 6bdb866c
      Christoph Wurst authored 
      
The initial message cache sync estimates the UID range of the next
message page so it can fetch messages efficiently. There was an
uncovered edge case where the next page might not return any message,
simple because there is a larger gap between the existing UIDs.
Therefore the sync process was aborted as incomplete over and over as it
could not make any progress and the number of known messages never
reached the total number of messages on IMAP. This patch changes the
logic so it retries the next page if the current one is empty. To break
a possibly infinite recursion there is a check about reaching the
highest UID reported by the server.

Signed-off-by: default avatarChristoph Wurst <christoph@winzerhof-wurst.at>
      6bdb866c
  25. Apr 24, 2020
  27. Apr 16, 2020
  29. Apr 14, 2020
  31. Apr 07, 2020
    • Christoph Wurst's avatar
      Lower the memory footprint of the initial message cache sync · 72a5703e
      Christoph Wurst authored 
      
The initial message sync has to fetch potentially large amounts of data
and insert that into the database. To work around limitations with sync
requests triggered by web requests the process had already been made
interruptable and resumable. This means we never insert all the data
right away. Yet, the IMAP code fetched all UIDs before we capped it to a
maximum number of results per sync attempt. Depending on the mailbox
size this operation could require and allocate a lot of memory. On some
setup with lower memory limits, the process was aborted by the web
server due to a php memory exhaustion.

This patch modifies the IMAP code to optimize the memory usage by
limiting the amount of data that is fetched with each initial sync
attempt. The algorithm works as follows.

IMAP allows us to search in a range with a lower an upper bound UID.
While we know the highest known UID from the current cache values, we
can't derive the range for the next page from that as UIDs are not
continuous but might have holes due to deleted messages. If we assume
that messages of a mailbox are roughly distributed equally across the
assigned UIDs we can guess the max UID for the next range.
So we ask the server for min and max UIDs. The min or our known highest
UID is always the lower bound. Then we can calculate the distribution
rate from the min, max and number of messages and build the upper bound.
On everage this will fetch about the expected number of messages. It
could be more, but it could also be less. It shouldn't matter in most
cases.

Signed-off-by: default avatarChristoph Wurst <christoph@winzerhof-wurst.at>
      72a5703e
  33. Mar 26, 2020
  35. Mar 24, 2020
  37. Mar 23, 2020
  39. Mar 20, 2020
  41. Feb 26, 2020
  43. Feb 10, 2020
  45. Feb 06, 2020
  47. Jan 31, 2020
  49. Jan 08, 2020
  51. Oct 08, 2019
  53. Oct 07, 2019
  55. Oct 01, 2019
  57. Sep 27, 2019
  59. Sep 26, 2019
  61. Sep 06, 2019
  63. Sep 05, 2019