Skip to content
Snippets Groups Projects
Select Git revision
0 results
Search by author
  • Any Author
0 authors
  1. Apr 16, 2020
  3. Apr 14, 2020
  5. Apr 07, 2020
    • Christoph Wurst's avatar
      Lower the memory footprint of the initial message cache sync · 72a5703e
      Christoph Wurst authored 
      
The initial message sync has to fetch potentially large amounts of data
and insert that into the database. To work around limitations with sync
requests triggered by web requests the process had already been made
interruptable and resumable. This means we never insert all the data
right away. Yet, the IMAP code fetched all UIDs before we capped it to a
maximum number of results per sync attempt. Depending on the mailbox
size this operation could require and allocate a lot of memory. On some
setup with lower memory limits, the process was aborted by the web
server due to a php memory exhaustion.

This patch modifies the IMAP code to optimize the memory usage by
limiting the amount of data that is fetched with each initial sync
attempt. The algorithm works as follows.

IMAP allows us to search in a range with a lower an upper bound UID.
While we know the highest known UID from the current cache values, we
can't derive the range for the next page from that as UIDs are not
continuous but might have holes due to deleted messages. If we assume
that messages of a mailbox are roughly distributed equally across the
assigned UIDs we can guess the max UID for the next range.
So we ask the server for min and max UIDs. The min or our known highest
UID is always the lower bound. Then we can calculate the distribution
rate from the min, max and number of messages and build the upper bound.
On everage this will fetch about the expected number of messages. It
could be more, but it could also be less. It shouldn't matter in most
cases.

Signed-off-by: default avatarChristoph Wurst <christoph@winzerhof-wurst.at>
      72a5703e
  7. Mar 26, 2020
  9. Mar 24, 2020
  11. Mar 20, 2020
  13. Feb 26, 2020
  15. Feb 10, 2020
  17. Jan 31, 2020
  19. Jan 08, 2020
  21. Oct 08, 2019
  23. Sep 27, 2019
  25. Sep 06, 2019
  27. Sep 05, 2019
  29. Sep 04, 2019
  31. May 08, 2019
  33. Jul 30, 2018
  35. Feb 02, 2018
  37. May 23, 2017
  39. Apr 25, 2017
  41. Feb 22, 2017
  43. Dec 16, 2016
  45. Aug 04, 2016
  47. May 09, 2016
  49. Apr 12, 2016
  51. Feb 07, 2016
  53. Sep 17, 2015
  55. Aug 29, 2015