Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

* Orphaned mailboxes
* Orphaned messages
* Orphaned aliases
* Orphaned collected recipients

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: GretaD <gretadoci@gmail.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

We want to compare the recipient type, not the ID.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

The initial message cache sync estimates the UID range of the next
message page so it can fetch messages efficiently. There was an
uncovered edge case where the next page might not return any message,
simple because there is a larger gap between the existing UIDs.
Therefore the sync process was aborted as incomplete over and over as it
could not make any progress and the number of known messages never
reached the total number of messages on IMAP. This patch changes the
logic so it retries the next page if the current one is empty. To break
a possibly infinite recursion there is a check about reaching the
highest UID reported by the server.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

UID validity is a way for the IMAP server to signal that the UIDs - that
are usually persistend - have changed. Whenever this happens clients
have to wipe their cached data and read the IMAP data anew. Since making
the initial sync process resumable, it does not clear the messages of a
mailbox as these might be from a previous sync attempt. Hence, we have
to manually clear the cache on a UID validity change before we invoke
the initial sync routine.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: GretaD <gretadoci@gmail.com>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

This is helpful to determine whether the initial sync just takes long or
got stuck.

Ref https://github.com/nextcloud/mail/issues/2976#issuecomment-618833115



Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

When we just had one implementation of a group service we could just
inject the implementation when the interface is used. However, as
another implementation was added, it is necessary to explicitly require
the implementations to be able to use all of them.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

We have special handling for IncompleteSyncException in the controller,
but since the exception handling always wrapped the original exception
in a service exception, the type cast to the incomplete sync exception
didn't work. Now, it just doesn't wrap unless it's something other than
a service exception.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

The initial message sync has to fetch potentially large amounts of data
and insert that into the database. To work around limitations with sync
requests triggered by web requests the process had already been made
interruptable and resumable. This means we never insert all the data
right away. Yet, the IMAP code fetched all UIDs before we capped it to a
maximum number of results per sync attempt. Depending on the mailbox
size this operation could require and allocate a lot of memory. On some
setup with lower memory limits, the process was aborted by the web
server due to a php memory exhaustion.

This patch modifies the IMAP code to optimize the memory usage by
limiting the amount of data that is fetched with each initial sync
attempt. The algorithm works as follows.

IMAP allows us to search in a range with a lower an upper bound UID.
While we know the highest known UID from the current cache values, we
can't derive the range for the next page from that as UIDs are not
continuous but might have holes due to deleted messages. If we assume
that messages of a mailbox are roughly distributed equally across the
assigned UIDs we can guess the max UID for the next range.
So we ask the server for min and max UIDs. The min or our known highest
UID is always the lower bound. Then we can calculate the distribution
rate from the min, max and number of messages and build the upper bound.
On everage this will fetch about the expected number of messages. It
could be more, but it could also be less. It shouldn't matter in most
cases.

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Cyrille Bollu <cyrpub@bollu.be>
Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>
Co-authored-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>

* Remove the heavy-weight virtual flagged mailbox
* Implement a favorites inbox as a simple filtered view
* Refactor the sync logic so it work with filtered mailboxes

Signed-off-by: Christoph Wurst <christoph@winzerhof-wurst.at>