Wir arbeiten an einem neuen Produkt, welches unter anderem Emails per IMAP abfragt. Wie bei einem Email-Client wird verfolgt, welche Emails vorhanden sind: Zum einen müssen neue Emails auftauchen, es muss allerdings auch erkannt werden, wenn zum Beispiel eine Email verschoben oder gelöscht wird.
Der erste Prototyp ist ziemlich langsam gewesen. Daher habe ich versucht, Beispiele (in Java) zu finden, wie man effizient viele Emails per IMAP abfragen kann.
Leider habe ich viele ähnliche Fragen, weniger Antworten und noch weniger Beispiele gefunden. Die meisten Beispiele waren Snippets ohne aufrufenden Code, und ließen sich nicht ohne weiteres lokal starten.
Ein lauffähiges Beispiel hätte mir sehr geholfen, also habe ich zum Ausprobieren eines erstellt, um ein paar Zahlen zu bekommen. Wer möchte, darf den untenstehenden Beispiel-Quellcode gerne verwenden.
English Version
We are working on a new product (more about that in a future post) which accesses emails via IMAP. Very much like an email client it keeps track of existing emails. Not only should new emails appear, also changes to old ones (i.e. if they're moved around folders or are deleted) must be detected.
The first prototype has been very slow and I started searching for Java code examples about how to load large amounts of emails efficiently via IMAP.
Unfortunately I found a lot of questions, few answers and even less code samples. Most examples have been short snippets, put outside context which would not run just locally.
Some running sample code to start from would have been really helpful. So I built one to play around and get some numbers – attached below. Feel free to use the sample for yourself!
package de.sandstormmedia;
import com.sun.mail.imap.IMAPFolder;
import javax.mail.*;
import java.util.Properties;
/**
* This example prints all messages (emails) from an IMAP-folder to the console.
* <p/>
* The messages are fetched in chunks of maximal 500 messages from latest message
* to oldest message.
*/
public class IMAPExample {
public static void main(String[] args) {
String server = readLine("server", "imap.example.net");
String username = readLine("username", "user@example.net");
/*
* !!! NOTE: The password is displayed in the console and CAN BE READ by anyone watching. !!!
*/
String password = System.console().readLine("password: ");
try {
listMessages(server, username, password);
} catch (Throwable e) {
e.printStackTrace();
}
}
private static String readLine(String message, String defaultValue) {
String value = System.console().readLine(message + " (default: " + defaultValue + "): ");
if (value.isEmpty()) {
value = defaultValue;
}
return value;
}
private static void listMessages(String url, String username, String password) throws MessagingException {
Properties props = System.getProperties();
props.setProperty("mail.store.protocol", "imaps");
Session session = Session.getDefaultInstance(props, null);
Store store = session.getStore("imaps");
try {
System.out.println("Connecting to IMAP server: " + url);
store.connect(url, username, password);
Folder root = store.getDefaultFolder();
Folder[] folders = root.list();
System.out.println("Select a folder");
for (int i = 0; i < folders.length; i++) {
System.out.println("\t" + folders[i].getName());
}
String folderName = readLine("folder", "INBOX");
IMAPFolder folder = (IMAPFolder) store.getFolder(folderName);
long afterFolderSelectionTime = System.nanoTime();
int totalNumberOfMessages = 0;
try {
if (!folder.isOpen()) {
folder.open(Folder.READ_ONLY);
}
/*
* Now we fetch the message from the IMAP folder in descending order.
*
* This way the new mails arrive with the first chunks and older mails
* afterwards.
*/
long largestUid = folder.getUIDNext() - 1;
int chunkSize = 500;
for (long offset = 0; offset < largestUid; offset += chunkSize) {
long start = Math.max(1, largestUid - offset - chunkSize + 1);
long end = Math.max(1, largestUid - offset);
/*
* The next line fetches the existing messages within the
* given range from the server.
*
* The messages are not loaded entirely and contain hardly
* any information. The Message-instances are mostly empty.
*/
long beforeTime = System.nanoTime();
Message[] messages = folder.getMessagesByUID(start, end);
totalNumberOfMessages += messages.length;
System.out.println("found " + messages.length + " messages (took " + (System.nanoTime() - beforeTime) / 1000 / 1000 + " ms)");
/*
* If we would access e.g. the subject of a message right away
* it would be fetched from the IMAP server lazily.
*
* Fetching the subjects of all messages one by one would
* produce many requests to the IMAP server and take too
* much time.
*
* Instead with the following lines we load some information
* for all messages with one single request to save some
* time here.
*/
beforeTime = System.nanoTime();
// this instance could be created outside the loop as well
FetchProfile metadataProfile = new FetchProfile();
// load flags, such as SEEN (read), ANSWERED, DELETED, ...
metadataProfile.add(FetchProfile.Item.FLAGS);
// also load From, To, Cc, Bcc, ReplyTo, Subject and Date
metadataProfile.add(FetchProfile.Item.ENVELOPE);
// we could as well load the entire messages (headers and body, including all "attachments")
// metadataProfile.add(IMAPFolder.FetchProfileItem.MESSAGE);
folder.fetch(messages, metadataProfile);
System.out.println("loaded messages (took " + (System.nanoTime() - beforeTime) / 1000 / 1000 + " ms)");
/*
* Now that we have all the information we need, let's print some mails.
* This should be wicked fast.
*/
beforeTime = System.nanoTime();
for (int i = messages.length - 1; i >= 0; i--) {
Message message = messages[i];
long uid = folder.getUID(message);
boolean isRead = message.isSet(Flags.Flag.SEEN);
if (!isRead) {
// new messages are green
System.out.print("\u001B[32m");
}
System.out.println("\t" + uid + "\t" + message.getSubject());
if (!isRead) {
// reset color
System.out.print("\u001B[0m");
}
}
System.out.println("Listed message (took " + (System.nanoTime() - beforeTime) / 1000 / 1000 + " ms)");
}
} finally {
if (folder.isOpen()) {
folder.close(true);
}
}
System.out.println("\nListed all " + totalNumberOfMessages + " messages (took " + (System.nanoTime() - afterFolderSelectionTime) / 1000 / 1000 + " ms)");
} finally {
store.close();
}
}
}
