What it means
Base IMAP is strictly request and response: the client asks, the server answers, the connection goes quiet. To notice a new message the client has to ask again, which means either checking every few seconds and wasting most of those checks, or checking every few minutes and telling the user their mail is late.
IDLE, defined in RFC 2177 in 1997 and supported by essentially every IMAP server since, inverts that for as long as the client wants. The client issues IDLE, the server answers with a continuation line rather than a completion, and the connection stays open with the client silent. From then until the client sends DONE, the server is free to push untagged responses the instant something changes in the selected mailbox.
It is a small extension with an outsized effect: it is the difference between a CRM that shows a customer reply while the agent is still reading the previous one, and a CRM that shows it two minutes later.
The protocol, exactly
Three details trip up almost every first implementation. First, the server response to IDLE is a command continuation request, a line starting with +, not a tagged OK. The tagged completion only arrives after you end the idle. Second, while idling the connection is not yours to use: you may send nothing except the termination. Third, the termination is the bare word DONE followed by CRLF, with no tag, no arguments and no command name.
Everything the server sends in between is untagged. Those responses are the actual payload of the feature, and handling them correctly is where most of the work is.
Sequence numbers are a trap
EXISTS and EXPUNGE both speak in message sequence numbers, which are positions in the mailbox rather than stable identities. A single EXPUNGE renumbers every message above it, immediately, and it can arrive while you are still processing an earlier response. Fetching by sequence number after that gets you the wrong message, silently, and no error is ever raised.
The correct pattern is to record UIDVALIDITY and UIDNEXT when you select the mailbox, then treat every EXISTS as a signal to run UID FETCH n:* from your stored UIDNEXT, and never to use a sequence number in a command at all. UIDVALIDITY is the escape hatch the server has for saying "forget everything": if it changes, all your stored UIDs are void and the mailbox needs a full resynchronisation.
Even the UID is not an identity across folders. A message moved to another folder gets a new UID there, and on Gmail the same message is visible under several folders at once. That is why the durable key is the Message-ID, with the UID acting only as a cursor.
Connections are the scarce resource
One IDLE watches one mailbox, so a naive design opens a connection per folder per account. Gmail allows 15 simultaneous IMAP connections per account, shared with the user own phone and laptop, and other providers are stricter. Cross the line and you get * BYE and a period during which reconnecting makes it worse.
In practice: idle on the inbox only, poll the other folders on a slow cursor, and cap connections per account rather than per worker. After a deploy, reconnect with jitter. A hundred workers restarting simultaneously and each opening two connections looks exactly like an attack, and the provider responds accordingly.
The failure that is hardest to see
An idle connection that is quiet and an idle connection that is dead look identical from the application side. TCP will not necessarily tell you: a middlebox can drop the flow without sending a reset, so your socket stays open forever and no bytes ever arrive. This is why the 29-minute re-IDLE is not optional. Add TCP keepalives, add an application-level timer that tears the connection down and rebuilds it, and add a slow reconciliation sweep that fetches by UID range regardless of what the push path has told you.
Authentication failures have their own signature. Since Google withdrew basic password access, a plain password produces AUTHENTICATIONFAILED with a link to a help page rather than anything descriptive, and the fix is an App Password or OAuth with XOAUTH2. Treat that response as a permanent credential error that needs the user, not as a transient error to retry, or you will keep hammering a mailbox that will never let you in.
Related concepts
- Message-ID: the identity that UIDs cannot provide.
- Webhook: the push model you would use instead if every provider offered one.
- Bounce rate: bounce messages arrive over the same sync path as everything else.
- DKIM: the authentication headers you read off the messages this connection delivers.
How Pinlyx handles it
Mailbox sync holds one IDLE connection per connected account on the inbox, with the Sent folder tracked on its own cursor so a message you sent is never re-imported as an inbound reply. The idle is recycled well inside the 29-minute window, reconnects use backoff with jitter so a deploy does not stampede the provider, and a periodic UID-range sweep reconciles anything the push path missed. Credential failures are separated from transient ones and surface as a reconnect prompt on the mailbox rather than a silent retry loop.