rmail

Component Overview

rmail is a single-process asyncio application composed of three protocol servers, a shared backend, a delivery queue, and supporting utilities. All I/O is non-blocking; only UID allocation uses a short-duration file lock.

External clients
       |
       +-- SMTP (25 / 465 / 587) ──── SMTPSession ────┐
       |                                               |
       +-- IMAP (143 / 993) ──────── IMAPSession ─────+──── FileMailStoreBackend ──── Filesystem
       |                                               |           (shard dirs, .eml files)
       +-- Exchange API (9002 / 9003) ─── ExchangeAPI ┘
                                              |
                                     JWT Auth / RateLimiter / Validate

Package Structure

IMAP Session Composition

IMAPSession uses Python MRO-based mixin composition to keep the file size manageable:

class IMAPSession(_MailboxCommandsMixin, _MessageCommandsMixin):
    # Core: auth, CAPABILITY, dispatch, TLS negotiation
    # _MailboxCommandsMixin: SELECT, EXAMINE, LIST, LSUB, IDLE, STATUS, RENAME, CREATE, DELETE, SUBSCRIBE
    # _MessageCommandsMixin: FETCH, STORE, SEARCH, APPEND, COPY, MOVE, EXPUNGE, UID, ID, ENABLE

Similarly, ExchangeAPI inherits from _APIHandlersMixin, separating routing logic from handler implementations.

TLS Security Model

When a TLS certificate is configured, AUTH capabilities are hidden from cleartext EHLO and CAPABILITY responses. This prevents credentials from being transmitted before TLS is negotiated.

Cleartext EHLO (TLS configured):
  250-mail.example.com
  250-SIZE 52428800
  250-8BITMIME
  250-STARTTLS           ← advertised
  250 ENHANCEDSTATUSCODES
  (AUTH not listed)

After STARTTLS:
  250-mail.example.com
  250-SIZE 52428800
  250-AUTH PLAIN LOGIN   ← now advertised
  250-8BITMIME
  250 ENHANCEDSTATUSCODES

The same logic applies to IMAP CAPABILITY: AUTH=PLAIN and AUTH=LOGIN are omitted on cleartext connections when TLS is configured.

Storage Layout

Messages are stored as individual .eml files. Flags are encoded in the filename so flag changes require only an atomic os.rename() — no index file is read or rewritten.

mailstore/
└── domains/
    └── example.com/
        ├── domain.json
        ├── aliases.json
        └── users/
            └── alice/
                ├── user.json
                └── mailboxes/
                    ├── INBOX/
                    │   ├── status.json          # uid_validity, uid_next
                    │   ├── 0000/                # UIDs 1–999
                    │   │   ├── 00000001_.eml    # no flags
                    │   │   └── 00000042_AFS.eml # Answered + Flagged + Seen
                    │   └── 0001/                # UIDs 1000–1999
                    │       └── 00001000_S.eml
                    ├── Sent/
                    │   └── status.json
                    ├── Drafts/
                    │   └── status.json
                    └── Trash/
                        └── status.json

Filename Format

{uid:08d}_{flags}.eml

Flag characters (alphabetically sorted): A = \Answered, D = \Deleted, F = \Flagged, S = \Seen, T = \Draft.

Shard Calculation

Shard directory = uid // 1000, zero-padded to 4 digits. Supports 10M+ UIDs per mailbox with at most 1000 files per directory.

Concurrency Model

All protocol handlers run as asyncio coroutines within a single event loop. Blocking I/O is limited to the short-duration UID allocation lock; all other storage operations are non-blocking on modern filesystems.

Delivery Flow

Inbound SMTP messages are passed to a DeliveryQueue backed by a configurable number of async worker coroutines. Each worker calls FileMailStoreBackend.append_message(), which allocates a UID and writes the .eml file atomically.

SMTP DATA received
    → DeliveryQueue.enqueue(message)
        → worker coroutine
            → alias resolution (aliases.json)
            → user lookup (auth)
            → quota check (user.quota_mb)
            → FileMailStoreBackend.append_message()
                → fcntl.flock (UID alloc)
                → write {uid:08d}_.eml
                → fcntl.unlock
            → IMAP IDLE push notification

Authentication

User credentials are stored per-user as user.json containing a PBKDF2-SHA256 hash (300,000 iterations) and a random salt. The Exchange API issues stateless JWT tokens signed with HMAC-SHA256 using the exchange_api.token_secret from config. Token expiry is 3600 seconds. Any process sharing the same secret can validate tokens, enabling multi-process deployment. Users have enabled and admin boolean fields; disabled accounts are rejected at login; admin accounts can manage aliases via the API.