Webmail

The module is a plugin that sits between the browser (which cannot speak IMAP) and the mailbox's Dovecot/Postfix (ADR 0012 §1). It stores no mail and no mailbox password: one live IMAP connection per signed-in mailbox lives in the plugin process; the password is kept in memory for the life of the session (needed to reconnect and to authenticate SMTP/ManageSieve) and zeroed on close. Libraries: emersion go-imap v2, go-smtp, go-message, go-sasl (MIT, in THIRD_PARTY_NOTICES.md); the HTML sanitiser is our own on golang.org/x/net/html.

Layout: gw/ (gateway, no platform types, unit-tested against in-process IMAP/SMTP servers), sanitize/ (HTML mail sanitiser), cp/ (thin action handlers, DB for identities/preferences), schemas/, migrations/.

Sessions

webmail.session.open {address, password} -> {session, folders, expires_at, ...}. Every other action takes that session id. A session belongs to the panel principal that opened it (Actor.principal_id); nobody else can use or close it. It expires after 30 min idle (settings session_idle_minutes) or 12 h absolute, whichever first; closing it first delivers any message still inside its undo window. Max 10 sessions per principal. Failed sign-ins: 5 misses in 10 min lock that (principal, address) for 10 min; failures are audited. The IMAP host comes from mail.client.config called as the caller (so mail's access rules apply), never from the request. SSO (webmail.session.sso) checks the caller can see the mailbox (mail.mailbox.list), asks the mail module for its Dovecot master user (mail.sso.prepare, answered to the webmail module only; the master password is made on first use and kept in the mail module's vault) and signs in as address*rcmaster. A mailbox of another account (staff "Open") needs a fresh step-up, the session lasts at most 30 minutes, the audit log records who opened which mailbox (webmail.session.sso.staff) and the event webmail.sso.opened goes to the mailbox's account.

Module settings (JSON, schema schemas/settings.json)

host_override (dial this host, e.g. 127.0.0.1) + server_name (certificate name to verify), skip_tls_verify (self-signed mail cert, lab), imap_port/imap_security, smtp_port/smtp_security, sieve_port, max_attachment_mb (25), session_idle_minutes, session_max_hours, live_updates (IDLE, default on).

Actions (all permission-checked, schema-validated; see module.yaml)

Session: session.open/.sso/.close/.list/.watch. Folders: folder.list/.create/.rename/.delete/.empty/.subscribe. Reading: message.list, thread.list, message.search (folder * = every folder), message.get, message.source, attachment.get, message.reply_template. Organising: message.flags/.move/.delete/.archive/.spam. Composing: draft.save, message.send, send.cancel, upload.chunk/.discard. Rules: rules.get/.validate/.set, vacation.get/.set. Preferences: identity.list/.set/.delete, settings.get/.set, sender.trust. Events (owner-scoped, to the hosting account): webmail.mailbox.changed {session,address,folder,kind:exists|expunge|flags|sent,exists}, webmail.send.failed {detail}.

Reading mail safely (AC-webmail-08, -38)

sanitize.Clean parses with the HTML5 algorithm, then rebuilds a new tree from an allow-list; nothing is copied through unchecked. Dropped with content: script, style, iframe/frame, object/embed/applet, svg/math (any foreign namespace), template, noscript, head/title/meta/link/base, video/audio/source, select/textarea. Unwrapped (tags gone, text kept): form, input, button, label and every unknown element. No id, class, name, on*, srcset, formaction, background. Links: only http/https/mailto/tel survive, target=_blank rel="noopener noreferrer nofollow", and links[] reports each target with mismatch (visible text names another host) and idn (punycode look-alike). Inline style is reduced to presentational properties; any value with url(, expression, javascript, @import, \, var( is dropped; position, float, z-index are not allowed. Images: cid: -> data: URI of the inline part (raster only, <= 1 MB each, 3 MB total); data:image/svg never; http(s) images are **blocked** (data-rc-blocked, counted in remote_blocked) unless load_remote or the sender is on the trusted list (sender.trust); 1x1 / hidden images are tracking pixels: always dropped, counted in tracking_pixels. CSS url() never loads, even with opt-in. Test corpus: ~55 XSS vectors (OWASP classics, mXSS namespace tricks, Outlook conditional comments, null bytes, entity- encoded schemes), a tokenising oracle that re-parses the output and checks every tag/attribute/URL, and a fuzz target. **The UI must still render html in a sandboxed iframe (sandbox="", no allow-scripts/same-origin) with CSP default-src 'none'; img-src data: https:; style-src 'unsafe-inline'** — the sanitiser is layer one of two. Attachments: names are stripped of paths/control/RTL-override characters; executable extensions are flagged (dangerous); text/html, SVG, XML and JS attachments are served as application/octet-stream; chunked reads (attachment.get, 1 MiB default, 4 MiB max, base64) with the decoded part cached for the session; cap max_attachment_mb. Sender authentication shown from Authentication-Results (spf/dkim/dmarc).

Conversations and search

thread.list uses server THREAD=REFERENCES (Dovecot) and falls back to grouping the newest 1000 messages by Message-ID/In-Reply-To/References and "Re:"-normalised subject. message.search parses operators (from: to: cc: subject: body: has:attachment is:unread|read|starred|answered label: before: after: larger: smaller:) into IMAP SEARCH; the remaining words are TEXT (Dovecot FTS when a plugin is configured). has:attachment is the heuristic Content-Type: multipart/mixed. Counts in lists are exact; pages are by sequence window (newest first).

Sending

SMTP submission with the mailbox's own login (587 STARTTLS; Postfix enforces login = sender, the refusal is returned as a readable conflict with smtp_code). Addresses/subject are validated (no CR/LF/NUL injection), Bcc only in the envelope, outgoing HTML is run through the sanitiser (scripts out, cid: kept), text part derived when missing, multipart/mixed > alternative > related structure, attachments from chunked uploads (spooled 0600 under the module data dir, wiped on session close; offset must match so a retry cannot duplicate bytes) or forwarded from the original. After sending: copy to Sent, \Answered / $Forwarded on the original, draft removed. delay_seconds 0-30 = undo window held in memory; send.cancel returns the message to Drafts; if a delayed send fails it is saved to Drafts and webmail.send.failed is emitted. Scheduled send is below ("Scheduled send and snooze"); read-receipt handling is not done.

Across folders (w11-24)

message.search and thread.list take folder *: every selectable folder except Trash, Junk, the Snoozed holding folder and a virtual "All mail" is searched with the server's own SEARCH (gw/allfolders.go); in:<folder|role> narrows it (and is the only way to search Trash or Junk). The newest 300 matches of each folder are looked at; a mail that lives in two folders (you sent it to yourself) is shown once. With threads: true the matches are grouped into conversations by Message-ID / In-Reply-To / References / "Re:"-subject across folders, and a conversation that matches is shown whole (its other messages are pulled in from the same folders), so a reply in Sent joins the mail it answers; every message carries its folder. No full-text index is installed by the mail module (no fts plugin in the Dovecot config), so the free text is a header + body scan; the result says so (full_text: false) and the UI shows "Header and body scan" with the reason. The result flips to full_text: true the day Dovecot advertises SEARCH=FUZZY.

Scheduled send and snooze (w11-24)

Both are database queues owned by this module (migration 0002: scheduled, snoozed), worked by one loop in the plugin process (10 s tick, cp/queue.go), so a closed browser or an expired webmail session loses nothing.

• send.schedule {message, send_at, timezone} builds the finished RFC 5322 message now (same validation as send, so a bad address fails at once, attachments are inside the stored message) and stores it. A time zone name is required and is kept with the row; the instant is stored in UTC. 5 s to 366 days ahead, 100 per mailbox. The draft it came from is removed.
• When due, the loop signs in with the Dovecot master user (mail.sso.prepare as the system actor, ADR 0018; the server comes from mail.client.config, now listed in system_calls) through an unregistered session (gw.OpenOffline) and uses the same hand-off as an immediate send (gw.DeliverRaw: submission, copy to Sent, \Answered/$Forwarded, draft removed). Postfix still enforces login = sender, so a scheduled message cannot do what an immediate one cannot. A 5xx answer marks the row failed (kept in Scheduled with the server's reason, webmail.send.failed event); a network error retries with 1, 2, 4, 8, 16 minutes, then fails. A claim is FOR UPDATE SKIP LOCKED; a sender that died mid-send is retried after 10 min.
• scheduled.list, scheduled.edit (puts the message back into Drafts with its attachments and deletes the row), scheduled.cancel. Sent rows are history for 7 days with the body emptied.
• Undo send is unchanged: a 0-30 s hold in memory of the session (it is delivered when the session closes). It is not durable across a restart of the plugin; the scheduled queue is.
• message.snooze {folder, uids, wake_at, timezone} moves the messages to the folder Snoozed (created on first use; a normal folder, so every client and phone sees it) and records the Message-ID. At the time the loop moves the message back to the folder it came from, adds the keyword $Snoozed and clears \Seen, so it arrives as unread, newest, marked. A message deleted meanwhile just is not there. snooze.list, snooze.cancel (back now).

Mail connections per mailbox (w11-36)

The mail server allows ten connections per user and address. A mailbox session holds the command connection only while it works (given back after 45 s without work and reopened by the next operation) and one IDLE connection (push) only while a page is looking: the page sends session.watch every 30 s, and a session that heard nothing for 100 s parks IDLE and wakes by itself on the next request. A second sign-in to the same mailbox from the same panel session returns the existing session. All sessions of one mailbox share a budget of five connections (gw/conns.go); an open that cannot get one within 8 s answers "too many open mail connections", never "wrong password".

Calendar (w11-36)

Events live in the mailbox's own CalDAV calendars on the contacts-and-calendar service (the same sign-in and confinement as contacts; the service's port is read from the mail module's mail.client.config, never written here). gw/ical.go is the strict iCalendar reader/writer (512 KB per object, 8 MB and 3000 events per import, text cleaned like mail, ATTACH and other unmodelled lines never taken from mail or an import) and the repeat-rule expander (DAILY, WEEKLY, MONTHLY, YEARLY with INTERVAL, COUNT, UNTIL, BYDAY with ordinals, BYMONTHDAY, BYMONTH, BYSETPOS; finer frequencies refused; 200 000 iterations and 2000 occurrences per query; a window is at most 400 days). A stored rule is capped at COUNT=5000, because the calendar service itself expands a rule when it stores the event. Times are kept in the event's own zone (VTIMEZONE written), repeat rules are evaluated on the wall clock of that zone, and the screen draws everything in one display zone chosen in the toolbar. "This event / this and following / all" are RECURRENCE-ID overrides, UNTIL + a new series, and the master. gw/invite.go: an iCalendar part in a mail shows Accept / Maybe / Decline; the answer is the standard REPLY to the ORGANIZER of the invitation (shown to the person, and re-checked on the server), sent through the mailbox's own send path; updates, cancellations and guests' answers are applied only when the organiser matches. Reminders: a loop looks at the calendars of mailboxes whose Calendar was opened (cal_watch) once a minute through the master user and raises one bell notice per due reminder (core.notice.raise, grouped after four); only the last 15 minutes are considered, so an import of past events raises nothing; cal_fired stops a second notice. Nothing is mailed.

Contacts (w11-35) and the calendar to come

The mail stack has no DAV, so contacts are a separate small service, Radicale (Python, GPL-3.0, plain files per mailbox; radicale on Debian/Ubuntu, radicale3 from EPEL 9/10 on the RHEL family). It is off until an administrator runs mail.dav.set {enabled: true} (high risk: plan, step-up, a job). That installs the package (system.apt.ensure), then the agent op mail.dav.service writes /etc/radicale/config, the sign-in plugin /etc/radicale/rc/rc_auth.py, a systemd drop-in and the mail host's certificate for it, and starts it on port 5232 with TLS, all interfaces. mail.client.config reports dav (enabled, host, port) and the webmail Contacts screen shows an explanation (and, for an administrator, a "Switch on" button) while it is off.

• Sign-in is the mail stack's own, never the panel session. The plugin does an IMAP login against Dovecot on 127.0.0.1:993 with what the client sent: the mailbox address and its password (an app password later: whatever Dovecot accepts), or address*masteruser and the master password, which is how the gateway signs in for a person who opened webmail without typing a password. The user the service sees is the mailbox address (everything after * is dropped), and rights = owner_only confines it to /<address>/. Proven by lab/tests/dav-isolation.sh (two mailboxes, 16 attempts) and by two mailboxes in the webmail.
• The gateway is a DAV client (gw/dav.go): it addresses only /<own address>/contacts/ (MKCOL on first use), over the same verified TLS as IMAP (the mail node's pinned self-signed certificate or a CA one), with the session's own credentials. Actions: webmail.contacts.list/save/delete/import/export/group. Optimistic concurrency by ETag (a phone's change in between gives a clear conflict, not a lost update).
• vCards are untrusted data (gw/vcard.go): one card at most 256 KB, an import 8 MB / 3000 cards, every value stripped of control and bidi-override characters and truncated, e-mail addresses validated, only http(s) URLs kept, group names without separators, what the webmail writes back is generated (vCard 3.0), never echoed; properties the webmail does not model (PHOTO, X-*) are kept as opaque lines so a phone's data survives an edit and are never interpreted or fetched. A card UID that is path-shaped is replaced.
• Recipient suggestions come from the address book first, then from the people in the loaded mail.
• Not yet: app passwords (Dovecot's passwd-file passdb takes one password per user; needs a second passdb or SQL), the service behind the web server on 443 with /.well-known/carddav (Radicale itself redirects /.well-known/carddav to / on its own port), certificate renewal reaching the service's copy, the firewall opening for 5232, a calendar screen (w11-36 uses the same service: CalDAV collections are allowed already).

Rules (ManageSieve) and vacation

Rules are structured JSON (conditions on from/to/cc/subject/header/size/body; actions fileinto/copy/mark read/flag/label/ redirect/discard/stop), rendered to Sieve by BuildSieve with every value quoted/validated and the rules embedded in a comment for exact round trip, stored as script rc-webmail and activated through port 4190; the server's own syntax error is returned verbatim. The rules script is the user's active script; the out-of-office reply lives in a separate file the mail module owns (~/.rc-vacation.sieve, a per-user before script), so the two never share a file (mail/012). rules.set never switches off another active script (one written by an older install or another client): it is pulled into rc-webmail with include :personal before the rules (BuildSieveWith), as is any script the previous rc-webmail already included; replace_foreign drops them on purpose. Saving a rule therefore cannot lose an out-of-office reply. vacation.set calls mail.autoresponder.set/.delete as the caller and mirrors the values in settings.

Data (Postgres schema of the module)

identities (owner account + mailbox, email, name, reply-to, signatures; limit key webmail.identities 20), settings (<= 16 KiB JSON per mailbox, vacation mirror under key vacation), trusted_senders. Keyed by the hosting account so a mailbox shared by sub-users shows one set.

Verified

See docs/tasks/w4/w4-06-webmail-backend.handover.md (unit + lab results).

Not done / next

Calendar (CalDAV on the same service), app passwords, one-click unsubscribe POST (needs an SSRF-safe fetcher), push, S/MIME/PGP, delegation, attachment zip, FTS tuning/benchmark on 100k mailboxes, message preview snippets (IMAP PREVIEW), remembered sessions via the vault, a panel-session-id binding (the actor has no session id; binding is to the principal), per-folder IDLE for several folders at once, Sieve include integration with the mail module (above).

Remote images (w11-13)

The panel's content-security policy (img-src 'self' data:) is inherited by the sandboxed frame that shows a message, so a remote https: image never loads in a browser, whatever the frame's own policy says (proved in Chromium by web/apps/panel/scripts/ui-browser.test.mjs). The panel's policy is not loosened. When the person presses "Show images" (or trusts the sender) the gateway fetches each image itself (gw/inlineimg.go: the egress guard refuses private, loopback and metadata addresses, no redirects, PNG/JPEG/GIF/WebP/AVIF sniffed from the bytes, SVG never, 2 MiB each, 40 and 8 MiB per message) and puts it into the message as a data: image.