Skip to content

fix(Auth): normalize and compare email addresses case-insensitively - #314

Open
jakeaturner wants to merge 1 commit into
stagingfrom
fix/email-normalization
Open

jakeaturner wants to merge 1 commit into
stagingfrom
fix/email-normalization

Conversation

@jakeaturner

Copy link
Copy Markdown
Collaborator

Email addresses entered LibreOne through eight paths and were stored exactly as received, leaving case matching to whatever collation MySQL defaulted to. Mixed casing leaked into outbound mail, synced library user records, CAS bridge JWT payloads, and email_events correlation.

All addresses now share one canonical form: trim, Unicode NFC, lowercase. Diacritics are preserved, so josé@x.com and jose@x.com remain distinct.

  • Add normalizeEmail() in email.ts, shared by the server and browser bundle
  • Add emailValidator to validators/shared.ts and apply it across the auth, user, and store schemas so controllers receive canonical input
  • Wire query validation onto GET /users/principal-attributes and GET /auth/cas-interrupt-check, which previously had none
  • Normalize the external IdP JWT, CAS bridge principal, back-channel SLO lookup, and both getAttrMatchKey call sites
  • Normalize the email operands in the user search predicate and relevance ranking
  • Add @BeforeValidate hooks on User, EmailVerification, and EmailEvent
  • Normalize client form input in AuthForm, NewEmailForm, and password recovery
  • Add migrations/0003_normalize_email_case.sql: collision detection, backfill, and utf8mb4_0900_as_ci collation (case-insensitive, accent-sensitive)

Requires migration 0003 before deploy. Run its passes in order and resolve any case-variant duplicates by hand before the backfill.

Email addresses entered LibreOne through eight paths and were stored exactly as
received, leaving case matching to whatever collation MySQL defaulted to. Mixed
casing leaked into outbound mail, synced library user records, CAS bridge JWT
payloads, and email_events correlation.

All addresses now share one canonical form: trim, Unicode NFC, lowercase.
Diacritics are preserved, so josé@x.com and jose@x.com remain distinct.

- Add normalizeEmail() in email.ts, shared by the server and browser bundle
- Add emailValidator to validators/shared.ts and apply it across the auth, user,
  and store schemas so controllers receive canonical input
- Wire query validation onto GET /users/principal-attributes and
  GET /auth/cas-interrupt-check, which previously had none
- Normalize the external IdP JWT, CAS bridge principal, back-channel SLO lookup,
  and both getAttrMatchKey call sites
- Normalize the email operands in the user search predicate and relevance ranking
- Add @BeforeValidate hooks on User, EmailVerification, and EmailEvent
- Normalize client form input in AuthForm, NewEmailForm, and password recovery
- Add migrations/0003_normalize_email_case.sql: collision detection, backfill, and
  utf8mb4_0900_as_ci collation (case-insensitive, accent-sensitive)

Requires migration 0003 before deploy. Run its passes in order and resolve any
case-variant duplicates by hand before the backfill.
@github-actions

Copy link
Copy Markdown

Severity: Medium — Migration line 56: The backfill and collision check only lowercase addresses; application lookups now also trim whitespace. Existing addresses with leading whitespace become unreachable through email lookup. Apply the same normalization during backfill and collision detection before enabling normalized lookups.

Severity: Medium — CAS validation line 98: joi.string() rejects empty strings. Requests containing registeredService= or service= now return HTTP 400 instead of reaching the existing no-service redirect logic. Allow empty values or convert them to absent values.

Tests were not run because dependencies are unavailable.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The migration has an unresolved duplicate-key failure path, and additional collation, documentation, and validator issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 High severity · 2 Low severity

Open (3)
What changed in this PR

This pull request standardizes email addresses using trim, Unicode NFC normalization, and lowercasing across authentication, persistence, integrations, client forms, and existing database records.

Changes:

  • Adds shared email normalization and validation across server and client flows.
  • Updates authentication, CAS, IdP, search, event, and model paths.
  • Adds regression tests and a manual migration with collision checks and collation updates.
File Summary / final review notes
server/​validators/​user.ts Applies canonical email validation to user inputs.
server/​validators/​store.ts Applies canonical email validation to store inputs.
server/​validators/​shared.ts Moderate (1 vote): Non-email external subject IDs are trimmed; opaque values should remain unchanged.
server/​validators/​auth.ts Applies canonical email validation to authentication inputs.
server/​tests/​users.spec.ts Covers user normalization behavior.
server/​tests/​snsEvents.spec.ts Covers SNS event handling.
server/​tests/​email.unit.ts Covers shared email normalization.
server/​tests/​auth.spec.ts Covers authentication normalization.
server/​routes/​users.ts Adds validation for user query parameters.
server/​routes/​auth.ts Adds validation for authentication query parameters.
server/​models/​User.ts Adds persistence normalization for users.
server/​models/​EmailVerification.ts Adds persistence normalization for email verifications.
server/​models/​EmailEvent.ts Adds persistence normalization for email events.
server/​helpers.ts Re-exports shared normalization helpers.
server/​controllers/​UserController.ts Normalizes user lookups and search operands.
server/​controllers/​SnsEventController.ts Normalizes SNS event email handling.
server/​controllers/​EmailVerificationController.ts Normalizes verification lookups.
server/​controllers/​AuthController.ts Normalizes authentication and external identity flows.
pages/​passwordrecovery/​+Page.vue Normalizes password recovery input.
migrations/​README.md Nit (2 votes): Document 0003 as a manual, multi-pass exception to the generic migration command.
migrations/​0003_normalize_email_case.sql Critical (3 votes): Pass 2 can violate the (ses_message_id, event_type, email) unique index for existing case variants; add collision detection before updates. Moderate (1 vote): Use accent-sensitive or binary comparison for collision keys. Moderate (1 vote): Align model column metadata with the new collations or document the exception. Nits (2 and 1 votes): Correct the identical accent example and document the manual migration procedure.
email.ts Provides the shared email normalizer.
components/​registration/​AuthForm.vue Normalizes registration form input.
components/​account_management/​NewEmailForm.vue Normalizes new-email form input.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +53 to +54
-- email_verifications rows expire after 24 hours and email_events has no unique
-- constraint on email alone, so neither needs a collision pre-check.
-- on User, EmailVerification, and EmailEvent. This file brings existing rows in line
-- and makes the database enforce the same policy.
--
-- Diacritics are preserved on purpose: jose@x.com and jose@x.com (with an accent) are
Comment thread migrations/README.md
| --- | --- | --- | --- |
| `0001_users_search_indexes.sql` | | | |
| `0002_api_user_password_write_permission.sql` | | | |
| `0003_normalize_email_case.sql` | | | |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants