Skip to content

feat: warn about duplicate recipients in CSV uploads (#3319)#2708

Open
smcmurtry wants to merge 1 commit into
mainfrom
fix/3319-warn-duplicate-recipients
Open

feat: warn about duplicate recipients in CSV uploads (#3319)#2708
smcmurtry wants to merge 1 commit into
mainfrom
fix/3319-warn-duplicate-recipients

Conversation

@smcmurtry

Copy link
Copy Markdown
Contributor

Warn about duplicate recipients in CSV uploads — closes cds-snc/notification-planning#3319

Summary

Surfaces a non-blocking warning on the bulk-send preview page when the
uploaded CSV contains the same recipient more than once. Senders can review
the warning, download a list of duplicates, and either proceed with the send
or go back and fix their list.

What's new

Backend

  • app/utils.py — new get_warnings_for_csv() helper that returns
    user-facing warning strings (currently: duplicate recipients). Letters are
    exempt because multiple recipients can legitimately share an address.
  • app/main/views/send.py — wires the new warnings into the
    _check_messages data dict, alongside count_of_duplicate_recipients and
    count_of_duplicate_recipient_rows.
  • New route download_duplicate_recipients that returns a CSV containing
    only the duplicate rows. The CSV is built server-side from the original S3
    upload (which is scoped to the authenticated user via
    @user_has_permissions("send_messages", restrict_admin_usage=True)), so
    duplicates are only ever visible to the sender — privacy preserved.

UI

  • app/templates/views/check/ok.html — a yellow "warning" banner is
    rendered at the top of the bulk-send review page when duplicates are
    detected. Includes:
    • a count-aware message (singular/plural EN+FR)
    • an additional row count when one recipient appears many times
    • a "Download a list of duplicate recipients" link
    • wrapped in aria-live="polite" for screen-reader users
    • data-testid="duplicate-recipients-warning" for Cypress hooks

Translations

EN/FR strings for the warning are added to app/translations/csv/fr.csv. The
generated en.csv / messages.po files are produced by make babel and are
gitignored.

English French
1 recipient appears in your list more than once. They will receive the message multiple times. 1 destinataire figure plusieurs fois dans votre liste. Cette personne recevra le message plusieurs fois.
{} recipients appear in your list more than once. They will receive the message multiple times. {} destinataires figurent plusieurs fois dans votre liste. Ces personnes recevront le message plusieurs fois.
{} rows are duplicates of an earlier row. {} lignes sont des doublons d'une ligne précédente.
Download a list of duplicate recipients Télécharger la liste des destinataires en double

Detection rules (delivered by notifications-utils)

  • Case-insensitive — Alice@Example.com and ALICE@EXAMPLE.COM are duplicates.
  • Whitespace-insensitive — leading/trailing whitespace is ignored.
  • Phone numbers normalised to E.164 — 6502532222, +1 650-253-2222 and
    650 253 2222 recognised as the same recipient.
  • Bad-recipient and missing-recipient rows skipped.
  • Letters excluded (recipients can legitimately share an address).
  • Only the second and later copies of a recipient are flagged.

Tests

  • New tests/app/main/test_warnings_for_csv.py covers
    get_warnings_for_csv for all branches (no duplicates, singular, plural,
    letter exemption, row-count subline).
  • The detection logic itself is exhaustively tested in the companion
    notifications-utils PR.

A11y / privacy / security

  • A11y — banner is in an aria-live="polite" region and uses the
    existing accessible banner_wrapper macro.
  • Bilingualism — EN+FR strings added.
  • Privacy — the duplicates CSV is generated server-side from the
    authenticated user's own upload and is never persisted; only the sender can
    download it.
  • Security — no new attack surface. The route is permission-gated,
    filename is sanitised via SanitiseASCII, and the response uses the
    existing Spreadsheet helper.

Companion PR (must merge first)

Depends on duplicate-recipient detection in
notification-utils #fix/3319-warn-duplicate-recipients
(version 53.2.25).

Closes cds-snc/notification-planning#3319.

Surfaces a non-blocking warning on the bulk-send preview page when the
uploaded CSV contains the same recipient more than once. The warning is
case-insensitive, ignores leading/trailing whitespace, and (for SMS)
treats phone numbers in different formats as equivalent. Letters are
excluded because multiple recipients can legitimately share an address.

Changes:
- new get_warnings_for_csv helper in app/utils.py
- new download_duplicate_recipients route serves a CSV of duplicate
  rows so the sender can review them before sending; the response is
  built server-side from the original upload, so duplicates are visible
  only to the authenticated sender (privacy)
- ok.html renders the warning in a banner inside an aria-live=polite
  region, including a link to download the duplicates list
- EN/FR translation strings added to fr.csv (en.csv and messages.po
  files are regenerated by 'make babel')
- unit tests for get_warnings_for_csv

Depends on notifications-utils with duplicate-recipient detection
(see corresponding PR in notification-utils).

Refs: cds-snc/notification-planning#3319
@github-actions

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[admin/utils] Bulk-send CSV uploads do not warn about duplicate recipients

1 participant