[PM-33948] Add draft project changes by jprusik · Pull Request #4 · bitwarden/map-the-web

jprusik · 2026-03-24T21:34:32Z

🎟️ Tracking

📔 Objective

This PR primarily pulls in the working state from the draft project at https://github.com/jprusik/map-the-web-draft (you can also get a sense for how the releases look there)

The project provides curated JSON "Maps" that describe websites. These changes represent the form Map which describes form field locations using CSS selectors, enabling consumers (like the Bitwarden browser extension) to locate fields without relying solely on heuristics.

The READMEs cover most of the guidance (which represents the most important part of this review), but I'll add a few notes about some decisions made along the way:

I initially represented Maps as json files, but switched to jsonc for some ad-hoc annotation flexibility; the release builds drop any comment
we're using a custom >>> combinator in CSS selectors to describe iframe and shadow dom boundaries. There are some examples of prior art: WebDriverIO | Puppeteer | originated in (abandoned) w3 spec
~~the key names for selectors are still a bit in flux (as called out in the README and schema); I opted to come back to them to fight scope creep (see PM-34086)~~ edit: I've addressed this in 93e3152 to get us started with something more concrete
The authoring guidance initially allowed for a broader range of values to indicate "not relevant" and "no information". I personally found that got confusing quickly, and so added constraints around empty values not being allowed as an authoring concern. In our consuming app, the type documentation still defends against these (invalid) cases (and defines how we handle them), since we cannot enforce authoring guidelines of other map sources (which we will allow).
For IDNs using unicode, I felt it was easier for human authors to standardize around keys with unicode values over punycode representations. We build to the punycode representations, so that ostensibly avoids problems with consumer normalization. There's a risk of homoglyph confusion, but since we're expecting to hand-validate entries, I didn't feel that to be a large risk at this stage.
Release versions and semver for schemas give us flexibility to adopt changes to the schema without breaking existing consumers' implementations; we build to all schema versions of all maps for each release version. To summarize; release version represents the growth of the Map data/content, schema versions represent the growth of the Map shapes.
- Note, this does mean writing downgrade/migration scripts for the build process if/when a map data source moves to a new major (breaking) schema version
Prettier breaks Github alert syntax, so we don't use it for markdown files. The current Remark tooling is used only for linting, not formatting, for the same reason. Maybe worth revisiting at some point.
www. prefixed hosts technically can behave differently than their non-www counterpart. This scenario is described in the guidance (only include www in the host key name when needed), but the checks only warn about this scenario right now. It's edgy enough that I don't think we need to dive deeply into it, but if we run into these cases commonly, we might re-visit the tooling.
Additional work for this project is tracked in the Epic here: PM-32062 - as we uncover additional tasks for the initial release, we should add them there.

jprusik · 2026-03-24T22:49:26Z

+- describing the age of individual host entries
+- annotating entries (descriptions of why a particular selector is needed, etc.)
+- form rendering timings
+- distinguishing URLs by query string and/or fragment that affect rendered form content
+- describing dynamically-renamed fields (e.g. sites that randomize attribute values on each render)
+- form inputs that represent user confirmation (e.g. agree to terms of service, etc)
+- indicators of irrelevant data at the form field level


Some of these limitations are addressable, but require a level of consideration that would expand the scope of this effort considerably. We should identify the level of impact of these (and identify other cases) in our testing.

jprusik · 2026-03-24T22:57:11Z

Checkmarx and Sonar scans are expected to fail as we get the repo configured.

MGibson1

Looking really good. There are a few items in here that I think need to change before we can commit to schemas.

MGibson1 · 2026-03-25T00:21:22Z

+      if (host.startsWith("www.")) {
+        console.warn(
+          `\x1b[33mWarning: ${file} - host key "${host}" uses a www. prefix. ` +
+            `Author under the non-www host as canonical unless hosts differ. ` +


This line is unnecessarily terse to the point of being unclear

Suggested change

`Author under the non-www host as canonical unless hosts differ. ` +

`Prefer adding maps without the "www." prefix, unless rules differ between the "www." and un-prefixed. domains` +

MGibson1 · 2026-03-25T00:24:55Z

+
+  const validate = ajv.compile(schema);
+  if (!validate(data)) {
+    console.error(`\x1b[31mValidation failed: ${file}\x1b[0m`);


It wouldn't be that big a deal to pick up chalk, would it?

Another dependency is another attack vector 😭 😂

MGibson1 · 2026-03-25T00:29:08Z

+using CSS selectors. It enables consuming applications to locate specific fields
+without relying on heuristic determinations or page-specific detection logic.
+
+This Map describes **the content of a page**. It does not prescribe or imply how


Suggested change

This Map describes **the content of a page**. It does not prescribe or imply how

This Map describes **the semantic content of a page**. It does not prescribe or imply how

I'm worried this term is overloaded for this context; we indeed describe non-semantic elements (e.g. div) to give them semantic meaning they otherwise don't have. What I was trying to get at with the emphasis here was that we're not, for example, interested in describing page interactions or programmatic code behaviours (at least not in this Map at this time)

edit: The revised form properties (93e3152) now describe form actions as an independent concern.

Maybe this is obvious to a consumer, but the lift that semantic is doing in that suggestion isn't to distinguish from actions, but to make it clear that this map does not describe content directly, just the meaning behind whatever content is presented.

Pretty pedantic, I don't think I'm too fussed about this one.

Not a gigantic pedantic semantic, granted

MGibson1 · 2026-03-25T00:30:45Z

+
+This Map describes **the content of a page**. It does not prescribe or imply how
+a consumer of this Map should behave. Additionally, the term "form" here
+describes the user-facing concept, which may or may not utilize the HTML `form`


Describe the concept for clarity. I took a stab at what I suspect you mean. Surprisingly hard to describe tersely.

Suggested change

describes the user-facing concept, which may or may not utilize the HTML `form`

describes the user-facing concept of users supplying data to a web site, which may or may not utilize the HTML `form`

I think this suggestion works; the form element spec talks about "palpable content", which is an interesting idea in the mix here.

MGibson1 · 2026-03-25T00:41:09Z

+## Pathnames
+
+The `pathnames` object maps URL pathnames to page-specific entries. Pathnames
+must start with `/`.


Is this enforced?

It's captured in the schema (which we run validation against) as a regex pattern here:
https://github.com/bitwarden/map-the-web/pull/4/changes#diff-1d367c40e031794e17463f106698bdc17b6b8ce8127cc0d3b14b50d5dcb5085cR72-R73

MGibson1 · 2026-03-25T00:45:12Z

+When `category` is omitted, the form's purpose is unspecified. Consumers should
+not infer purpose from the absence of a category.


This should be unallowed in the schema, without a category, you are missing critical semantic information. As you point out, it renders data unusable, so why have it?

As you point out, it renders data unusable

Perhaps I'm misunderstanding, but It does not; it optionally enriches it. The purpose of a form lacking the contextual information of category can still be inferred by the input values grouped by the form concept (which is what our consumer already does heuristically).

to me it sounds like you're saying "don't infer the purpose of a form field" and then followed it up with "we do".

While I understand that you're saying you're pulling in contextual information around the field to infer it's purpose, that is not the purpose of these maps. That's describing our normal autofill process. I don't see value in a field in the map having no category, since the reason for the map to exist to the specify the purpose of a field.

Ah, what I'm after is that the category absence itself does not have meaning (but consumers can still infer categories/context from the fields).

But perhaps it should not be optional. My intent was to cover categories we've not yet captured, but we probably want to capture them as we go. We can always revisit it later.

addressed in 692d32e (category is no longer optional)

MGibson1 · 2026-03-25T00:47:52Z

+The `selectors` object maps keys to arrays of CSS selectors. Each key is an
+opaque identifier whose meaning is defined by the consuming application. The Map


Each key is an opaque identifier

I don't think this is going to fly, I understand it's hard to meaningfully list all options, but we need a defined set of known meanings and maybe support arbitrary data.

As written, the repository could become weird infighting where each application repeats work to appease some esoteric internal concern.

Yeah, totally - as I called out in the PR summary; I intentionally deferred this, but it needs to be finalized before a release (of the project). Our consumers aren't currently making use of it, so I wanted to get a sense of the practical value add of this before committing to it.

Thinking about this more, I see no reason we cannot solidify what we have now. We can always extend the schema during evaluation and have the ability to version the schema after release if needed.

I'll update the requirements and definitions of the selectors.

addressed in 93e3152

MGibson1 · 2026-03-25T00:49:54Z

+
+The boundary type is determined by the element matched before `>>>`:
+
+- **No `iframe` tag** in the preceding segment → shadow DOM boundary


And if there is no preceding segment? (this is probably not possible, you'd just use html, but idk, just noticed a potentially missing arm)

I believe a host node will be required for both cases (iframe and shadow host). If the shadow root is at the root of the page somehow, yeah I'd expect it to be represented as html >>> nodetag - so for the purposes of the doc; we'd always expect something preceding (and following!) the >>> combinator (never naked). I'll clarify this point in the doc.

addressed here: https://github.com/bitwarden/map-the-web/pull/4/changes#diff-a95d9819c625a153f046b2ff8607e66c6986a7d4587452c6713c87e019850c5dR373

MGibson1 · 2026-03-25T14:06:12Z

+  manifest.maps[map.name] = {
+    version: map.data.version,
+    files: [relative(DIST, outDataFile), relative(DIST, outSchemaFile)],
+  };


This manifest schema will not work once you have to publish multiple versions of the same map, which I believe is a backward compatibility goal.

addressed in cc8086b

jprusik · 2026-03-25T18:47:49Z

+  - [Null and Empty Semantics](#null-and-empty-semantics)
+  - [Authoring Guidelines](#authoring-guidelines)
+
+## Limitations


Not to be conflated with goals - some of these are limitations we actively do want to eliminate, but may require additional work in follow up tasks; others are constraints of necessity

MGibson1 · 2026-04-06T23:07:00Z

+using CSS selectors. It enables consuming applications to locate specific fields
+without relying on heuristic determinations or page-specific detection logic.
+
+This Map describes **the content of a page**. It does not prescribe or imply how


Maybe this is obvious to a consumer, but the lift that semantic is doing in that suggestion isn't to distinguish from actions, but to make it clear that this map does not describe content directly, just the meaning behind whatever content is presented.

Pretty pedantic, I don't think I'm too fussed about this one.

MGibson1 · 2026-04-06T23:10:15Z


 ## Data Structure Overview

 ```json


changing to jsonc may fix this red formtting of the comment. Probably not the ellipse, though

addressed in 692d32e

MGibson1 · 2026-04-06T23:14:20Z

 }
 ```

+### Field Keys


Does adding a field key constitute a schema version bump? What about removing? Renaming?

I think you should define some process here since it's likely to occur. We can head off some confusion by thinking through it now and preparing.

addressed in 2aa15e8

MGibson1 · 2026-04-06T23:17:38Z

+> [!IMPORTANT]
+> A combined expiration field (`cardExpirationDate`) is not the same as separate
+> month and year fields (`cardExpirationMonth` / `cardExpirationYear`). Use the
+> key that matches the actual input structure on the page.


nit: different structure to this callout vs address, phone, birthday, etc

addressed in 692d32e

MGibson1 · 2026-04-06T23:22:02Z

+Order is significant within a sequence. The map does not specify how the value
+is split across the elements.


Again, it feels like it does and needs to. Looking back at the name, birthday, etc, all of those define where and how to split data. This isn't really a special case.

I suppose some malicious site may come out that splits things such that the first digit goes in the first, two in the next, and the remaining in a third... That may just mean fields that lack pre-defined division rules need some way to express them.

This can be a schema improvement for later, though.

Yeah, I was also thinking about SSN entry forms and partial entry forms like SSN and credit card confirmation and totp-like separated entries for phone numbers. I played with trying to abstract that split more definitively, but ended up with some unwieldy patterns. I think I'm leaning toward the opinion that grouped partial information deserves it's own descriptor when it's not a uniform distribution, but I agree we can come back to it when we've had more time using the map.

some malicious site may come out

If we're curating a map for the site, it means we can't represent something on the page (that we'd otherwise want to), at worst. If the site/page doing something actively malicious, we have other mechanisms to communicate/defend against that.

differsthecat

This is awesome, thank you, @jprusik ! 🚀

differsthecat · 2026-04-08T12:27:40Z

+There is presently no mechanism embedded within the Forms Map for:
+
+- describing the age of individual host entries
+- annotating entries (descriptions of why a particular selector is needed, etc.)


Non-blocking: The file format for forms is jsonc, which should allow us to add comments, so we could annotate an entry with comments (https://jsonc.org/), unless that messes with the validation of the entries.

github-actions · 2026-04-09T13:04:13Z

Checkmarx One – Scan Summary & Details – ff9e969d-ffdf-475a-acd9-2931dd5fa30b

Great job! No new security vulnerabilities introduced in this pull request

jprusik · 2026-04-10T13:44:34Z

Merged as discussed with @MGibson1 - we will continue to refine and will have at least one final review before public release

jprusik added 5 commits March 24, 2026 15:43

pull in draft repo changes

6ba1ca1

additional cleanup

1c4b0a1

add guidance and tooling for hosts with unicode characters

0c9247e

add release version validation

eb76188

update docs

d6e83cb

jprusik self-assigned this Mar 24, 2026

jprusik commented Mar 24, 2026

View reviewed changes

address workflow linting feedback

571e01e

jprusik requested review from MGibson1 and differsthecat March 24, 2026 22:58

MGibson1 suggested changes Mar 25, 2026

View reviewed changes

jprusik commented Mar 25, 2026

View reviewed changes

update forms map doc

f26559c

jprusik force-pushed the pm-33948 branch from 5bb4838 to f26559c Compare March 25, 2026 22:06

jprusik added 5 commits April 3, 2026 14:17

split selectors to dedicated concepts

93e3152

enable multi-input value descriptions

cec1c3d

cleanup

1d3cb2d

handle search form case gap

958855b

clarify authoring www-prefixed cases

ae6a406

jprusik force-pushed the pm-33948 branch from 5ab1260 to ae6a406 Compare April 6, 2026 18:13

jprusik added 3 commits April 6, 2026 15:14

update schema build and release strategy

cc8086b

cleanup

b974406

additional guidances

48408c7

jprusik requested a review from MGibson1 April 6, 2026 20:29

MGibson1 reviewed Apr 6, 2026

View reviewed changes

jprusik force-pushed the pm-33948 branch from 692d32e to 2aa15e8 Compare April 7, 2026 12:03

jprusik added 2 commits April 7, 2026 08:31

clarify guidance in docs

5e69fc5

add versioning guidance

02d8e62

jprusik force-pushed the pm-33948 branch from 2aa15e8 to 02d8e62 Compare April 7, 2026 12:44

jprusik requested a review from MGibson1 April 7, 2026 14:01

clarify host key inheritence strategy

c1697da

jprusik marked this pull request as ready for review April 7, 2026 14:37

jprusik requested a review from a team as a code owner April 7, 2026 14:37

differsthecat approved these changes Apr 8, 2026

View reviewed changes

jprusik merged commit 10ef7ed into main Apr 9, 2026
23 of 35 checks passed

jprusik deleted the pm-33948 branch April 9, 2026 22:31

jprusik mentioned this pull request Apr 10, 2026

[PM-33948] Address workflow failure #6

Merged

	`Author under the non-www host as canonical unless hosts differ. ` +
	`Prefer adding maps without the "www." prefix, unless rules differ between the "www." and un-prefixed. domains` +

	This Map describes the content of a page. It does not prescribe or imply how
	This Map describes the semantic content of a page. It does not prescribe or imply how

	describes the user-facing concept, which may or may not utilize the HTML `form`
	describes the user-facing concept of users supplying data to a web site, which may or may not utilize the HTML `form`

		When `category` is omitted, the form's purpose is unspecified. Consumers should
		not infer purpose from the absence of a category.

		The `selectors` object maps keys to arrays of CSS selectors. Each key is an
		opaque identifier whose meaning is defined by the consuming application. The Map


		The boundary type is determined by the element matched before `>>>`:

		- No `iframe` tag in the preceding segment → shadow DOM boundary

		Order is significant within a sequence. The map does not specify how the value
		is split across the elements.

Uh oh!

Conversation

jprusik commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎟️ Tracking

📔 Objective

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik commented Mar 24, 2026

Uh oh!

MGibson1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jprusik Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

jprusik commented Mar 24, 2026 •

edited

Loading

jprusik Mar 25, 2026 •

edited

Loading

jprusik Mar 25, 2026 •

edited

Loading

jprusik Mar 25, 2026 •

edited

Loading

jprusik Mar 25, 2026 •

edited

Loading

jprusik Mar 25, 2026 •

edited

Loading

jprusik Apr 7, 2026 •

edited

Loading

github-actions Bot commented Apr 9, 2026 •

edited

Loading