Migrate Specberus to TypeScript and improve API surface

.c8rc

@@ -1,5 +1,6 @@
 {
     "all": true,
-    "exclude": ["public/*", "test/*", "tools/*"],
+    "exclude": ["public/*", "test/*", "**/*.d.ts"],
+    "extension": [".ts"],
     "reporter": ["html"]
 }

.cspell.json

@@ -4,6 +4,7 @@
         "ˈspɛk",
         "apikey",
         "badterms",
+        "basenames",
         "Beihang",
         "blocklist",
         "capi",
@@ -17,7 +18,6 @@
         "deniak",
         "discr",
         "DNOTE",
-        "doasync",
         "doctypes",
         "domhandler",
         "dvcs",
@@ -108,26 +108,20 @@
         "**/*.ttf",
         "**/*.woff",
         "**/*.svg",
-        ".nyc_output/**",
         ".github/**",
         "coverage/**",
         "test/**/*.html",
         "package-lock.json",
         "tsconfig.json",
+        "{lib,test}/**/*.{d.ts,js}",
         "node_modules/**",
         "design/**"
     ],
-    "ignoreRegExpList": ["/require\\(.*\\);/"],
+    "ignoreRegExpList": ["(FIXME|TODO|XXX)\\(.[^\\)]+\\)"],
     "overrides": [
         {
             "filename": ["package.json"],
-            "words": [
-                "capi",
-                "doasync",
-                "metaviewport",
-                "nodesecurity",
-                "vulns"
-            ]
+            "words": ["capi", "metaviewport", "nodesecurity", "vulns"]
         }
     ],
     "ignoreWords": ["en", "gb"]

.github/dependabot.yml

@@ -18,8 +18,6 @@ updates:
         update-types: ['version-update:semver-minor']
       - dependency-name: 'husky'
         update-types: ['version-update:semver-minor']
-      - dependency-name: 'nodemon'
-        update-types: ['version-update:semver-minor']
   - package-ecosystem: github-actions
     directory: '/'
     schedule:

.gitignore

@@ -25,7 +25,10 @@ scratch
 # mac files
 .DS_Store
 */.DS_Store
-.vscode/
 
-.nyc_output
-.eslintcache
+# TS build output
+app.js
+*.d.ts
+!lib/types.d.ts
+lib/**/*.js
+test/**/*.js

.husky/pre-commit

@@ -1,4 +1,4 @@
 #!/bin/sh
 . "$(dirname "$0")/_/husky.sh"
 
-npx lint-staged && npx tsc && npm run spelling
+npx lint-staged && npx tsc --noEmit && npm run spelling

.prettierignore

@@ -1,3 +1,2 @@
 *.html
 *.handlebars
-.nyc_output

.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "js/ts.tsdk.path": "node_modules/typescript/lib"
+}

README.md

@@ -22,16 +22,15 @@ Specberus is a [Node.js](https://nodejs.org/en/) application, [distributed throu
 Alternatively, you can clone [the repository](https://github.com/w3c/specberus) and run:
 
 ```bash
-$ npm install -d
+$ npm install
 ```
 
 In order to get all the dependencies installed. Naturally, this requires that you have a reasonably
 recent version of Node.js installed.
 
 ## 2. Running
 
-Currently there is no shell to run Specberus. Later we will add both Web and CLI interfaces based
-on the same core library.
+Specberus runs as a web server, providing both HTML form UI and API endpoints.
 
 ### Syntax and command-line parameters
 
@@ -51,6 +50,20 @@ $ npm start
 $ npm start 3001
 ```
 
+**Note:** `npm start` relies on JavaScript files, which must first be built via `npm run build`.
+This step is not necessary when running the live development server or tests (see below).
+
+#### Auto reload when developing
+
+Run `npm run live [PORT]` when developing. The app will automatically reload when changes happen.
+
+Examples:
+
+```bash
+$ npm run live
+$ npm run live 3001
+```
+
 ### Environment variables
 
 #### `DEBUG`
@@ -85,50 +98,27 @@ unauthenticated requests to 60 per hour.
 GH_TOKEN=github_pat_... npm start
 ```
 
-### Auto reload when developing
-
-Run `npm run live` when developing. The app will automatically reload when changes happen.
-
-```bash
-$ npm run live
-
-$ npm run live 3001
-```
-
 ## 3. Testing
 
-#### 1. Simple test
-
-Testing is done using mocha. Simply run:
+### 1. Simple test
 
-```bash
-$ mocha
-```
-
-from the root and you will be running the test suite. Mocha can be installed with:
+Run:
 
 ```bash
-$ npm install -g mocha
+$ npm test
 ```
 
-#### 2. SKIP_NETWORK
-
-Some of the tests can on occasion take a long time, or fail outright because a remote service is
-unavailable. To work around this, you can set SKIP_NETWORK:
-
-```bash
-$ SKIP_NETWORK=1 mocha
-```
+from the root to run the test suite.
 
-#### 3. Run testserver
+### 2. Run testserver
 
-The testcase document can run independently
+The testcase document server can run independently:
 
 ```bash
 $ npm run testserver
 ```
 
-#### 4. Run certain test
+### 3. Run certain test
 
 Add process env before `npm run test` and `describe.only()` to run single test.
 
@@ -156,12 +146,12 @@ $ RULE=copyright TYPE=noCopyright npm run test
 
 ## 4. JS API
 
-The interface you get when you `import { Specberus } from "specberus"` is that from `lib/validator`.
+The interface you get when you `import { Specberus } from "specberus"` is that from `lib/specberus`.
 `Specberus` is a class configured for operation in the Node.js environment.
 
 (See also [the REST API](#5-rest-api).)
 
-## Creating a Validator instance
+## Creating a Specberus instance
 
 ```js
 import { Specberus } from 'specberus';
@@ -172,26 +162,26 @@ const specberus = new Specberus();
 
 ### `validate(options)`
 
-This method takes an object with the following fields:
+This method returns a Promise that resolves with errors, warnings, and informative messages resulting from relevant checks.
+
+`options` is an object accepting the following fields:
 
-- `url`: URL of the content to check. One of `url`, `source`, `file`, or `document` must be
+- `url`: URL of the content to check. One of `url`, `source`, or `file` must be
   specified and if several are they will be used in this order.
 - `source`: A `String` with the content to check.
 - `file`: A file system path to the content to check.
-- `document`: A DOM `Document` object to be checked.
 - `profile`: A profile object which defines the validation. Required. See below.
-- `events`: An event sink which supports the same interface as the Node.js `EventEmitter`. Required. See
-  below for the events that get generated.
 
 ### `extractMetadata(options)`
 
-This method eventually extends `this` with metadata inferred from the document.
-Once the [event `end-all`](#validation-events) is emitted, the metadata should be available in a new property called `meta`.
+This method returns a Promise that resolves with metadata inferred from the document.
 
-The `options` accepted are equal to those in `validate()`, except that a `profile` is not necessary and will be ignored (finding out the profile is one of the
-goals of this method).
+The `options` accepted are equal to those in `validate()`, with the following differences:
 
-`this.meta` will be an `Object` and may include up to 20 properties described below:
+- Optional `additionalMetadata` property, which performs additional checks (e.g. errata URL)
+- No `profile` or `validation` properties (this method can be used to _determine_ profile)
+
+The resolved object's `metadata` property points to an object with up to 20 properties, described below:
 
 - `profile`
 - `title`: The (possible) title of the document
@@ -216,7 +206,7 @@ goals of this method).
 
 If some of these pieces of metadata cannot be deduced, that key will not exist, or its value will not be defined.
 
-This is an example of the value of `Specberus.meta` after the execution of `Specberus.extractMetadata()`:
+The following is an example of the value of the `metadata` object after the execution of `Specberus.extractMetadata()`:
 
 ```json
 {
@@ -238,8 +228,8 @@ This is an example of the value of `Specberus.meta` after the execution of `Spec
 
 Similar to the [JS API](#4-js-api), Specberus exposes a REST API via HTTP too.
 
-The endpoint is `<host>/api/`.
-Use either `url` or `file` to pass along the document (neither `source` nor `document` are allowed).
+The base path is `<host>/api/`.
+Use either `url` or `file` to pass along the document (`source` is not allowed).
 
 Note: If you want to use the public W3C instance of Specberus, you can replace `<host>` with `https://www.w3.org/pubrules`.
 
@@ -411,47 +401,63 @@ Profiles that are identical to its parent profile, ie that do not add any new ru
 
 ## 7. Validation events
 
-For a given checking run, the event sink you specify will be receiving a bunch of events as
-indicated below. Events are shown as having parameters since those are passed to the event handler.
-
-- `start-all(profile-name)`: Fired first to indicate that the profile's checking has started.
-- `end-all(profile-name)`: Fired last to indicate that the profile's checking has completed. When
-  you receive this you are promised that all testing operations, including asynchronous ones, have
-  terminated.
-- `done(rule-name)`: Fired when a specific rule has finished processing, including its asynchronous
-  tasks.
-- `ok(rule-name)`: Fired to indicate that a rule has succeeded. There is only one `ok` per rule.
-  There cannot also be `err` events but there can be `warning` events.
-- `err(error-name, data)`: Fired when an error is detected. The `data` contains further details,
-  that depend on the error but _should_ feature a `message` field. There can be multiple errors for
-  a given rule. There cannot also be `ok` events but there can be `warning`s.
-- `warning(warnings-name, data)`: Fired for non-fatal problems with the document that may
-  nevertheless require investigation. There may be several for a rule.
-- `info(info-name, data)`: Fired for additional information items detected by the validator.
-- `metadata(key, value)`: Fired for every piece of document metadata found by the validator.
-- `exception(message)`: Fired when there is a system error, such as a _File not found_ error. `message`
-  contains details about this error. All exceptions are displayed on the error console in addition to
+When using the JS API, the Specberus instance will fire various events.
+The `Specberus` class extends [`EventEmitter`](https://nodejs.org/dist/latest/docs/api/events.html#class-eventemitter),
+so listeners can be registered using the `on` API.
+
+```js
+const specberus = new Specberus();
+specberus.on('error', (rule, data) => {
+    // ...
+});
+```
+
+Events listed below are expressed with parameters to reflect what is passed to the event handler.
+
+- `done(ruleName)`: Fired when a specific rule has finished processing, including its asynchronous
+  tasks. This fires regardless of whether the rule passes, fails, or encounters an unexpected
+  system error (exception).
+- `err(rule, data)`: Fired when an error is detected. There can be multiple errors for each rule.
+    - `rule` contains information on which rule failed validation;
+      always includes a `name` string, and may also include `rule` and `section` strings
+    - `data` contains further details; always includes `key` and `detailMessage` strings,
+      and optionally includes an `extra` object with additional fields that vary by error
+- `warning(rule, data)`: Fired for non-fatal problems with the document that may
+  nevertheless require investigation. There can be multiple warnings for each rule.
+  `rule` and `data` follow the same format as for `err` events.
+- `info(rule, data)`: Fired for additional information items detected by the validator.
+  `rule` and `data` follow the same format as for `err` and `warning` events.
+- `exception({ message })`: Fired when there is an unexpected system error, such as a
+  _File not found_ error. The event passes an object with a `message` property containing
+  details about this error. All exceptions are displayed on the error console in addition to
   this event being fired.
 
 ## 8. Writing rules
 
-Rules are simple modules that just expose a `check(sr, cb)` method. They receive a Specberus object
-and a callback, use the Specberus object to fire validation events and call the callback when
-they're done.
+Rules are simple modules that expose a `check(context)` method. They receive a context object,
+which they use to examine the document and fire validation events. They return a promise which
+resolves on completion (regardless of pass or fail) or rejects on unexpected system error
+(exception). Usually, they are written as `async` functions to automatically handle the
+resolve vs. reject distinction.
+
+The context object includes the following APIs useful for validation:
 
-The Specberus object exposes the following API that's useful for validation:
+### Properties
 
 - `source`. The HTML source of the document being processed
 - `url`. The URL of the document being processed, only applicable if `options.url` was specified
-- `error`, `warn`, `info`. Methods for firing respective levels of events to the instance's sink.
+- `version`. The Specberus version.
+
+### Methods
+
+- `error`, `warn`, `info`. Methods for firing respective levels of events on the instance.
   All three methods accept the same arguments:
     - `rule` object: at minimum, an object with a `name` string. May also contain `rule` and `section` strings.
     - `key` string: specifies the precise occurrence within the particular `rule`
     - `extra` object (optional): any additional fields to include within the event
-- `version`. The Specberus version.
-- `checkSelector(selector, rule-name, cb)`. Some rules need to do nothing other than to check that a
-  selector returns some content. For this case, the rule can just call this method with the selector
-  and its callback, and Specberus will conveniently take care of all the rest.
+- `checkSelector(selector, ruleName)`. Some rules need to do nothing other than to check that a
+  selector returns some content. This handles checking the selector, reporting an error if it is
+  not found, or throwing an error if the selector is invalid.
 - `norm(text)`. Returns a whitespace-normalized version of the text.
 - `getDocumentDate()`. Returns a Date object that matches the document's date as specified in the
   headers' `stateElement` (id="w3c-state").