diff --git a/CHANGELOG.md b/CHANGELOG.md
index 725a773d..c03812fd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -50,6 +50,8 @@
`429 Too Many Requests` for a full remote browser service, and exposes it as `#status`
### Fixed
+- `Node#property` and `Node#attribute` read any name, a quote in it no longer breaks the script, and `property`
+ returns a `Ferrum::Node` for a property holding one instead of `{}` [#673]
- `evaluate`, `execute` and friends raise `JavaScriptError` when JS throws something other than an `Error`, e.g.
`throw "smth"`, instead of returning it. Returning an `Error` object no longer crashes with `NoMethodError`
- `evaluate` keeps `null` and `undefined` in arrays and returns `NaN`, `Infinity`, `-0` and `BigInt` instead of `nil`,
@@ -117,6 +119,9 @@
loaded (on CDP's `Network.loadingFinished`, not `Network.responseReceived`, so `exchange.response.body` is always
available), yielding the request's `Network::Exchange`. Doesn't require `network.intercept` to be set up, and is
never fired for requests that fail to load [#294]
+- `#evaluate`/`#execute` take named arguments that become the script's parameters, `page.evaluate("a + b", a: 1,
+ b: 2)`, a function declaration as the script with positional arguments, `await`, and `timeout:` [#673]
+- `#evaluate_handle` returning a `Ferrum::RemoteObject`, and `Node#execute`/`Node#evaluate_handle` [#673]
- `Ferrum::Browser#quit`/`Ferrum::Browser::Process#stop` accept `wait: false` to return immediately and run process
killing and user-data-directory cleanup on a background thread instead of blocking; the call returns the `Thread`
so callers can `#join` it if they need cleanup to have finished, e.g. before process exit or before reusing a
@@ -132,6 +137,10 @@
- `Ferrum::Page::Stream#stream` now closes the CDP stream handle (`IO.close`) once it's been fully read, so streams
opened for `Ferrum::Browser#pdf` and `Ferrum::Page::Tracing#record` no longer keep their backing storage alive in
the browser; a failed `IO.close` is raised to the caller.
+- `#evaluate_async`, `#evaluate_func` and `#evaluate_on` are deprecated, and scripts whose result will change in
+ 0.21 warn with a rewrite. 0.21 removes the old behavior, and `FERRUM_JS_API=new` runs the new one now, see
+ [upgrading](docs/13-javascript.md#upgrading-to-021) [#673]
+- `Ferrum::ScriptTimeoutError` inherits from `Ferrum::TimeoutError` [#673]
### Fixed
- An out-of-process iframe, e.g. Chrome's PDF viewer for an embedded PDF, appearing while a web worker was connected
diff --git a/README.md b/README.md
index 12358157..6fd5adb3 100644
--- a/README.md
+++ b/README.md
@@ -85,6 +85,55 @@ JS
browser.quit
```
+Pass arguments by name, so the script can use parameters instead of digging
+through `arguments[0]`:
+
+```ruby
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+```
+
+A script that is a function or arrow declaration is used as-is and takes its
+arguments positionally, in the order it declares them:
+
+```ruby
+page.evaluate(<<~JS, 1, 2)
+ function(a, b) {
+ const sum = a + b;
+ return sum * 3;
+ }
+JS
+```
+
+`await` works directly, and a function declaration's returned promise is
+awaited:
+
+```ruby
+page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+```
+
+Pass `timeout:` (seconds, defaulting to the page timeout) to bound how long the
+script may take before `Ferrum::ScriptTimeoutError` is raised. `timeout:` is the
+only reserved keyword.
+
+Use `#execute` when you only want the side effects, `#evaluate_handle` when you
+want to keep a value in the browser and pass it back in later, and
+`Ferrum::Node#evaluate` to run a script with `this` bound to an element:
+
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
+
+page.at_css("input").evaluate("this.value")
+```
+
+Scripts written for the old API keep working, and Ferrum warns, with a suggested
+rewrite, about any whose result will change. The next release, 0.21, removes the
+old behavior, so run your suite with `FERRUM_JS_API=new` to check it's ready. See
+[upgrading to 0.21](docs/13-javascript.md#upgrading-to-021).
+
Do any mouse movements you like:
```ruby
diff --git a/docs/13-javascript.md b/docs/13-javascript.md
index 1c8d028f..93fc1aaa 100644
--- a/docs/13-javascript.md
+++ b/docs/13-javascript.md
@@ -4,16 +4,29 @@ sidebar_position: 13
# JavaScript
-#### evaluate(expression, \*args)
+#### evaluate(expression, \*args, timeout: nil, \*\*named)
-Evaluate and return result for given JS expression
+Evaluate JavaScript and return the result, converted to Ruby.
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+* expression `String` JavaScript returning the value of its last statement, or a function/arrow declaration, which
+ is used as-is
+* args positional arguments, a declaration's parameters or `arguments[0]` and up. Can't be mixed with named ones
+* timeout `Numeric` seconds to wait for the script and any promise it returns, defaults to the page timeout. Raises
+ `Ferrum::ScriptTimeoutError` when exceeded, which inherits from `Ferrum::TimeoutError`
+* named named arguments, which become the parameters of a script that isn't a declaration. A `Ferrum::Node` arrives
+ as the live element, but can't be put inside an Array or Hash. `timeout` can't be a parameter name
```ruby
-page.evaluate("[window.scrollX, window.scrollY]")
+page.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+page.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
+page.evaluate(<<~JS, 3) # => 9
+ function(c) {
+ const sum = 1 + 2;
+ return sum * c;
+ }
+JS
```
The result is converted to Ruby: objects become `Hash`, arrays and node lists `Array`, DOM nodes `Ferrum::Node`,
@@ -21,31 +34,59 @@ dates ISO 8601 strings, `NaN` and `Infinity` `Float`, `BigInt` `Integer`, `null`
without a Ruby counterpart, such as functions, `Map`, `Set` or `window`, become `{}`. A reference back to an
enclosing object, e.g. `a.self = a`, becomes `Ferrum::CyclicObject`.
-#### evaluate_async(expression, wait_time, \*args)
-
-Evaluate asynchronous expression and return result
+#### evaluate_handle(expression, \*args, timeout: nil, \*\*named) : `Ferrum::RemoteObject`
-* expression `String` should be valid JavaScript
-* wait_time How long we should wait for Promise to resolve or reject
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+Same as `evaluate`, but keeps the value in the browser and returns a reference you can pass back in as an argument.
+Call `#value` to convert it to Ruby, and `#release` to free it.
```ruby
-page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
```
-#### execute(expression, \*args)
+#### execute(expression, \*args, timeout: nil, \*\*named) : `Boolean`
+
+Run JavaScript for its side effects. The script is used as a function body, so statements need no wrapping.
-Execute expression. Doesn't return the result
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+```
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+`Ferrum::Node` has the same three methods, with `this` bound to the element. Arrow functions don't get `this`, so use a
+`function` declaration when you need it.
```ruby
-page.execute(%(1 + 1)) # => true
+page.at_css("input").evaluate("this.value")
+page.at_css("input").execute("this.value = text", text: "hello")
```
+#### Upgrading to 0.21
+
+Scripts written for the old API run exactly as before, and Ferrum warns, pointing at your code, about any whose result
+will change. Calls that only the new API can make, e.g. with named arguments, already run the new way.
+
+* 0.20 runs old-style scripts the old way and warns. Set `FERRUM_JS_API=new` to run everything the new way.
+* 0.21 removes the old way, along with `evaluate_async`, `evaluate_func` and `evaluate_on`. There's no switch back.
+
+Warnings only cover the scripts and results your suite actually runs into, so before upgrading to 0.21, run the suite
+once with `FERRUM_JS_API=new`. Set `FERRUM_DEPRECATION_WARNINGS=raise` to turn the warnings into errors while
+migrating, or `0` to silence them. A library built on Ferrum should pass named arguments or a function declaration,
+which run the new way in both 0.20 and 0.21.
+
+| Script | < 0.21 | >= 0.21 | Notes |
+|---|---|---|---|
+| `evaluate("arguments[0].foo", foo: 1)` | `1` | `nil` | keywords are named parameters now, write `evaluate("foo", foo: 1)` |
+| `evaluate("[arguments[0], arguments[1]]", 1, foo: 2)` | `[1, { "foo" => 2 }]` | `ArgumentError` | positional and keyword arguments can't be mixed, pass `{ foo: 2 }` in braces |
+| `evaluate("a(); b")` | `a()`'s value, `b` never runs | `b`'s value | every statement runs, the last one's value is returned |
+| `evaluate("\n window.ran = true\n")` | `nil`, nothing runs | `true` | the line break no longer ends the `return`, check it's safe to run |
+| `evaluate("Promise.resolve(1)")` | `{}` | `1` | promises are awaited, `void` keeps one from being awaited |
+| `node.evaluate("this.parentNode")` | `{}` | `Ferrum::Node` | |
+| `node.evaluate("new Date(0)")` | `{}` | `"1970-01-01T00:00:00.000Z"` | |
+| `evaluate_async("arguments[0](1)", 5)` | `1` | `NoMethodError` | `evaluate("await …", timeout: 5)` |
+| `evaluate_func("function(a) { return a }", 1)` | `1` | `NoMethodError` | `evaluate("function(a) { return a }", 1)`, with `on:` use `node.evaluate` |
+| `evaluate_on(node: node, expression: "this.id")` | `"main"` | `NoMethodError` | `node.evaluate("this.id")` |
+| `evaluate_on(node: link, expression: "this.click()", wait: 0.1)` | waits for the new page | `NoMethodError` | `page.wait_for_navigation!(0.1) { link.evaluate("this.click()") }` |
+
#### evaluate_on_new_document(source, \*\*options) : `String`
Runs JavaScript in every new document of the page and its in-process frames, before the document's own scripts.
@@ -67,7 +108,7 @@ JS
#### isolated_world(name) : `IsolatedWorld`
Returns the frame's isolated world with this name, its own JavaScript globals over the frame's DOM, so the page's
-scripts can't see or override what runs there. It has `evaluate`, `evaluate_async`, `execute` and `evaluate_func`,
+scripts can't see or override what runs there. It has `evaluate`, `evaluate_handle` and `execute`,
takes and returns nodes, and comes back empty in every new document. On a page it's the main frame's world, use
`frame.isolated_world(name)` for an iframe. It's the same world `evaluate_on_new_document` runs in with
`world_name:`.
diff --git a/docs/21-workers.md b/docs/21-workers.md
index 7b8d3d10..32fc6e58 100644
--- a/docs/21-workers.md
+++ b/docs/21-workers.md
@@ -36,7 +36,7 @@ Returns service worker targets registered in this context. Unlike `workers`, the
are not connected to -- call `attach_target` first if you want to interact with one.
```ruby
-page.evaluate_async(%(navigator.serviceWorker.register("/sw.js").then(arguments[0])), 5)
+page.evaluate(%(await navigator.serviceWorker.register("/sw.js")))
sleep 0.1
@@ -91,7 +91,7 @@ Same API as `page.console` -- see [JavaScript](/docs/ferrum/javascript#console).
worker.console.map(&:text) # => ["from worker"]
```
-#### evaluate / execute / evaluate_async
+#### evaluate / execute / evaluate_handle
Same API as on `Page`/`Frame` -- see [JavaScript](/docs/ferrum/javascript). A worker has a single execution context,
so there's no frame to target.
diff --git a/lib/ferrum.rb b/lib/ferrum.rb
index bf1567af..db746feb 100644
--- a/lib/ferrum.rb
+++ b/lib/ferrum.rb
@@ -7,8 +7,10 @@
require "ferrum/utils/platform"
require "ferrum/utils/elapsed_time"
require "ferrum/utils/attempt"
+require "ferrum/utils/deprecate"
require "ferrum/errors"
require "ferrum/browser"
+require "ferrum/remote_object"
require "ferrum/node"
#
diff --git a/lib/ferrum/browser.rb b/lib/ferrum/browser.rb
index 5b50e758..2e56b0ea 100644
--- a/lib/ferrum/browser.rb
+++ b/lib/ferrum/browser.rb
@@ -37,7 +37,8 @@ class Browser
screenshot pdf mhtml viewport_size device_pixel_ratio
start_screencast stop_screencast
frames frame_by main_frame
- evaluate evaluate_on evaluate_async execute evaluate_func isolated_world
+ evaluate evaluate_handle execute
+ evaluate_on evaluate_async evaluate_func isolated_world
add_script_tag add_style_tag bypass_csp
evaluate_on_new_document remove_script_on_new_document
on position position=
diff --git a/lib/ferrum/errors.rb b/lib/ferrum/errors.rb
index 298db890..57b95cec 100644
--- a/lib/ferrum/errors.rb
+++ b/lib/ferrum/errors.rb
@@ -87,7 +87,7 @@ def awaited
end
# Raised when an evaluated script takes too long to return a value.
- class ScriptTimeoutError < Error
+ class ScriptTimeoutError < TimeoutError
#
# Explains that the evaluated script timed out.
#
diff --git a/lib/ferrum/frame.rb b/lib/ferrum/frame.rb
index c53c344d..affa06a1 100644
--- a/lib/ferrum/frame.rb
+++ b/lib/ferrum/frame.rb
@@ -167,12 +167,11 @@ def parent
# frame.body # =>
lol
#
def content=(html)
- evaluate_async(%(
+ execute(<<~JS, html: html)
document.open();
- document.write(arguments[0]);
+ document.write(html);
document.close();
- arguments[1](true);
- ), @page.timeout, html)
+ JS
@page.document_node_id
end
alias set_content content=
diff --git a/lib/ferrum/frame/dom.rb b/lib/ferrum/frame/dom.rb
index 3cf1492b..8ff8b0c2 100644
--- a/lib/ferrum/frame/dom.rb
+++ b/lib/ferrum/frame/dom.rb
@@ -27,32 +27,38 @@ class Frame
#
module DOM
SCRIPT_SRC_TAG = <<~JS
- const script = document.createElement("script");
- script.src = arguments[0];
- script.type = arguments[1];
- script.onload = arguments[2];
- document.head.appendChild(script);
+ function(url, type) {
+ const script = document.createElement("script");
+ script.src = url;
+ script.type = type;
+ document.head.appendChild(script);
+ return new Promise(resolve => script.onload = resolve);
+ }
JS
SCRIPT_TEXT_TAG = <<~JS
- const script = document.createElement("script");
- script.text = arguments[0];
- script.type = arguments[1];
- document.head.appendChild(script);
- arguments[2]();
+ function(content, type) {
+ const script = document.createElement("script");
+ script.text = content;
+ script.type = type;
+ document.head.appendChild(script);
+ }
JS
STYLE_TAG = <<~JS
- const style = document.createElement("style");
- style.type = "text/css";
- style.appendChild(document.createTextNode(arguments[0]));
- document.head.appendChild(style);
- arguments[1]();
+ function(content) {
+ const style = document.createElement("style");
+ style.type = "text/css";
+ style.appendChild(document.createTextNode(content));
+ document.head.appendChild(style);
+ }
JS
LINK_TAG = <<~JS
- const link = document.createElement("link");
- link.rel = "stylesheet";
- link.href = arguments[0];
- link.onload = arguments[1];
- document.head.appendChild(link);
+ function(url) {
+ const link = document.createElement("link");
+ link.rel = "stylesheet";
+ link.href = url;
+ document.head.appendChild(link);
+ return new Promise(resolve => link.onload = resolve);
+ }
JS
#
@@ -160,7 +166,7 @@ def xpath(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector, within)
end
#
@@ -187,7 +193,7 @@ def at_xpath(selector, within: nil)
return xpath.snapshotItem(0);
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector, within)
end
#
@@ -214,7 +220,7 @@ def css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector, within)
end
#
@@ -241,7 +247,7 @@ def at_css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector, within)
end
#
@@ -307,17 +313,17 @@ def wait_for_selector(css: nil, xpath: nil, within: nil, timeout: @page.timeout,
# browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
#
def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
- expr, *args = if url
- [SCRIPT_SRC_TAG, url, type]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [SCRIPT_TEXT_TAG, content, type]
- end
+ if url
+ evaluate(SCRIPT_SRC_TAG, url, type)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(SCRIPT_TEXT_TAG, content, type)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
#
@@ -333,17 +339,17 @@ def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
# browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
#
def add_style_tag(url: nil, path: nil, content: nil)
- expr, *args = if url
- [LINK_TAG, url]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [STYLE_TAG, content]
- end
+ if url
+ evaluate(LINK_TAG, url)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(STYLE_TAG, content)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
end
end
diff --git a/lib/ferrum/frame/runtime.rb b/lib/ferrum/frame/runtime.rb
index b2cce4df..794a7c6c 100644
--- a/lib/ferrum/frame/runtime.rb
+++ b/lib/ferrum/frame/runtime.rb
@@ -1,6 +1,8 @@
# frozen_string_literal: true
require "singleton"
+require "ferrum/frame/script"
+require "ferrum/frame/runtime/legacy"
module Ferrum
#
@@ -21,19 +23,145 @@ def inspect
class Frame
#
- # Evaluates and executes JavaScript in a frame's execution context via
+ # Runs JavaScript in a frame's execution context via
# `Runtime.callFunctionOn`, converting arguments and return values
# between Ruby and JS, and resolving object/array/node results into Ruby
# equivalents. A reference back to an enclosing object becomes
# {CyclicObject}.
#
+ # There are three entry points, all of which accept the script in the
+ # same two shapes and pass arguments the same way:
+ #
+ # * {#evaluate} returns the value, serialized into Ruby.
+ # * {#evaluate_handle} returns a {RemoteObject} that stays in the browser.
+ # * {#execute} discards the value and returns `true`.
+ #
+ # ## Script shapes
+ #
+ # A script is either code that gets wrapped in a function for you and
+ # returns the value of its last statement, or a function declaration,
+ # which is used as-is.
+ #
+ # page.evaluate("window.scrollY")
+ # page.evaluate("const a = 1\na + 1")
+ # page.evaluate("function() { const a = 1; return a + 1 }")
+ #
+ # Arrow functions keep the `this` of where they're defined, so on a
+ # {Node} use a `function` declaration when the script needs `this`.
+ #
+ # ## Arguments
+ #
+ # Keyword arguments become the function's parameters, so the script can
+ # name what it receives instead of digging through `arguments[0]`:
+ #
+ # page.evaluate("a + b", a: 1, b: 2)
+ #
+ # Each keyword is sent as its own protocol argument, so a {Node} arrives
+ # in JavaScript as the live element rather than as serialized JSON.
+ #
+ # A function declaration names its parameters itself, so it takes
+ # positional arguments, in the order it declares them:
+ #
+ # page.evaluate("function(a, b) { return a + b }", 1, 2)
+ #
+ # Positional arguments also work with an expression, landing in
+ # `arguments[0]` and up, but can't be mixed with keyword ones.
+ #
+ # `timeout:` is reserved for Ferrum, so a variable of the script can't
+ # take that name.
+ #
+ # ## Promises
+ #
+ # A returned promise, or any object with a `then` method, is awaited, so
+ # `async`/`await` works directly and there is no separate asynchronous
+ # method:
+ #
+ # page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+ #
+ # ## Timeouts
+ #
+ # If the script hasn't settled within `timeout:` seconds (defaulting to
+ # the page's timeout) a {ScriptTimeoutError} is raised.
+ #
+ # The timeout is how long Ruby waits for Chrome's answer, so a page that
+ # replaces `Promise` or `setTimeout` can't defeat it, but it doesn't stop
+ # the script. A promise that never settles is harmless, while a
+ # synchronous endless loop keeps the page's JavaScript busy, so later
+ # calls on that page time out too. Other pages aren't affected.
+ #
+ # ## How a call runs
+ #
+ # All three methods, and their {Node} counterparts through
+ # {#evaluate_in}, take the same steps and differ only in a mode:
+ # `:value`, `:handle` or `:none`.
+ #
+ # 1. **Parse.** {Script} reads the source once, skipping strings,
+ # template literals, regular expressions and comments. It finds the
+ # code without surrounding comments and semicolons, whether it's a
+ # function or arrow declaration, where its last statement starts, and
+ # whether it uses `await`.
+ # 2. **Arguments.** `timeout:` is taken out of the keywords.
+ # Positional arguments are passed as they are. Named ones become the
+ # parameters of the function an expression is wrapped in, and raise
+ # for a declaration, which names its parameters itself.
+ # 3. **Wrap.** A declaration is sent as it is. An expression becomes
+ # `function() { return () }`, `async` when it uses
+ # `await`. Of several statements, only the last is returned, as in
+ # `function() { a(); return (b) }`. {#execute} uses the script as
+ # written as the function body instead.
+ # 4. **Call.** One `Runtime.callFunctionOn` in the frame's context, or
+ # with `this` bound to a node or remote object. When it returns a
+ # promise or a plain object, a second call awaits it, which isn't
+ # retried after a navigation, as the script has already run. {Node}
+ # and {RemoteObject} arguments are passed by reference.
+ # 5. **Result.** {#evaluate} converts Chrome's deep serialization into
+ # Ruby, {#evaluate_handle} wraps an object in a {RemoteObject}, and
+ # {#execute} returns `true`. A thrown value raises {JavaScriptError}.
+ #
+ # For example, this call:
+ #
+ # page.evaluate("await fetch(url).then(r => r.status)", url: "/ping")
+ #
+ # is an expression that uses `await`, so it's sent as:
+ #
+ # async function(url) {
+ # return (
+ # await fetch(url).then(r => r.status)
+ # );
+ # }
+ #
+ # with the arguments `[{ value: "/ping" }]`. Chrome answers with a
+ # promise, a second call awaits it and gets
+ # `{ type: "number", value: 200 }`, which comes back as
+ # `200`. A declaration takes positional arguments instead:
+ #
+ # node.evaluate("function(prefix, suffix) { return prefix + this.id + suffix }", "#", "!")
+ #
+ # is sent unchanged, with `this` bound to the node and the arguments
+ # `["#", "!"]`, and comes back as `"#main!"`.
+ #
+ # ## Scripts written for the old API
+ #
+ # Until Ferrum 0.21, a call that only the old API could have made runs
+ # the old way through {Legacy} and warns when its result will change.
+ # That's a call without named arguments, or one whose script reads
+ # `arguments` or also passes positional ones, unless the script is a
+ # declaration or uses `await`, or the method is {#evaluate_handle} or
+ # {Node#execute}.
+ # `FERRUM_JS_API=new` sends every call through the steps above.
+ #
module Runtime
+ include Legacy
+
INTERMITTENT_ATTEMPTS = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
INTERMITTENT_SLEEP = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
DEEP_SERIALIZATION = {
serialization: "deep",
additionalParameters: { maxNodeDepth: 0, includeShadowTree: "none" }
}.freeze
+ # Describes a primitive or node result, which a handle can't stand for,
+ # without serializing an object that stays in the browser.
+ SHALLOW_SERIALIZATION = DEEP_SERIALIZATION.merge(maxDepth: 0).freeze
UNSERIALIZABLE_NUMBERS = {
"NaN" => Float::NAN,
"-0" => -0.0,
@@ -42,170 +170,286 @@ module Runtime
}.freeze
#
- # Evaluate and return result for given JS expression.
+ # Evaluates JavaScript and returns the result, serialized into Ruby.
#
# @param [String] expression
- # The JavaScript to evaluate.
+ # A JavaScript expression, or a function declaration.
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @param [Array] positional
+ # Arguments reachable as `arguments[0]` and up. Can't be combined
+ # with named ones.
+ #
+ # @param [Hash] keywords
+ # Named arguments, which become the function's parameters, plus:
+ #
+ # @option keywords [Numeric, nil] :timeout
+ # How long to wait for the script, and any promise it returns, to
+ # settle, in seconds. Defaults to the page timeout.
+ #
+ # @return [Object]
+ # The result. DOM nodes come back as {Node}, arrays and plain objects
+ # are converted recursively, and cyclic values become {CyclicObject}.
+ #
+ # @raise [ScriptTimeoutError]
+ # The script didn't settle within `timeout`.
+ #
+ # @raise [JavaScriptError]
+ # The script threw.
#
# @example
- # browser.evaluate("[window.scrollX, window.scrollY]")
+ # browser.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+ # browser.evaluate("a + b", a: 1, b: 2) # => 3
+ # browser.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
#
- def evaluate(expression, *args)
- expression = format("function() { return %s }", expression)
- call(expression: expression, arguments: args)
+ def evaluate(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :value)
end
#
- # Evaluate asynchronous expression and return result.
+ # Same as {#evaluate}, but returns a {RemoteObject} that keeps the value
+ # in the browser instead of serializing it. Handles can be passed back
+ # in as arguments. Primitives are returned as-is, and DOM nodes as
+ # {Node}, since both are already usable from Ruby.
#
- # @param [String] expression
- # The JavaScript to evaluate.
- #
- # @param [Integer] wait
- # How long we should wait for Promise to resolve or reject.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [RemoteObject, Node, Object]
#
# @example
- # browser.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+ # list = page.evaluate_handle("document.querySelectorAll('li')")
+ # page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
#
- def evaluate_async(expression, wait, *args)
- template = <<~JS
- function() {
- return new Promise((__f, __r) => {
- try {
- arguments[arguments.length] = r => __f(r);
- arguments.length = arguments.length + 1;
- setTimeout(() => __r(new Error("timed out promise")), %s);
- %s
- } catch(error) {
- __r(error);
- }
- });
- }
- JS
-
- expression = format(template, wait * 1000, expression)
- call(expression: expression, arguments: args, awaitPromise: true)
+ def evaluate_handle(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :handle)
end
#
- # Execute expression. Doesn't return the result.
+ # Runs JavaScript for its side effects and discards the result. Unlike
+ # {#evaluate}, the script is used as a function body rather than an
+ # expression, so multiple statements need no wrapping.
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [Boolean]
+ # Always `true`.
#
# @example
- # browser.execute(%(1 + 1)) # => true
+ # browser.execute("window.scrollBy(0, 100)") # => true
+ # browser.execute(<<~JS, url: "/next")
+ # history.pushState({}, "", url);
+ # window.dispatchEvent(new Event("popstate"));
+ # JS
#
- def execute(expression, *args)
- expression = format("function() { %s }", expression)
- call(expression: expression, arguments: args, handle: false, returnByValue: true)
+ def execute(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :none)
true
end
#
- # Evaluates a raw JS function declaration (unlike {#evaluate}, which
- # wraps the given expression in one), optionally on a specific remote
- # object instead of the frame's global execution context.
+ # @api private
#
- # @param [String] expression
- # A JS function declaration, e.g. `"function(a, b) { return a + b }"`.
+ # Backs {Node#evaluate}, {Node#evaluate_handle}, {Node#execute} and
+ # {RemoteObject#value}. Runs the script with `this` bound to `object`,
+ # using the same argument and script conventions as {#evaluate}.
#
- # @param [Array] args
- # Arguments to pass to the function.
+ # @param [Node, RemoteObject] object
+ # The node or object to bind `this` to.
#
- # @param [Node, nil] on
- # Remote object to invoke the function on.
+ # @param [Symbol] mode
+ # `:value`, `:handle` or `:none`.
#
- def evaluate_func(expression, *args, on: nil)
- call(expression: expression, arguments: args, on: on)
+ # @return [Object]
+ #
+ def evaluate_in(object, expression, positional, keywords, mode: :value)
+ run(expression, positional, keywords, mode: mode, on: object)
end
+ private
+
#
- # Evaluates an expression against a given node's remote object (+this+
- # refers to the node), returning the raw JS value rather than
- # resolving it to a {Node}/Hash/Array.
+ # Parses the script, turns the arguments into a parameter list and
+ # values, wraps the script into the function declaration that's sent,
+ # and hands it to {#call}. `mode` decides both how the result is
+ # converted and whether the script's value is kept.
#
- # @param [Node] node
- # The node to evaluate the expression on.
+ def run(expression, positional, keywords, mode:, on: nil)
+ script = Script.new(expression)
+ if legacy?(script, positional, keywords, mode: mode, on: on)
+ return run_legacy(script, positional, keywords, mode: mode, on: on)
+ end
+
+ named = keywords.except(:timeout)
+ params, values = arguments_for(script, positional, named)
+ call(wrap(script, params, mode), values, on: on, mode: mode, timeout: script_timeout(keywords[:timeout]))
+ end
+
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # Normalizes the two argument styles into the parameter names of the
+ # wrapping function and the values to send:
#
- # @param [Boolean] by_value
- # Whether to return the plain JS value instead of a handle.
+ # * positional ones: no names, values reachable as `arguments[0]` and up,
+ # or as a declaration's own parameters;
+ # * named ones for an expression: their names and their values. A
+ # declaration names its parameters itself, so it takes positional
+ # ones only.
#
- # @param [Integer] wait
- # Passed through to the underlying `Runtime.callFunctionOn` command.
+ def arguments_for(script, positional, named)
+ if positional.any?
+ raise ArgumentError, "Pass arguments either positionally or by name, not both" if named.any?
+
+ return [[], positional]
+ end
+
+ return [[], []] if named.empty?
+ if script.declaration?
+ raise ArgumentError, "A function declaration names its own parameters, so pass its arguments " \
+ "positionally: evaluate(\"function(a, b) { … }\", 1, 2)"
+ end
+
+ [named.keys.map(&:to_s), named.values]
+ end
+
+ #
+ # Builds the function declaration that is actually sent to the browser:
+ # the caller's declaration as-is, or an expression wrapped in one.
#
- def evaluate_on(node:, expression:, by_value: true, wait: 0)
- options = { handle: true }
- expression = format("function() { return %s }", expression)
- options = { handle: false, returnByValue: true } if by_value
- call(expression: expression, on: node, wait: wait, **options)
+ # wrap(Script.new("a + b"), %w[a b], :value)
+ # # => "function(a, b) { return (a + b\n);\n}"
+ # wrap(Script.new("await x()"), [], :none)
+ # # => "async function() { await x()\n}"
+ #
+ def wrap(script, params, mode)
+ return script.code if script.declaration?
+
+ # A body keeps its trailing semicolons, which can be statements of
+ # their own, as in `while (busy());`. Otherwise the last statement is
+ # returned, parenthesized so that automatic semicolon insertion can't
+ # end the `return`. The script starts on the first line, so that error
+ # locations match its lines.
+ head, last = script.split_last_statement
+ body = if mode == :none then script.source
+ elsif last.empty? then ""
+ elsif last.match?(Script::STATEMENT) then script.code
+ else "#{head}return (#{last}\n);"
+ end
+ "#{'async ' if script.awaits?}function(#{params.join(', ')}) { #{body}\n}"
end
- private
+ # Waiting for the response is what bounds the script, so a page that
+ # replaces `Promise` or `setTimeout`, or throttles timers, can't defeat
+ # it.
+ def script_timeout(timeout)
+ (timeout || @page.timeout).to_f
+ end
- def call(expression:, arguments: [], on: nil, wait: 0, handle: true, **options)
+ #
+ # Sends the declaration with `Runtime.callFunctionOn`, retrying while
+ # the frame's context is being replaced unless `this` is bound to an
+ # existing node, and converts the result.
+ #
+ # @param [Symbol] mode
+ # `:value` resolves the result into Ruby, `:handle` returns a
+ # {RemoteObject}, and `:none` discards it.
+ #
+ def call(declaration, values, on:, mode:, timeout:)
# do not rescue -> retry if we operate on an existing node
errors = on ? [] : [NodeNotFoundError, NoExecutionContextError]
- Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
- params = options.dup
+ serialization = mode == :value ? DEEP_SERIALIZATION : SHALLOW_SERIALIZATION
+ response = Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
+ call_function(declaration, values, call_target(on), serialization, timeout)
+ end
+ handle_error(response, declaration)
- params = params.merge(objectId: resolve_node(on)) if on
- params = params.merge(serializationOptions: DEEP_SERIALIZATION) if handle
+ result = response["result"]
+ result = await_promise(result, serialization, timeout) if thenable?(result)
+ case mode
+ when :none then nil
+ when :handle then handle_remote_object(result)
+ else handle_response(result)
+ end
+ end
- if params[:executionContextId].nil? && params[:objectId].nil?
- params = params.merge(executionContextId: execution_id!)
- end
+ def call_function(declaration, values, target, serialization, timeout)
+ @page.command("Runtime.callFunctionOn",
+ timeout: timeout, slowmoable: true, serializationOptions: serialization,
+ functionDeclaration: declaration, arguments: prepare_args(values), **target)
+ rescue TimeoutError
+ raise ScriptTimeoutError
+ end
- response = @page.command("Runtime.callFunctionOn",
- wait: wait, slowmoable: true,
- **params.merge(functionDeclaration: expression,
- arguments: prepare_args(arguments)))
- handle_error(response)
- response = response["result"]
+ # A promise, or a plain object that may be one, such as the promises of
+ # zone.js or Bluebird, which Chrome doesn't report as `promise`.
+ def thenable?(result)
+ result["type"] == "object" && [nil, "promise"].include?(result["subtype"])
+ end
- handle ? handle_response(response) : response["value"]
+ # Awaited outside the retries: by now the script has run, and running
+ # it again after a navigation would repeat what it did. Awaiting an
+ # object that isn't thenable returns it as it is.
+ def await_promise(promise, serialization, timeout)
+ response = @page.command("Runtime.callFunctionOn",
+ timeout: timeout, awaitPromise: true, serializationOptions: serialization,
+ functionDeclaration: "function() { return this }", objectId: promise["objectId"])
+ handle_error(response)
+ response["result"]
+ rescue TimeoutError
+ raise ScriptTimeoutError
+ end
+
+ def call_target(on)
+ if on.is_a?(RemoteObject)
+ { objectId: on.remote_id }
+ elsif on
+ { objectId: resolve_node(on) }
+ else
+ { executionContextId: execution_id! }
end
end
- # FIXME: We should have a central place to handle all type of errors
- def handle_error(response)
+ # Chrome reports no location for a script that doesn't compile, so the
+ # function it compiled is added, wrapper included.
+ def handle_error(response, declaration = nil)
details = response["exceptionDetails"]
return unless details
- case details.dig("exception", "description")
- when /\AError: timed out promise/
- raise ScriptTimeoutError
- else
- raise JavaScriptError, details
+ if declaration && details["stackTrace"].nil? && details.dig("exception", "className") == "SyntaxError"
+ description = "#{details.dig('exception', 'description')}\n in: #{declaration[0, 500]}"
+ details = details.merge("exception" => details["exception"].merge("description" => description))
end
+ raise JavaScriptError, details
+ end
+
+ def handle_remote_object(result)
+ return RemoteObject.new(self, @page, result) if result["objectId"] && result["subtype"] != "node"
+
+ handle_response(result)
end
def handle_response(response)
deserialize(response["deepSerializedValue"])
end
- def deserialize(serialized, references = {}, path = [])
+ # Chrome sends a shared object in full once and refers to it after
+ # that, so each one is converted once and reused, as the same object.
+ def deserialize(serialized, references = {}, path = [], converted = {})
reference = serialized["weakLocalObjectReference"]
if reference
return cyclic_object if path.include?(reference)
+ return converted[reference] if converted.key?(reference)
serialized = references[reference] ||= serialized
path += [reference]
end
+ result = convert(serialized, references, path, converted)
+ converted[reference] = result if reference
+ result
+ end
+
+ def convert(serialized, references, path, converted)
value = serialized["value"]
case serialized["type"]
when "undefined", "null", "symbol"
@@ -217,9 +461,9 @@ def deserialize(serialized, references = {}, path = [])
when "bigint"
Integer(value)
when "array", "nodelist", "htmlcollection"
- value.map { |v| deserialize(v, references, path) }
+ value.map { |v| deserialize(v, references, path, converted) }
when "object"
- value.to_h { |k, v| [k, deserialize(v, references, path)] }
+ value.to_h { |k, v| [k, deserialize(v, references, path, converted)] }
when "node"
Node.new(self, value)
else
@@ -237,16 +481,35 @@ def resolve_node_options
def prepare_args(args)
args.map do |arg|
- if arg.is_a?(Node)
+ if nested_reference?(arg)
+ raise ArgumentError, "A Ferrum::Node or Ferrum::RemoteObject inside an Array or Hash can't be passed " \
+ "to JavaScript. Pass each one as its own argument"
+ end
+
+ case arg
+ when Node
{ objectId: resolve_node(arg) }
- elsif arg.is_a?(Hash) && arg["objectId"]
- { objectId: arg["objectId"] }
+ when RemoteObject
+ { objectId: arg.remote_id }
+ when Hash
+ arg["objectId"] ? { objectId: arg["objectId"] } : { value: arg }
else
{ value: arg }
end
end
end
+ # A protocol argument is either a value or a reference, so a node in
+ # an array or hash would only arrive as its `to_s`.
+ def nested_reference?(value)
+ children = case value
+ when Array then value
+ when Hash then value.values
+ else return false
+ end
+ children.any? { |v| v.is_a?(Node) || v.is_a?(RemoteObject) || nested_reference?(v) }
+ end
+
def cyclic_object
CyclicObject.instance
end
diff --git a/lib/ferrum/frame/runtime/legacy.rb b/lib/ferrum/frame/runtime/legacy.rb
new file mode 100644
index 00000000..1f33eaf8
--- /dev/null
+++ b/lib/ferrum/frame/runtime/legacy.rb
@@ -0,0 +1,336 @@
+# frozen_string_literal: true
+
+module Ferrum
+ class Frame
+ module Runtime
+ #
+ # The JavaScript evaluation API as it was before named arguments, kept
+ # so that scripts written for it return exactly what they used to. A
+ # call takes this path unless `FERRUM_JS_API=new` is set or the call
+ # uses something only the new API has: named arguments, a function
+ # declaration, `await`, or one of the new methods.
+ #
+ # Whatever runs here is checked for what Ferrum 0.21 will do
+ # differently, which is warned about with a suggested rewrite. 0.21
+ # removes this module, leaving only the new way.
+ #
+ # The script is parsed by {Script} as for the new way, to route the
+ # call and to spot what changes. An old-style call then runs as it did
+ # in Ferrum 0.19:
+ #
+ # 1. **Arguments.** Keyword arguments, `timeout:` included, are
+ # appended to the positional ones as one trailing object.
+ # 2. **Wrap.** The source is wrapped as it is, in
+ # `function() { return }`, or `function() { }` for
+ # {#execute}. A line break after `return` ends that statement, and
+ # only the first statement of `"a(); b"` runs.
+ # 3. **Call.** One `Runtime.callFunctionOn` that doesn't await
+ # promises, bounded by the page timeout, which raises a plain
+ # {TimeoutError}. {#evaluate_async} races the promise against a
+ # timer in the page instead.
+ # 4. **Check.** Before the result is converted, it's compared with what
+ # 0.21 will do: its subtype for promises, nodes and dates, its
+ # properties for nodes in an array, and the parsed script for
+ # statements separated by a semicolon. Nothing is
+ # run on the result, so getters and proxy traps don't fire.
+ # 5. **Warn.** Each difference warns once per call site, naming the
+ # first caller outside installed gems, then the result is converted
+ # as before: a promise comes back as `{}`.
+ #
+ module Legacy
+ # Marker rejected browser-side by {#evaluate_async} when it times out.
+ TIMEOUT_MARKER = "timed out promise"
+
+ NEXT_VERSION = "Ferrum 0.21"
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {#evaluate}, which awaits
+ # promises: `page.evaluate("await thing()", timeout: wait)`.
+ #
+ # Evaluates an asynchronous expression, appending a resolve callback as
+ # the last entry of `arguments`.
+ #
+ # @param [String] expression
+ # The JavaScript to evaluate.
+ #
+ # @param [Integer] wait
+ # How long to wait for the promise to settle, in seconds.
+ #
+ # @param [Array] args
+ # Additional arguments, reachable as `arguments[0]` and up. The
+ # resolve callback follows them.
+ #
+ # @return [Object]
+ #
+ def evaluate_async(expression, wait, *args)
+ Utils::Deprecate.warn(
+ "#{self.class}#evaluate_async will be removed in Ferrum 0.21. Use #evaluate, which waits for a " \
+ "returned promise, instead of calling the last of `arguments`: " \
+ "evaluate(\"await new Promise(resolve => setTimeout(resolve, 100))\", timeout: #{wait})."
+ )
+
+ template = <<~JS
+ function() {
+ return new Promise((__f, __r) => {
+ try {
+ arguments[arguments.length] = r => __f(r);
+ arguments.length = arguments.length + 1;
+ setTimeout(() => __r(new Error("#{TIMEOUT_MARKER}")), %s);
+ %s
+ } catch(error) {
+ __r(error);
+ }
+ });
+ }
+ JS
+
+ expression = format(template, wait * 1000, expression)
+ legacy_call(expression: expression, arguments: args, awaitPromise: true)
+ end
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {#evaluate}, which takes a
+ # function declaration and named arguments, or {Node#evaluate}.
+ #
+ # Evaluates a raw JS function declaration, optionally on a specific
+ # remote object instead of the frame's global execution context.
+ #
+ # @param [String] expression
+ # A JS function declaration, e.g. `"function(a, b) { return a + b }"`.
+ #
+ # @param [Array] args
+ # Arguments to pass to the function.
+ #
+ # @param [Node, nil] on
+ # Remote object to invoke the function on.
+ #
+ # @return [Object]
+ #
+ def evaluate_func(expression, *args, on: nil)
+ Utils::Deprecate.warn(
+ "#{self.class}#evaluate_func will be removed in Ferrum 0.21. Use #evaluate, which takes a function " \
+ "declaration: evaluate(\"function(a, b) { return a + b }\", 1, 2). " \
+ "For on:, use node.evaluate."
+ )
+
+ legacy_call(expression: expression, arguments: args, on: on)
+ end
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {Node#evaluate}.
+ #
+ # Evaluates an expression against a given node's remote object (+this+
+ # refers to the node).
+ #
+ # @param [Node] node
+ # The node to evaluate the expression on.
+ #
+ # @param [String] expression
+ # The JavaScript to evaluate.
+ #
+ # @param [Boolean] by_value
+ # Whether to return the plain JS value instead of resolving it.
+ #
+ # @param [Integer] wait
+ # Passed through to the underlying `Runtime.callFunctionOn` command.
+ #
+ # @return [Object]
+ #
+ def evaluate_on(node:, expression:, by_value: true, wait: 0)
+ replacement = 'node.evaluate("this.value")'
+ replacement = "page.wait_for_navigation!(#{wait}) { #{replacement} }" if wait.positive?
+ Utils::Deprecate.warn("#{self.class}#evaluate_on will be removed in Ferrum 0.21. Use #{replacement} instead.")
+
+ options = { handle: true }
+ expression = format("function() { return %s }", expression)
+ options = { handle: false, returnByValue: true } if by_value
+ legacy_call(expression: expression, on: node, wait: wait, **options)
+ end
+
+ private
+
+ # Calls that only the new API could have made go the new way.
+ def legacy?(script, positional, keywords, mode:, on:)
+ return false if ENV.fetch("FERRUM_JS_API", nil) == "new"
+ return false if mode == :handle || (on && (mode == :none || !on.is_a?(Node)))
+ return false if script.declaration? || script.awaits?
+
+ keywords.empty? || positional.any? || script.uses?("arguments")
+ end
+
+ # Keywords used to be collected into a trailing Hash, `timeout:`
+ # included, so they're passed on exactly that way.
+ def run_legacy(script, positional, keywords, mode:, on:)
+ warn_about_keywords(script, positional, keywords)
+ arguments = keywords.empty? ? positional : [*positional, keywords]
+
+ if mode == :none
+ legacy_execute(script, arguments)
+ elsif on
+ legacy_evaluate_on(script, arguments, on)
+ else
+ legacy_evaluate(script, arguments)
+ end
+ end
+
+ def legacy_evaluate(script, arguments)
+ warn_about_line_break(script)
+ response = legacy_call(expression: format("function() { return %s }", script.source),
+ arguments: arguments, handle: :raw, serializationOptions: DEEP_SERIALIZATION)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :value)
+ warn_about_statements(script)
+ handle_response(response["result"])
+ end
+
+ def legacy_evaluate_on(script, arguments, node)
+ warn_about_line_break(script)
+ response = legacy_call(expression: format("function() { return %s }", script.source),
+ arguments: arguments, on: node, handle: :raw)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :node)
+ warn_about_statements(script)
+ serialize(response["result"])
+ end
+
+ def legacy_execute(script, arguments)
+ response = legacy_call(expression: format("function() { %s }", script.source),
+ arguments: arguments, handle: :raw)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :none)
+ release(response["result"])
+ true
+ end
+
+ # What `returnByValue` used to give, fetched afterwards so that the
+ # result could be inspected first.
+ def serialize(result)
+ object_id = result["objectId"]
+ return result["value"] unless object_id
+
+ response = @page.command("Runtime.callFunctionOn", objectId: object_id, returnByValue: true,
+ functionDeclaration: "function() { return this }")
+ response.dig("result", "value")
+ ensure
+ release(result)
+ end
+
+ def release(result)
+ object_id = result&.dig("objectId")
+ @page.command("Runtime.releaseObject", objectId: object_id) if object_id
+ rescue BrowserError
+ nil
+ end
+
+ def legacy_call(expression:, arguments: [], on: nil, wait: 0, handle: true, **options)
+ # do not rescue -> retry if we operate on an existing node
+ errors = on ? [] : [NodeNotFoundError, NoExecutionContextError]
+
+ Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
+ params = options.dup
+
+ params = params.merge(objectId: resolve_node(on)) if on
+ params = params.merge(serializationOptions: DEEP_SERIALIZATION) if handle == true
+
+ if params[:executionContextId].nil? && params[:objectId].nil?
+ params = params.merge(executionContextId: execution_id!)
+ end
+
+ response = @page.command("Runtime.callFunctionOn",
+ wait: wait, slowmoable: true,
+ **params.merge(functionDeclaration: expression,
+ arguments: prepare_args(arguments)))
+ next response if handle == :raw
+
+ legacy_handle_error(response)
+ response = response["result"]
+
+ handle ? handle_response(response) : response["value"]
+ end
+ end
+
+ def legacy_handle_error(response)
+ details = response["exceptionDetails"]
+ return unless details
+
+ description = details.dig("exception", "description").to_s
+ raise ScriptTimeoutError if description.start_with?("Error: #{TIMEOUT_MARKER}")
+
+ raise JavaScriptError, details
+ end
+
+ def warn_about_keywords(script, positional, keywords)
+ return if keywords.empty?
+
+ if positional.any?
+ warn_about(script, "Mixing positional and keyword arguments will raise ArgumentError in #{NEXT_VERSION}, " \
+ "where keywords will become named parameters instead of a trailing object. Pass the " \
+ "object in braces, evaluate(\"…\", 1, { foo: 1 }), or name every argument.")
+ else
+ warn_about(script, "Keyword arguments will become named parameters in #{NEXT_VERSION} instead of " \
+ "arriving as one object in `arguments[0]`. Name the object instead: " \
+ "evaluate(\"options.foo\", options: { foo: 1 }).")
+ end
+ end
+
+ def warn_about_line_break(script)
+ return unless script.leading_line_break?
+
+ warn_about(script, "This script never runs: the line break before it ends the `return` it's placed " \
+ "after, so the result is always nil. #{NEXT_VERSION} will run it and return its value. " \
+ "Check it's safe to run, or remove it.")
+ end
+
+ def warn_about_statements(script)
+ return if script.leading_line_break? || !script.several_statements?
+
+ warn_about(script, "This script is more than one statement, and today only the first one runs. " \
+ "#{NEXT_VERSION} will run them all and return the value of the last one. Check the " \
+ "rest is safe to run, or remove it.")
+ end
+
+ def warn_about_result(script, response, kind)
+ result = response["result"]
+ case result["subtype"]
+ when "promise" then warn_about_promise(script, kind)
+ when "node" then warn_about_node(script, "a DOM node", "Ferrum::Node") if kind == :node
+ when "date" then warn_about_node(script, "a Date", "its ISO 8601 string") if kind == :node
+ when "array" then warn_about_nodes(script, result) if kind == :node
+ end
+ end
+
+ def warn_about_promise(script, kind)
+ if kind == :none
+ warn_about(script, "This script returns a promise, which #execute will wait for in #{NEXT_VERSION}. " \
+ "Drop the `return`, or use `void`, to keep it from waiting.")
+ else
+ warn_about(script, "This script returns a promise, which comes back as {} today. #{NEXT_VERSION} " \
+ "will wait for it and return its value, or raise if it rejects. Prefix the " \
+ "expression with `void` to keep it from waiting.")
+ end
+ end
+
+ def warn_about_node(script, value, replacement)
+ warn_about(script, "Node#evaluate returns #{value} as {} today, and will return #{replacement} in " \
+ "#{NEXT_VERSION}. Check the code that uses the result.")
+ end
+
+ def warn_about_nodes(script, result)
+ nodes = properties(result, ownProperties: true).any? { |p| p.dig("value", "subtype") == "node" }
+ warn_about_node(script, "DOM nodes in an array", "Ferrum::Node") if nodes
+ end
+
+ def properties(result, **options)
+ @page.command("Runtime.getProperties", objectId: result["objectId"], **options)["result"]
+ end
+
+ def warn_about(script, message)
+ excerpt = script.code.lines.first.to_s.strip
+ excerpt = "#{excerpt[0, 60]}…" if excerpt.length > 60 || script.code.lines.size > 1
+ Utils::Deprecate.warn("#{message} Set FERRUM_JS_API=new to run it the #{NEXT_VERSION} way now." \
+ "\n script: #{excerpt}", user_code: true)
+ end
+ end
+ end
+ end
+end
diff --git a/lib/ferrum/frame/script.rb b/lib/ferrum/frame/script.rb
new file mode 100644
index 00000000..99bc56d5
--- /dev/null
+++ b/lib/ferrum/frame/script.rb
@@ -0,0 +1,289 @@
+# frozen_string_literal: true
+
+require "strscan"
+
+module Ferrum
+ class Frame
+ #
+ # @api private
+ #
+ # A script passed to {Runtime#evaluate} and friends. Reads just enough
+ # JavaScript to tell a function declaration from an expression and to
+ # find where the code and its statements end, skipping strings, template
+ # literals, regular expressions and comments so that a `//` inside
+ # `"http://…"` isn't mistaken for one.
+ #
+ # The source is scanned once, in {#initialize}, keeping a stack of open
+ # brackets and template literals. On the way it records where the code
+ # starts and ends, where the first top-level `(…)` opens and closes,
+ # where the first top-level `{…}` closes, where the last statement
+ # starts, and whether `await` appears. Every question below is answered
+ # from those positions.
+ #
+ # @example
+ # script = Script.new("// sum\nasync function(a, b = 1) { return a + b };")
+ # script.code # => "async function(a, b = 1) { return a + b }"
+ # script.declaration? # => true
+ # script.awaits? # => false
+ #
+ class Script
+ # A JavaScript identifier, Unicode letters included.
+ IDENTIFIER = /[\p{L}\p{Nl}_$][\p{L}\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}$]*/
+
+ FUNCTION_DECLARATION = /\A(?:async\s+)?function[\s*(]/
+ SINGLE_PARAMETER_ARROW = /\A(?:async\s+)?#{IDENTIFIER}\s*=>/
+ ASYNC_PREFIX = /\A(?:async\s*)?\z/
+ ARROW = /\A\s*=>/
+
+ COMMENT = %r{//[^\n]*|/\*.*?(?:\*/|\z)}m
+ NOISE = /\s+|#{COMMENT}/
+ STRING = /"(?:\\.|[^"\\\n])*"?|'(?:\\.|[^'\\\n])*'?/m
+ TEMPLATE_CHUNK = /(?:\\.|[^`\\$]|\$(?!\{))*/m
+ REGEXP = %r{/(?:\\.|\[(?:\\.|[^\]\\\n])*\]|[^/\\\n\[])+/[A-Za-z]*}
+ WORD = /[\p{Alnum}_$]+/
+
+ # Characters ending a value, after which a `/` is a division.
+ VALUE_END = %r{[\p{Alnum}_$)\]\}"'`/]}
+
+ # Words after which a `/` starts a regular expression, not a division.
+ REGEXP_KEYWORDS = %w[return typeof instanceof in of new delete void throw case do else yield await].freeze
+
+ # Words that continue the statement on the line before them.
+ CONTINUING_WORDS = %w[in instanceof of else catch finally].freeze
+
+ # `x++` ends a value, and `++x` starts one.
+ INCREMENTS = %w[++ --].freeze
+
+ # Words whose condition in parentheses is followed by the statement they
+ # control, which may start on the next line.
+ CONTROL_WORDS = %w[if for while with].freeze
+
+ # Words after which a `{` opens an object, not a block.
+ OBJECT_KEYWORDS = (REGEXP_KEYWORDS - %w[do else]).freeze
+
+ # A statement that isn't an expression, so it has no value to return.
+ STATEMENT = /\A(?:const|let|var|if|for|while|do|switch|try|return|throw|break|continue)\b/
+
+ # @return [String] the script as given.
+ attr_reader :source
+
+ # @return [String] the script without surrounding whitespace, comments
+ # and trailing semicolons.
+ attr_reader :code
+
+ #
+ # @param [String] source
+ #
+ def initialize(source)
+ @source = source.to_s
+ @code_start = @code_end = 0
+ @words = Set.new
+ @scanned = @source.valid_encoding? && (@source.encoding == Encoding::UTF_8 || @source.ascii_only?)
+ @scanned ? scan : @code_end = @source.bytesize
+ @code = @source.byteslice(@code_start...@code_end)
+ @wrapped = Script.new(@code[1...-1]) if parenthesized?
+ end
+
+ #
+ # Whether the script is a complete function or arrow declaration,
+ # rather than an expression that happens to begin with one, such as the
+ # immediately invoked `function() { … }()`.
+ #
+ # @return [Boolean]
+ #
+ def declaration?
+ return false unless @scanned
+ return @wrapped.declaration? if @wrapped
+
+ arrow? || (@code.match?(FUNCTION_DECLARATION) && @body_end == @code_end)
+ end
+
+ #
+ # Whether the script uses `await`, and so has to run in an `async`
+ # function.
+ #
+ # @return [Boolean]
+ #
+ def awaits?
+ uses?("await")
+ end
+
+ #
+ # Whether the script mentions the given word as code, rather than as a
+ # property name or inside a string or comment.
+ #
+ # @param [String] word
+ #
+ # @return [Boolean]
+ #
+ def uses?(word)
+ @words.include?(word)
+ end
+
+ #
+ # Whether the code is several statements, separated by a top-level
+ # semicolon, as in `"a(); b"`, or by a line break that can only end a
+ # statement, as in `"a()\nb"`, unlike `"a\n.b()"`.
+ #
+ # @return [Boolean]
+ #
+ def several_statements?
+ !@last_statement_start.nil?
+ end
+
+ #
+ # The code split before its last statement, so that the value of that
+ # one can be returned: `"a = 1\nb"` gives `["a = 1\n", "b"]`.
+ #
+ # @return [Array(String, String)]
+ #
+ def split_last_statement
+ return ["", @code] unless several_statements?
+
+ [@source.byteslice(@code_start...@last_statement_start), @source.byteslice(@last_statement_start...@code_end)]
+ end
+
+ #
+ # Whether a line break comes before the code, which right after a
+ # `return` ends that statement.
+ #
+ # @return [Boolean]
+ #
+ def leading_line_break?
+ @scanned && !@code.empty? && @source.byteslice(0...@code_start).include?("\n")
+ end
+
+ private
+
+ # Whether the code is one parenthesized group, as in `(function() {})`.
+ def parenthesized?
+ @paren_open == @code_start && @paren_close == @code_end - 1
+ end
+
+ def arrow?
+ return true if @code.match?(SINGLE_PARAMETER_ARROW)
+ return false unless @paren_close
+
+ @source.byteslice(@code_start...@paren_open).match?(ASYNC_PREFIX) &&
+ @source.byteslice((@paren_close + 1)..).match?(ARROW)
+ end
+
+ def scan
+ scanner = StringScanner.new(@source)
+ stack = []
+ previous = nil
+ ended = line_break = false
+
+ until scanner.eos?
+ if (length = scanner.skip(NOISE))
+ line_break ||= @source.byteslice(scanner.pos - length, length).include?("\n")
+ next
+ end
+
+ start = scanner.pos
+ top = stack.empty?
+ token = scan_token(scanner, stack, previous)
+ new_line = line_break && top && new_line_statement?(previous, token, scanner.peek(1))
+ @last_statement_start = start if token != ";" && (ended || new_line)
+ track_block(token, previous, stack)
+ track_condition(token, previous, stack)
+ ended = token == ";" && (ended || stack.empty?)
+ previous = token
+ line_break = false
+ @code_start = start if @code_end.zero?
+ @code_end = scanner.pos unless previous == ";"
+ end
+ end
+
+ # A value, or `x++`, followed on the next line by a word, a string, `!`,
+ # `~`, `++` or `--` can't continue it, so the line break ends the statement.
+ def new_line_statement?(previous, token, following)
+ return true if @block_end && ["(", "["].include?(token)
+ return false if @condition_end
+ return false if previous.nil? || REGEXP_KEYWORDS.include?(previous) || CONTINUING_WORDS.include?(token)
+
+ ends = previous[-1].match?(VALUE_END) || INCREMENTS.include?(previous)
+ ends && (token.match?(/\A[\p{Alnum}_$"'~]/) || INCREMENTS.include?(token) || (token == "!" && following != "="))
+ end
+
+ # Remembers whether the last top-level `}` closed a block, whose end
+ # ends the statement even before a `(` or `[` on the next line.
+ def track_block(token, previous, stack)
+ @block = block_start?(previous) if token == "{" && stack == ["{"]
+ @block_end = token == "}" && stack.empty? && @block
+ end
+
+ # Remembers whether the last top-level `)` closed the condition of an
+ # `if`, `for` or `while`, whose statement goes on even after a line break.
+ def track_condition(token, previous, stack)
+ @condition = CONTROL_WORDS.include?(previous) if token == "(" && stack == ["("]
+ @condition_end = token == ")" && stack.empty? && @condition
+ end
+
+ # A `{` after `)`, `=>`, a statement or a name opens a block, as in
+ # `function f() {`, `class A {` or `if (a) {`.
+ def block_start?(previous)
+ return true if previous.nil? || %w[) > ; }].include?(previous)
+
+ previous.match?(/\A#{WORD}\z/) && !OBJECT_KEYWORDS.include?(previous)
+ end
+
+ def scan_token(scanner, stack, previous)
+ if scanner.scan(WORD)
+ @words << scanner.matched unless previous == "."
+ scanner.matched
+ elsif scanner.scan(STRING) || (regexp_allowed?(previous) && scanner.scan(REGEXP))
+ scanner.matched
+ elsif scanner.skip(/`/)
+ scan_template(scanner, stack)
+ else
+ scan_punctuator(scanner, stack)
+ end
+ end
+
+ def scan_punctuator(scanner, stack)
+ char = scanner.getch
+ return char + scanner.getch if %w[+ -].include?(char) && scanner.peek(1) == char
+
+ track_parens(char, scanner.pos - 1, stack)
+ case char
+ when "{", "(", "[" then stack.push(char)
+ when ")", "]" then stack.pop
+ when "}"
+ opened = stack.pop
+ return scan_template(scanner, stack) if opened == :template
+
+ @body_end ||= scanner.pos if opened == "{" && stack.empty?
+ end
+ char
+ end
+
+ # Remembers where the first top-level parenthesized group, the
+ # parameter list of a declaration, opens and closes.
+ def track_parens(char, position, stack)
+ return if @paren_close
+
+ case char
+ when "(" then @paren_open ||= position if stack.empty?
+ when ")" then @paren_close = position if @paren_open && stack == ["("]
+ end
+ end
+
+ def scan_template(scanner, stack)
+ scanner.skip(TEMPLATE_CHUNK)
+ if scanner.skip(/\$\{/)
+ stack.push(:template)
+ else
+ scanner.skip(/`/)
+ end
+ "`"
+ end
+
+ def regexp_allowed?(previous)
+ return true if previous.nil? || REGEXP_KEYWORDS.include?(previous)
+
+ previous.length == 1 && !previous.match?(VALUE_END)
+ end
+ end
+ end
+end
diff --git a/lib/ferrum/node.rb b/lib/ferrum/node.rb
index 755d5561..67e4af44 100644
--- a/lib/ferrum/node.rb
+++ b/lib/ferrum/node.rb
@@ -37,6 +37,7 @@ class Node
# The node's properties from Chrome's deep serialization.
#
def initialize(frame, properties)
+ @runtime = frame
@page = frame.page
@target_id = @page.target_id
@backend_node_id = properties["backendNodeId"]
@@ -242,9 +243,9 @@ def scroll_into_view
#
# @return [Boolean]
def in_viewport?(of: nil)
- function = <<~JS
- function(element, scope) {
- const rect = element.getBoundingClientRect();
+ evaluate(<<~JS, of)
+ function(scope) {
+ const rect = this.getBoundingClientRect();
const [height, width] = scope
? [scope.offsetHeight, scope.offsetWidth]
: [window.innerHeight, window.innerWidth];
@@ -254,7 +255,6 @@ def in_viewport?(of: nil)
rect.right <= width;
}
JS
- page.evaluate_func(function, self, of)
end
#
@@ -375,7 +375,7 @@ def value
# page.at_css("input").property("value") # => "Foo"
#
def property(name)
- evaluate("this['#{name}']")
+ evaluate("this[name]", name: name)
end
alias [] property
@@ -392,7 +392,7 @@ def property(name)
# page.at_css("input").attribute("value") # => "Foo"
#
def attribute(name)
- evaluate("this.getAttribute('#{name}')")
+ evaluate("this.getAttribute(name)", name: name)
end
#
@@ -401,15 +401,14 @@ def attribute(name)
# @return [Array]
#
def selected
- function = <<~JS
- function(element) {
- if (element.nodeName.toLowerCase() !== 'select') {
+ evaluate(<<~JS)
+ function() {
+ if (this.nodeName.toLowerCase() !== 'select') {
throw new Error('Element is not a