feat(cloud): add TestingBot real device cloud provider - #283
Open
jochen-testingbot wants to merge 1 commit into
Open
jochen-testingbot wants to merge 1 commit into
jochen-testingbot wants to merge 1 commit into
Conversation
Adds TestingBot as a sixth cloud device farm alongside BrowserStack, Sauce Labs, LambdaTest, AWS Device Farm and Firebase Test Lab. It implements the existing CloudProvider interface, so nothing on the shared cloud path in internal/cli/test.go changes. - UploadApp: POST /v1/storage (multipart "file"), returns the tb:// URL used as the appium:app capability. - ListDevices: GET /v1/devices, mapping platform_name/version onto the common Device shape. - StartSession/StopSession: Appium W3C hub at hub.testingbot.com, with credentials passed in tb:options as TestingBot documents. - GetSessionArtifacts: GET /v1/tests/<id>/assets for video and screenshots, polled up to 30s since assets are processed asynchronously. Credentials resolve from "key"/"secret" (TestingBot's own naming), falling back to the generic "username"/"access_key" so --cloud-key and --cloud-secret keep working without touching the CLI credential map. Like Sauce Labs and LambdaTest, the provider runs in relay mode; direct port forwarding would require the TestingBot Tunnel binary and returns an error pointing at --relay. Endpoints and payloads were taken from the TestingBot OpenAPI spec (api.testingbot.com/v1/openapi.json) and the Appium support docs. Signed-off-by: Jochen Delabie <jochen@testingbot.com>
|
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds TestingBot as a sixth cloud device farm, alongside BrowserStack, Sauce Labs, LambdaTest, AWS Device Farm and Firebase Test Lab.
Disclosure: I work at TestingBot. This is a vendor integration for our own platform, written against the public API — happy to adjust anything that doesn't fit how you'd like third-party providers handled here.
Why
TestingBot is a real device cloud in the same category as the three farms already supported, and
CloudProvideris a clean extension point, so this slots in without disturbing existing providers.How
internal/cloud/testingbot.goimplements the existingCloudProviderinterface. The only change to shared code is onecaseinNewProvider— nothing on the cloud path ininternal/cli/test.gomoves.UploadAppPOST /v1/storage(multipartfile) →tb://<appkey>forappium:appListDevicesGET /v1/devices→name/platform_name/versionStartSession/StopSessionhub.testingbot.com, credentials intb:optionsGetSessionArtifactsGET /v1/tests/<id>/assets→ video + screenshotsEndpoints and payload shapes were taken from the TestingBot OpenAPI spec (
api.testingbot.com/v1/openapi.json) and the Appium support docs, not from memory.Two decisions worth your review:
key/secret(TestingBot's own naming, matchingTESTINGBOT_KEY/TESTINGBOT_SECRET) and falls back to the genericusername/access_key. That keeps--cloud-key/--cloud-secretandprobe.yamlworking without editing the shared credential map intest.go, which would have touched all five existing providers. Happy to flip to generic-only if you'd rather keep one convention.ForwardPortpasses through whenRelayURLis set and otherwise returns an error pointing at--relay, exactly as Sauce Labs and LambdaTest do. Direct forwarding would need the TestingBot Tunnel binary; that can be a follow-up if it's wanted.Artifact collection polls up to 30s because TestingBot processes assets asynchronously — same shape as the existing BrowserStack video poll.
Tests
internal/cloud/testingbot_test.gocovers credential resolution (both naming schemes and the failure cases), upload, device listing, capability construction for Android and iOS, the WebDriver error path, relay/directForwardPort, artifact polling, and session teardown.To test against
httptestI made the base and hub URLs struct fields defaulting to the package constants. The other providers hardcode consts and currently have no tests; I kept that change insidetestingbot.gorather than refactoring them.make test— all 17 packages passgo build ./...andgo vetclean;gofmtclean on the added files (firebase.goandlambdatest.goare gofmt-dirty onmainalready; left alone to keep this focused)Not exercised against a live TestingBot account in this branch: request/response shapes are spec-verified, not wire-verified.
Also updated
README provider table,
probe.yaml.example, the--cloud-providerflag help, thecloud.providerconfig comment, and the VS Code extension (setting enum, positionalenumDescriptions, run-profile dropdown, extension README setup section).No CHANGELOG entry — happy to add one under
[Unreleased]in whatever wording you prefer if that's part of your flow.