Repository navigation
Move the JSF test suite into modules/dataverse-jsf-tests - #12832
Closed
ashmanda-cs wants to merge 252 commits into
Closed
ashmanda-cs wants to merge 252 commits into
ashmanda-cs wants to merge 252 commits into
Conversation
- Add Playwright output dirs (test-results, playwright-report, blob-report, playwright/.cache, playwright/.auth) - Add .s2-dataverse-id runtime state file - Add sensitive-data/ directory - Add .DS_Store macOS noise - Remove obsolete .output, .pnpm-store, .vite/ entries - Minor comment cleanups
- Set initial version to 6.10.1 tracking Dataverse release versioning - Add files allowlist (lib/, tests/, scripts/, docs/, config files) - Remove dead main field (no index.js exists) - Add publishConfig access public for unscoped publish - Add playwright/dataverse/e2e/testing keywords
- workers: 1, fullyParallel: false for safe serial execution - slowMo: 200ms for auditable pacing - Always-on video, screenshot, and trace for compliance recording - Three browser projects (chromium, firefox, webkit) with explicit dependency chain to enforce sequential ordering - Per-browser auth file paths derived from BASE_URL slug
Derives a per-endpoint, per-browser auth storage state file path from BASE_URL so sessions never collide across instances or browsers.
…ER env resolution
Set max-parallel: 1 on the language matrix, so a push takes one runner instead of four from the organization's shared concurrency limit. Each language is still its own independent job (fail-fast: false), in the listed order. The run takes about 8 minutes instead of 5, still well under the API and JSF tests.
…QSS#12782 docker-compose-dev.yml now builds gdcc/keycloak from conf/keycloak instead of pulling a published image. The build job built it, but only saved the app and configbaker images, so the test jobs tried to pull gdcc/keycloak from Docker Hub and the stack failed to start. Save every gdcc/* image the build produced, so images added to the compose file later travel along too.
The dump and upload steps ran with always(), so when the stack failed to start they failed too (connection refused on port 6300), adding a second red step next to the real cause. Run them only if the stack step succeeded and the run wasn't cancelled; they still run after test failures, so coverage from failing runs is kept.
Start the Java port: a pom.xml for Playwright for Java and JUnit 5, using the same Java, JUnit and Hamcrest versions as IQSS/dataverse, and a Java-oriented .gitignore (build output and saved login sessions). `mvn verify` runs the *JT tests once per browser (Chromium, Firefox, then WebKit), each as its own Failsafe run; -Ddataverse.test.jsf.browser=<name> runs only one. The upload fixtures move from tests/test-data to src/test/resources/jsf.
JsfTestConfig is the single place settings are read: system properties (dataverse.test.baseurl, dataverse.test.jsf.*) with the existing environment variable names as fallback, so nothing is read from a file. Alongside it: per-endpoint session file paths, unique names for created data, absolute paths to the upload fixtures, a PrimeFaces AJAX-idle wait, a retry-until-passes helper (Playwright Test's toPass()), and the open-a-menu-or-dialog helper that re-clicks until it opens.
Port the builtin, shibboleth-direct and incommon-seamlessaccess adapters, the shared IdP credentials step and the Duo MFA handler.
Port the collection, collection form, dataset, dataset form, permissions and templates page objects. The collection Edit button is found through the stable #dataverse-info-dropdown menu rather than the first button whose name contains "Edit".
@JsfTest runs a test method in the browser chosen by dataverse.test.jsf.browser, in a fresh, logged-in browser context with a trace, a video and a final screenshot under target/playwright. The browser is installed on demand (only the one being run), launched and logged in once per run; a failed login fails every test in that run with the cause instead of retrying. JsfFixtures replaces the Playwright Test fixtures: createCollection, createDataset, trackDataset, cleanup and step. A collection's cleanup is registered before its form is submitted, so a failure after Dataverse has created it still deletes it.
Account menu (My Data, Notifications, Account Information, API token) and the collection tests: create, publish, roles, dataset templates, guestbooks and theme/widgets. Each test creates its own throwaway collection and works there, never in the root collection.
File upload (plain and tabular-ingest formats), downloading all files, the Locally FAIR download, citation downloads (EndNote, RIS, BibTeX) and the dataset lifecycle: create, edit metadata, edit and replace a file, publish. The Locally FAIR test skips itself unless its feature flag is on.
Browse, search, version history, dataset and file permissions, and the
Preview URL test, which checks the message Dataverse shows to a visitor
of the URL ("You are viewing a preview of this unpublished dataset
version"). The custom-license test skips itself unless its feature flag
is on.
Rewrite the README and developer guide for Maven and Playwright for Java: setup, the single-browser flag, running one class or method, the configuration reference (system property and environment variable for each setting), Playwright tooling, how the suite fits into IQSS/dataverse's CI, and how to fold it into that repository. The test specifications, auth, test-data and backlog docs now point at the Java classes.
The Java port replaces the Playwright Test specs, the lib/ helpers, the npm and TypeScript configuration, .env.example (settings are now system properties or environment variables) and the script that stitched test videos together.
Port the suite to Java (Playwright for Java + JUnit 5)
run-jsf-tests now checks out gdcc/dataverse-jsf-tests at the java branch (a ref input, so it can be pinned) and runs the Java port of the suite: install the browser's system libraries with Playwright's Java CLI, then mvn verify for that browser only. Node.js, npm ci and the separate authentication step are gone; Java and Maven come from the job's setup-maven. The JUnit reports and the per-test traces, videos and screenshots are uploaded as jsf-test-results-<browser>.
SKIP_PREFLIGHT and DOTENV_CONFIG_QUIET were only read by the Node suite. The Java suite reads the remaining variables (BASE_URL, LOGIN_ADAPTER, DV_USERNAME, DV_PASSWORD, DV_FULL_NAME, ROOT_DATAVERSE, LOCALLY_FAIR_ENABLED).
IQSS#12782 The JSF tests now run the Java port of gdcc/dataverse-jsf-tests (java branch) with Maven. Say so, how CI runs it per browser, and that running it locally needs Java and Maven but no Node.js.
The Java JSF suite writes JUnit reports, so each browser job now publishes them with the same action as the API job, as a check and PR comment named "JSF Test Results (<browser>)". A failing JSF test shows up by name on the pull request instead of only as a failed job. The JSF jobs get checks: write and pull-requests: write for this; Maven Tests already grants them to the call.
IQSS#12782 Also drop the claim that the suites finish at about the same time: the JSF chain (Chromium, then Firefox and WebKit) takes longer than the API tests.
Instead of a check and PR comment per browser, a JSF Test Results job runs after the Chromium, Firefox and WebKit jobs, downloads their JUnit reports and publishes them together. Each test counts once, with a run per browser; report_individual_runs reports a failure in a single browser on its own, pointing at that browser's report. Only this job needs write access to checks and pull requests, so the JSF jobs go back to read-only.
-Ddataverse.test.jsf.browser=<name> skips the other two browsers' Failsafe runs, but the single verify execution still read all three summary files and failed on the missing ones, even when every test passed. Give each browser its own verify execution, skipped together with that browser's run. A full mvn verify still runs all three browsers before checking any of them, so one browser failing doesn't stop the next.
fix(build): verify only the browser that ran
The JSF jobs failed only in the suite's own failsafe verify step, which read all three browsers' summaries in a single-browser run. That is fixed in gdcc/dataverse-jsf-tests#15 (java branch), which the workflow checks out at run time.
The suite now lives in this repository, so run-jsf-tests runs it in place with mvn -f modules/dataverse-jsf-tests instead of checking out gdcc/dataverse-jsf-tests. The JSF jobs add the module's pom.xml to the Maven cache key so its dependencies (Playwright) are cached too.
The module's README and developer guide no longer describe a separate repository: how to run it from a clone of IQSS/dataverse (cd into the module or use -f modules/dataverse-jsf-tests), that the main build doesn't include it, how the Dataverse Tests workflow runs it, and where its history from gdcc/dataverse-jsf-tests can be found. The README also explains running the suite against an installation from a supported machine (including for platforms Playwright doesn't support, such as RHEL), and warns against pointing it at production.
…sting guide IQSS#12797 How to run it from a clone (mvn -f modules/dataverse-jsf-tests verify, per browser), that it can test an installation from any machine Playwright supports, including for RHEL servers, and a warning not to point it at production.
…IQSS#12797 The IQSS#12782 note no longer points at gdcc/dataverse-jsf-tests; the new note covers where the suite lives now and how to run it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
How it was before
What this PR introduces
modules/dataverse-jsf-tests, imported with its full history (git subtree add) from thejavabranch of gdcc/dataverse-jsf-tests: Java, with Playwright for Java and JUnit 5.mvn -f modules/dataverse-jsf-tests verify, optionally with-Ddataverse.test.jsf.browser=<browser>.How it helps
Business value
Which issue(s) this PR closes:
Special notes for your reviewer:
git log.tests/jsf/.Suggestions on how to test this:
mvn -f modules/dataverse-jsf-tests verify -Ddataverse.test.jsf.browser=chromiumagainst a running dev stack (see the module's README).Does this PR introduce a user interface change? If mockups are available, please link/include them here:
No.
Is there a release notes update needed for this change?:
Yes, for developers:
doc/release-notes/12797-merge-jsf-tests.md.Additional documentation:
modules/dataverse-jsf-tests/README.mdanddocs/03_developer_guide.md.