feat(databases): reveal the ending of stored secret values on demand by gabotorresruiz · Pull Request #45997 · apache/superset

gabotorresruiz · 2026-11-04T03:31:16Z

SUMMARY

In the database modal, the Secure extra editor shows masked secrets with no way to tell which value is stored. Editors who rotate a credential need to confirm which value is live. This PR surfaces the safe tail that the backend PR exposes, where the masked value is edited, using the same field pattern as every other field in the Security section.

Stacked on #45996, which adds GET /api/v1/database/<pk>/encrypted_extra_hints/ and the engine information this UI reads; the first four commits here are that PR and this diff shrinks to the last commit once it merges.

What changes:

For a saved database, when the server enables hints, one field per stored secret sits under the editor, labelled with the engine's name for it ("JWT Token") and built from the modal's own password field: Input.Password with visibilityToggle, the same control the SSH tunnel fields use. ExtraOptions resolves the sensitive paths the connection payload reports (engine_information.encrypted_extra_sensitive_fields on GET /api/v1/database/<pk>/connection, which the backend PR narrows to the fields actually stored as strings, labelled by the declaration or the key name) against the masked JSON in the editor and renders one SecureExtraHints field per path that holds the mask, when db.id is set and common.conf.ENCRYPTED_EXTRA_HINT_CHARS is above 0. Declared patterns from older backends still resolve, including wildcards.
Each field reuses the modal's StyledInputContainer: label from the engine, a read-only Input.Password whose prefix is a fixed run of bullets and whose value is only the ending, antd's own eye as the single control, helper "Stored at connect_args.jwt. Edits to Secure extra above are not reflected until saved."
Each eye reveals only its own field: the first click fetches the hints endpoint once, later eyes reuse that response, and hiding one field leaves the others as they are. Nothing else on screen changes: the label, helper and input are the same nodes before and after, the bullets are a static prefix so the toggle only swaps the ending between masked and clear (a pixel diff of the bullet region across the toggle is empty), and the input disables the font's contextual alternates. A hidden field never holds its ending in the DOM. Nothing is requested before the first click.
When the server withholds the ending because the secret is shorter than four times the configured length, the field says "Too short to show safely" with a tooltip stating the threshold; a field the server returned no hint for says "Not available". A failed request shows an inline warning under the field whose eye sent it and keeps the eye usable.
The fields survive typing in the editor: while the Secure extra JSON is mid-edit and unparseable, the last resolved list stays mounted with whatever was revealed; removing the mask still hides the field. After a save that keeps the modal open (an extension onSave error), the fields remount so a rotated secret is fetched again rather than showing the old ending.
Components come from @superset-ui/core; strings go through t() and are added to the translation template; the weekly catalog sync propagates them to the locale files.

Compatibility: no API or config change; the field reads the config key the backend PR exposes and never renders with the server default (0). No change to the editor or the save flow.

Review focus: the gating and path resolution in ExtraOptions, the single-request guard, the remount key after a save, and the helper wording that keeps unsaved edits from being misread.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before (hints disabled, the editor shows only the mask):

Before: Secure extra editor with the masked value and no field

After, field hidden and revealed (only the value and the eye change):

Secret too short to reveal, with the tooltip explaining the threshold:

Too short to show safely, tooltip with the minimum length

Two stored secrets on one connection, each with its own ending:

Two fields, JWT Token and Password, both revealed

Backend unavailable: inline warning, field stays hidden, the eye retries:

Warning under the field when the request fails

Reveal, hide and reopen:

TESTING INSTRUCTIONS

Automated:

SecureExtraHints.test.tsx: fields rendered hidden with their final label and helper and no request; exact URL requested once; the first eye reveals only its field and the other stays bullets with its ending never on the page; hiding one keeps the other; exactly one eye per field; a failed eye warns under its own field and the other can still fetch; only the triggering field shows loading, in the slot; suffix matched by path, not position; too short; no matching hint; double click sends one request; no count rendered; too-short tooltip states the threshold; remount; the bullets are a static prefix and only the ending switches between masked and clear; hidden fields are password inputs with an empty value; no inner border; works when effects run twice as in strict mode; keyboard reveal keeps focus on the eye; console stays clean in isolation; plus the path resolver cases (dotted, wildcard, nested wildcard, masked parent with a declared leaf, concrete path keeps its label over a wildcard, non-mask values, invalid JSON, declared order).
DatabaseModal/index.test.tsx: edit mode reads the secret fields from the connection payload, not the plain GET.
ExtraOptions.test.tsx: fields per masked declared path; absent for config 0, create mode, no mask, no declared match; a transiently invalid JSON keeps the mounted fields and their revealed state; a new secrets generation remounts them; index.test.tsx: a save that keeps the modal open drops the revealed endings.
npm run type: 0 errors.

Also verified in the local stack: a service-account PEM key reveals a body ending that changes on rotation; a connection on an engine without declared fields shows only its string secrets with key-name labels; two secrets on one connection reveal independently.

Manual, in a local stack with the backend PR and ENCRYPTED_EXTRA_HINT_CHARS = 5:

Edit a Druid connection whose Secure extra holds a JWT, open Advanced > Security: a "JWT Token" field with bullets sits under the editor and no hints request has been made.
Click the eye: one request, the bullets gain the last five characters, nothing else moves. Click again: bullets only.
Save without touching the secret and reopen: the stored value is unchanged and the same ending shows. Paste a new token, save, reopen: the new ending shows.
A connection with a short secret shows "Too short to show safely"; a read-only user sees no edit affordance; stopping the backend shows the inline warning and the eye retries once it is back.

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

netlify · 2026-11-04T03:33:26Z

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	`64727d8`
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6af25d863d73f15d310c75b0
😎 Deploy Preview	https://deploy-preview-45997--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

codecov · 2026-11-04T04:05:06Z

Codecov Report

❌ Patch coverage is 90.82569% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.39%. Comparing base (a131fcb) to head (87d88fd).

Files with missing lines	Patch %	Lines
superset/utils/json.py	75.00%	3 Missing and 4 partials ⚠️
superset/db_engine_specs/base.py	64.70%	4 Missing and 2 partials ⚠️
superset/databases/api.py	78.57%	2 Missing and 1 partial ⚠️
...uperset/commands/database/encrypted_extra_hints.py	89.47%	1 Missing and 1 partial ⚠️
superset/models/core.py	75.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #45997      +/-   ##
==========================================
+ Coverage   65.35%   65.39%   +0.03%     
==========================================
  Files        3083     3085       +2     
  Lines      211224   211441     +217     
  Branches    49803    49858      +55     
==========================================
+ Hits       138054   138265     +211     
+ Misses      69992    69989       -3     
- Partials     3178     3187       +9

Flag	Coverage Δ
hive	`34.06% <43.61%> (+<0.01%)`	⬆️
javascript	`81.62% <100.00%> (+0.03%)`	⬆️
mysql	`51.83% <78.72%> (+0.02%)`	⬆️
postgres	`51.86% <78.72%> (+0.02%)`	⬆️
presto	`35.70% <43.61%> (+<0.01%)`	⬆️
python	`52.15% <78.72%> (+0.02%)`	⬆️
sqlite	`51.53% <71.27%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

msyavuz · 2026-11-06T15:21:38Z

+              {typeof db?.id === 'number' && secretFields.length > 0 && (
+                <SecureExtraHints
+                  // A different database remounts, so no revealed hint survives.
+                  key={db.id}


db.id doesn't change on save, so when the modal stays open after a PUT (extension onSave error) a rotated secret that gets re-masked still shows the old ending from the cached hints. Key on changed_on or clear hints on save.

@msyavuz agreed, the id alone is not enough once the modal survives a save. The modal now bumps a generation counter after every successful update, and the hints component is keyed on the id plus that counter, so a rotated secret is fetched again on the next click instead of showing the old ending. There is a test for the save-then-reveal path.

msyavuz · 2026-11-06T15:21:38Z

                  </div>
                </div>
              </StyledInputContainer>
+              {typeof db?.id === 'number' && secretFields.length > 0 && (


Any keystroke that makes the editor JSON unparseable unmounts SecureExtraHints, dropping revealed state and refetching once it parses again. Intended?

@msyavuz not intended, only the mask check was. The fields now keep the last successfully resolved list while the editor JSON is mid-edit, so a stray keystroke no longer unmounts them or drops what was revealed. Once the JSON parses again the fresh resolution applies, so removing the mask still hides the field.

msyavuz · 2026-11-06T15:21:38Z

+          <StyledInputContainer key={path} data-test="secure-extra-hint">
+            <div className="control-label">{label}</div>
+            <div className="input-container">
+              <Input


Input.Password with visibilityToggle={{ visible, onVisibleChange }} is the eye pattern used by SSHTunnelForm; this hand-rolls it and uses count.show as a right slot.

@msyavuz fair, and it works. The first version used Input.Password with the whole value in it, and the browser-drawn dots redrew when toggling, which looked broken next to a value that is mostly bullets. Moving the bullets into a static prefix and keeping only the ending as the value fixes that: the prefix never changes, the toggle only swaps the five characters, and a pixel diff across the toggle shows no change in the bullet region. The field is now Input.Password with visibilityToggle, like the tunnel form, with antd's toggle as the single control and our icons rendered inside it; the count slot keeps the right-hand note.

Add a helper that reports, for every declared sensitive path of a database's encrypted_extra, the last few characters of the stored string, so a masked secret can be told apart from another one (for example after a token rotation). The number of characters comes from the new ENCRYPTED_EXTRA_HINT_CHARS setting (0 disables, which is the default) and is exposed to the frontend. At most 25% of a trimmed value is ever revealed: a value shorter than four times the configured length gets no suffix. The length of the value is never reported, because the ending is withheld below that threshold precisely so a short secret gives nothing away, and a length would still identify it. Non-string values are skipped, wildcard paths expand per match, and output is sorted by path. BaseEngineSpec.encrypted_extra_hints builds on encrypted_extra_sensitive_field_paths and takes labels from encrypted_extra_sensitive_fields when it is a dict. A value that is a single PEM block (a service account or key-pair private key) always ends with the same "-----END ...-----" footer, so for those the suffix and the 25% cap use the base64 body with line breaks, newline escapes and the trailing "=" padding removed, so the revealed characters are real base64 characters. The setting is documented in Configuring Superset and announced in UPDATING.md.

Add unhappy-path tests: unicode and trailing-newline values, a stored value equal to the mask literal, non-object or malformed JSON, and a 1 MB secret next to thousands of other keys.

…ra fields Add GET /api/v1/database/<pk>/encrypted_extra_hints/, which returns the label, path and short suffix of each sensitive field stored in a database's encrypted_extra, so an editor can tell which secret is stored without it being revealed. The suffix length comes only from the ENCRYPTED_EXTRA_HINT_CHARS config; 0 or less returns an empty list. The route maps to can_write on Database, so only users who can edit a connection can call it. Calls go to the event log with the database id and never the hint values, and the response is dumped through the schema so nothing beyond the three declared keys can leave the server. The length of a stored value is not returned: it could identify a short secret whose ending is withheld. The published OpenAPI spec gains the new path and its two schemas.

…formation Add the JSONPath and label of sensitive encrypted_extra fields to the engine information returned by the database endpoints. The connection modal needs to know, before any click, which Secure extra fields are stored secrets and what to call them, so it can render a field per stored secret up front and let the reveal control only change the value. The database GET, the list and the available endpoint carry the engine's declared patterns. The connection endpoint, which the edit modal loads and which requires write access to databases, replaces them with the fields stored for that database: every string value at a sensitive path, keyed by its concrete path. An engine on the catch-all "$.*" rule therefore shows one entry per stored string key instead of the pattern, and objects are left out. The label is the declared one when the path is declared exactly, else the key name. A missing or malformed encrypted_extra gives an empty map. Readers of a database see nothing about what is stored. BaseEngineSpec gains encrypted_extra_sensitive_field_labels(), which maps each path to its declared label (or the last path segment) with keys sorted. The secrets every engine adds get readable labels unless the engine declares its own: "OAuth2 client secret" for the OAuth2 client secret, and "Embedded guest password", "Embedded guest private key (PEM)" and "Embedded guest private key passphrase" for the embedded guest credential. encrypted_extra_hints reads its labels from the same method, and a new get_sensitive_field_paths helper shares its path resolution with the hints, so both list the same fields. Database.stored_sensitive_fields() builds the per-database map. The published OpenAPI spec documents the new key.

Database editors cannot tell which secret is saved in Secure extra, because the modal only shows the masked value. When a token is rotated or pasted into the wrong connection, the only way to check was to replace the secret blindly. Render one read-only field per stored secret under Secure extra. The fields are known before any request: the modal matches the masked values of Secure extra against the sensitive fields listed in the engine information of the connection payload, so each field shows its final label and its storage path from the start. A concrete path keeps its own label when a wildcard also matches it. Each field is a read-only password input whose value is empty while hidden, after a static run of bullets, and its eye is the password input's own toggle (focusable, Enter and Space, labelled Show or Hide), which changes only that field. The first eye clicked sends one request to the encrypted_extra_hints endpoint, with a spinner on the right of the field; an eye clicked while that request is pending waits on it and loads too. The answer is kept, so later eyes reveal their fields without a second request. A revealed field shows the last characters of its secret after the bullets, which do not move, and its eye hides it again without touching the other fields. A failed request warns under the field whose eye sent it and can be retried from any eye. The label and the helper text stay the same across a reveal. A secret too short to reveal says so, with a tooltip giving the minimum length, which is four times ENCRYPTED_EXTRA_HINT_CHARS. Nothing is fetched until an eye is clicked, so opening the modal never exposes even a partial secret. The fields render only when editing a saved database whose Secure extra still holds a masked value at a declared path and the ENCRYPTED_EXTRA_HINT_CHARS config is above zero. The hints live in local state: a hidden field drops its ending from the page, and closing the modal discards them, as does a save that keeps the modal open, so an ending never outlives the stored value. While Secure extra is being edited and briefly does not parse as JSON, the fields of its last valid JSON stay, revealed or not. Configuring Superset now says that the edit modal shows one field per masked secret, with an eye that reveals its ending on request.

gabotorresruiz requested review from Vitor-Avila and betodealmeida November 4, 2026 03:31

pull-request-size Bot added the size/XXL label Nov 4, 2026

github-actions Bot added api Related to the REST API doc Namespace | Anything related to documentation labels Nov 4, 2026

gabotorresruiz force-pushed the feat/encrypted-extra-hints branch from c3bb13c to 10285f4 Compare November 4, 2026 03:39

gabotorresruiz marked this pull request as ready for review November 4, 2026 04:23

gabotorresruiz requested review from michael-s-molina, rusackas, sadpandajoe and villebro as code owners November 4, 2026 04:23

gabotorresruiz force-pushed the feat/encrypted-extra-hints branch 2 times, most recently from da13dd2 to 9163ce7 Compare November 6, 2026 01:32

msyavuz reviewed Nov 6, 2026

View reviewed changes

github-actions Bot added the requires:rebase Requires rebasing on top of current master label Nov 7, 2026

gabotorresruiz force-pushed the feat/encrypted-extra-hints branch 2 times, most recently from 01b92f7 to 594f2bd Compare November 9, 2026 18:39

gabotorresruiz requested a review from msyavuz November 9, 2026 19:54

github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Nov 9, 2026

gabotorresruiz added 2 commits November 9, 2026 20:47

test(databases): cover edge cases for encrypted_extra hints

2214a1f

Add unhappy-path tests: unicode and trailing-newline values, a stored value equal to the mask literal, non-object or malformed JSON, and a 1 MB secret next to thousands of other keys.

gabotorresruiz force-pushed the feat/encrypted-extra-hints branch from 594f2bd to 64727d8 Compare November 9, 2026 23:55

gabotorresruiz added 3 commits November 9, 2026 21:31

gabotorresruiz force-pushed the feat/encrypted-extra-hints branch from 64727d8 to 87d88fd Compare November 10, 2026 00:35

Navigation Menu

Repository navigation

feat(databases): reveal the ending of stored secret values on demand - #45997

feat(databases): reveal the ending of stored secret values on demand#45997
gabotorresruiz wants to merge 5 commits into
apache:masterfrom
gabotorresruiz:feat/encrypted-extra-hints

gabotorresruiz commented Nov 4, 2026 •

edited

Loading

Uh oh!

netlify Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

msyavuz Nov 6, 2026

Uh oh!

gabotorresruiz Nov 9, 2026

Uh oh!

msyavuz Nov 6, 2026

Uh oh!

gabotorresruiz Nov 9, 2026

Uh oh!

msyavuz Nov 6, 2026

Uh oh!

gabotorresruiz Nov 9, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

gabotorresruiz commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

netlify Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for superset-docs-preview ready!

Uh oh!

codecov Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

msyavuz Nov 6, 2026

Choose a reason for hiding this comment

Uh oh!

gabotorresruiz Nov 9, 2026

Choose a reason for hiding this comment

Uh oh!

msyavuz Nov 6, 2026

Choose a reason for hiding this comment

Uh oh!

gabotorresruiz Nov 9, 2026

Choose a reason for hiding this comment

Uh oh!

msyavuz Nov 6, 2026

Choose a reason for hiding this comment

Uh oh!

gabotorresruiz Nov 9, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gabotorresruiz commented Nov 4, 2026 •

edited

Loading

netlify Bot commented Nov 4, 2026 •

edited

Loading

codecov Bot commented Nov 4, 2026 •

edited

Loading