Repository navigation
feat(databases): warn when a stored JWT is expired or expiring - #46036
Open
gabotorresruiz wants to merge 7 commits into
Open
gabotorresruiz wants to merge 7 commits into
gabotorresruiz wants to merge 7 commits into
Conversation
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
github-actions
Bot
added
i18n
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #46036 +/- ##
==========================================
+ Coverage 65.35% 65.41% +0.05%
==========================================
Files 3083 3087 +4
Lines 211224 211629 +405
Branches 49803 49915 +112
==========================================
+ Hits 138054 138429 +375
- Misses 69992 70009 +17
- Partials 3178 3191 +13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
gabotorresruiz
marked this pull request as ready for review
November 5, 2026 12:30
gabotorresruiz
requested review from
geido,
michael-s-molina,
rusackas,
sadpandajoe and
villebro
as code owners
November 5, 2026 12:30
github-actions
Bot
removed
i18n:japanese
gabotorresruiz
force-pushed
the
feat/encrypted-extra-expiry-warning
branch
3 times, most recently
from
November 6, 2026 01:32
1321ce5 to
2a76caf
Compare
msyavuz
reviewed
Nov 6, 2026
|
|
||
| const [tabKey, setTabKey] = useState<string>(DEFAULT_TAB_KEY); | ||
| // Bumped to ask the Advanced tab to open and scroll to Secure extra. | ||
| const [secureExtraFocusRequest, setSecureExtraFocusRequest] = useState(0); |
Member
There was a problem hiding this comment.
This is never reset in onClose, and the "already shown" guard lives in ExtraOptions which unmounts on close. After the link is used once, every later edit of any database auto-expands Security and scrolls on first Advanced open.
Contributor
Author
There was a problem hiding this comment.
@msyavuz you are right, that would have fired on every later edit. The request is now a pending flag next to a counter that only moves forward: the link sets it, the options pane clears it once it has opened and scrolled, and closing the modal clears it too, so nothing replays on a later edit while a second click in the same session still works (a plain reset of the counter on close turned out to break that second click). Tests cover both: a second database opened after the link keeps the panel collapsed, and the link used twice across a close and reopen opens it both times.
| `encrypted_extra`, give no entry. | ||
| """ | ||
| try: | ||
| config = json.loads(self.encrypted_extra or "{}") |
Member
There was a problem hiding this comment.
/connection now decrypts and parses encrypted_extra twice back to back (stored_sensitive_fields then this). Share one loader?
Contributor
Author
There was a problem hiding this comment.
@msyavuz done, both maps now read from one parsed copy of encrypted_extra.
|
|
||
| // Expired, or expiring today or within the warning window. A live token whose | ||
| // day the browser clock already passed still reads as today. | ||
| const isListed = (expiresAt: string, expired: boolean) => |
Member
There was a problem hiding this comment.
isListed accepts negative days but isExpiringSoon requires >= 0, so across a midnight clock skew the alert says "expires today" while the revealed field shows nothing. Intended?
Contributor
Author
There was a problem hiding this comment.
@msyavuz not intended. Both now use the same predicate: anything the browser clock has passed or that falls within the window counts, so the alert and the field agree across a skewed clock.
github-actions
Bot
added
the
requires:rebase
gabotorresruiz
force-pushed
the
feat/encrypted-extra-expiry-warning
branch
2 times, most recently
from
November 9, 2026 18:39
a45e55a to
3d82c63
Compare
github-actions
Bot
removed
the
requires:rebase
Add a helper that reports, for every declared sensitive path of a database's encrypted_extra, the last few characters of the stored string, so a masked secret can be told apart from another one (for example after a token rotation). The number of characters comes from the new ENCRYPTED_EXTRA_HINT_CHARS setting (0 disables, which is the default) and is exposed to the frontend. At most 25% of a trimmed value is ever revealed: a value shorter than four times the configured length gets no suffix. The length of the value is never reported, because the ending is withheld below that threshold precisely so a short secret gives nothing away, and a length would still identify it. Non-string values are skipped, wildcard paths expand per match, and output is sorted by path. BaseEngineSpec.encrypted_extra_hints builds on encrypted_extra_sensitive_field_paths and takes labels from encrypted_extra_sensitive_fields when it is a dict. A value that is a single PEM block (a service account or key-pair private key) always ends with the same "-----END ...-----" footer, so for those the suffix and the 25% cap use the base64 body with line breaks, newline escapes and the trailing "=" padding removed, so the revealed characters are real base64 characters. The setting is documented in Configuring Superset and announced in UPDATING.md.
Add unhappy-path tests: unicode and trailing-newline values, a stored value equal to the mask literal, non-object or malformed JSON, and a 1 MB secret next to thousands of other keys.
gabotorresruiz
force-pushed
the
feat/encrypted-extra-expiry-warning
branch
from
November 9, 2026 23:55
3d82c63 to
98bd5c7
Compare
…ra fields Add GET /api/v1/database/<pk>/encrypted_extra_hints/, which returns the label, path and short suffix of each sensitive field stored in a database's encrypted_extra, so an editor can tell which secret is stored without it being revealed. The suffix length comes only from the ENCRYPTED_EXTRA_HINT_CHARS config; 0 or less returns an empty list. The route maps to can_write on Database, so only users who can edit a connection can call it. Calls go to the event log with the database id and never the hint values, and the response is dumped through the schema so nothing beyond the three declared keys can leave the server. The length of a stored value is not returned: it could identify a short secret whose ending is withheld. The published OpenAPI spec gains the new path and its two schemas.
…formation Add the JSONPath and label of sensitive encrypted_extra fields to the engine information returned by the database endpoints. The connection modal needs to know, before any click, which Secure extra fields are stored secrets and what to call them, so it can render a field per stored secret up front and let the reveal control only change the value. The database GET, the list and the available endpoint carry the engine's declared patterns. The connection endpoint, which the edit modal loads and which requires write access to databases, replaces them with the fields stored for that database: every string value at a sensitive path, keyed by its concrete path. An engine on the catch-all "$.*" rule therefore shows one entry per stored string key instead of the pattern, and objects are left out. The label is the declared one when the path is declared exactly, else the key name. A missing or malformed encrypted_extra gives an empty map. Readers of a database see nothing about what is stored. BaseEngineSpec gains encrypted_extra_sensitive_field_labels(), which maps each path to its declared label (or the last path segment) with keys sorted. The secrets every engine adds get readable labels unless the engine declares its own: "OAuth2 client secret" for the OAuth2 client secret, and "Embedded guest password", "Embedded guest private key (PEM)" and "Embedded guest private key passphrase" for the embedded guest credential. encrypted_extra_hints reads its labels from the same method, and a new get_sensitive_field_paths helper shares its path resolution with the hints, so both list the same fields. Database.stored_sensitive_fields() builds the per-database map. The published OpenAPI spec documents the new key.
Database editors cannot tell which secret is saved in Secure extra, because the modal only shows the masked value. When a token is rotated or pasted into the wrong connection, the only way to check was to replace the secret blindly. Render one read-only field per stored secret under Secure extra. The fields are known before any request: the modal matches the masked values of Secure extra against the sensitive fields listed in the engine information of the connection payload, so each field shows its final label and its storage path from the start. A concrete path keeps its own label when a wildcard also matches it. Each field is a read-only password input whose value is empty while hidden, after a static run of bullets, and its eye is the password input's own toggle (focusable, Enter and Space, labelled Show or Hide), which changes only that field. The first eye clicked sends one request to the encrypted_extra_hints endpoint, with a spinner on the right of the field; an eye clicked while that request is pending waits on it and loads too. The answer is kept, so later eyes reveal their fields without a second request. A revealed field shows the last characters of its secret after the bullets, which do not move, and its eye hides it again without touching the other fields. A failed request warns under the field whose eye sent it and can be retried from any eye. The label and the helper text stay the same across a reveal. A secret too short to reveal says so, with a tooltip giving the minimum length, which is four times ENCRYPTED_EXTRA_HINT_CHARS. Nothing is fetched until an eye is clicked, so opening the modal never exposes even a partial secret. The fields render only when editing a saved database whose Secure extra still holds a masked value at a declared path and the ENCRYPTED_EXTRA_HINT_CHARS config is above zero. The hints live in local state: a hidden field drops its ending from the page, and closing the modal discards them, as does a save that keeps the modal open, so an ending never outlives the stored value. While Secure extra is being edited and briefly does not parse as JSON, the fields of its last valid JSON stay, revealed or not. Configuring Superset now says that the edit modal shows one field per masked secret, with an eye that reveals its ending on request.
Revealing the ending of a stored secret tells an editor which token is stored. It does not tell them whether that token still works, and an expired JWT fails only when a query runs. The reveal now also says when a stored JWT expires, read from its exp claim. Backend: `jwt_expiry` in `superset.utils.json` decodes a three-segment base64url value with PyJWT's unverified decode (header and payload must be JSON objects, and every segment must be canonical base64url) and reads only `exp`: any finite number in range, as RFC 7519 allows a non-integer one. The signature is not verified (the server has no key; the claim is informational) and no other claim is read, returned or logged. Anything else, a PEM key or a plain password for example, gives nulls. It never raises. Each hint from `encrypted_extra_hints` gains `expires_at` (UTC) and `expired`, computed against the server clock, and both are null for values that are not such a JWT. The response schema, the OpenAPI spec and the engine spec README describe the two keys. Frontend: in the revealed field's slot, `Expired <date>` in the danger colour after an error icon, or `Expires <date>` in default text after a warning icon when the token expires today or within 7 calendar days, or the browser clock has already passed it without the server calling it expired, and in secondary text beyond that (the icon carries the severity, so the text stays readable), with the date in the app's locale-aware short format and a tooltip saying it is read from the exp claim. A field without a readable expiry shows nothing there, and the too-short and not-available notes keep the slot when they apply. The three new strings are added to messages.pot; the weekly translation sync workflow propagates the template into the catalogs. Configuring Superset notes the expiry.
A database whose stored JWT had expired kept failing every query until someone found the cause. The edit modal can say so as soon as it opens, before any click and without a new request: the connection payload it already loads now carries when each stored token expires. - Database.stored_secret_expirations() maps each stored sensitive field holding a JWT with an exp claim to its expiry and whether it has passed, read from the unverified exp claim. Other secrets, such as PEM keys and passwords, are left out. It and stored_sensitive_fields() share one parse of encrypted_extra, kept until the stored value changes. - GET /api/v1/database/<pk>/connection sets it as engine_information.encrypted_extra_expirations, next to the stored sensitive fields. The plain GET, the list and available do not carry it, so readers learn nothing new and pay nothing. - EngineInformationSchema documents the map; the OpenAPI spec gains the EncryptedExtraExpiration component. - StoredSecretExpiryAlert shows one dismissible alert at the top of the edit modal, above the Basic and Advanced tabs, with one line per stored secret that has expired or expires within 7 calendar days: "The stored JWT Token expired on <date>.", "The stored JWT Token expires in 5 days (<date>)." or, on its last day, "The stored JWT Token expires today at <time>." Expired lines come first, then by expiry date. Secrets sharing a label, such as two JWT Token paths of one engine, add their path in parentheses. The alert is an error when any listed token has expired and a warning otherwise. Days are counted between calendar dates in the browser's time zone, the same dates the line shows. Nothing shows in create mode or when no entry qualifies. Whether a token has expired is decided on the server. - The alert ends with one action line, "Paste a new token into Secure extra and click Finish.", and a "Go to Secure extra" link that opens the Advanced tab, expands Security and scrolls to the editor once the layout settles. Each click bumps a counter that only grows and marks the request pending; the Advanced pane clears it once the scroll is checked, and closing the modal clears it too. A pane that stays mounted across close and reopen still sees the next click as new, and a pane mounted afresh never replays an old one. The Secure extra editor re-measures itself on the frame after Security opens and again after that scroll: ace mounted inside a panel that is still opening can keep a stale size and paint an empty editor. The token lines also stay under the Secure extra editor, each after an error or warning icon (expired text in the danger colour, expiring text in the default colour, so it stays readable), so they remain in view while the token is replaced. - The lines come from expiryLines in DatabaseModal/expiry.ts, next to the window and the calendar-day count the revealed field already uses for its warning icon, so the alert, the note and the field agree. - The warning does not depend on ENCRYPTED_EXTRA_HINT_CHARS: it carries no secret material, only when a token expires, and it is the part that prevents an outage, so an install with the reveal off still gets it. - The new strings are in messages.pot; the weekly translation sync workflow propagates the template into the catalogs.
gabotorresruiz
force-pushed
the
feat/encrypted-extra-expiry-warning
branch
from
November 10, 2026 00:35
98bd5c7 to
26d8cf6
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
An expired JWT in a connection's Secure extra fails every query until someone opens the connection and notices. This PR warns the editor the moment the database modal opens, before any tab, panel or reveal click, when a stored JWT is expired or expires within seven days.
Stacked on #46035 (itself stacked on #45997); the first six commits belong to those PRs and this diff shrinks to the last commit once they merge.
What changes:
Database.stored_secret_expirations()returns, for each stored sensitive field whose value is a JWT withexp, its expiry and whether it is past, using the same decoder as the hints (payload only,exponly, never raises). It shares one parsed copy ofencrypted_extrawithstored_sensitive_fields(), so/connectiondecrypts and parses the value once.GET /api/v1/database/<pk>/connection, the write-gated payload the edit form loads, carries it asengine_information.encrypted_extra_expirations. The plain database GET, the list andavailabledo not carry it, and tests pin that, so readers see nothing new and SQL Lab bootstrap does no extra work.StoredSecretExpiryAlertcomponent renders once above the tabs of the modal in edit mode, so it is visible from both Basic and Advanced, using the same wrapper and alert styling as the modal's other alerts. One dismissible alert lists each affected secret by its label:The stored JWT Token expired on Nov 4, 2026.,The stored JWT Token expires in 5 days (Nov 10, 2026).or, on its last day,The stored JWT Token expires today at 5:50 PM.Expired tokens come first, then by expiry date. Days are counted on the calendar, so the number always agrees with the date beside it. When two stored secrets share a label (an engine with two JWT fields) each line adds its path in parentheses so they can be told apart with the reveal off. The alert is an error when any listed token has expired and a warning otherwise, matching the colour of the revealed field. Nothing renders in create mode, without the map, or when no secret is expired or within seven days.Paste a new token into Secure extra and click Finish., and aGo to Secure extralink that switches to the Advanced tab, expands the Security panel and scrolls to the editor once the tab and panel animations have settled, then re-measures the JSON editor so it paints after opening inside the animating panel. The request is a pending flag next to a forward-only counter: the link sets it, the options pane clears it once handled, and closing the modal clears it, so nothing replays on a later edit and a second click in the same session still works. The same token lines also stay under the Secure extra editor, in the error or warning colour, so the problem is still marked once the alert has scrolled out of view or been dismissed.DatabaseModal/expiry.ts, shared with the revealed field, so the alert, the note and the field agree, also for a token the browser clock has passed but the server has not yet marked expired.Compatibility: no config change. The warning does not depend on
ENCRYPTED_EXTRA_HINT_CHARS, because it carries no secret material (an expiry date, not characters of the token) and it is the part that prevents the outage, so an installation with the reveal switched off still gets it. An older frontend ignores the new key.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Expired token, first open of the modal, nothing expanded:
After clicking Go to Secure extra: the Advanced tab, the Security panel open, the editor in view with the expiry note under it:
Two stored secrets with the same label, one expired and one expiring, told apart by their paths:
Token expiring later the same day:
Token expiring in 5 days: the note under the editor and the revealed field share the warning icon:
Token 60 days out, no warning:
Create mode, no warning:
TESTING INSTRUCTIONS
Automated:
tests/unit_tests/models/core_test.py: expirations for an expiring and an expired token with a PEM key alongside (only the two JWTs listed), six empty or malformed cases, only declared fields considered,/connectionparses the stored value once.tests/unit_tests/databases/api_test.py:/connectioncarries the map; the plain GET, the list andavailabledo not.DatabaseModal/index.test.tsx: expired and 7-day tokens warn as soon as the modal renders in edit mode without opening any tab or panel, several secrets in one alert, the link opens Secure extra on the Advanced tab, closing the modal forgets the request, no warning at 8 days, in create mode, or without the map.StoredSecretExpiryAlert.test.tsx: error versus warning type, the action line once with its link, shared labels told apart by path with expired first, distinct labels bare, the same-day time, and the path fallback for an unlabelled field.ExtraOptions.test.tsx: the note under the editor for an expired and an expiring token, nothing without expirations, and a focus request expanding the Security panel and scrolling to the editor.Manual:
expis yesterday in a Druid connection's Secure extra, open Edit: a red alert shows above the tabs with no hints request in the network tab; clickGo to Secure extra: the Advanced tab opens on the Security panel with the editor in view and the same line under it; dismiss the alert; reopen: it is back.expires today at <time>. In 60 days: no alert, no note.ENCRYPTED_EXTRA_HINT_CHARS = 0: the alert and the note under the editor still show, the reveal field does not.ADDITIONAL INFORMATION