Skip to content

feat(databases): make the connection-dialect blocklist configurable per deployment - #45020

Open
sha174n wants to merge 5 commits into
apache:masterfrom
sha174n:feat/configurable-db-dialect-allowlist
Open

sha174n wants to merge 5 commits into
apache:masterfrom
sha174n:feat/configurable-db-dialect-allowlist

Conversation

@sha174n

@sha174n sha174n commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

SUMMARY

check_sqlalchemy_uri() blocks a fixed set of SQLAlchemy dialects (sqlite, shillelagh, duckdb) because they can resolve local filesystem paths at query time. Today the only lever an operator has to relax that is PREVENT_UNSAFE_DB_CONNECTIONS=False, which disables the whole check. That is all-or-nothing: an operator who deliberately runs duckdb/MotherDuck has to drop the guard for every other dialect too.

This PR adds per-dialect granularity. A new config setting lets an operator opt a specific dialect back in while the rest of the blocklist keeps applying.

Mechanism

  • New config ALLOWED_UNSAFE_DB_DIALECTS: set[str] = set() (next to PREVENT_UNSAFE_DB_CONNECTIONS). It holds base dialect names, the part of the drivername before any +driver.
  • In check_sqlalchemy_uri(), when a blocklist entry matches, the base dialect is compared against the allowlist; if the operator listed it, that one dialect is permitted and the loop moves on. Everything else (the superset/ENABLE_SUPERSET_META_DB handling, the raised error) is unchanged.
  • Entries are matched on the base dialect, lower-cased and with any +driver suffix and surrounding whitespace stripped, so duckdb, DuckDB and duckdb+duckdb_engine all opt in the duckdb dialect (and the MotherDuck cloud URI, which rides the same dialect). A None/unset value is treated as no allowlist. This mirrors how the blocklist regexes already treat +driver suffixes.
  • The allowlist only covers the filesystem dialects it is meant to relax (sqlite, shillelagh, duckdb, via ALLOWLISTABLE_DIALECTS). The feature-flag-gated superset meta-database block is deliberately excluded, so this setting cannot be used to bypass ENABLE_SUPERSET_META_DB, and a future blocklist entry is not allowlistable unless added on purpose.

Backward compatibility

Defaults to an empty set, so behaviour is identical to today unless an operator explicitly opts a dialect in. No effect when PREVENT_UNSAFE_DB_CONNECTIONS is False (the check is skipped entirely in that case). This complements the existing all-or-nothing flag rather than replacing it.

TESTING INSTRUCTIONS

tests/integration_tests/security/analytics_db_safety_tests.py adds coverage through the real check_sqlalchemy_uri (no mocking of the check):

  • a dialect added to ALLOWED_UNSAFE_DB_DIALECTS is accepted, including the MotherDuck URI and the +duckdb_engine driver;
  • allowing one dialect does not re-enable the others (per-dialect, not global);
  • the default empty allowlist preserves existing behaviour.
pytest tests/integration_tests/security/analytics_db_safety_tests.py

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@sha174n
sha174n marked this pull request as ready for review October 6, 2026 09:59
@sha174n
sha174n marked this pull request as draft October 6, 2026 09:59
@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.28%. Comparing base (a6f30aa) to head (c8fbd6b).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #45020      +/-   ##
==========================================
- Coverage   65.35%   65.28%   -0.07%     
==========================================
  Files        3083     3083              
  Lines      211333   211339       +6     
  Branches    49823    49824       +1     
==========================================
- Hits       138110   137983     -127     
- Misses      70041    70170     +129     
- Partials     3182     3186       +4     
Flag Coverage Δ
hive 34.05% <50.00%> (+<0.01%) ⬆️
mysql 51.81% <100.00%> (+<0.01%) ⬆️
postgres ?
presto 35.69% <50.00%> (+<0.01%) ⬆️
python 52.02% <100.00%> (-0.11%) ⬇️
sqlite 51.52% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codeant-ai-for-open-source

codeant-ai-for-open-source Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

CodeAnt  CodeAnt PR Risk

🟢  Low Risk

  • The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: c8fbd6b3fd69

@netlify

netlify Bot commented Oct 6, 2026

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit 76fea0f
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6ac53ecadb425900088dee13
😎 Deploy Preview https://deploy-preview-45020--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@sha174n
sha174n marked this pull request as ready for review November 10, 2026 15:11
@sha174n
sha174n force-pushed the feat/configurable-db-dialect-allowlist branch from 106e685 to 10ae81e Compare November 10, 2026 16:27
sha174n and others added 5 commits November 11, 2026 11:36
…er deployment

check_sqlalchemy_uri() blocks a fixed set of SQLAlchemy dialects
(sqlite, shillelagh, duckdb) because they can resolve local filesystem
paths at query time. Today the only way to relax that is to turn the
whole check off via PREVENT_UNSAFE_DB_CONNECTIONS=False, which is
all-or-nothing.

Add ALLOWED_UNSAFE_DB_DIALECTS, a config set of base dialect names
(the part before any "+driver") an operator opts into for their
deployment. A listed dialect is accepted; every other entry in the
blocklist still applies. Defaults to an empty set, so behaviour is
unchanged unless an operator explicitly opts in. No effect when
PREVENT_UNSAFE_DB_CONNECTIONS is False, since the check is skipped
entirely in that case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The allowlist must not relax the feature-flag-gated superset meta-database
block; intersect operator entries with the allowlistable filesystem dialects.
…t allowlist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sha174n
sha174n force-pushed the feat/configurable-db-dialect-allowlist branch from d89da14 to c8fbd6b Compare November 11, 2026 11:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants