Repository navigation
fix(mcp): validate temporal SIMPLE filter values before they reach SQL - #46110
Draft
aminghadersohi wants to merge 3 commits into
Draft
aminghadersohi wants to merge 3 commits into
aminghadersohi wants to merge 3 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #46110 +/- ##
==========================================
- Coverage 65.35% 65.35% -0.01%
==========================================
Files 3083 3084 +1
Lines 211224 211260 +36
Branches 49803 49813 +10
==========================================
+ Hits 138054 138063 +9
- Misses 69992 70019 +27
Partials 3178 3178
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
A SIMPLE filter on a temporal column accepted any value. Through the MCP chart tools and
query_dataset:{"column": "ds", "op": ">=", "value": "2000"}was read as epoch milliseconds and becameds >= '1970-01-01 00:00:02', so "since 2000" silently returned nearly everything."not-a-date"reached SQL as a string literal (an empty result on SQLite, a database error on engines with typed timestamps).Reproduced with the real
SqlaTable.get_sqla_queryon a SQLiteDATETIMEcolumn:"2000"rendersWHERE ds >= '1970-01-01 00:00:02',"not-a-date"rendersWHERE ds >= 'not-a-date'.The conversion lives in
ExploreMixin.filter_values_handler, which reads an all-digit string on a temporal column as epoch milliseconds and passes every other string through. That is deliberate (drill-to-detail and cross-filters depend on it, see #40180 and thetest_temporal_*_filter_*tests intests/unit_tests/models/helpers_test.py), so the shared query path is left unchanged. The check is added in the MCP layer instead, where the dataset's column types are known:invalid_temporal_filter_valuehelper (superset/mcp_service/common/temporal_filter_values.py). For=,!=,<,<=,>,>=,IN,NOT INon a temporal column, strings must be ISO 8601 and numbers are epoch milliseconds. An all-digit string is rejected as ambiguous between a year and an epoch. Booleans and unparseable strings are rejected.LIKE-style operators andIS NULL/IS NOT NULLare not checked.DatasetValidator._validate_filter_referencesapplies it tofiltersandfilters_secondaryof typed chart configs. Typed configs ingenerate_chart,update_chart,update_chart_preview, andgenerate_explore_linkfunnel throughvalidate_against_datasetand return anINVALID_TEMPORAL_FILTER_VALUEvalidation error before querying invalid temporal comparators.get_chart_previewreads saved/native chart state rather than a typed config and is unchanged.query_datasetapplies the same check to itsfilters.Not changed: non-temporal columns, valid ISO dates and datetimes, numeric epoch milliseconds, and null checks all behave as before. Direct chart data API requests (
/api/v1/chart/data) are not validated here, because there epoch-millisecond strings are an intended input.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A, backend only.
TESTING INSTRUCTIONS
The rejection regressions fail with the two modified call sites restored to master (9 original cases and 6 mixed-timeseries cases, including
filters_secondary,filters_b, and case-folded column names). The four test files above pass locally: 334 passed, 1 skipped; acceptance cases also pin unchanged valid-input behavior. Manually: callgenerate_chartorquery_datasetwith a filter{"column": "<temporal col>", "op": ">=", "value": "2000"}and observe a validation error suggesting2000-01-01;"2000-01-01"and946684800000still run.ADDITIONAL INFORMATION