Skip to content

feat(export): include applied filters as a separate file in chart CSV exports - #46090

Draft
Madan015 wants to merge 4 commits into
apache:masterfrom
jchong9:feature/export-filters-separate-csv
Draft

Madan015 wants to merge 4 commits into
apache:masterfrom
jchong9:feature/export-filters-separate-csv

Conversation

@Madan015

@Madan015 Madan015 commented Nov 9, 2026 •

Copy link
Copy Markdown

SUMMARY

First of five PRs for SIP-237 (#45332). It adds an Include filters as a separate file choice to every chart CSV export that the backend builds. Picking it downloads a ZIP containing the usual data CSV and a filters.csv that lists the filters applied to the query that produced the data. Without filter metadata keeps the current export exactly as it is.

Request field: ChartDataQueryContextSchema gets a top-level filter_metadata field with the values none (the default) and separate. separate is only accepted with result_format: csv, so any other format gets a 400 error instead of silently ignoring it. The option is stored on QueryContext and is not part of the cache key.

Filter rows are built on the backend: The new superset/utils/export_metadata.py builds the rows in _materialize_full_payload() from the query that actually ran, so the file matches the data:

  • Filters that could not apply, according to applied_filters, are left out.
  • Conditions are written as translatable words, for example "is greater than or equal to" or "is one of".
  • Lists longer than 100 values show the first 100 followed by "and N more".
  • Time ranges show the dates they resolved to, for example Last week (2026-10-26 to 2026-11-02). A dashboard time range that replaces a chart's saved range is marked as a dashboard filter. Charts saved with the legacy time column are covered.
  • Custom SQL WHERE and HAVING clauses get their own rows. They are left out for embedded guests who are not allowed to view the query, following redact_sensitive_query_fields().
  • Every cell goes through the existing escape_value() formula protection, and line breaks become spaces.
  • filters.csv uses the CSV_EXPORT encoding and separator, so it matches the data file. The default utf-8-sig encoding keeps Excel displaying non-ASCII text correctly.
  • The rows are only built when the request asks for them, because this function runs for every chart drawn on screen.

The file: It starts with an "Exported from" line naming the chart and an "Exported at" line with the UTC time. These are followed by the columns Filter, Condition, Value and Source, where Source is Chart filter, Dashboard filter or Custom SQL. If no filters applied, the file says "No filters were applied". A chart with several queries puts query_1.csv, query_2.csv and so on in the ZIP, with one filters.csv whose duplicate rows are removed.

Streaming: A CSV that streams cannot be packed into a ZIP. When the frontend knows a CSV will stream (CSV_STREAMING_ROW_THRESHOLD), the separate file choice is greyed out with a tooltip, and "Without filter metadata" still streams as before. Pivoted CSV and the drill views never stream, so they are never greyed out. If the backend would otherwise decide to stream after the query has run, or an API caller sends a large expected_rows, the response is still the ZIP, because the data can still be read normally.

Menus: The two choices appear under:

  • Explore's Export All Data: CSV, original CSV and pivoted CSV.
  • The standalone chart's Export to .CSV.
  • A dashboard chart's Download menu: CSV, full CSV and pivoted CSV.
  • Drill to detail and Drill by: Export to CSV.

All the menus build their items from one shared helper, src/utils/exportFilterMetadata.tsx. Excel, JSON and Export Current View are unchanged.

Docs: The API docs (docs/static/resources/openapi.json) are regenerated for the new field. No page in docs/ describes the chart export menus, so there is no user guide to update.

Drill export names: Drill to detail and Drill by exports previously sent no chart name, so their files were named after the chart type (superset_echarts_timeseries_bar_…). They now send the drilled chart's name with a prefix, for example Drill to detail: Genders by State or Drill by: Genders by State. That name is used for both the downloaded file (superset_Drill_to_detail_Genders_by_State_…) and the "Exported from" line.

Later PRs in this series, not included here:

  • PR 2: a Filters worksheet for chart Excel exports.
  • PR 3: filters as comment lines in chart CSVs, including streamed CSVs.
  • PR 4: a Filters worksheet for dashboard Excel exports.
  • PR 5: filters for exports built in the browser, Pivoted Excel and Export Current View.

Notes for reviewers:

  • Time ranges in "Applied filters": A chart's "Applied filters" popup can list a dashboard time range that the query never used. This happens when the chart has neither a time filter nor a legacy time column, because get_time_filter_status() marks the range as applied whenever the dataset has a date column. filters.csv only lists a time range the query actually filtered on.
  • Dashboard filter names: filters.csv lists a dashboard filter under the column it filters, using the dataset's display name, with "Dashboard filter" as its Source. It does not show the name the filter was given on the dashboard.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before: Explore's Export All Data → CSV downloads straight away.
image

After: CSV opens the two choices.
image

After: the separate file choice greyed out for a CSV that will stream.
image

After: a dashboard chart's Download menu.
image

After: the ZIP's contents, and filters.csv opened in Excel.
image

image

TESTING INSTRUCTIONS

Automated tests

  1. Backend: pytest tests/unit_tests/utils/test_export_metadata.py tests/unit_tests/charts/test_chart_data_api.py tests/unit_tests/charts/test_schemas.py tests/unit_tests/common/test_query_actions.py
  2. Frontend, from superset-frontend: npm run test -- src/utils/exportFilterMetadata src/explore src/dashboard/components/SliceHeaderControls src/dashboard/components/gridComponents/Chart src/components/Chart/DrillDetail src/components/Chart/DrillBy

Explore

  1. Open a table chart on birth_names in aggregate mode, grouped by name with the metric SUM(num), and add these filters:
    • Simple: gender equals boy
    • Simple: state not equal to Québec
    • Simple: name not equal to =1+1
    • Simple: num_boys ≥ 100
    • Time range on ds: a custom range, for example 2000-01-01 to 2005-01-01. The sample data covers 1965–2008, so relative ranges such as "Last week" return no rows.
    • Custom SQL WHERE, written over two lines:
      name LIKE 'J%'
      OR state = 'Québec'
    • Custom SQL HAVING: SUM(num) > 500
  2. Go to ⋯ → Download → Export All Data → CSV → Include filters as a separate file.
  3. Check that the ZIP holds the data CSV and filters.csv. Open filters.csv in Excel and check that it lists:
    • gender / equals / boy / Chart filter
    • state / does not equal / Québec, with Québec displayed correctly
    • name / does not equal / '=1+1, which stays as text rather than becoming a formula
    • num_boys / is greater than or equal to / 100
    • ds / is within / the range followed by its resolved dates
    • Custom SQL / WHERE / (name LIKE 'J%' OR state = 'Québec'), on one line. The line break becomes a space, so there may be extra spaces where it was.
    • Custom SQL / HAVING / (SUM(num) > 500)
  4. Export with Without filter metadata. It should download a plain CSV, unchanged from master.
  5. Check that Excel, JSON and Export Current View have no new choices.

Other menus

  1. On a pivot table, check both original CSV and pivoted CSV.
  2. Check the standalone chart, at /explore/?slice_id=<id>&standalone=1&show_download=true.
  3. On a dashboard with a native filter and a time range filter applied, export a chart's CSV with the separate file. The dashboard filters should show Source "Dashboard filter", and the time row should show the dashboard's range.
  4. Try Drill to detail and Drill by on a dashboard chart. "Exported from" should show the prefix followed by the chart's name, for example Drill by: Genders by State, and the file name should match, for example superset_Drill_by_Genders_by_State_….

Grey-out and full CSV

  1. Set CSV_STREAMING_ROW_THRESHOLD = 100 and turn on the ALLOW_FULL_CSV_EXPORT feature flag, which adds the dashboard's Export to full .CSV item for table charts. Then restart.
  2. Use a table chart whose query returns at least 100 rows. The check uses the number of rows the query returned, not the row limit.
  3. Check that:
    • The separate file choice is greyed out with a tooltip in Explore, the standalone chart, and the dashboard's CSV and full CSV items.
    • Without filter metadata still streams.
    • On a pivot table returning 100 rows or more, original CSV is greyed out but pivoted CSV is not, because a pivoted CSV never streams.
    • The drill views are not greyed out.
    • A chart returning fewer than 100 rows keeps the choice available.

API

  1. POST /api/v1/chart/data with "filter_metadata": "separate" and "result_format": "xlsx" should return a 400 error.

ADDITIONAL INFORMATION

  • Has associated issue: Part of [SIP-237] Include Applied Filter Settings in Excel and CSV Exports #45332
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@github-actions github-actions Bot added api Related to the REST API doc Namespace | Anything related to documentation utsc2026 UTSC 2026 course contributions labels Nov 9, 2026
@netlify

netlify Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit cd3b512
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6af2858b1f3fdf3bf7efc28a
😎 Deploy Preview https://deploy-preview-46090--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov

codecov Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 40.00000% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.32%. Comparing base (a131fcb) to head (cd3b512).

Files with missing lines Patch % Lines
superset/charts/data/api.py 21.42% 22 Missing ⚠️
superset/common/query_actions.py 25.00% 5 Missing and 1 partial ⚠️
superset/charts/schemas.py 71.42% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #46090      +/-   ##
==========================================
- Coverage   65.35%   65.32%   -0.04%     
==========================================
  Files        3083     3084       +1     
  Lines      211224   211312      +88     
  Branches    49803    49804       +1     
==========================================
- Hits       138054   138034      -20     
- Misses      69992    70098     +106     
- Partials     3178     3180       +2     
Flag Coverage Δ
hive 34.04% <36.00%> (-0.02%) ⬇️
mysql 51.77% <40.00%> (-0.04%) ⬇️
postgres 51.80% <40.00%> (-0.04%) ⬇️
presto 35.68% <40.00%> (-0.02%) ⬇️
python 52.09% <40.00%> (-0.04%) ⬇️
sqlite 51.48% <40.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jchong9
jchong9 force-pushed the feature/export-filters-separate-csv branch from f0e0de1 to 850dddd Compare November 9, 2026 06:20
@github-actions github-actions Bot added i18n Namespace | Anything related to localization i18n:spanish Translation related to Spanish language i18n:italian Translation related to Italian language i18n:french Translation related to French language i18n:chinese Translation related to Chinese language i18n:japanese Translation related to Japanese language i18n:german Translation related to German language i18n:russian Translation related to Russian language i18n:korean Translation related to Korean language i18n:dutch i18n:slovak i18n:slovenian i18n:ukrainian i18n:portuguese i18n:brazilian i18n:traditional-chinese i18n:persian i18n:czech i18n:latvian i18n:arabic i18n:catalan labels Nov 9, 2026
Madan015 and others added 4 commits November 9, 2026 21:46
… exports

Backend: adds the filter_metadata request field, builds the filter rows from
the query that ran, and returns the data CSV and filters.csv as a ZIP.
Frontend: adds the two CSV choices to the Explore, standalone, dashboard and
drill menus, and greys out the separate file when the CSV will stream.

Co-authored-by: Jeremy Chong <chong.jeremy@hotmail.com>
@jchong9
jchong9 force-pushed the feature/export-filters-separate-csv branch from cca133b to cd3b512 Compare November 10, 2026 02:46

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Related to the REST API doc Namespace | Anything related to documentation i18n:arabic i18n:brazilian i18n:catalan i18n:chinese Translation related to Chinese language i18n:czech i18n:dutch i18n:finnish i18n:french Translation related to French language i18n:german Translation related to German language i18n:italian Translation related to Italian language i18n:japanese Translation related to Japanese language i18n:korean Translation related to Korean language i18n:latvian i18n:maori i18n:persian i18n:polish i18n:portuguese i18n:romanian i18n:russian Translation related to Russian language i18n:serbian i18n:serbian-latin i18n:slovak i18n:slovenian i18n:spanish Translation related to Spanish language i18n:tamil i18n:thai i18n:traditional-chinese i18n:turkish i18n:ukrainian i18n Namespace | Anything related to localization size/XXL utsc2026 UTSC 2026 course contributions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants