[JunaidAhamed-7777]

SUMMARY

Preserve valid dates and times when datetime normalization falls back to parsing a column with mixed formats.

When neither a configured format nor a sampled common format is available, _process_datetime_column passes format=None, exact=False to pandas. Pandas infers a format from the first value and applies it to the column. Other valid values can become NaT, and a timestamp can lose its time of day without raising an error.

Use format="mixed" in that fallback so each value is parsed using its own format. Omit exact, which pandas does not allow with mixed, and remove the obsolete format-inference warning suppression. Configured formats, cached format mappings, common-format detection, and epoch parsing keep their existing paths.

The existing mixed-format test checked warnings and dtype but not the values. Strengthen it and add coverage for reversed row order, null/invalid values, microseconds, timezone conversion, and configured-format precedence. Add a short note to the dataset column documentation.

Nearby work: #45823 changes this helper to handle mixed timezone offsets. Its fallback still uses format=None, exact=False; this change addresses mixed string formats and does not change timezone policy.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Backend-only change. For ['2023-01-01', '01/02/2023', '2023-03-01 12:00:00']:

• Before: January 1 midnight, NaT, March 1 midnight.
• After: January 1 midnight, January 2 midnight, March 1 noon.

TESTING INSTRUCTIONS

In a Superset development environment:

import pandas as pd
from superset.utils.core import DateColumn, normalize_dttm_col

frame = pd.DataFrame({
    "date": ["2023-01-01", "01/02/2023", "2023-03-01 12:00:00"]
})
normalize_dttm_col(frame, (DateColumn(col_label="date"),))
print(frame["date"].tolist())

Verify that all three dates survive and the third retains noon. Reverse the input order and verify the same timestamps in reverse order.

pytest tests/unit_tests/utils/test_date_parsing.py \
  tests/unit_tests/utils/test_core.py \
  tests/unit_tests/datasets/test_datetime_format_detector.py \
  tests/unit_tests/db_engine_specs/test_crate.py -q

Local results on Python 3.12.14 and pandas 3.1.0:

• 221 tests passed.
• The final datetime tests run against unchanged master at 8527530ace produced four regression failures and 25 passes.
• Staged pre-commit run passed, including mypy and Ruff. Direct pylint on the changed production file passed with the repository configuration.

The full repository suite, docs build, and browser validation were not run.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	ef0492c
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6af1637eae95ec5f75179e10
😎 Deploy Preview	https://deploy-preview-46092--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codeant-ai-for-open-source]

Suggestion: When fallback input contains timestamps with different UTC offsets, pandas 3 raises ValueError because utc=False; it escapes normalization and can fail the request.

Assessment: 🟠 Major · 🔁 Occurrence: Sometimes · 🏷️ Api mismatch

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** superset/utils/core.py
**Line:** 2160:2165
**Comment:**
	*Api Mismatch: When fallback input contains timestamps with different UTC offsets, pandas 3 raises `ValueError` because `utc=False`; it escapes normalization and can fail the request.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai-for-open-source]

✅ CodeAnt verified this suggestion was addressed in subsequent commits and marked this thread resolved as of ef0492c.

The fallback now catches pandas’ “Mixed timezones” ValueError and retries parsing with utc=True, so timestamps with different offsets can be normalized.

_{If that's not right, unresolve this thread and CodeAnt will leave it open.}

[codeant-ai-for-open-source]

  CodeAnt PR Risk

🟡  Medium Risk

• The PR needs attention before merging.
• Mixed-format columns with timestamps carrying different UTC offsets may raise during parsing and fail the request; the added timezone test covers only a consistent offset.

Assessed commit: 3f7d32c697da