[brunocarlin]

Polars inputs previously encountered pandas-specific edits, statistics and result pipelines. This contribution moves eager dataframe processing to narwhals.stable.v2, retaining native pandas/Polars/Arrow model batches and typed result storage while preserving the existing writable pandas API. Refs #575.

Python >=3.10 is proposed, dropping 3.9; Python 3.14 is primary. Runtime minimums are pandas 1.5.3 and Narwhals 2.0, without speculative future-major caps. Only actual tested versions are compatibility evidence. Current stable pandas is 3.1.0; no pandas 4 release exists. The available 3.2 development build is tested separately. Maintainer review of the minimum-version change is requested.

Scope and compatibility

• Native importance, breakdown/interactions, DALEX Shapley, CP and aggregate profiles, metrics/diagnostics, Aspect/triplots, classification/regression fairness, Arena resources/EDA and dataset loading.
• Plot preparation also retains native tables across those families. Fairness console decisions use native score tables and numerical vectors, with an explicit pandas text-rendering boundary.
• Default pandas views remain writable and authoritative after access; result_as() provides independent typed exports. Incompatible mixed scalar/large-integer columns require explicit partitions rather than silent narrowing. Native observation snapshots retain precise metadata.
• Legacy private helpers adapt shared kernels instead of maintaining duplicate pandas algorithms. Tests cover indexes, dtypes, nested metadata, nullable edits, tie order and serial/process behavior. Nonlinear hierarchical-Shapley and index-sensitive callback cases preserve upstream semantics.
• Real sklearn/LightGBM regression and binary pipelines exercise original-feature preprocessing, prediction, exports, figures and pickle. Model-owned numerical/tensor adapters remain explicit. No mandatory Polars or Arrow dependency is added for pandas-only users.

Validation status

Latest source af6b862 streamlines Gaussian numerical work, clean-float importance means and CP metadata writes, and completes native fairness console preparation. Local checks pass: 177 targeted kernel cases, 490 integration/plot cases, 65 new Gaussian/console cases, 107 minimum-stack cases and 107 pandas-nightly cases. All 29 numerical outputs and 114 private/text outputs match unmodified upstream. Exact-source full CI is running; the preceding checkpoint does not certify these latest changes.

Validated checkpoint fdb23b0: 22/22 fork CI jobs passed. Downloaded artifacts verify:

• 15 complete suites: 1,778 passed / 18 optional PPS skips each; all 18 PPS cases pass separately in their compatible environment.
• Eight PyTorch and eight TensorFlow CPU cases; 12 installed-wheel core workflows without Polars, Arrow, sklearn, SHAP, LIME or statsmodels.
• 29 upstream numerical outputs, maximum error 8.88e-16; 106 helper contracts; 15 actual upstream pickle objects.
• 174 figure comparisons: 168 complete payloads and six triplots normalized only for redundant unlabelled collinear hover samples. Vertices, labelled points, tooltips and layout still match. The hover-density exception reflects removal of browser layout resolution.
• Python 3.10–3.14, Linux/macOS, Windows 3.14, pandas 3, minimum dependencies and required Linux 3.14t. Actual GIL state remains disabled after all 21 dependency imports. Unforced SHAP import enables it; forced-disabled passing tests do not certify arbitrary compiled-extension concurrency. No extension safety flags are patched.

The stronger nonlinear reference case exposed an RNG-reset error missed by earlier linear examples; fdb23b0 fixes the shared serial stream across hierarchy cuts. Comparisons use a documented NumPy spelling shim for the untouched reference. Consolidation report and source fingerprints · Integration map.

Benchmarks and regressions

Fresh alternating-source measurements are running for af6b862, using identical saved fitted models, data/target hashes, seeds and settings. They include conversion from original Polars frames, explanation fitting and compatible result publication; elapsed phases, sampled RSS and numerical outcomes are retained. Results will be reported with their exact source, including regressions.

Earlier end-to-end measurements belong to 080f946, not these later commits. They reported native Polars advantages for the measured original-Polars workflows, but pandas importance remained about 20% slower and partial/conditional explanation calculations about 45–50% slower than untouched upstream. Wide-frame Polars RSS was higher. Those results remain historical until remeasurement. Full previous trials and limitations.

Separate plotting-only measurements are source-stamped: CP and fairness. The fairness case includes a pandas timing regression and pandas/Polars RSS increases. Plotting gains do not establish faster explanation fitting.

Boundaries and review

Pandas remains required for writable legacy results, index/dtype/scalar formatting and the external PPS call. NumPy/SciPy, estimator and tensor boundaries are explicit. PPS dependency constraints and SHAP's automatic GIL fallback remain upstream limitations. Exact incompatible axes require separate selections; automatic interpolation retains its floating-grid policy. Raw LightGBM dictionary/Enum inputs and older sklearn/modern-Polars combinations need the documented model-owned bridges. Lazy/distributed/GPU execution, other engines, multiclass, arbitrary objects and arbitrary extension concurrency are uncertified.

Compatibility reference is pinned to upstream 77ea3ad9580a3b68aaa94d8dca75d11bd527848f. No upstream default-branch writes or merge are performed. Fork CI does not imply approval of upstream workflow execution. This PR remains draft and unmerged for the author's manual review. Prepared with AI assistance (Codex); no maintainer approval of AI-assisted contributions is implied.