Skip to content

Run the subqueries of a where query before MongoDB and Neo4j compare with them - #16824

Closed
borinquenkid wants to merge 10 commits into
fix/h7-criteria-query-args-8.0.xfrom
fix/gorm-aggregate-subquery-8.0.x
Closed

borinquenkid wants to merge 10 commits into
fix/h7-criteria-query-args-8.0.xfrom
fix/gorm-aggregate-subquery-8.0.x

Conversation

@borinquenkid

Copy link
Copy Markdown
Member

Fixes #16809

Stacked on #16823 (chain: #16816, #16817, #16818, #16820, #16821, #16822, #16823). no ci until the chain reaches 8.0.x.

Problem

Person.where { age > avg(age) } and the other aggregate subqueries documented for where put a DetachedCriteria in the comparison. SQL datastores run it as a subquery. MongoDB and Neo4j compared the property with the criteria object and returned no rows, without an error (the issue reports a MissingMethodException for some forms). inList / notIn against a subquery, exists and the all / some forms had the same gap.

Changes

  • Query.resolveSubqueries(Junction) (new, protected, @since 8.0.1) runs each subquery and returns a copy of the criteria that holds the results. The criteria passed in are not changed, so a reused where query runs its subqueries again each time.
    • ==, !=, >, >=, <, <= take the one value the subquery returns (an empty result compares with null);
    • inList takes every row, and notIn becomes a negated inList;
    • a comparison against a subquery that returns more than one row, exists, notExists and the all / some forms throw InvalidDataAccessResourceUsageException naming the form or the property.
  • MongoQuery.executeQuery and Neo4jQuery.executeQuery call it first.
  • Docs: where reference and upgrade guide section 26.16.

Two queries instead of one $expr pipeline or a Cypher subquery: the result is a snapshot taken when the query runs, which is what a where query already is on the SQL datastores' first read. A native $expr / CALL translation can replace the substitution later without changing behavior.

Tests

  • AggregateSubquerySpec for MongoDB (12) and Neo4j (9): avg with >, >=, <, <=, the .of { } closure, max, min, sum, ==, !=, negation, ||, inList, notIn, the multi-row and exists errors, and a repeat run that sees new data. All fail without the change (10 of 10 and 7 of 7 when I checked before adding the later cases).
  • Full suites: grails-data-mongodb-core 1036 tests and grails-data-neo4j-core 623, 0 failures; grails-datastore-core 348, 0 failures. codeStyle, validateRepositoryConventions and rat pass.
  • PITest (feat(build-logic): add opt-in PITest mutation testing support and skill #16819, cross-module): 43 mutants in the new Query code, 43 killed.

Not in this PR

  • A native $expr / Cypher translation (see above).
  • count as an aggregate comparison is untested here and not documented.

…with them

A where query such as Person.where { age > avg(age) } puts a DetachedCriteria
in the comparison. MongoDB and Neo4j compared against that object and matched
no rows. Query.resolveSubqueries runs the subqueries first and returns a copy
of the criteria holding their results, so the query is correct and runs the
subquery again on every execution. Comparisons take the single value of the
subquery, in and notIn take every row, and exists, notExists and the all and
some forms throw an InvalidDataAccessResourceUsageException that names the
form.

Fixes #16809

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@borinquenkid

Copy link
Copy Markdown
Member Author

Replaced by #16875, which has only this change on 8.0.x. This branch sat on top of the chain #16816 to #16823, so its diff against 8.0.x was the whole chain. Closing; the review history here was Copilot's only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants