Skip to content

[CALCITE-7614] UNNEST of an unqualified struct-rooted array path fails validation: "Column 's.s' not found"#5033

Merged
mihaibudiu merged 1 commit into
apache:mainfrom
takaaki7:CALCITE-7614
Jun 24, 2026
Merged

[CALCITE-7614] UNNEST of an unqualified struct-rooted array path fails validation: "Column 's.s' not found"#5033
mihaibudiu merged 1 commit into
apache:mainfrom
takaaki7:CALCITE-7614

Conversation

@takaaki7

Copy link
Copy Markdown
Contributor

Jira Link

CALCITE-7614

Changes Proposed

When the operand of UNNEST is an unqualified identifier whose leading component is a PEEK_FIELDS struct column, validation fails:

-- table T has column S: STRUCT<arr ARRAY<VARCHAR>> with StructKind.PEEK_FIELDS

SELECT * FROM t AS r CROSS JOIN UNNEST(r.s.arr) AS x;  -- OK (table-qualified)
SELECT * FROM t        CROSS JOIN UNNEST(s.arr)   AS x;  -- FAILS: "Column 'S.S' not found in table 'T'"

Root cause

DelegatingScope.fullyQualify qualifies the unqualified operand by first resolving the table that owns the PEEK_FIELDS column. That resolution re-enters validation of the UNNEST namespace, which expands the same operand SqlIdentifier and rewrites its name list in place to the fully-qualified form (SqlValidatorImpl.validateIdentifierSqlIdentifier.assignNamesFrom). Control then returns to fullyQualify, which re-qualifies the now-already-qualified identifier, prepending the table/struct segment a second time and producing the doubled S.S.

The table-qualified form is unaffected because re-qualifying r.s.arr is idempotent.

Fix

DelegatingScope.fullyQualify now snapshots the identifier before the re-entrant resolution and builds the qualified PEEK_FIELDS identifier from that snapshot, so it always works from the original names. The change is a no-op when no re-entrant rewrite occurs. The production change is confined to DelegatingScope.java.

Tests

  • New SqlValidatorTest.testUnnestPeekFieldsArrayColumn covering qualified/unqualified × CROSS JOIN/comma-join.
  • Added a PEEK_FIELDS struct type containing an array field (Fixture.peekArrayType) and a DEPT_NESTED_PEEK(DEPTNO, S) table in the mock catalog (MockCatalogReaderSimple), since no such column existed.
  • Updated SqlAdvisorTest's expected SALES completion list to include the new table.

./gradlew :core:test passes (15649 tests, 0 failed).

@takaaki7

Copy link
Copy Markdown
Contributor Author

@julianhyde could you review this small DelegatingScope.fullyQualify fix when you have a moment? Thanks!

int size = identifier.names.size();
int i = size - 1;
// Snapshot: resolution below may rewrite identifier's names in place [CALCITE-7614]
final SqlIdentifier identifierSnapshot =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this "originalIdentifier"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@mihaibudiu mihaibudiu added the LGTM-will-merge-soon Overall PR looks OK. Only minor things left. label Jun 21, 2026
@mihaibudiu

Copy link
Copy Markdown
Contributor

CI is not happy, can you take a look?

@takaaki7

Copy link
Copy Markdown
Contributor Author

Fixed.

@mihaibudiu

Copy link
Copy Markdown
Contributor

Please squash the commits so we can merge this.

…s validation: "Column 's.s' not found"

When the operand of UNNEST is an unqualified identifier whose leading
component is a PEEK_FIELDS struct column (e.g. UNNEST(s.arr) where s is a
PEEK_FIELDS struct containing array field arr), validation failed with
"Column 's.s' not found in table 't'", while the table-qualified form
UNNEST(r.s.arr) worked.

While DelegatingScope.fullyQualify resolves the qualifying table for the
unqualified operand, that resolution re-enters validation of the UNNEST
namespace, which rewrites the very same operand identifier's names in
place to the fully-qualified form. fullyQualify then re-qualified the
already-qualified identifier, duplicating the struct-column segment and
producing the doubled "s.s".

Fix fullyQualify to qualify a copy of the identifier taken before the
re-entrant resolution, so the PEEK_FIELDS branch always works from the
original names.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@sonarqubecloud

Copy link
Copy Markdown

@takaaki7

Copy link
Copy Markdown
Contributor Author

Squashed

@mihaibudiu mihaibudiu merged commit 3809e7c into apache:main Jun 24, 2026
19 checks passed
@xiedeyantu

Copy link
Copy Markdown
Member

@takaaki7 Please note that Calcite does not recommend using agents like Claude as collaborators; please only include your own contributions in subsequent submissions.

@mihaibudiu

Copy link
Copy Markdown
Contributor

I don't think we have a policy not to use agents. A tool that solves the problem is a good tool. But the users should scrutinize code from the agents carefully before submitting a PR, ensuring that it is compact and clear. Sometimes these agents solve simple problems in very complicated ways. You cannot expect other reviewers to clean up after agents, you have to do it yourself before you submit.

@takaaki7

Copy link
Copy Markdown
Contributor Author

Thanks for the feedbacks. Understood on the co-author tags. I'll properly review things on my end to avoid AI slop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LGTM-will-merge-soon Overall PR looks OK. Only minor things left.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants