feat(db): consolidate resource content onto a binary-safe /api/db/resource#59
Conversation
…cutover Per review: a skipped (it.skip) test asserting existdb-openapi's /api/db/resource honors expand-xincludes server-side. When the open path cuts over to that endpoint (see Scope/notes), db:load-document's own serialization handling goes away; this documents the expected contract and becomes a tripwire when it lands. Skipped until the consolidated db-core (eXist-db/existdb-openapi#55 and eXist-db/existdb-openapi#59) is in eXide's test environment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
92ccced to
59a8a35
Compare
874adaa to
7901900
Compare
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | -107 |
| Duplication | -2 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
line-o
left a comment
There was a problem hiding this comment.
A much wanted change to the API for sure.
Let's work on the implementation details.
7901900 to
08d5b8e
Compare
|
[This response was prompted by Joe, drafted by Claude Code, and reviewed by Joe.] Thanks @line-o — both addressed.
Colon instead of a dot on the wire ( |
…t-db#38) Adds GET/PUT /api/db/resource/{path} — a binary-safe, roaster-native resource transport, the clean alternative to the JSON-envelope /api/db/resource (which is text/base64-only). Closes the binary side of eXist-db#35/eXist-db#38 without a controller workaround (Juri's concern about an /api/* side door). - GET streams a binary resource's raw bytes with response:stream-binary (NOT via the serializer, which would emit base64 text and corrupt it — the established roaster pattern, the same path eXist's REST server uses); an XML/text resource is returned as a node so roaster serializes it once with its stored mime (returning a pre-serialized string would be XML-escaped a second time). - PUT stores the raw request body via db-core (binary bodies arrive intact; mime inferred from the name), returning { stored, runPath } (201/200). No exist-core or roaster change needed — binary transport works on stock eXist via the existing response:stream-binary. (Zero-copy streaming of very large stored binaries is a separate exist-core optimization, tracked with exist-strategy.) Self-contained Cypress coverage (raw round-trip not base64-mangled, 201/200 + stored/runPath, XML serialized with mime, 404); full db suite stays green (59 + 4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…in headers, split tests Per @line-o's review of eXist-db#56: - Move the path-in-URL raw endpoint from /api/db/resource/{path} to /api/resource/{path}: the {path} already begins with /db, so the old form produced a redundant /api/db/resource/db/apps/... Handlers unchanged. - meta=full now returns the resource's metadata as X-Resource-* response headers (owner/group/mode/size/created/last-modified; acl JSON-encoded in X-Resource-Acl, omitted when empty) instead of flattening it into the JSON body. db-core still returns metadata in the result map for in-process callers; only the HTTP wrapper lifts it into headers, so the body stays the resource content. - Split the combined serialization it() blocks into one-behavior-each, and update the meta=full test to assert headers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ource
Retire the JSON envelope and the path-in-URL endpoints; all resource content now
flows through the query-param /api/db/resource, identical in shape to its db
siblings.
- GET /api/db/resource?path= → raw content: binary streamed as-is; XML/text
serialized from the node tree (eXist has no raw byte form for XML). Content-Type
is the stored mime. download=true → Content-Disposition: attachment.
- PUT /api/db/resource?path= → raw request body (binary-safe); mime from
Content-Type or inferred from the name; returns { path } (201/200) so the caller
can reconcile name normalization.
- Serialization params: full W3C vocabulary + eXist extensions via the output:
namespace (eXist-db/exist#6447 — expand-xincludes, highlight-matches,
add-exist-id, process-xsl-pi, jsonp, insert-final-newline). Unsupported params
on a pre-#6447 eXist return a clean 400.
- Removed: JSON envelope, meta=full + X-Resource-* headers, runPath (derivable
client-side), the /api/resource/{path} and /api/db/resource/{path} endpoints,
and dbc:get-run-path.
- Metadata stays at GET /api/db/properties.
Supersedes eXist-db#38, eXist-db#56; folds + extends eXist-db#48. Depends on eXist-db#54, eXist-db#55.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- db.cy.js: store via a raw store() helper (path/&mime in the query, raw body); GET assertions read the raw body, not a JSON envelope; fold the binary/source/ download cases in; drop the runPath and meta=full cases (both removed); add the missing-path 400 and the eXist-extension-param graceful-degradation cases. - README: document /api/db/resource as the single binary-safe content endpoint and the client-side runPath derivation formula. (db-binary.cy.js removal folded into the preceding code commit.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…op #6447 dependency) db-core's serialization-params emitted the eXist extension params (expand-xincludes, highlight-matches, add-exist-id, process-xsl-pi, jsonp, insert-final-newline) in the W3C output: namespace, which only reaches fn:serialize with eXist-db/exist#6447. eXist already honors these as exist:-namespaced (http://exist.sourceforge.net/NS/exist) children of output:serialization-parameters, so emit the extensions there instead; standard W3C params stay in output:. expand-xincludes=no is now reachable on any eXist — the data-loss-safe read for round-tripping <xi:include> documents — without #6447. - db-core.xqm: choose the namespace per param (extensions -> exist:, standard -> output:) - api.json: drop the "requires #6447" notes from the six extension params - db.xqm: reword the serialization-error comment (no longer #6447-specific) - db.cy.js: replace the weak "honored-or-400" test with a real expand-xincludes round-trip (=no preserves <xi:include>, =yes expands it) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mp to 0.12.0 The W3C serialization parameters (indent, omit-xml-declaration, …) keep their plain names, but the implementation-defined eXist extensions now carry an "exist." prefix on the wire (exist.expand-xincludes, exist.highlight-matches, exist.add-exist-id, exist.process-xsl-pi, exist.jsonp, exist.insert-final-newline). This makes the W3C-vs-eXist split explicit in the query contract and mirrors the serialization model (W3C output: namespace vs the exist: namespace). A dot is used rather than a colon so the names stay clean for OpenAPI tooling / SDK codegen. db-core reads each eXist extension from its exist.-prefixed key but still emits the canonical local-name in the exist: namespace, so the serialized output is unchanged. api.json renames the six parameters; the resource spec updates the expand-xincludes case accordingly. info.version -> 0.12.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…gacy-name read-compat in GET Cleanup surfaced once eXist core (eXist-db/exist#6497) fixed the raw binary PUT, which had been masking these in the cypress suite. - create-collection: strict by default — a missing parent is now a clean 409 (was a raw exception). Pass `recursive: true` to create intermediate collections (mkdir -p). api.json gains the `recursive` body property and the 409 response; db-core gains the recursive arity + dbc:ensure-collection-path. - db:error-response now pins the response media-type to application/json. An error mapped to a status the route doesn't declare in api.json otherwise hits roaster's application/xml fallback and the error map fails to serialize (SENR0001). Hardens every db error path. See eeditiones/roaster#127. - db:get-resource resolved its existence check / binary streaming with dbc:to-stored, so it 404'd on legacy full-encoded names that dbc:get-resource / dbc:properties resolve. Made dbc:resolve-stored public and use it in the wrapper. - tests: migrate the remaining store call-sites (permissions setup; query.cy.js, query_pool_reuse, query_scope before-hooks) off the pre-consolidation {path,content,mime-type} envelope to the raw transport; make the raw-transport block self-sufficient; add strict-409 + recursive create coverage. Full cypress suite green (194/194) against an eXist build carrying #6497. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PUT /api/db/resource delivers content as bytes (octet-stream transport), and dbc:store passed those bytes straight to xmldb:store. When the target mime routes through eXist's XML parser (application/xml, text/html, image/svg+xml, any +xml, or an XML-class name when mime is inferred), handing it a binary value fails — it errors on some builds and, on the current existdb/existdb:latest, blocks the request while holding a broker, so a run of XML stores exhausts the pool and every later request hangs. Decode the bytes to a string first for XML-class targets so the parser sees the actual markup; non-XML targets (json, xquery, images, octet-stream, …) still store the bytes verbatim. Adds a dbc:xml-parsed-target helper keyed on the explicit mime, else the resource-name extension. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses Juri's review — db:get-resource nested four levels deep (try → if/else → if/else → if/else). Extract db:resolve-and-stream, db:stream-content, and db:stream-serialized, each encapsulating one level, so the flow reads top to bottom. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… wire Addresses Juri's review — the eXist serializer extensions were wire-named with an "exist." dot prefix; switch to "exist:" (e.g. ?exist:expand-xincludes=no) so the wire name mirrors the actual exist: serialization namespace. Colons are valid in query-parameter and OpenAPI parameter names, and SDK codegen sanitizes either form, so the earlier dot rationale doesn't hold. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eXist-db#61 merged to develop at 0.13.0, colliding with this branch's 0.13.0; bump to 0.14.0 so the api-version guard passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
08d5b8e to
8bd1157
Compare
[This PR was co-authored with Claude Code. -Joe]
Summary
Consolidates all resource-content transport onto a single, binary-safe
/api/db/resource, and retires the JSON-envelope shape. The/api/db/resourceendpoint, in the/api/dbfamily, is identical in form to its siblings (collection/move/copy/properties/sync). This design supersedes the path-in-URL binary endpoints (#38, #56) and folds in the latest serialization work (#48), extending it to the full parameter vocabulary.This is the outcome of a design review (with @line-o on #38 and #56, and in coordination with eXide, dashboard-next, and existdb-oxygen-plugin): the path-in-URL
/api/resource/{path}broke the query-param convention of the/api/dbfamily (it dangled under "default" in Swagger and doubleddbin the URL), and the JSON envelope wasn't relied on by any client. A review of the clients confirmed they could drop it.Rebased onto current
develop— the decoded-UTF-8 names and db-core work it was originally stacked behind have since merged, so the diff here is the consolidation itself.The consolidated contract
GET /api/db/resource?path=/db/…— returns the resource's raw content:Content-Type= the stored mime type.method,indent,omit-xml-declaration,encoding,media-type,item-separator,standalone, …) plus eXist's extensions (expand-xincludes,highlight-matches,add-exist-id,process-xsl-pi,jsonp,insert-final-newline). Omitted params fall through to the conf.xml defaults — defaults are a fallback, never a ceiling.download=true→Content-Disposition: attachment.PUT /api/db/resource?path=/db/…[&mime=…]— stores a raw request body (binary-safe):&mimequery param, else it is inferred from the resource name; passmime=application/octet-streamto force raw-byte storage of content that would otherwise be parsed. The request's ownContent-Typeis transport only (HTTP clients send unpredictable defaults, so&mimekeeps it deterministic).{ "path": "<canonical decoded path>" }so the caller can reconcile any server-side name normalization.DELETE /api/db/resource?path=/db/…— unchanged.Removed
meta=fulland theX-Resource-*metadata headers./api/resource/{path}and/api/db/resource/{path}path-in-URL endpoints.runPathfrom every response (db-coreget-resourceandstore);dbc:get-run-pathdeleted.Note: runPath is derivable client-side and documented in the README:
Metadata
GET /api/db/properties?path=— clients fetch it separately (a review of the clients confirmed this is preferred; content reads need only mime + binary-ness, derivable fromContent-Type).Encoding
The endpoint speaks decoded UTF-8 on the wire (eXist-db/exist#6463): a client sends and receives
café.xml, nevercaf%C3%A9.xml. The?path=form aligns better with this than path-in-URL did — a single query token roaster decodes cleanly, with no per-segment encoding ambiguity. db-core'sto-stored/to-display/resolve-storedmap to/from the canonical stored form (and stay read-compatible with legacy full-encoded names). Known Phase-1 limitation: literal%in a name is ambiguous (eXist-db/exist#6463), to be made bijective later.Serialization params — no eXist-core dependency
The eXist-specific serialization params (notably
expand-xincludes) are emitted asexist:-namespaced children ofoutput:serialization-parameters(thehttp://exist.sourceforge.net/NS/existnamespace), which eXist already honors — so they work on any eXist, with no dependency on eXist-db/exist#6447.expand-xincludes=noon read (the data-loss-safe round-trip of<xi:include>documents) is reachable everywhere.Testing
db.cy.js): raw GET round-trip (binary bytes intact, XQuery source not executed), serialization params (W3C + eXist extensions, including a realexpand-xincludescase), raw PUT create/overwrite with canonical-path echo, DELETE, decoded-UTF-8 name round-trips, non-/db400. Full suite green (200/200) against a cleanexistdb/existdb:latestcontainer using CI's install → restart →cypress runsequence.dbc:storedecodes the raw octet-stream body to text for XML-class targets before handing it toxmldb:store, so an XML mime never receives a binary value (which errored on some builds and blocked the request on others); non-XML content is stored byte-for-byte.