Skip to content

bigquery, bigquery-derive: hide macro implementation details behind #[doc(hidden)] pub mod __private before 1.0 #7449

Open

Description

@coryan

Summary

google-cloud-bigquery and google-cloud-bigquery-derive currently expose procedural-macro implementation details directly in google-cloud-bigquery's documented public API (google_cloud_bigquery::query) and emit calls directly to user-facing methods instead of routing generated code through a #[doc(hidden)] pub mod __private module.

Because google-cloud-bigquery was recently bumped to 1.0.0 on main (#7424, not yet released), shipping 1.0.0 as-is will freeze macro-internal types and trait methods under 1.x SemVer.

Problem Details

Per Rust proc-macro conventions (serde::__private, thiserror::__private, etc.), README.md ("any public symbol that is not part of the public API have the #[doc(hidden)] attribute set"), and doc/contributor/howto-guide-handwritten-client-api.md:

Generated code calls only a #[doc(hidden)] pub mod __private module in the runtime crate, never the user-facing API. Otherwise every method used by the generated code is frozen, and cargo-semver-checks cannot detect violations.

Currently:

  1. Derive-only types and methods are exposed in google_cloud_bigquery::query:
    • google_cloud_bigquery::query::SqlValue (src/bigquery/src/query/from_sql.rs, re-exported in src/bigquery/src/query.rs) is not returned or accepted by any user-facing Row or client API; it only exists to pass cell data into FromSql::from_value.
    • SqlValue::take explicitly documents that it is "Used by the #[derive(FromSql)] macro to extract struct fields", yet neither SqlValue nor SqlValue::take is marked #[doc(hidden)].
    • FromSql has a rustdoc warning ("Do not implement this trait directly. ... New methods (such as Arrow conversion methods) may be added in future versions, which would break manual implementations"), but FromSql is not sealed via a #[doc(hidden)] supertrait and FromSql::from_value(value: SqlValue) is not #[doc(hidden)]. Prose warnings do not exclude public traits or methods from SemVer or cargo-semver-checks.
  2. Generated code calls user-facing methods and unrooted paths directly (src/bigquery-derive/src/lib.rs):
    • #[derive(FromRow)] expands to row.take(...) on google_cloud_bigquery::query::Row and spells std::convert::TryFrom, std::result::Result, google_cloud_bigquery::query::Row, and google_cloud_bigquery::error::RowError without leading ::.
    • #[derive(FromSql)] expands to value.take(...) on google_cloud_bigquery::query::SqlValue and spells google_cloud_bigquery::query::{FromSql, SqlValue} and google_cloud_bigquery::error::ConvertError without leading ::.
    • Using dot-call syntax (row.take(...), value.take(...)) on unprefixed row / value locals instead of fully qualified function calls in __private freezes user-facing method signatures and risks method-resolution ambiguity if an in-scope extension trait defines .take(...).
    • Generated impl blocks are not enclosed in const _: () = { ... };.
  3. Crate-level rustdoc on google-cloud-bigquery-derive publishes maintainer policy:
    • src/bigquery-derive/src/lib.rs uses //! doc comments for internal # Compatibility / ## Contract tests CI instructions instead of // comments, and does not advise users to import FromRow / FromSql from google_cloud_bigquery::query rather than depending on google-cloud-bigquery-derive directly.
  4. Version skew note from chore(bigquery): bump to 1.0.0 #7424:
    • chore(bigquery): bump to 1.0.0 #7424 bumped google-cloud-bigquery from 0.19.0 to 1.0.0 while leaving google-cloud-bigquery-derive at 0.2.0. When migrating google-cloud-bigquery-derive to call ::google_cloud_bigquery::__private, google-cloud-bigquery-derive must also take a breaking bump (0.3.0 or 1.0.0) so google-cloud-bigquery 0.19.0 (which depends on ^0.2.0) does not resolve a 0.2.x patch release that references __private.

Expected Behavior

  • Add #[doc(hidden)] pub mod __private at the root of google-cloud-bigquery exposing only the internal types, traits, re-exports (Result, TryFrom), and helper functions needed by google-cloud-bigquery-derive.
  • Remove SqlValue from google_cloud_bigquery::query's public exports.
  • Keep google_cloud_bigquery::query::FromSql publicly nameable as a trait bound (for Row::get<T: FromSql>, Row::take<T: FromSql>, and generic callers), while sealing it / moving from_value onto a #[doc(hidden)] supertrait in __private so manual external implementations are prevented at the type/doc level rather than only by a prose warning.
  • Update google-cloud-bigquery-derive to emit const _: () = { ... }; blocks referencing only ::core::... / ::google_cloud_bigquery::__private::... via fully qualified function calls.
  • Convert the maintainer # Compatibility section in src/bigquery-derive/src/lib.rs from //! to // comments and add brief user-facing crate docs pointing to google_cloud_bigquery::query::{FromRow, FromSql}.
  • Take a breaking version bump on google-cloud-bigquery-derive alongside google-cloud-bigquery 1.0.0.

Activity

  1. added
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    api: bigqueryIssues related to the BigQuery API.
    priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.
    on Nov 11, 2026
  2. dbolduc commented on Nov 11, 2026

    @dbolduc
    Member

    I am pretty sure that (1) is bogus.

    • We try to avoid pub mod __internal at all costs, because it is still part of the public API. I have to think it is preferable to use the real public APIs.
    • Things like SqlValue::take() should not be hidden. Applications can and should use them. Our macro just helps them save a lot of boilerplate code.
    • We obviously can't seal the trait. If we did, how would we generate code in external crates that impl it? I assume the suggestion is to seal it with a __private trait, but that has the same problems of the first bullet.

    Most of (2) seems legit. It seems like we should harden the code with the fully qualified names. And TIL the const impl block trick. This stuff feels more like a cleanup to me.

    (3) is accurate, but a harmless clean up.

    (4) is only applicable if we do (1) which I don't think we should do.

  3. added
    type: cleanupAn internal cleanup or hygiene concern.
    and removed
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.
    on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigqueryIssues related to the BigQuery API.type: cleanupAn internal cleanup or hygiene concern.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions