Skip to content

fix(DONT MERGE): duckdb>=1.4.1 typing & warnings - #3189

Closed
dangotbanned wants to merge 38 commits into
mainfrom
fix-duckdb-1-4-1-typing
Closed

dangotbanned wants to merge 38 commits into
mainfrom
fix-duckdb-1-4-1-typing

Conversation

@dangotbanned

@dangotbanned dangotbanned commented Oct 9, 2025 •

Copy link
Copy Markdown
Member

Will close #3188

What type of PR is this? (check all applicable)

  • 💾 Refactor
  • ✨ Feature
  • 🐛 Bug Fix
  • 🔧 Optimization
  • 📝 Documentation
  • ✅ Test
  • 🐳 Other

Related issues

If you have comments or can explain your changes, please do so below

Note

Typing is "fixed"
Still need to open some issues/PRs upstream in duckdb (#3189 (comment))

@dangotbanned dangotbanned linked an issue Oct 9, 2025 that may be closed by this pull request
1 of 2 tasks
@dangotbanned dangotbanned added fix typing duckdb Issue is related to duckdb backend labels Oct 9, 2025
@dangotbanned dangotbanned added fix typing blocked duckdb Issue is related to duckdb backend labels Oct 9, 2025
Comment thread pyproject.toml Resolved
Comment thread narwhals/_duckdb/utils.py Resolved

@FBruzzesi FBruzzesi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dangotbanned - I personally have mixed feeling for all the patching that is happening in the typing module. It's roughly 100 lines to take care of 6 issues that arguably duckdb should deal with?! Isn't there also the risk to lose track of potential future updates/improvement they might do just because we are taking care of all these details? I am no expert but I will give a try to simplifying it a bit

I am 1000% onboarded on all the remaining changes

Comment thread narwhals/_duckdb/dataframe.py Resolved
Comment thread narwhals/_duckdb/utils.py
lit = duckdb.ConstantExpression
"""Alias for `duckdb.ConstantExpression`."""

# TODO @dangotbanned: Raise an issue upstream on `Expression | str` too narrow

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I notice that as well! It's worse than having Any 😢

@dangotbanned dangotbanned Oct 10, 2025 •

Copy link
Copy Markdown
Member Author

@dangotbanned dangotbanned Oct 10, 2025 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FBruzzesi!!!! I wasn't expecting it to support all of this, but here's the proof 😂

from __future__ import annotations

import datetime as dt  # noqa: F811
import decimal
import uuid
from typing import Any, Callable, cast  # noqa: F811

import duckdb  # noqa: F811
import numpy as np  # noqa: F811
import pandas as pd

lit = cast("Callable[[Any], duckdb.Expression]", duckdb.ConstantExpression)


def try_lit(*inputs: Any) -> None:
    for value in inputs:
        print(f"{value}: {lit(value)}")


try_lit(
    None,
    pd.NaT,
    pd.NA,
    np.ma.masked,
    True,
    False,
    0,
    -999_991,
    100_000_000_000_000,
    1e98,
    decimal.Decimal("2933.957546"),
    uuid.uuid5(uuid.NAMESPACE_URL, "hello"),
    dt.datetime(2034, 1, 23, 4, 2, 56),
    dt.time(1, 2, 3, 4),
    dt.date(1023, 3, 1),
    dt.timedelta(50),
    "i am a string",
    bytearray(range(4)),
    memoryview(b"i was a bytestring"),
    b"i am a bytestring",
    ["list", "of", "str"],
    ("tuple", "of", "str"),
    {"dict": "str"},
    np.arange(10),
    np.datetime64(dt.date(2000, 3, 1)),
    duckdb.UnsignedIntegerValue(50),
)
None: NULL
NaT: NULL
<NA>: NULL
--: NULL
True: true
False: false
0: 0
-999991: -999991
100000000000000: 100000000000000
1e+98: 1e+98
2933.957546: 2933.957546
074171de-bc84-5ea4-b636-1135477620e1: '074171de-bc84-5ea4-b636-1135477620e1'::UUID
2034-01-23 04:02:56: '2034-01-23 04:02:56'::TIMESTAMP
01:02:03.000004: '01:02:03.000004'::TIME
1023-03-01: '1023-03-01'::DATE
50 days, 0:00:00: '50 days'::INTERVAL
i am a string: 'i am a string'
bytearray(b'\x00\x01\x02\x03'): '\x00\x01\x02\x03'::BLOB
<memory at 0x00000154FFEA3100>: 'i was a bytestring'::BLOB
b'i am a bytestring': 'i am a bytestring'::BLOB
['list', 'of', 'str']: ['list', 'of', 'str']
('tuple', 'of', 'str'): ['tuple', 'of', 'str']
{'dict': 'str'}: {'dict': 'str'}
[0 1 2 3 4 5 6 7 8 9]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
2000-03-01: '2000-03-01'::DATE
50: 50

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread narwhals/_duckdb/utils.py Resolved
Comment thread narwhals/_duckdb/typing.py Resolved
Comment thread narwhals/_duckdb/typing.py Resolved
Comment thread narwhals/_duckdb/typing.py Resolved
Comment thread narwhals/_duckdb/utils.py Resolved
@dangotbanned

Copy link
Copy Markdown
Member Author

I personally have mixed feeling for all the patching that is happening in the typing module. It's roughly 100 lines to take care of 6 issues that arguably duckdb should deal with?! Isn't there also the risk to lose track of potential future updates/improvement they might do just because we are taking care of all these details?

@FBruzzesi ahh I thought I was safe in draft 😂

I vaguely mentioned in a comment somewhere (that I can't seem to find 🤦‍♂️) about needing to upstream some stuff to duckdb.

All of (https://github.com/narwhals-dev/narwhals/pull/3189/files#diff-3e3b75f6b9584445d40b9b7564a703fb4399c24094039fef6d8ab041d43954c7) is essentially experimenting with what we need for most of typing to be "fixed" - so I can refer to it in an issue 🙂

I still need to work out the right signature for these:

  • LambdaExpression
  • ConstantExpression

I thought (once I've fixed them on our side) they might be an easier first issue to raise over there 🙏

@dangotbanned dangotbanned changed the title fix: duckdb>=1.4.1 typing & warnings fix(DONT MERGE): duckdb>=1.4.1 typing & warnings Oct 11, 2025
@dangotbanned

This comment was marked as resolved.

@dangotbanned dangotbanned removed the ibis Issue is related to ibis backend label Oct 15, 2025
dangotbanned added a commit that referenced this pull request Jan 31, 2026
@dangotbanned

Copy link
Copy Markdown
Member Author

Superseded by #3433

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duckdb Issue is related to duckdb backend fix typing upstream issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

typing: Re-pin & fix duckdb==1.4.1

3 participants