PyErr not Cleared

What happens?

In testing threading/free threading, I observed that the PyError is not cleared on a UBIGINT conversion failure. This occurred when testing test_all_types.py, but is masked by the test exit and GIL.

duckdb-python/src/duckdb_py/native/python_conversion.cpp

Lines 963 to 967 in c535a23

    
           if (conversion_target.id() == LogicalTypeId::UBIGINT) { 
        
           	throw InvalidInputException("Python Conversion Failure: Value out of range for type %s", 
        
           	                            conversion_target); 
        
           } 
        
           PyErr_Clear();

* The PyErr_Clear() at 967 is really just at the wrong location.

The specific fix is adding PyErr_Clear() here. But, this may be an issue in other locations, such as:

duckdb-python/src/duckdb_py/native/python_conversion.cpp

Lines 975 to 977 in c535a23

    
           } else if (value == -1 && PyErr_Occurred()) { 
        
           	throw InvalidInputException("An error occurred attempting to convert a python integer"); 
        
           } else {

To Reproduce

PyErr_Clear() should be called before throwing an exception, after a PyErr_Occurred.

I can work on a reproducer, if needed, but I'm not sure I'll be able to against a GIL-enabled build.

OS:

Linux

DuckDB Package Version:

1.4.0dev

Python Version:

3.14

Full Name:

Paul T

Affiliation:

Iqmo

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a source build

Did you include all relevant data sets for reproducing the issue?

No - Other reason (please specify in the issue body)

Did you include all code required to reproduce the issue?

Yes, I have

Did you include all relevant configuration to reproduce the issue?

Yes, I have

I crafted a test to hit this path:

import duckdb

val = duckdb.Value(18446744073709551616, 'UBIGINT')
print(val)

duckdb.sql("select $1", params=[val]).show()

But I'm not sure why not clearing the error is a problem, can you expand on this test to show why it becomes a problem?

Also, if this is a problem, can't we handle it generically, in the exception translation logic?

	py::register_exception_translator([](std::exception_ptr p) { // NOLINT(performance-unnecessary-value-param)
		try {
			if (p) {
				std::rethrow_exception(p);
			}
		} catch (const duckdb::Exception &ex) {
			duckdb::ErrorData error(ex);
			PyThrowException(error, HTTP_EXCEPTION.ptr());
		} catch (const py::builtin_exception &ex) {
			// These represent Python exceptions, we don't want to catch these
			throw;
		} catch (const std::exception &ex) {
			duckdb::ErrorData error(ex);
			if (error.Type() == ExceptionType::INVALID) {
				// we need to pass non-DuckDB exceptions through as-is
				throw;
			}
			PyThrowException(error, HTTP_EXCEPTION.ptr());
		}
	});

Do a check for:

if (PyErr_Occurred()) {
    PyErr_Clear();
}

in here?

This looks like it should work:

db::LogicalType>(ele=handle @ 0x000000016fdfbb00, result=0x000000016fdfbe38, param=0x0000000146797658, nan_as_null=true) at python_conversion.cpp:963:9
   960                                          OP::HandleUnsignedBigint(result, param, unsigned_value);
   961                                          break;
   962                                  }
-> 963                                  if (conversion_target.id() == LogicalTypeId::UBIGINT) {
   964                                          throw InvalidInputException("Python Conversion Failure: Value out of range for type %s",
   965                                                                      conversion_target);
   966                                  }
Target 0: (python3) stopped.
(lldb) p (PyObject *)PyErr_Occurred()
(PyObject *) 0x00000001008e28c8
(lldb) br del
About to delete all breakpoints, do you want to do that?: [Y/n] y
All breakpoints removed. (1 breakpoint)
(lldb) b exceptions.cpp:384
Breakpoint 5: where = _duckdb.cpython-311-darwin.so`duckdb::RegisterExceptions(pybind11::module_ const&)::$_0::operator()(std::exception_ptr) const + 160 at exceptions.cpp:384:43, address = 0x000000010b7a1798
(lldb) b exceptions.cpp:390
Breakpoint 6: where = _duckdb.cpython-311-darwin.so`duckdb::RegisterExceptions(pybind11::module_ const&)::$_0::operator()(std::exception_ptr) const + 308 at exceptions.cpp:390:14, address = 0x000000010b7a182c
(lldb) c
Process 91001 resuming
Process 91001 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 5.1
    frame #0: 0x000000010b7a1798 _duckdb.cpython-311-darwin.so`duckdb::RegisterExceptions(pybind11::module_ const&)::$_0::operator()(this=0x000000016fdfccc7, p=(__ptr_ = 0x0000000116104100)) const at exceptions.cpp:384:43
   381                          }
   382                  } catch (const duckdb::Exception &ex) {
   383                          duckdb::ErrorData error(ex);
-> 384                          PyThrowException(error, HTTP_EXCEPTION.ptr());
   385                  } catch (const py::builtin_exception &ex) {
   386                          // These represent Python exceptions, we don't want to catch these
   387                          throw;
Target 0: (python3) stopped.
(lldb) p (PyObject *)PyErr_Occurred()
(PyObject *) 0x00000001008e28c8

Result for PyErr_Occurred is the same, so this happens directly after, before pybind11 does anything with the PyErr

Reset PyErr before throwing (duckdb#69)

Navigation Menu

Repository navigation

PyErr not Cleared #66

What happens?

To Reproduce

OS:

DuckDB Package Version:

Python Version:

Full Name:

Affiliation:

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

Did you include all relevant data sets for reproducing the issue?

Did you include all code required to reproduce the issue?

Did you include all relevant configuration to reproduce the issue?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if (conversion_target.id() == LogicalTypeId::UBIGINT) {
	throw InvalidInputException("Python Conversion Failure: Value out of range for type %s",
	conversion_target);
	}
	PyErr_Clear();

	} else if (value == -1 && PyErr_Occurred()) {
	throw InvalidInputException("An error occurred attempting to convert a python integer");
	} else {

PyErr not Cleared #66

Description

What happens?

To Reproduce

OS:

DuckDB Package Version:

Python Version:

Full Name:

Affiliation:

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

Did you include all relevant data sets for reproducing the issue?

Did you include all code required to reproduce the issue?

Did you include all relevant configuration to reproduce the issue?

Activity

Tishj commented on Sep 18, 2025

Tishj commented on Sep 18, 2025

Tishj commented on Sep 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions