we got a real-life case where the lines at https://github.com/jgallagher/rusqlite/blob/master/src/statement.rs#L625 cause a panic because sqlite3_column_text can indeed contain non-utf8. FWIW the sqlite3 docs https://www.sqlite.org/version3.html say this:
SQLite is not particular about the text it receives and is more than happy to process text strings that are not normalized or even well-formed UTF-8 or UTF-16. Thus, programmers who want to store IS08859 data can do so using the UTF-8 interfaces. As long as no attempts are made to use a UTF-16 collating sequence or SQL function, the byte sequence of the text will not be modified in any way.
In our case we are reading from a database that was created/populated by a non-rust program so we can't change the fact that a text column contains non-utf8. We'd like to handle the invalid case ourselves -- could the API be amended? any recommendations on how we could work around the issue?
we got a real-life case where the lines at https://github.com/jgallagher/rusqlite/blob/master/src/statement.rs#L625 cause a panic because sqlite3_column_text can indeed contain non-utf8. FWIW the sqlite3 docs https://www.sqlite.org/version3.html say this:
In our case we are reading from a database that was created/populated by a non-rust program so we can't change the fact that a text column contains non-utf8. We'd like to handle the invalid case ourselves -- could the API be amended? any recommendations on how we could work around the issue?