[dependabot]

Bumps duckdb from 1.5.6 to 2.0.0.

Release notes

Sourced from duckdb's releases.

DuckDB v2.0.0

duckdb-python 2.0.0 is the Python package for DuckDB 2.0. It supports Python >= 3.11 and drops Python 3.10. From Python 3.12 on, one wheel per platform serves every Python version. The binding layer was rewritten on nanobind, query results stream instead of being collected up front, the relational API was rebuilt on SQL text, and the way DataFrames, Arrow data and polars frames are read was replaced. Many long-standing bugs are fixed, and moving data in and out of Python is much faster. The engine is DuckDB 2.0.0; see its release notes for what changed in SQL and storage.

Highlights

• Moving data is much faster. Numeric exports to numpy, pandas, Arrow and polars take a quarter to two fifths of the 1.5 time; in-memory Arrow, polars and pandas 3 inputs are scanned on all threads, 8x to 20x faster; Arrow UDFs run about 20x faster; draining an Arrow reader on 16 threads takes an eighth of the time. (#693, #697, #700)
• Results stream. con.execute() starts the query and you read rows as you fetch them; con.complete() and con.abort() finish or cancel an open result, and an open result or Arrow reader owns the connection until it is done.
• The relational API is rebuilt on SQL text. A relation is a lazy query, bound at every step so its schema is known; rel.sql_query() shows exactly what will run. Many new methods and expression helpers (col, lit, when, window functions). One semantic change: a plain string in an expression is a value, not a column name. (#697)
• A new Appender for fast bulk inserts: with con.appender("t") as a: a.append_row(...), under a microsecond per row. (#661)
• One stable ABI wheel per platform for Python 3.12 and later, so a new Python release works on day one. Python 3.15 is supported. (#672, #673, #684)
• Arrow export on con.execute() results changed: fetch_arrow_table, fetch_record_batch and arrow(n) are gone; use con.sql(query).to_arrow_table() or .to_arrow_reader(). (#702)
• The experimental Spark API is brought up to date: it is now compatible with PySpark 4.2.0 and 3.5.9 (#686). We would love your input on whether to continue support or to drop it in future releases.

Upgrading from 1.x: what to check

1. Make sure you run Python 3.11 or later.
2. If you call fetch_arrow_table, fetch_record_batch, arrow(n) or pl() on the result of con.execute(), switch to con.sql(query).to_arrow_table(), to_arrow_reader() or pl(). Libraries that call the removed names on an execute result need an update. (#702)
3. If you call fetchone(), fetchmany(), fetch_df_chunk() or execute() on a relation, use rel.limit(n).fetchall(), rel.to_arrow_reader(), rel.materialize(), or run the query through con.execute(). (#697)
4. If you compare a relation column with a plain string, col("s") == "x" now compares with the text x. Wrap column names in col(). (#697)
5. If you rely on import duckdb having opened the default database, note that it happens on the first duckdb.sql() or duckdb.connect() call instead. (#680)
6. If you pass Decimal values with a positive exponent, such as Decimal("1E+2"), they now arrive as the number they denote, 100, where 1.5 produced 1.00. (#654)
7. If you write files with overwrite=False, an existing file is now refused instead of replaced. (#668)
8. If a test or script depends on str(rel) being free, note that it runs the query each time now.

Breaking changes

Arrow export on DB-API results (#702)

Results of con.execute() are now read row by row and are not kept in memory, so they cannot hand their data to Arrow for free. fetch_arrow_table, fetch_record_batch, arrow(rows_per_batch) and pl() on a DB-API result are removed; they had been deprecated. to_arrow_table() and to_arrow_reader() remain, also as duckdb.to_arrow_table() and duckdb.to_arrow_reader(), but they materialize the rows twice, once as the result and once as Arrow, and emit a UserWarning pointing at the relational path, con.sql(query).to_arrow_table(), which we recommend instead. The reader no longer streams: its first batch arrives after the whole query ran. After a partial fetchone or fetchmany the Arrow call returns the remaining rows instead of raising. With preserve_insertion_order off, the row order can differ between calls. duckdb.arrow(n) with an integer is gone; duckdb.arrow(arrow_object) still builds a relation.

# 1.5
table = con.execute("SELECT * FROM t").fetch_arrow_table()
reader = con.execute("SELECT * FROM t").fetch_record_batch(10_000)
2.0
table = con.sql("SELECT * FROM t").to_arrow_table()
reader = con.sql("SELECT * FROM t").to_arrow_reader(10_000)

Relations are SQL text (#697)

The relational API is a query building API, not an execution API. A relation is a piece of SQL text with a schema bound once when it is built, so every step knows its columns and types. It runs nothing until you ask for data, and it runs again on every read; rel.sql_query() returns exactly the text that runs, as one SELECT per group of steps and a common table expression where the order of clauses breaks. The engine's binder decides every column name and type; nothing is renamed by the client.

What you are most likely to notice:

• A bare string operand of an expression is a value, not a column. For one release a warning is raised when the string matches a column name; the warning goes in 2.1. A dotted string is not split into parts; pass the parts separately.

... (truncated)

Commits

• 83c7199 Build the dictionary test's indices with a 64 bit arange (#705)
• b12fd55 Build the dictionary test's indices with a 64 bit arange
• 6aa2ce3 Build the Linux and macOS wheels with clang and ThinLTO (#703)
• 2f0ac79 Pin submodule to v2.0.0 hash
• f7826ad Fix wrong values and lost parallelism in the pandas, numpy and Arrow input sc...
• 2f2a0dd Bring back to_arrow_table and to_arrow_reader on DB-API results over a re-fee...
• 6086a15 Keep the macOS wheels native, like the engine's release
• 07ee3e2 Build the Linux and macOS wheels with clang and ThinLTO
• a2c151f Register the linked extensions when the module loads, not only on import (#701)
• dd2a3e9 Read numpy masks a word at a time
• Additional commits viewable in compare view

[dependabot]

Labels

The following labels could not be found: dependencies, security. Please create them before Dependabot can add them to a pull request.

Please fix the above issues or remove invalid values from dependabot.yml.

[github-actions]

⚠️ This is a major version update and requires manual review before merging.

[github-actions]

⚠️ This is a major version update and requires manual review before merging.

[github-actions]

⚠️ This is a major version update and requires manual review before merging.