The csvql blog

Notes on building an analytical CSV query engine for AI agents.

2026-11-08

csvql vs DuckDB MCP Servers: Which Read-Only CSV Server Should an Agent Get?

Several MCP servers give Claude read-only SQL over CSV files, and most wrap DuckDB. What each one lets an agent do, what it refuses, what SQL it understands, and when DuckDB is the better choice.

2026-11-08

csvql vs Letting the Agent Read the CSV File Itself

An agent with a file tool can open a CSV and answer questions about it, up to a point. Where that stops, in tokens, memory and what the agent can see, and what changes when it runs SQL on the file instead.

2026-11-08

How to Let Claude Desktop Query a CSV File With SQL Without Reading the Whole File

Three commands to give Claude Desktop a read-only SQL tool over one folder of CSV and Excel files. The config, why --root is required, what the agent can and cannot do, and how to check it worked.

2026-11-08

Claude Code: SQL Over a Folder of CSV and Excel Files

One command to register a read-only SQL tool over the CSV, TSV and .xlsx files in a project folder, the queries the agent will write, joins across files, and the limits to know before it hits them.

2026-11-08

Hide PII Columns From an AI Agent With --deny-columns

Name the columns the agent must never read and every query that touches them is refused, while department, salary and region stay queryable. What it blocks, verified, and what it does not.

2026-11-02

Claude Desktop Starts MCP Servers in /, So the Working Directory Is Not a Sandbox

We checked where Claude Desktop starts MCP servers: the filesystem root. What that meant for csvql 3.0's default, what 3.0.1 does about it, and what it means for any MCP server. Plus LEFT JOIN and Excel files.

2026-11-01

csvql 3.0: Safe by Default for Agents, Private Columns, and Arithmetic Everywhere

Agents now stay in their folder without any flag, --deny-columns keeps names and emails out of the model, and arithmetic works in SELECT, WHERE and over aggregates. Plus four queries that used to return a wrong answer without an error.

2026-10-19

Read-Only SQL Access to CSV Files for AI Agents, Without Moving the Data

Agents are getting access to company data, and the usual two paths both move it: a database connection or an upload. The third path is to run a query engine next to the files and let the agent ask questions there. What read-only means for csvql in checkable terms, how to set it up locally and over SSH, and what it does not protect you from.

2026-10-09

csvql vs DuckDB vs polars on Raw CSV

Six query shapes, two file sizes, three engines, one query per process. csvql wins twelve of twelve against DuckDB and eleven of twelve against polars. The one it loses is more interesting than the eleven it wins.

2026-10-08

Our Tests Covered One Third of the Engine

103 differential checks against DuckDB, a nightly fuzzer, and a query that returned rows in the wrong order for fifty releases. The queries that catch it were already in the suite. The fixture was 0.65 MB and the bug starts at 5 MB.

2026-10-08

What npm Search Actually Indexes

Our 15 keywords did nothing. The README is not indexed. A package with 11 downloads a week outranked us. Here is what we measured, the one field that matters, and the before and after of rewriting it.

2026-10-05

csvql vs polars: The Memory Stays Flat

polars is what people move to when pandas gets slow, and it is genuinely fast. On the same queries csvql is 1.5 to 3.2x ahead of it, at a flat 75 MB of memory from a 73 MB file to a 4 GB one, against polars settling around 2 GB. Updated 2026-10-07 with polars 2.0 and two corrections.

2026-10-01

The Journey to 10x DuckDB on CSV Parsing and Querying

csvql reads unquoted CSV up to 12.7x faster than DuckDB on one thread. Add quotes and the lead used to vanish entirely. A running log of finding out why, the fixes that did not work, and the prefix XOR that did.

2026-09-30

Querying a 3 GB CSV with Jev Costs the Same as Querying 10 Rows

Jev turns an English question into SQL for a CSV without ever seeing the data. One question, files from 10 to 20 million rows, the same 3,322 tokens every time.

2026-09-29

DuckDB 2.0 Says It's 6x Faster on Windows. I Checked, and Found My Own Engine Was the Slow One.

On raw-CSV queries the DuckDB v2.0 alpha measured 11-18% faster than 1.5.5. The same benchmark showed csvql barely ahead on Windows, because it was copying the whole file into memory first. Fixing that took the Windows lead from 1.3x to 6x.

2026-09-23

Five Ways to Find Distinct Values in a 10 GB CSV. One of Them Was Mine, and It Was the Worst.

Python, Node, Rust, DuckDB and csvql on the same 11 GB file. csvql took more than 6 minutes and 4 GB of memory to do what DuckDB did in 5.7 seconds. The benchmark found a real bug, and fixing it took the query to 4.8 seconds.

2026-08-17

Auditing Our Own Claims Found a Real Bug

Checking a 4-month-old blog post's memory claim against real measurement turned up two things: the claim was wrong, and the honest re-measurement exposed a live silent-data-corruption bug in eight places across the query engine.

2026-08-13

Chasing qsv: What It Takes to Close a Single-Thread Parsing Gap

csvql started 1.9x behind qsv's Rust CSV parser on raw single-thread throughput. Two targeted fixes closed it to parity, one plausible-sounding optimization made things worse, and a bad first reading almost led to the wrong conclusion.

2026-08-13

For Structured Data, Retrieval Accuracy Isn't a Model Problem

The AI industry is spending real money on reranking and hybrid search to push retrieval accuracy up a few points. For a table, exact SQL already gets you 100%, for free.

2026-08-10

Closing the Loop: 18 of 19 Bugs Fixed, One Filed Honestly

The differential-testing bug hunt that started with one blog comment is done for now. 19 issues filed, 18 fixed and shipped across five point releases, one narrow gap filed rather than hidden.

2026-08-06

When Does Converting a CSV to Parquet Actually Pay Off?

Converting to Parquet makes every later query faster, but the conversion itself isn't free. Measured the real breakeven point: 20 to 50 queries against the same file before that upfront cost pays for itself.

2026-08-05

csvql vs pandas: Querying a CSV Without Loading It First

pandas needs to load the whole file into a DataFrame before you can ask it anything. csvql doesn't. That's 17 to 24x tuned, 50 to 76x against how most pandas code is actually written, plus about 4x less memory.

2026-08-04

One Comment on a Benchmark Post Turned Into 18 Filed Bugs

A reader suggested adversarial CSV testing against DuckDB instead of trusting the happy path. It found 18 real bugs in csvql, 12 already fixed and shipped.

2026-07-30

The Token Economics of Querying a CSV Instead of Pasting It

Pasting an 8 GB CSV into an LLM costs 230 million tokens and still doesn't fit. Querying it costs about 560 tokens, flat, regardless of file size.

2026-07-30

Why csvql joins beat DuckDB by 50x while aggregates only beat it by 8x

Two query types, two different bottlenecks. What the parallel join probe actually changed, and why the same trick doesn't move the needle on a plain COUNT(*).

2026-07-13

I Benchmarked My CSV Engine Against DuckDB on Its Own Dataset. It Found Two Bugs in Mine.

csvql queries raw CSV about 2.8x faster than DuckDB on an 8 GB file, using about 6x less memory and zero extra disk. The benchmark that proved it also surfaced two real bugs.