---
title: "Import files through a staging table · Agent Wranglers"
url: https://agentwranglers.se/playbook/data/transactions-and-concurrency/staged-imports
id: DAT-78
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1061
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Import files through a staging table

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Transactions and concurrency](https://agentwranglers.se/playbook/data/transactions-and-concurrency.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: DAT-78
- Level: Recommended
- Chapter: [Transactions and concurrency](https://agentwranglers.se/playbook/data/transactions-and-concurrency.md)
- Rule: Load each uploaded CSV or spreadsheet into a staging table first and check every row. Show the person each failing row with its line number, then apply the good rows in one transaction.
- Check: Import a test file with a byte order mark, a quoted comma, a line break in a quoted field and a bad line 7. The preview names line 7 alone, and the other rows land once. Import the same file again, and the row count stays the same.
- How: Parse with a CSV library, never by splitting on commas. RFC 4180 lets a quoted field hold commas, line breaks and doubled quotes. Read the file as UTF-8 and drop a leading byte order mark, which some spreadsheet programs write. Cap the rows and the bytes per file. Match each row on a natural key, such as an SKU. Then a second import updates rows instead of adding copies, as [Make writes safe to repeat](https://agentwranglers.se/playbook/api-design/reliability/idempotency-keys.md) says. Store the import's ID on every row it wrote, so one bad import can be undone whole. Keep cells that start with `=` as plain text, and escape them on the way out as [Neutralise formulas in spreadsheet exports](https://agentwranglers.se/playbook/application-security/input-and-injection/csv-injection.md) says.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### Python example

```python
import csv
import io

MAX_ROWS = 10_000

def read_import(raw: bytes):
    reader = csv.DictReader(io.StringIO(raw.decode("utf-8-sig"), newline=""))
    good, bad = [], []
    for row in reader:
        if len(good) + len(bad) == MAX_ROWS:
            raise ImportTooLarge(MAX_ROWS)
        problem = check_row(row)
        if problem:
            bad.append({"line": reader.line_num, "problem": problem})
        else:
            good.append(row)
    return good, bad
```

### Go example

```go
const maxRows = 10_000

func readImport(raw []byte) (good [][]string, bad []RowError, err error) {
	reader := csv.NewReader(bytes.NewReader(bytes.TrimPrefix(raw, []byte("\xef\xbb\xbf"))))
	header, err := reader.Read()
	if err != nil {
		return nil, nil, err
	}
	for {
		record, err := reader.Read()
		if err == io.EOF {
			return good, bad, nil
		}
		if err != nil {
			return nil, nil, err
		}
		if len(good)+len(bad) == maxRows {
			return nil, nil, ErrTooManyRows
		}
		line, _ := reader.FieldPos(0)
		if problem := checkRow(header, record); problem != "" {
			bad = append(bad, RowError{Line: line, Problem: problem})
		} else {
			good = append(good, record)
		}
	}
}
```

## Sources

- [RFC 4180 Common Format for CSV Files](https://www.rfc-editor.org/rfc/rfc4180)

## Related rules

- API-32 [Accept idempotency keys on POST that creates or charges](https://agentwranglers.se/playbook/api-design/reliability/idempotency-keys.md) in API design
- APS-99 [Neutralise formulas in spreadsheet exports](https://agentwranglers.se/playbook/application-security/input-and-injection/csv-injection.md) in Application security

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/data/transactions-and-concurrency/staged-imports">Agent Wranglers data and databases playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Make conditional writes](https://agentwranglers.se/playbook/data/transactions-and-concurrency/conditional-writes.md)
- [Transactions and concurrency](https://agentwranglers.se/playbook/data/transactions-and-concurrency.md)