mirror of
https://github.com/d3vyce/fastapi-toolsets.git
synced 2026-08-04 23:54:09 +00:00
chore: rework DB module (#324)
This commit is contained in:
d3vyce
+1
-1
@@ -167,7 +167,7 @@ user = await UserCrud.update(session, UserUpdate(credits=10), [User.id == user_i
|
||||
```
|
||||
|
||||
!!! warning
|
||||
`with_for_update` requires an open transaction. Wrap your call in `async with session.begin()` or use the `get_transaction` helper if you are not already inside one.
|
||||
`with_for_update` requires an open transaction. Wrap your call in `async with session.begin()` or use the `transaction` helper if you are not already inside one.
|
||||
|
||||
!!! note
|
||||
`NOWAIT` raises `sqlalchemy.exc.OperationalError` immediately if the row is locked rather than waiting.
|
||||
|
||||
+94
-57
@@ -7,96 +7,137 @@ SQLAlchemy async session management with transactions, table locking, advisory l
|
||||
|
||||
## Overview
|
||||
|
||||
The `db` module provides helpers to create FastAPI dependencies and context managers for `AsyncSession`, along with utilities for nested transactions, table locks, advisory locks, and polling for row changes.
|
||||
The `db` module is built around one object, [`Database`](../reference/db.md#fastapi_toolsets.db.Database), which owns the engine and sessionmaker and exposes the FastAPI dependency, a commit-before-response middleware, session/transaction context managers, and table locking. Free helpers cover savepoint-aware transactions, advisory locks, many-to-many association tables, and row-change polling.
|
||||
|
||||
## Session dependency
|
||||
## Setup
|
||||
|
||||
Use [`create_db_dependency`](../reference/db.md#fastapi_toolsets.db.create_db_dependency) to create a FastAPI dependency that yields a session and auto-commits on success:
|
||||
Create one `Database` for your app. Provide a **URL** (the facade builds and disposes the engine) or pass an existing **`engine=`** you own (e.g. for Alembic or `event.listen`). The session factory is built internally with `expire_on_commit=False`.
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker
|
||||
from fastapi_toolsets.db import create_db_dependency
|
||||
from fastapi import Depends, FastAPI
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
engine = create_async_engine(url="postgresql+asyncpg://...", future=True)
|
||||
session_maker = async_sessionmaker(bind=engine, expire_on_commit=False)
|
||||
from fastapi_toolsets.db import Database
|
||||
|
||||
get_db = create_db_dependency(session_maker=session_maker)
|
||||
db = Database("postgresql+asyncpg://postgres:postgres@localhost/app")
|
||||
|
||||
@router.get("/users")
|
||||
async def list_users(session: AsyncSession = Depends(get_db)):
|
||||
app = FastAPI()
|
||||
db.install(app) # commit middleware + engine disposal on shutdown
|
||||
|
||||
@app.get("/users")
|
||||
async def list_users(session: AsyncSession = Depends(db)):
|
||||
...
|
||||
```
|
||||
|
||||
## Session context manager
|
||||
The `Database` instance **is** the dependency: use it directly as `Depends(db)`. The whole request runs as a single transaction (CRUD writes use savepoints under it).
|
||||
|
||||
Use [`create_db_context`](../reference/db.md#fastapi_toolsets.db.create_db_context) for sessions outside request handlers (e.g. background tasks, CLI commands):
|
||||
## Committing before the response
|
||||
|
||||
[`db.install(app)`](../reference/db.md#fastapi_toolsets.db.Database) adds a middleware that commits the request's session when the response starts, after the endpoint returns and before the body is sent. With the middleware installed, the dependency does not commit again.
|
||||
|
||||
The request is committed as a single transaction:
|
||||
|
||||
- **Read-after-write**: a follow-up request sees the write.
|
||||
- **Atomicity**: multi-write endpoints roll back as a unit on failure.
|
||||
- **Errors roll back**: on a raised exception the session rolls back and nothing is committed.
|
||||
|
||||
Without `install`, the session commits in the dependency teardown, which runs after the response has been sent.
|
||||
|
||||
!!! warning "Streaming / SSE endpoints"
|
||||
For a `StreamingResponse` / `EventSourceResponse`, the commit fires at the **start** of the stream. A stream that **writes** must open a short-lived session per write with [`db.session()`](#session-context-manager); the start-time commit will not flush writes made later during the stream.
|
||||
|
||||
## Lifespan
|
||||
|
||||
`db.install(app)` disposes the engine on shutdown, composing around your own lifespan:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import create_db_context
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
db_context = create_db_context(session_maker=session_maker)
|
||||
@asynccontextmanager
|
||||
async def lifespan(app):
|
||||
await warm_cache() # your startup
|
||||
yield
|
||||
await flush_metrics() # your shutdown
|
||||
|
||||
app = FastAPI(lifespan=lifespan)
|
||||
db.install(app) # your shutdown runs first, then the engine is disposed
|
||||
```
|
||||
|
||||
If you have no lifespan of your own, [`db.lifespan`](../reference/db.md#fastapi_toolsets.db.Database) works standalone as `FastAPI(lifespan=db.lifespan)`. Engine disposal is idempotent and is a no-op when you passed your own `engine=`.
|
||||
|
||||
## Session context manager
|
||||
|
||||
Use [`db.session()`](../reference/db.md#fastapi_toolsets.db.Database) for sessions outside request handlers (e.g. background tasks, CLI commands). It commits on clean exit and rolls back on exception:
|
||||
|
||||
```python
|
||||
async def seed():
|
||||
async with db_context() as session:
|
||||
async with db.session() as session:
|
||||
...
|
||||
```
|
||||
|
||||
## Nested transactions
|
||||
## Transactions
|
||||
|
||||
[`get_transaction`](../reference/db.md#fastapi_toolsets.db.get_transaction) handles savepoints automatically, allowing safe nesting:
|
||||
[`transaction`](../reference/db.md#fastapi_toolsets.db.transaction) opens a transaction on a session, using a savepoint when one is already open so it nests safely:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import get_transaction
|
||||
from fastapi_toolsets.db import transaction
|
||||
|
||||
async def create_user_with_role(session=session):
|
||||
async with get_transaction(session=session):
|
||||
async def create_user_with_role(session):
|
||||
async with transaction(session):
|
||||
...
|
||||
async with get_transaction(session=session): # uses savepoint
|
||||
async with transaction(session): # uses a savepoint
|
||||
...
|
||||
```
|
||||
|
||||
When you have a `Database`, [`db.begin()`](../reference/db.md#fastapi_toolsets.db.Database) opens a session already inside a transaction:
|
||||
|
||||
```python
|
||||
async with db.begin() as session:
|
||||
session.add(User(name="ada")) # commits on exit, rolls back on exception
|
||||
```
|
||||
|
||||
## Table locking
|
||||
|
||||
[`lock_tables`](../reference/db.md#fastapi_toolsets.db.lock_tables) acquires PostgreSQL table-level locks before executing critical sections. It opens a **dedicated session** internally and yields it to the caller, so the lock is guaranteed to be released when the context exits:
|
||||
[`db.lock_tables`](../reference/db.md#fastapi_toolsets.db.Database) acquires PostgreSQL table-level locks for a critical section. It opens a dedicated session internally and releases the lock when the context exits:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import lock_tables, LockMode
|
||||
from fastapi_toolsets.db import LockMode
|
||||
|
||||
async with lock_tables(session_maker=session_maker, tables=[User], mode=LockMode.EXCLUSIVE) as session:
|
||||
async with db.lock_tables([User], mode=LockMode.EXCLUSIVE) as session:
|
||||
# No other transaction can modify User until this block exits
|
||||
...
|
||||
```
|
||||
|
||||
Available lock modes are defined in [`LockMode`](../reference/db.md#fastapi_toolsets.db.LockMode): `ACCESS_SHARE`, `ROW_SHARE`, `ROW_EXCLUSIVE`, `SHARE_UPDATE_EXCLUSIVE`, `SHARE`, `SHARE_ROW_EXCLUSIVE`, `EXCLUSIVE`, `ACCESS_EXCLUSIVE`.
|
||||
|
||||
Pass `timeout` to limit how long the lock waits before giving up. On timeout, a [`LockTimeoutError`](../reference/exceptions.md#fastapi_toolsets.exceptions.exceptions.LockTimeoutError) is raised instead of a raw database error:
|
||||
Pass `timeout` to limit how long the lock waits. On timeout, a [`LockTimeoutError`](../reference/exceptions.md#fastapi_toolsets.exceptions.exceptions.LockTimeoutError) is raised instead of a raw database error:
|
||||
|
||||
```python
|
||||
async with lock_tables(session_maker, [Order], timeout="2s") as session:
|
||||
async with db.lock_tables([Order], timeout="2s") as session:
|
||||
...
|
||||
```
|
||||
|
||||
## Advisory locking
|
||||
|
||||
[`advisory_lock`](../reference/db.md#fastapi_toolsets.db.advisory_lock) acquires a PostgreSQL session-level advisory lock. The lock is released explicitly when the context exits, regardless of whether the transaction has committed.
|
||||
[`advisory_lock`](../reference/db.md#fastapi_toolsets.db.advisory_lock) acquires a PostgreSQL session-level advisory lock on a session you provide. The lock is released when the context exits:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import advisory_lock
|
||||
|
||||
# Blocking exclusive lock — waits until the lock is free
|
||||
# Blocking exclusive lock: waits until the lock is free
|
||||
async with advisory_lock(session=session, key=42):
|
||||
...
|
||||
|
||||
# Non-blocking — yields False immediately if already held
|
||||
# Non-blocking: yields False immediately if already held
|
||||
async with advisory_lock(session=session, key=42, nowait=True) as acquired:
|
||||
if not acquired:
|
||||
raise HTTPException(409, "Resource is locked")
|
||||
|
||||
# Blocking with a timeout — raises LockTimeoutError if not acquired in time
|
||||
# Blocking with a timeout: raises LockTimeoutError if not acquired in time
|
||||
async with advisory_lock(session=session, key=42, timeout="5s"):
|
||||
...
|
||||
|
||||
# Shared — multiple readers allowed simultaneously, blocks exclusive writers
|
||||
# Shared lock: multiple readers allowed simultaneously, blocks exclusive writers
|
||||
async with advisory_lock(session=session, key=42, shared=True):
|
||||
...
|
||||
|
||||
@@ -106,11 +147,11 @@ async with advisory_lock(session=session, key=(1, user_id)):
|
||||
```
|
||||
|
||||
!!! note
|
||||
Advisory locks use PostgreSQL session-level functions (`pg_advisory_lock` / `pg_advisory_unlock`). The lock is tied to the database connection, not the SQLAlchemy transaction — it is released when the context exits, even if the surrounding transaction is still open.
|
||||
Advisory locks use PostgreSQL session-level functions (`pg_advisory_lock` / `pg_advisory_unlock`). The lock is tied to the database connection, not the SQLAlchemy transaction, so it is released when the context exits even if the surrounding transaction is still open.
|
||||
|
||||
## Row-change polling
|
||||
|
||||
[`wait_for_row_change`](../reference/db.md#fastapi_toolsets.db.wait_for_row_change) polls a row until a specific column changes value, useful for waiting on async side effects:
|
||||
[`wait_for_row_change`](../reference/db.md#fastapi_toolsets.db.wait_for_row_change) polls a row until a specific column changes value:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import wait_for_row_change
|
||||
@@ -120,7 +161,7 @@ await wait_for_row_change(
|
||||
session=session,
|
||||
model=Order,
|
||||
pk_value=order_id,
|
||||
columns=[Order.status],
|
||||
columns=["status"],
|
||||
interval=1.0,
|
||||
timeout=30.0,
|
||||
)
|
||||
@@ -128,28 +169,24 @@ await wait_for_row_change(
|
||||
|
||||
## Creating a database
|
||||
|
||||
!!! info "Added in `v2.1`"
|
||||
|
||||
[`create_database`](../reference/db.md#fastapi_toolsets.db.create_database) creates a database at a given URL. It connects to *server_url* and issues a `CREATE DATABASE` statement:
|
||||
[`create_database`](../reference/db.md#fastapi_toolsets.db.testing.create_database) (in `fastapi_toolsets.db.testing`) connects to *server_url* and issues a `CREATE DATABASE` statement:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import create_database
|
||||
from fastapi_toolsets.db.testing import create_database
|
||||
|
||||
SERVER_URL = "postgresql+asyncpg://postgres:postgres@localhost/postgres"
|
||||
|
||||
await create_database(db_name="myapp_test", server_url=SERVER_URL)
|
||||
```
|
||||
|
||||
For test isolation with automatic cleanup, use [`create_worker_database`](../reference/pytest.md#fastapi_toolsets.pytest.utils.create_worker_database) from the `pytest` module instead — it handles drop-before, create, and drop-after automatically.
|
||||
For test isolation with automatic cleanup, use [`create_worker_database`](../reference/pytest.md#fastapi_toolsets.pytest.utils.create_worker_database) from the `pytest` module, which handles drop-before, create, and drop-after.
|
||||
|
||||
## Cleaning up tables
|
||||
|
||||
!!! info "Added in `v2.1`"
|
||||
|
||||
[`cleanup_tables`](../reference/db.md#fastapi_toolsets.db.cleanup_tables) truncates all tables:
|
||||
[`cleanup_tables`](../reference/db.md#fastapi_toolsets.db.testing.cleanup_tables) (in `fastapi_toolsets.db.testing`) truncates all tables:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import cleanup_tables
|
||||
from fastapi_toolsets.db.testing import cleanup_tables
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
async def clean(db_session):
|
||||
@@ -159,50 +196,50 @@ async def clean(db_session):
|
||||
|
||||
## Many-to-Many helpers
|
||||
|
||||
SQLAlchemy's ORM collection API triggers lazy-loads when you append to a relationship inside a savepoint (e.g. inside `lock_tables` or a nested `get_transaction`). The three `m2m_*` helpers bypass the ORM collection entirely and issue direct SQL against the association table.
|
||||
The three `m2m_*` helpers modify a many-to-many association table with direct SQL, without loading the ORM collection.
|
||||
|
||||
### `m2m_add` — insert associations
|
||||
### `m2m_add`: insert associations
|
||||
|
||||
[`m2m_add`](../reference/db.md#fastapi_toolsets.db.m2m_add) inserts one or more rows into a secondary table without touching the ORM collection:
|
||||
[`m2m_add`](../reference/db.md#fastapi_toolsets.db.m2m_add) inserts one or more rows into a secondary table:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import lock_tables, m2m_add
|
||||
from fastapi_toolsets.db import m2m_add
|
||||
|
||||
async with lock_tables(session_maker, [Tag]) as session:
|
||||
async with db.lock_tables([Tag]) as session:
|
||||
tag = await TagCrud.create(session, TagCreate(name="python"))
|
||||
await m2m_add(session, post, Post.tags, tag)
|
||||
```
|
||||
|
||||
Pass `ignore_conflicts=True` to silently skip associations that already exist:
|
||||
Pass `ignore_conflicts=True` to skip associations that already exist:
|
||||
|
||||
```python
|
||||
await m2m_add(session, post, Post.tags, tag, ignore_conflicts=True)
|
||||
```
|
||||
|
||||
### `m2m_remove` — delete associations
|
||||
### `m2m_remove`: delete associations
|
||||
|
||||
[`m2m_remove`](../reference/db.md#fastapi_toolsets.db.m2m_remove) deletes specific association rows. Removing a non-existent association is a no-op:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import get_transaction, m2m_remove
|
||||
from fastapi_toolsets.db import m2m_remove, transaction
|
||||
|
||||
async with get_transaction(session):
|
||||
async with transaction(session):
|
||||
await m2m_remove(session, post, Post.tags, tag1, tag2)
|
||||
```
|
||||
|
||||
### `m2m_set` — replace the full set
|
||||
### `m2m_set`: replace the full set
|
||||
|
||||
[`m2m_set`](../reference/db.md#fastapi_toolsets.db.m2m_set) atomically replaces all associations: it deletes every existing row for the owner instance then inserts the new set. Passing no related instances clears the association entirely:
|
||||
[`m2m_set`](../reference/db.md#fastapi_toolsets.db.m2m_set) replaces all associations: it deletes every existing row for the owner instance then inserts the new set. Passing no related instances clears the association:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import get_transaction, m2m_set
|
||||
from fastapi_toolsets.db import m2m_set, transaction
|
||||
|
||||
# Replace all tags
|
||||
async with get_transaction(session):
|
||||
async with transaction(session):
|
||||
await m2m_set(session, post, Post.tags, tag_a, tag_b)
|
||||
|
||||
# Clear all tags
|
||||
async with get_transaction(session):
|
||||
async with transaction(session):
|
||||
await m2m_set(session, post, Post.tags)
|
||||
```
|
||||
|
||||
|
||||
@@ -134,7 +134,7 @@ SessionLocal = async_sessionmaker(engine, expire_on_commit=False, class_=EventSe
|
||||
```
|
||||
|
||||
!!! info "Callbacks fire on `session.commit()` only — not on savepoints."
|
||||
Savepoints created by [`get_transaction`](db.md) or `begin_nested()` do **not**
|
||||
Savepoints created by [`transaction`](db.md) or `begin_nested()` do **not**
|
||||
trigger callbacks. All events accumulated across flushes are dispatched once
|
||||
when the outermost `commit()` is called.
|
||||
|
||||
|
||||
@@ -107,10 +107,10 @@ url = worker_database_url("postgresql+asyncpg://user:pass@localhost/myapp", defa
|
||||
|
||||
## Manual table cleanup
|
||||
|
||||
[`cleanup_tables`](../reference/db.md#fastapi_toolsets.db.cleanup_tables) truncates all tables in a single statement and can be called directly when you need more control:
|
||||
[`cleanup_tables`](../reference/db.md#fastapi_toolsets.db.testing.cleanup_tables) truncates all tables in a single statement and can be called directly when you need more control:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import cleanup_tables
|
||||
from fastapi_toolsets.pytest import cleanup_tables
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
async def clean(db_session):
|
||||
|
||||
+20
-18
@@ -1,46 +1,48 @@
|
||||
# `db`
|
||||
|
||||
Here's the reference for all database session utilities, transaction helpers, and locking functions.
|
||||
Here's the reference for the `Database` facade, the transaction helper, locking
|
||||
functions, many-to-many helpers, and row-watching utilities.
|
||||
|
||||
You can import them directly from `fastapi_toolsets.db`:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db import (
|
||||
Database,
|
||||
LockMode,
|
||||
advisory_lock,
|
||||
cleanup_tables,
|
||||
create_database,
|
||||
create_db_dependency,
|
||||
create_db_context,
|
||||
get_transaction,
|
||||
lock_tables,
|
||||
m2m_add,
|
||||
m2m_remove,
|
||||
m2m_set,
|
||||
transaction,
|
||||
wait_for_row_change,
|
||||
)
|
||||
```
|
||||
|
||||
## ::: fastapi_toolsets.db.Database
|
||||
|
||||
## ::: fastapi_toolsets.db.transaction
|
||||
|
||||
## ::: fastapi_toolsets.db.LockMode
|
||||
|
||||
## ::: fastapi_toolsets.db.create_db_dependency
|
||||
|
||||
## ::: fastapi_toolsets.db.create_db_context
|
||||
|
||||
## ::: fastapi_toolsets.db.get_transaction
|
||||
|
||||
## ::: fastapi_toolsets.db.lock_tables
|
||||
|
||||
## ::: fastapi_toolsets.db.advisory_lock
|
||||
|
||||
## ::: fastapi_toolsets.db.wait_for_row_change
|
||||
|
||||
## ::: fastapi_toolsets.db.create_database
|
||||
|
||||
## ::: fastapi_toolsets.db.cleanup_tables
|
||||
|
||||
## ::: fastapi_toolsets.db.m2m_add
|
||||
|
||||
## ::: fastapi_toolsets.db.m2m_remove
|
||||
|
||||
## ::: fastapi_toolsets.db.m2m_set
|
||||
|
||||
## ::: fastapi_toolsets.db.wait_for_row_change
|
||||
|
||||
Admin and test helpers live in `fastapi_toolsets.db.testing`:
|
||||
|
||||
```python
|
||||
from fastapi_toolsets.db.testing import cleanup_tables, create_database
|
||||
```
|
||||
|
||||
## ::: fastapi_toolsets.db.testing.create_database
|
||||
|
||||
## ::: fastapi_toolsets.db.testing.cleanup_tables
|
||||
|
||||
Reference in New Issue
Block a user