feat: add startup retry with exponential backoff for Redis connection (#20)

This commit is contained in:
d3vyce
2026-05-05 20:21:43 +02:00
committed by GitHub
parent a36600c3e5
commit f586342811
5 changed files with 126 additions and 2 deletions
+1
View File
@@ -56,6 +56,7 @@ except DuplicateTaskError:
- **Explicit lock key** — pin any task to a fixed Redis key with `deduplication_key`, bypassing fingerprint computation entirely.
- **Partial fingerprint** — deduplicate on a subset of kwargs with `deduplication_key_fields`, ignoring irrelevant arguments.
- **Per-task opt-out** — disable deduplication for individual tasks with the `deduplication` label.
- **Startup resilience** — automatic reconnection with exponential backoff if Redis is unavailable at broker startup.
## License
+21
View File
@@ -21,6 +21,8 @@ broker = ListQueueBroker("redis://localhost:6379").with_middlewares(
| `default_deduplication` | `bool` | `True` | Whether deduplication is enabled for all tasks by default. Set `False` to opt-in per task instead of opting out. |
| `default_ttl` | `int` | `300` | Default lock TTL in seconds. Overridden per task with the `deduplication_ttl` label. |
| `key_prefix` | `str` | `"taskiq:deduplication"` | Prefix for all Redis lock keys. |
| `startup_retries` | `int` | `3` | Number of connection attempts during broker startup. |
| `startup_retry_delay` | `float` | `1.0` | Base delay in seconds between retries (exponential backoff: delay × 2^n). |
```python
broker = ListQueueBroker("redis://localhost:6379").with_middlewares(
@@ -29,10 +31,29 @@ broker = ListQueueBroker("redis://localhost:6379").with_middlewares(
default_deduplication=True,
default_ttl=60,
key_prefix="myapp:dedup",
startup_retries=5,
startup_retry_delay=0.5,
),
)
```
## Startup resilience
On startup the middleware verifies the Redis connection with a `PING`. If Redis is
temporarily unavailable, it retries with exponential backoff.
After all attempts are exhausted a `ConnectionError` is raised and the broker
fails to start.
Adjust `startup_retries` and `startup_retry_delay` to suit your deployment:
```python
RedisDeduplicationMiddleware(
redis_url="redis://localhost:6379",
startup_retries=5,
startup_retry_delay=2.0,
)
```
## How it works
When a task is dispatched, the middleware acquires a Redis lock keyed on the task's