Handle errors, retries and rate limits
What the SDK does when axonpush is unreachable, which errors it retries and which it raises, and how to configure all of it from the environment.
Observability that can take your application down is worse than no
observability. The SDKs default to failing open: a network problem produces a
warning and a None, not an exception in the middle of your agent.
Configuration problems are the opposite. A bad key or a malformed request raises, because those are your bugs and silence would hide them.
Fail-open
from axonpush import AxonPush
# fail_open defaults to True
with AxonPush() as client:
event = client.events.publish(
"web_search", {"query": "AI agents"}, channel_id="ch_...",
)
# axonpush unreachable → event is None, your code carries onFail-open covers transport failures only, DNS, connection refused, read
timeouts, and anything else that means the request never got an answer. Those
become APIConnectionError, and with fail_open=True the SDK logs a warning
and returns None (or [] for list operations) instead of raising.
An HTTP response, even a bad one, is an answer. 401, 422 and 429 all
raise regardless of fail_open, the API was reachable and told you something
you need to hear.
Set fail_open=False (or AXONPUSH_FAIL_OPEN=false) in tests, where you want a
missing event to be loud.
Framework integrations go further: they suppress everything, always. An
observability failure inside a LangChain callback or an OpenAI Agents hook can
never break the pipeline it is watching, whatever fail_open says.
The exception tree
AxonPushError
├── RetryableError → retried automatically
│ ├── APIConnectionError → transport failure (no HTTP response)
│ ├── RateLimitError → HTTP 429, carries retry_after
│ └── ServerError → HTTP 5xx
├── AuthenticationError → HTTP 401
├── ForbiddenError → HTTP 403
├── NotFoundError → HTTP 404
└── ValidationError → HTTP 422 (see below)The two SDKs disagree about 400, and it is worth knowing which one you are
holding. TypeScript raises ValidationError for both 400 and 422.
Python raises ValidationError for 422, or for any 4xx whose body
carries code: "validation_error", a plain 400 becomes the base
AxonPushError.
Several 400s are ones you will actually hit: an unknown environment slug, a
forbidden environment override, an unroutable OTLP request. In Python, catch
AxonPushError for those. Any status not listed above, 400, 408, 409,
413, is a base AxonPushError in Python and is not retried in either
SDK.
Every exception carries the backend’s structured error envelope:
| Attribute | From |
|---|---|
message | message in the body. A list of validation messages is joined with ; . |
status_code | The HTTP status. |
code | code in the body, the stable machine-readable identifier, e.g. unknown_environment, env_override_forbidden, slug_taken, env_cap_exceeded. |
hint | hint in the body, when the server has a remediation to offer. |
request_id | requestId in the body, falling back to the X-Request-Id header. |
retry_after | RateLimitError only, parsed from Retry-After. |
code and request_id are the two to log. Branch on code, not on the
message text.
from axonpush import AxonPush
from axonpush.exceptions import (
AuthenticationError,
AxonPushError,
RateLimitError,
ServerError,
)
with AxonPush() as client:
try:
client.events.publish("web_search", {"q": "…"}, channel_id="ch_...")
except RateLimitError as e:
log.warning("rate limited, retry after %ss (request %s)", e.retry_after, e.request_id)
except AuthenticationError as e:
log.error("bad credentials: %s", e.message)
except ServerError as e:
log.error("axonpush unavailable (request %s)", e.request_id)
except AxonPushError as e:
log.error("axonpush rejected the call: code=%s hint=%s", e.code, e.hint)Retries are already handled
You do not need a retry wrapper. The transport retries every RetryableError
before it ever reaches you.
- Retried on: connection errors,
429, and5xx, that is, anything the SDK classes as aRetryableError. Nothing else is retried. - Attempts:
max_retriesadditional attempts after the first. Default 3, so four calls in the worst case. - Backoff: a fixed ladder of 250 ms, 500 ms, 1 s, 2 s, 4 s, clamped at the last entry. Not exponential-with-jitter, a predictable ladder, so a stalled call has a knowable worst-case duration.
Retry-Afterwins. On a429carrying the header, the server’s number is used instead of the ladder.
A RateLimitError that reaches your code has therefore already been retried and
still failed. Backing off further in your own handler is reasonable; retrying
immediately is not.
The retry helpers run each request inside an OpenTelemetry context that sets
suppress_instrumentation, so an HTTP instrumentor on httpx, urllib3,
aiohttp or fetch does not create a span for the SDK’s own calls. Without
that, publishing a span would generate a span, which would be published. If you
see recursive telemetry, something is bypassing the SDK transport.
Configuration
Everything is settable in code or from the environment. Constructor arguments win; otherwise the variable is read.
| Variable | Type | Default |
|---|---|---|
AXONPUSH_API_KEY | string | |
AXONPUSH_TENANT_ID | string | , (AXONPUSH_ORG_ID also accepted) |
AXONPUSH_APP_ID | string | |
AXONPUSH_CHANNEL_ID | string | |
AXONPUSH_BASE_URL | string | https://api.axonpush.xyz |
AXONPUSH_ENVIRONMENT | string | |
AXONPUSH_TIMEOUT | seconds | 30 |
AXONPUSH_MAX_RETRIES | integer | 3 |
AXONPUSH_FAIL_OPEN | boolean | true |
AXONPUSH_CONTENT_CAPTURE | metadata_only / redacted / full | metadata_only |
AXONPUSH_REDACT_KEYS | comma-separated | empty |
AXONPUSH_MAX_CONTENT_LENGTH | characters | 4096 |
The last three are client-side redaction, applied before anything leaves the process. They mirror the server-side telemetry policy described in the privacy boundary, and both apply, the SDK strips first, the server strips again.
client = AxonPush(
fail_open=False,
max_retries=5,
timeout=10.0,
content_capture_mode="redacted",
redact_keys=["ssn", "card_number"],
)Recording a failure in the trace
The useful pattern is not retrying, it is making sure the failure is visible in the trace before you re-raise:
from axonpush import EventType, get_or_create_trace
from axonpush.exceptions import AxonPushError
trace = get_or_create_trace()
try:
result = call_external_tool()
except Exception as exc:
try:
client.events.publish(
"tool_failure",
{"error": str(exc), "tool": "web_search"},
channel_id="ch_...",
agent_id="researcher",
event_type=EventType.AGENT_ERROR,
)
except AxonPushError:
pass # never let the observability call mask the real failure
raiseThe inner except AxonPushError: pass matters even with fail_open=True,
because fail-open does not cover a 403 from a key that is missing the
publish scope.
An agent.error published this way is what
webhooks and alert rules fire on, and what shows up
red on the trace in Observe.
Async
import asyncio
from axonpush import AsyncAxonPush
from axonpush.exceptions import RateLimitError
async with AsyncAxonPush() as client:
try:
await client.events.publish("web_search", {"q": "…"}, channel_id="ch_...")
except RateLimitError as e:
await asyncio.sleep(e.retry_after or 1)AsyncAxonPush detects when the running event loop has changed and rebuilds its
HTTP client. A process that calls asyncio.run(...) more than once, a Lambda
handler, an SQS worker, a Celery task under eventlet, would otherwise hang on
the first request against a connection pool bound to a closed loop.
Next
- Get notified when your agent fails, pushing the failure somewhere
- Environments, the source of most
400s worth understanding - Events (Python SDK), the publish surface these errors come from
Get notified when your agent fails
Two different mechanisms, per-event webhooks that fire on every match, and alert rules that fire when a trace crosses a threshold. How to pick, how to verify the signature, and how to see what was delivered.
API reference
The axonpush HTTP API — publish events, manage channels, apps and keys, read traces and analytics. Every endpoint, grouped by resource.