axonpush
Recipes

Handle errors, retries and rate limits

What the SDK does when axonpush is unreachable, which errors it retries and which it raises, and how to configure all of it from the environment.

Observability that can take your application down is worse than no observability. The SDKs default to failing open: a network problem produces a warning and a None, not an exception in the middle of your agent.

Configuration problems are the opposite. A bad key or a malformed request raises, because those are your bugs and silence would hide them.

Fail-open

from axonpush import AxonPush

# fail_open defaults to True
with AxonPush() as client:
    event = client.events.publish(
        "web_search", {"query": "AI agents"}, channel_id="ch_...",
    )
    # axonpush unreachable → event is None, your code carries on

Fail-open covers transport failures only, DNS, connection refused, read timeouts, and anything else that means the request never got an answer. Those become APIConnectionError, and with fail_open=True the SDK logs a warning and returns None (or [] for list operations) instead of raising.

An HTTP response, even a bad one, is an answer. 401, 422 and 429 all raise regardless of fail_open, the API was reachable and told you something you need to hear.

Set fail_open=False (or AXONPUSH_FAIL_OPEN=false) in tests, where you want a missing event to be loud.

Framework integrations go further: they suppress everything, always. An observability failure inside a LangChain callback or an OpenAI Agents hook can never break the pipeline it is watching, whatever fail_open says.

The exception tree

AxonPushError
├── RetryableError            → retried automatically
│   ├── APIConnectionError    → transport failure (no HTTP response)
│   ├── RateLimitError        → HTTP 429, carries retry_after
│   └── ServerError           → HTTP 5xx
├── AuthenticationError       → HTTP 401
├── ForbiddenError            → HTTP 403
├── NotFoundError             → HTTP 404
└── ValidationError           → HTTP 422 (see below)

The two SDKs disagree about 400, and it is worth knowing which one you are holding. TypeScript raises ValidationError for both 400 and 422. Python raises ValidationError for 422, or for any 4xx whose body carries code: "validation_error", a plain 400 becomes the base AxonPushError.

Several 400s are ones you will actually hit: an unknown environment slug, a forbidden environment override, an unroutable OTLP request. In Python, catch AxonPushError for those. Any status not listed above, 400, 408, 409, 413, is a base AxonPushError in Python and is not retried in either SDK.

Every exception carries the backend’s structured error envelope:

AttributeFrom
messagemessage in the body. A list of validation messages is joined with ; .
status_codeThe HTTP status.
codecode in the body, the stable machine-readable identifier, e.g. unknown_environment, env_override_forbidden, slug_taken, env_cap_exceeded.
hinthint in the body, when the server has a remediation to offer.
request_idrequestId in the body, falling back to the X-Request-Id header.
retry_afterRateLimitError only, parsed from Retry-After.

code and request_id are the two to log. Branch on code, not on the message text.

from axonpush import AxonPush
from axonpush.exceptions import (
  AuthenticationError,
  AxonPushError,
  RateLimitError,
  ServerError,
)

with AxonPush() as client:
  try:
      client.events.publish("web_search", {"q": "…"}, channel_id="ch_...")
  except RateLimitError as e:
      log.warning("rate limited, retry after %ss (request %s)", e.retry_after, e.request_id)
  except AuthenticationError as e:
      log.error("bad credentials: %s", e.message)
  except ServerError as e:
      log.error("axonpush unavailable (request %s)", e.request_id)
  except AxonPushError as e:
      log.error("axonpush rejected the call: code=%s hint=%s", e.code, e.hint)

Retries are already handled

You do not need a retry wrapper. The transport retries every RetryableError before it ever reaches you.

  • Retried on: connection errors, 429, and 5xx, that is, anything the SDK classes as a RetryableError. Nothing else is retried.
  • Attempts: max_retries additional attempts after the first. Default 3, so four calls in the worst case.
  • Backoff: a fixed ladder of 250 ms, 500 ms, 1 s, 2 s, 4 s, clamped at the last entry. Not exponential-with-jitter, a predictable ladder, so a stalled call has a knowable worst-case duration.
  • Retry-After wins. On a 429 carrying the header, the server’s number is used instead of the ladder.

A RateLimitError that reaches your code has therefore already been retried and still failed. Backing off further in your own handler is reasonable; retrying immediately is not.

The retry helpers run each request inside an OpenTelemetry context that sets suppress_instrumentation, so an HTTP instrumentor on httpx, urllib3, aiohttp or fetch does not create a span for the SDK’s own calls. Without that, publishing a span would generate a span, which would be published. If you see recursive telemetry, something is bypassing the SDK transport.

Configuration

Everything is settable in code or from the environment. Constructor arguments win; otherwise the variable is read.

VariableTypeDefault
AXONPUSH_API_KEYstring
AXONPUSH_TENANT_IDstring, (AXONPUSH_ORG_ID also accepted)
AXONPUSH_APP_IDstring
AXONPUSH_CHANNEL_IDstring
AXONPUSH_BASE_URLstringhttps://api.axonpush.xyz
AXONPUSH_ENVIRONMENTstring
AXONPUSH_TIMEOUTseconds30
AXONPUSH_MAX_RETRIESinteger3
AXONPUSH_FAIL_OPENbooleantrue
AXONPUSH_CONTENT_CAPTUREmetadata_only / redacted / fullmetadata_only
AXONPUSH_REDACT_KEYScomma-separatedempty
AXONPUSH_MAX_CONTENT_LENGTHcharacters4096

The last three are client-side redaction, applied before anything leaves the process. They mirror the server-side telemetry policy described in the privacy boundary, and both apply, the SDK strips first, the server strips again.

client = AxonPush(
    fail_open=False,
    max_retries=5,
    timeout=10.0,
    content_capture_mode="redacted",
    redact_keys=["ssn", "card_number"],
)

Recording a failure in the trace

The useful pattern is not retrying, it is making sure the failure is visible in the trace before you re-raise:

from axonpush import EventType, get_or_create_trace
from axonpush.exceptions import AxonPushError

trace = get_or_create_trace()

try:
    result = call_external_tool()
except Exception as exc:
    try:
        client.events.publish(
            "tool_failure",
            {"error": str(exc), "tool": "web_search"},
            channel_id="ch_...",
            agent_id="researcher",
            event_type=EventType.AGENT_ERROR,
        )
    except AxonPushError:
        pass  # never let the observability call mask the real failure
    raise

The inner except AxonPushError: pass matters even with fail_open=True, because fail-open does not cover a 403 from a key that is missing the publish scope.

An agent.error published this way is what webhooks and alert rules fire on, and what shows up red on the trace in Observe.

Async

import asyncio

from axonpush import AsyncAxonPush
from axonpush.exceptions import RateLimitError

async with AsyncAxonPush() as client:
    try:
        await client.events.publish("web_search", {"q": "…"}, channel_id="ch_...")
    except RateLimitError as e:
        await asyncio.sleep(e.retry_after or 1)

AsyncAxonPush detects when the running event loop has changed and rebuilds its HTTP client. A process that calls asyncio.run(...) more than once, a Lambda handler, an SQS worker, a Celery task under eventlet, would otherwise hang on the first request against a connection pool bound to a closed loop.

Next