Kafka streaming
Consume SignalWise alerts in near real time from your Kafka topic.
SignalWise publishes alerts to topics on Confluent Cloud. You get read-only access to your topic(s).
What you receive
From Bridgewise, through a secure one-time link:
- Bootstrap server
- API key and secret (SASL username and password)
- Schema Registry URL, with its own key and secret
- Your topic name(s)
Topics
| Topic | Content |
|---|---|
alerts-reasoning-external | Shared feed: alerts with reasoning |
alerts-reasoning-external-<your-name> | Your dedicated feed, filtered to your universe and settings |
personalized-alerts-<your-name> | Personalized feed: one message per user (Personalized plan) |
user-portfolio-positions-external | Input: where you publish your users' holdings (Personalized plan) |
On staging, topic names start with stage-.
Connection settings
bootstrap.servers=<bootstrap-server>
security.protocol=SASL_SSL
sasl.mechanisms=PLAIN
sasl.username=<api-key>
sasl.password=<api-secret>
session.timeout.ms=45000Example consumer
from confluent_kafka import DeserializingConsumer, KafkaError, KafkaException
from confluent_kafka.schema_registry import SchemaRegistryClient
from confluent_kafka.schema_registry.json_schema import JSONDeserializer
TOPIC = "alerts-reasoning-external"
registry = SchemaRegistryClient({
"url": "<schema-registry-url>",
"basic.auth.user.info": "<registry-key>:<registry-secret>",
})
schema = registry.get_latest_version(f"{TOPIC}-value").schema.schema_str
consumer = DeserializingConsumer({
"bootstrap.servers": "<bootstrap-server>",
"security.protocol": "SASL_SSL",
"sasl.mechanisms": "PLAIN",
"sasl.username": "<api-key>",
"sasl.password": "<api-secret>",
"group.id": "my-signalwise-consumer",
"auto.offset.reset": "latest",
"isolation.level": "read_committed",
"value.deserializer": JSONDeserializer(schema, lambda obj, ctx: obj),
})
consumer.subscribe([TOPIC])
try:
while True:
msg = consumer.poll(timeout=1.0)
if msg is None:
continue
if msg.error():
if msg.error().code() == KafkaError._PARTITION_EOF:
continue
raise KafkaException(msg.error())
alert = msg.value() # already a dict
print(alert["eventMetadata"]["eventType"], alert["assetsDetails"]["tickerSymbol"])
finally:
consumer.close()Message wire format
Messages use the Confluent JSON wire format:
| Bytes | Content |
|---|---|
| 1 | Magic byte 0x00 |
| 2–5 | Schema ID |
| 6+ | JSON payload |
- Use a Schema Registry–aware deserializer when you can.
- Read the schema by subject and latest version, never by a hard-coded schema ID. Several schema IDs can be in use.
Best practices
Ignore transaction control records
Topics are written with Kafka transactions. Control records also start with 0x00, and a deserializer can return a single leftover byte (for example 0x08) instead of an alert. Use a client that supports read_committed, and skip records whose key is 4 bytes and value is 6 bytes. Do not filter on the byte value itself: it changes over time.
- Deduplicate on
uniqueEventId. It is the unique ID of the alert. Store it, and quote it in support requests. - Don't rely on field order or fixed lengths. Text fields can be longer than in the examples. Avoid tight database limits on names and texts.
- Expect new optional fields. Ignore fields you don't use.
- Cache logos.
logoUrlis a signed URL that expires. Download the image if you need it long term.