Skip to content
← 블로그

Where to run your 24/7 stream consumer, and how to keep it alive

dino.markets

A browser tab works well for watching a price gap open, but it can't catch one while you're asleep. The median cross-venue gap closes in about 9.2 seconds, and the connect ticket that gets you onto the stream expires in about 120 seconds. Nothing in that math leaves room for a process that starts back up on its own schedule. Whatever reads the stream needs to already be running when the gap opens, every time, including at 3am and after a power blip.

None of this touches order placement. This is developer data, not trading or financial advice. What follows is only about keeping a process alive.

The consumer core, in Python

The shape is the same wherever it runs: mint a ticket, connect, read events off the socket, and reconnect when the ticket runs out.

import json
import os
import time
import requests
import websocket

API_KEY = os.environ["DINO_API_KEY"]  # read from a 0600 env file, never hardcoded

def mint_ticket():
    resp = requests.post(
        "https://api.dino.markets/v1/stream/token",
        headers={"Authorization": f"Bearer {API_KEY}"},
    )
    return resp.json()

def run():
    while True:
        mint = mint_ticket()
        ws = websocket.create_connection(mint["ws_url"])
        ws.send(json.dumps({"id": 1, "connect": {"data": {"t": mint["ticket"]}}}))
        try:
            while True:
                # one WS frame can carry several newline-separated objects
                for line in ws.recv().split("\n"):
                    if not line:
                        continue
                    msg = json.loads(line)
                    if msg == {}:
                        ws.send("{}")  # server ping, pong straight back or the connection drops
                        continue
                    data = (msg.get("push") or {}).get("pub", {}).get("data")
                    if data and data.get("type") == "price":
                        handle_gap(data)
        except Exception:
            pass
        finally:
            ws.close()
        time.sleep(1)

def handle_gap(event):
    # forward each parsed event to wherever your alerting or storage lives
    print(event)

if __name__ == "__main__":
    run()

The ping handling inside the loop matters as much as the reconnect logic around it: the server checks for a live client roughly every 25 seconds, and a process that doesn't answer gets disconnected long before a 120-second ticket would have expired on its own. The three hosts below only change what keeps this loop running; the loop itself stays the same.

A Mac mini you already own

If there's a Mac mini or an old Mac sitting somewhere already, launchd will keep this script running across crashes and reboots without any extra software. A short plist with KeepAlive set does the job (these keys go inside the plist's top-level <dict>):

<key>Label</key><string>markets.dino.stream-consumer</string>
<key>ProgramArguments</key>
<array>
  <string>/bin/sh</string>
  <string>-c</string>
  <string>set -a; . /usr/local/etc/dino/env; set +a; exec /usr/bin/python3 /usr/local/bin/consumer.py</string>
</array>
<key>KeepAlive</key><true/>
<key>RunAtLoad</key><true/>
<key>StandardErrorPath</key><string>/tmp/dino-consumer.err</string>

The plist never carries the key itself. A wrapper sources it from an env file on disk (chmod 600) before the script starts, so nothing sensitive sits in a config file launchd loads. When the socket drops, the process's own reconnect loop mints a new ticket and reconnects; launchd only steps in if the whole process dies outright.

A cheap VPS

A small box works the same way, with systemd standing in for launchd. A unit file with Restart=on-failure restarts the process if it exits, and journalctl gives you the logs without setting anything else up:

[Unit]
Description=dino.markets stream consumer
After=network-online.target

[Service]
EnvironmentFile=/etc/dino/env
ExecStart=/usr/bin/python3 /opt/dino/consumer.py
Restart=on-failure
RestartSec=2

[Install]
WantedBy=multi-user.target

/etc/dino/env holds DINO_API_KEY=sk_live_... and should be chmod 600, owned by the user the service runs as. Reconnects happen inside the script; systemd only restarts the process if it exits on its own. Check on it with journalctl -u dino-consumer -f.

Cloudflare Durable Objects

A Durable Object skips the box entirely. There's no OS to patch and no plist or unit file, since Cloudflare runs the process for you. The tradeoff is billing. Durable Objects have a hibernation API that stops billing an idle object while it waits for an incoming connection, but that only covers sockets other people open to reach it. A consumer here runs the other direction: your object holds an outbound WebSocket open to our stream, and an outbound socket doesn't hibernate. For an always-on consumer, the object stays active continuously and bills accordingly. That's worth knowing before you pick this option; check Cloudflare's own pricing page for the actual number. Reconnects work the same way: mint another ticket and reopen the socket when it runs out.

What you already own versus what you want to operate

What you're running it onWhat you're signing up to operate
Mac minihardware already on the deskyour own power and network uptime
VPSa small, predictable monthly billpatching the OS yourself
Durable Objectsnothing to patchcontinuous billing for an always-open socket

None of these is wrong. If a Mac mini is already on the desk, it costs nothing extra to try first, and a VPS is worth reaching for once uptime starts to matter more than convenience. Durable Objects earns its keep once patching a box yourself is no longer worth your time.

The stream this consumer reads is on Basic and up. A Free key connects too, but only to a small curated sample of markets, not the full multi-sport feed this loop is built for. The docs cover the full connect flow.

Once a consumer is running somewhere, the last piece is usually turning a qualifying gap into something that reaches you directly instead of sitting in a log file. That's covered next, in sending a Telegram alert the moment a gap opens.

블로그의 다른 글