Data / News

News data

Digital asset and macro news flow, collected continuously, collapsed to one item per event, and summarized into text that stands on its own. Structured enough to join against market data, readable enough to put in front of a user.

What this data is

A stream of discrete events — an exchange hack, a fund flow figure, a regulatory decision, a macro print that moves risk assets — each represented once, with a timestamp and a summary. The feed is market-wide rather than per-asset: it describes what happened, not what happened to a specific ticker.

The current window holds the most recent 200 items, which in practice spans about ten to twelve days of flow.

The processing chain

News is the category where raw input is furthest from useful output. The same event arrives from a dozen outlets within an hour, in a dozen registers, with a dozen headlines. Everything below exists to get from that to one row.

Sources
crypto outletsmacro wiresprimary announcements
Articles
fetchextract textrecord publication time
Deduplicationthe hard part
cluster near-identical coveragekeep one canonical itemretain source count
Entity extraction
organisationsprotocolsassetsamounts
Asset & topic mapping
entity → canonical assetevent → category
Summarization
self-contained summaryjargon explained inline
Delivery
hourly feedapplications

Deduplication

Deduplication is not string matching on headlines. Two outlets covering the same exploit will share almost no headline tokens; two unrelated exploits in the same week will share most of them. Clustering therefore has to work on what the article is about — the entities involved, the figures quoted, the time the event occurred — rather than on how it is worded.

The count of contributing sources is retained, and it is informative in its own right: an event covered by eighteen outlets is a different signal from one covered by two. That count is what the narrative layer ranks on.

Asset mapping

Mapping an extracted entity to a canonical asset is the same problem as entity resolution, in a harder form: prose does not carry identifiers. "Cronos" in an article can mean the chain, the token, or the company. Mapping is therefore conservative — an ambiguous mention stays unmapped rather than being attached to the wrong asset, because a wrong asset attribution propagates into every downstream view.

Sentiment and relevance

Items carry an editorial tone that can be classified, and a relevance ordering driven by recency and source breadth. Both are treated as ranking aids for presentation, not as tradeable signals, and neither is exposed as a standalone score.

Update frequency

Hourly. Measured across a full day of the live feed, an item is present in every hour bucket from 00 to 23 UTC. Freshness of the newest item is therefore typically under an hour.

Available fields

idintegerStable identifier of the news item. Monotonically increasing, usable as a cursor.
generatedMessagestringA generated plain-language summary of the event, written to be self-contained: it names the entities involved and explains jargon inline rather than assuming it.
mediaUrlurlIllustrative image for the item, served from our own media origin.
createdDatetimestampUTC timestamp at which the item entered the feed.
news feed item
[
  {
    "id": 14825,
    "generatedMessage": "Crypto investment funds attracted $3.2 billion in net inflows last week, the largest weekly amount since October 2025 …",
    "mediaUrl": "https://image.example/og97bk.jpg",
    "createdDate": "2026-08-31T05: 00: 04.011Z"
  }
]

Limitations

  • Rolling window, not an archive. The feed exposes the most recent 200 items. It is not a historical news corpus and is not intended for long-range backtesting.
  • Summaries are generated. Item text is machine-written from source coverage. It is written to be accurate and conservative, but it is a derived artefact and can be wrong where the underlying coverage was wrong or contradictory.
  • No article redistribution. Source article text is not stored or served. What the feed carries is a summary of the event, which is what keeps the feed clear of publisher content rights. Full-text access is a matter for the publishers.
  • Coverage bias. English-language and crypto-native outlets dominate the input. Regional events surface later and less completely.
  • Unmapped items exist. An event with no unambiguous asset is delivered without an asset link. That is a deliberate choice in favour of precision.

Which applications use it

The feed backs the flash news surface on moonboard.ai and supplies the article flow that the narrative clustering runs over.

Related