Alternative data: job ads, Reddit, X, satellites and card payments

In depth12 min read · 32 sources

Alternative data is information about companies and markets beyond prices, financial statements and official statistics: job ads, posts on Reddit and X, satellite images, card payments, app downloads or location data. It is often available long before the quarterly figures, but it is expensive, patchy and legally delicate. Here you will learn how the most important types work, what the research shows about their value and why any edge wears off quickly.

What alternative data is

Traditional data are prices and trading volumes on the exchange, financial statements and quarterly reports, and macro data such as inflation or the labour market. They are released according to fixed rules, to everyone at the same time. Alternative data arise as a by-product: when companies advertise jobs, customers pay by card, people post or satellites photograph the earth. They are available earlier, they are raw (only processing turns them into a signal) and they are unevenly distributed, because good datasets cost a lot. Most of the time the aim is nowcasting: estimating what the next quarterly report will say. A signal is only worth something to the extent that it differs from analysts' expectations and is not yet reflected in the price.

Web scraping: job ads, prices, reviews

In web scraping, programs read web pages at regular intervals. This is cheap but fragile: any redesign of a page can interrupt the data feed.

• Job ads reveal growth and strategy. According to Gutiérrez et al. (2020), the change in the number of online ads is linked to future business performance, especially when the ads signal growth rather than replacement; investors react positively to them. For the market as a whole, the ratio of job openings to employment is, according to Kothari and O’Doherty (2023), one of the strongest known predictors of the equity risk premium. But an ad is not a hire. It can appear twice, stand for several positions or stay online long after the position has been filled.
• Online prices: in the Billion Prices Project, Cavallo and Rigobon (2016) collected 5 million prices a day from more than 300 retailers in 50 countries up to 2010; in most countries, the indices built from them move in line with official consumer prices. They show inflation and discounts, but not what is bought.
• Product reviews: Huang (2018) analysed more than 14.5 million customer reviews on Amazon from 2004 to 2015. Unusually good reviews predicted sales and earnings surprises; a long/short strategy based on them earned roughly 55.7 to 73.0 basis points of excess return a month. The risk: paid-for reviews.

Social media and forums: Reddit, X, Stocktwits

On Reddit, for instance in the WallStreetBets forum, on X (formerly Twitter) and on Stocktwits, investors write about what they are buying. This yields two signals: attention (how often a stock is mentioned) and sentiment, measured with sentiment analysis. Dictionaries alone are misleading here: according to Loughran and McDonald (2011), almost three quarters of the words that a widely used general-purpose dictionary classifies as negative are not meant negatively in annual reports. Today, machine learning and large language models (LLMs) help.

The evidence is mixed. According to Antweiler and Frank (2004), forum posts help to predict volatility; their effect on returns is statistically significant but economically small. Bollen et al. (2011) became famous: sentiment measures from Twitter posts in 2008 were said to predict the daily direction of the Dow Jones with a hit rate of 87.6%. Lachanski and Pav (2017) could not reproduce this; an effect appeared only in the original period and not out of sample. That is consistent with data snooping. On WallStreetBets, detailed buy recommendations predicted returns before the GameStop squeeze, but no longer afterwards, while posts about price pressure and attention increased (Bradley et al. 2024).

Meme stocks in 2021

According to SEC (2021, p. 18), GameStop, the best-known meme stock, closed at $347.51 on 27 January 2021, more than 1,600% above its closing price on 11 January, accompanied by intense interest in forums such as WallStreetBets. Short interest reached 122.97% of the free float (SEC 2021, p. 21). Even so, according to the report, a short squeeze, i.e. covering purchases by short sellers, was not the main driver:

Here the forums moved the price rather than predicting it. If you measure their sentiment, you are measuring part of the events themselves. According to Lopez-Lira and Tang (2026), language models that rate headlines predicted the price drift after news, especially for small stocks and bad news; as the models spread, the returns fell.

Satellite images: car parks, oil tanks, night lights, fields

• Car parks: according to Katona et al. (2025), investors with access to satellite data on large US retailers traded profitably, especially ahead of poor quarterly results. Short sales became better informed, retail investors' purchases worse informed, and liquidity around the reports fell.
• Oil tanks: the shadow on the roof of a floating roof tank shows the fill level. According to Mukherjee et al. (2021), such estimates take away part of the surprise in the weekly US inventory data. Ships can also be tracked; see ship data.
• Night lights: Henderson et al. (2012) measure growth by brightness at night (night lights data). In countries with weak statistics, the new estimate of annual growth differed from the official figures by up to 3 percentage points.
• Fields: images in several spectral bands show how green fields are, which gives an indication of harvests. Freely available images with a resolution of 10 m are often enough for this (ESA 2015), but not for cars and tank roofs.

Limits: clouds and the cost of resolution. In addition, a car park shows nothing of online sales.

Transaction data: card payments and e-mail receipts

Transaction data come from the card payments of a panel, a sample of cardholders whose spending is passed on in anonymised form. From this, a retailer's sales are extrapolated, often long before the end of the quarter. E-mail receipts go further: with the users' consent, apps read order confirmations from their inboxes. How much such data can show is demonstrated by Chetty et al. (2024) with a public database of anonymised company data on consumption, sales and employment during the pandemic. Pitfalls: people who pay cash are missing, a changing panel shifts the extrapolation, and without effective anonymisation these are personal data.

Worked example: sales from a card panel

Apps, web traffic, locations and news

Estimated app downloads and web traffic show how many people use a service. According to Da et al. (2011), Google searches measure attention, presumably above all that of retail investors: more searches predicted higher prices for two weeks and a reversal within a year. Geolocation data from smartphones show visits to shops or plants. Froot et al. (2017) built real-time measures of retailers' sales from, among other sources, around 50 million mobile devices; these explained sales and earnings surprises and delivered an average excess return of 3.4% around the quarterly results.

News can be read by machines in real time. According to Tetlock (2007), pessimism in a newspaper column predicted downward price pressure that later reversed: sentiment rather than new information. ESG data from news and reports are inconsistent, as the widely diverging ESG ratings show (Berg et al. 2022); more on this under ESG ETFs.

How much the origin matters is shown by the US Securities and Exchange Commission's first fraud case against a provider of alternative data (SEC press release 2021): the company sold estimates of app usage and promised to use confidential company data only in anonymised form. In reality, from late 2014 to mid-2018 it adjusted its estimates using non-anonymised data, and it settled with the regulator for a $10 million penalty.

Types of data compared

Lead time and cost are qualitative assessments, not measured values.

For advanced readers: what is a signal worth?

The value of a dataset can be estimated with the fundamental law of active management of Grinold (1989): the information ratio depends on forecasting accuracy, the information coefficient (IC), and on breadth, the number of independent bets per year. Clarke et al. (2002) added the transfer coefficient: constraints such as a short-selling ban mean that only part of the forecasting skill reaches the portfolio.

• Look-ahead: providers can revise histories or adjust panels retrospectively. A backtest needs point-in-time data, i.e. the data as they stood at the time. Otherwise there is a risk of look-ahead bias; if bankruptcies and takeovers are missing, survivorship bias comes on top.
• Overfitting: out of thousands of variables, one always fits in hindsight. Harvey et al. (2016) require a t-statistic above 3.0 for new factors, and Bailey et al. (2016) estimate the probability of backtest overfitting (see backtest overfitting).
• Known factors: check whether a signal merely replicates what momentum, size or quality already explain.
• Capacity: according to McLean and Pontiff (2016), published signals earn more in illiquid stocks with high idiosyncratic risk, i.e. exactly where little money can be moved.

What the research shows

• Prices become more informative: according to Zhu (2019), alternative data lower the cost of acquiring information; managers trade their own company's shares opportunistically less often and invest more efficiently.
• Unequal access: paying investors gain, while retail investors trade on worse information (Katona et al. 2025); official data lose their monopoly on surprises (Mukherjee et al. 2021).
• Signals decay: for 97 published return predictors, returns according to McLean and Pontiff (2016) were 26% lower out of sample and 58% lower after publication (alpha decay). The same pattern appears with WallStreetBets and language models.
• No free advantage: according to the Grossman-Stiglitz paradox, markets cannot be perfectly efficient if information costs money. Those who pay are rewarded for their effort and costs. More under efficient market hypothesis.

Law and ethics: data protection, scraping, insider rules

An overview, not legal advice. Data protection: according to European Commission (2026), location data, IP addresses, cookie identifiers and smartphone advertising IDs are also personal data. Anonymised data only fall outside the GDPR if the anonymisation is irreversible; pseudonymised data remain personal data. If you buy card, e-mail or location data, you need to know how consent and anonymisation were obtained.

Scraping: in Germany, Section 44b of the Copyright Act (Urheberrechtsgesetz, UrhG) (UrhG 1965) permits text and data mining of lawfully accessible works; the copies must be deleted afterwards, and rights holders can reserve the use in machine-readable form online. Terms of use apply in addition. In the US case hiQ v. LinkedIn, according to the appeals court's rulings, reading public profiles probably did not fall under the anti-hacking law CFAA; in 2022, however, the district court ruled that hiQ had breached the terms of use through scraping and fake accounts, and the settlement provided for $500,000 and a permanent injunction (Proskauer Rose 2022).

Insider rules: under Art. 7 of the Market Abuse Regulation (MAR), inside information is precise information that has not been made public about an issuer or a financial instrument and that could have a significant effect on the price; Art. 14 prohibits trading on it (CJEU 2026, paras. 5–7). Under recital 28, analyses and estimates based on publicly available data are not in themselves inside information (BaFin 2026). On 16 April 2026, the Court of Justice of the European Union took a narrow view of when information is public: according to CJEU (2026), only after disclosure under Art. 17 MAR. Communication to a limited group is not enough, nor is mere inspection on request.

As an assessment, this means: if you count car parks or collect prices yourself, you are analysing public data. Data from inside the company are the delicate part: confidential metrics, leaked plans, communications to a few people. For every dataset, check who collected it and with what permission.

Limits and common mistakes

• Costs before benefits: an expensive dataset needs a lot of capital before it pays off, so in a private portfolio it hardly ever does.
• Data quality and short histories: panels change, scrapers break down, and a few years of data cannot separate small advantages from chance.
• Crowding: the more investors buy the same data, the faster the information is reflected in the price.
• Ambiguity: full car parks can mean discounts, many job ads high staff turnover, good sentiment a speculative bubble.
• Origin: data from an unclear source can violate data protection, contract or insider law.

For private investors, alternative data mainly explain why prices react even before the quarterly figures. They are useful as background, but hardly as a buy or sell signal. This article is not investment advice.

How Portfolio Lab calculates it

The Sentiment page calculates its own fear and greed index from six components based on market data, each expressed as a percentile rank over the last two years, as soon as at least four components have values. In addition, there is the put/call ratio of the options on three large index ETFs as a daily snapshot, and speculators' positioning from the COT reports with a COT index over three years; subsequent returns are counted from publication, three days after the reference date (methodology). The Ships page (beta) counts tankers and cargo ships in twelve areas at chokepoints, using AIS messages from a terrestrial receiver network. It records positions and counts, not volumes (methodology).

Portfolio Lab does not analyse posts from X or Reddit, because large-scale access requires contracts with the platforms and fees. The app does not use card data, satellite images or location data either. Sentiment and ship data are not used as signals in backtests.

This article explains a method; it is not investment or tax advice. Worked examples are simplified and rounded. All terms in the glossary.