News and sentiment: methodology for filings, central bank tone, and backtest
News and sentiment belong together on the News page: it collects releases from official sources and classifies each one, companies by the sentiment of their press release and the type of filing, central banks by the tone of their statement. This methodology describes where the releases come from, how sentiment and tone are calculated, how they turn into tactical signals and the News trading backtest, and where the limitations lie.
Sources and sections
All items come from the authorities' own feeds and pages; the list shows the title, time and our own classification with a link to the original. Each item has its own page: the classification in sentences and the authority's text, read from its site on first access; the Fed, the BLS and the ECB allow reproduction with attribution, Destatis also in excerpts with a source reference. For company press releases the page shows only an excerpt without tables. General business news is missing because it may only be shown under a license. The server reads the feeds in the background every 15 minutes; if the last fetch is older when the page is opened, the page reads them again.
- Central banks: press releases of the Fed (without enforcement actions and bank approvals) and the ECB (press releases, speeches, interviews).
- Macro: press releases of Destatis and the latest numbers of the main series of the US Bureau of Labor Statistics (BLS), one release per series and month.
- Companies: the latest Form 8-K filings with the SEC, only from the 1,500 largest listed companies according to the SEC's directory.
- Dates: FOMC meetings, monetary policy meetings of the ECB Governing Council, and the release dates of US consumer prices and the employment report from the agencies' calendars.
Type of filing and warnings
A Form 8-K lists numbered sections (items). They determine the title and group: results of operations (Item 2.02), agreements and deals (Items 1.01, 1.02, 2.01, 2.03, 5.01), management (Items 5.02, 5.03, 5.07). A warning is a termination of a material agreement (Item 1.02), bankruptcy (Item 1.03), a cybersecurity incident (Item 1.05), an accelerated obligation (Item 2.04), an impairment (Item 2.06), a notice of delisting (Item 3.01), and prior financial statements that can no longer be relied on (Item 4.02). A warning comes from the type of filing, not from the text: press releases sound positive even when the news is bad.
Sentiment per sentence
The press release attached to the filing as an exhibit is scored with VADER Hutto and Gilbert (2014), a dictionary with rules for negation, intensifiers, and punctuation. Over entire press releases VADER is almost always close to its maximum; per sentence the scores vary much more. That is why the mean of the sentences counts: without the section on forward-looking statements and non-GAAP measures at the end, without table rows, and only sentences with 6 to 80 words.
- VADER score per sentence (−1 to +1), averaged over the press release
- at least three earlier filings of the same company, otherwise a fixed benchmark of 0.25
If sentiment is at least 0.05 above the benchmark, the filing counts as positive, at least 0.05 below as negative, in between as neutral. On the stock page, the price reaction on the first trading day after the filing is shown alongside; if it was received after 16:00 New York time, the next trading day counts.
Central bank tone
For FOMC statements and ECB monetary policy decisions, the page counts words from two of our own lists: those that stand for tighter monetary policy (such as elevated inflation, strong growth, a rate hike) and those for looser policy (such as weaker growth, easing inflation, a cut). It also reads the decision: rate up, down, or unchanged.
- number of words from the two lists in the statement
- dampens short statements, which would otherwise be fully tight or fully loose with just a few words
Compared with the bank's previous statement, a statement counts as tighter if the rate rises or the tone rises by at least 0.15, looser in the opposite case, otherwise unchanged. For tactical allocation there are two signals per bank since 1999: tone as 50 · (1 + tone) and rate direction as 50 plus 12.5 for each rate hike and minus 12.5 for each cut in the last four decisions. Each statement applies from the trading day after until the next one. Older ECB decisions only state the rates; their tone is then neutral, and the rate direction counts.
News trading backtest
The News trading backtest takes the filings of today's 500 largest listed US companies from 23 August 2004, the first day of the item for earnings (Item 2.02). Twelve kinds are available: earnings, other press releases (Item 7.01 or 8.01 without earnings, such as guidance, buybacks or dividends), insider purchases and nine kinds of Form 8-K by item, such as changes of directors or officers (Item 5.02), completed acquisitions (Item 2.01), impairments (Item 2.06) or non-reliance on prior financial statements (Item 4.02).
For each kind it shows the excess return of all filings over the market after 1, 2, 3, 5, 10, 20 or 60 trading days from the first close after the filing. It also splits the filings into fifths: by sentiment (earnings and press releases), by the reaction on the filing day or, for insider purchases, by amount. The fifths are formed per calendar quarter with at least 25 filings; if there are fewer than eight such quarters, per calendar year (at least three years), otherwise over the whole period. The bet buys the top fifth and shorts the bottom fifth, equally weighted and without costs.
Insider purchases are open-market purchases (code P) from Forms 4 in the SEC's quarterly data sets from 2006, combined per company and filing date. The data sets only give the day, so the purchase is at the close of the next trading day. Only derived values are stored for each filing. The nightly run reads for at most 45 minutes in its regular round and then continues until 8 a.m.: first the lists and the insider purchase data sets (at most twelve quarters per night), then the sentiment, first of all earnings and then of the other press releases, newest first; a quarter of the time is kept for prices.
What research shows
Tetlock (2007) found that pessimistic tone in a newspaper column predicts downward pressure on prices that later reverses. Loughran and McDonald (2011) showed that general dictionaries misclassify financial texts, because words like liability or tax are neutral there, and built their own dictionary for annual reports; it can only be used commercially with a license and is therefore not used here. For central banks, Lucca and Trebbi (2009) showed that the tone of statements can be measured automatically and that changes in wording move rate expectations. Comparing with the company's own history and taking warnings from the type of filing offset part of the weaknesses of a general dictionary.
Limitations
- General dictionary: VADER does not know financial language; a word like risk always counts as negative, even in boilerplate.
- Official sources only: no wire services, no newspapers; links to major business media are under sources for further reading.
- Survivorship bias in the backtest: the companies are today's largest; companies that no longer exist are missing. There are therefore hardly any filings on bankruptcy or delisting.
- Our own word lists for tone: rough and in English; the Fed and ECB change their style over the years.
- Not advice: sentiment and tone are classifications, not recommendations.
Frequently asked questions
Where does the news come from?
From feeds and pages of the Fed, the ECB, Destatis, the US Bureau of Labor Statistics, and the SEC. General business news is only available with a license; until then the page links to major business media.
Why is almost every press release positive?
Because companies report their numbers in positive language. That is why the page compares each filing with the same company's earlier ones and takes warnings from the type of filing, not from the text.
What does tighter mean for the Fed and the ECB?
The statement contains more words for tighter monetary policy than the previous one, or the rate was raised. Looser means the opposite.
Can I use central bank tone in tactical allocation?
Yes, in signal rotation as the tone or rate direction of the Fed and the ECB, each on a scale from 0 to 100.
Why are European companies missing?
For European companies there is no equivalent free source with real-time filings and a machine-readable type of filing.
Literature
- Hutto, C.; Gilbert, Eric (2014): VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. AAAI (Proceedings of the International AAAI Conference on Web and Social Media, Band 8). doi:10.1609/icwsm.v8i1.14550
- Loughran, Tim; McDonald, Bill (2011): When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks. The Journal of Finance 66 (1), pp. 35–65. doi:10.1111/j.1540-6261.2010.01625.x
- Lucca, David O.; Trebbi, Francesco (2009): Measuring Central Bank Communication: An Automated Approach with Application to FOMC Statements. NBER Working Paper 15367. doi:10.3386/w15367
- Tetlock, Paul C. (2007): Giving Content to Investor Sentiment: The Role of Media in the Stock Market. The Journal of Finance 62 (3), pp. 1139–1168. doi:10.1111/j.1540-6261.2007.01232.x
This page describes how Portfolio Lab calculates; past results are not a forecast. All terms in the glossary.