readability README

The project's README from github.com/mozilla/readability. As one markdown file: README.md.

Readability.js

A standalone version of the readability library used for Firefox Reader View.

Installation

Readability is available on npm:

npm install @mozilla/readability

You can then require() it, or for web-based projects, load the Readability.js script from your webpage.

Basic usage

To parse a document, you must create a new Readability object from a DOM document object, and then call the parse() method. Here's an example:

var article = new Readability(document).parse();

If you use Readability in a web browser, you will likely be able to use a document reference from elsewhere (e.g. fetched via XMLHttpRequest, in a same-origin <iframe> you have access to, etc.). In Node.js, you can use an external DOM library.

API Reference

new Readability(document, options)

The options object accepts a number of properties, all optional:

parse()

Returns an object containing the following properties:

The parse() method works by modifying the DOM. This removes some elements in the web page, which may be undesirable. You can avoid this by passing the clone of the document object to the Readability constructor:

var documentClone = document.cloneNode(true);
var article = new Readability(documentClone).parse();

isProbablyReaderable(document, options)

A quick-and-dirty way of figuring out if it's plausible that the contents of a given document are suitable for processing with Readability. It is likely to produce both false positives and false negatives. The reason it exists is to avoid bogging down a time-sensitive process (like loading and showing the user a webpage) with the complex logic in the core of Readability. Improvements to its logic (while not deteriorating its performance) are very welcome.

The options object accepts a number of properties, all optional:

The function returns a boolean corresponding to whether or not we suspect Readability.parse() will succeed at returning an article object. Here's an example:

/*
    Only instantiate Readability  if we suspect
    the `parse()` method will produce a meaningful result.
*/
if (isProbablyReaderable(document)) {
    let article = new Readability(document).parse();
}

Node.js usage

Since Node.js does not come with its own DOM implementation, we rely on external libraries like jsdom. Here's an example using jsdom to obtain a DOM document object:

var { Readability } = require('@mozilla/readability');
var { JSDOM } = require('jsdom');
var doc = new JSDOM("<body>Look at this cat: <img src='./cat.jpg'></body>", {
  url: "https://www.example.com/the-page-i-got-the-source-from"
});
let reader = new Readability(doc.window.document);
let article = reader.parse();

Remember to pass the page's URI as the url option in the JSDOM constructor (as shown in the example above), so that Readability can convert relative URLs for images, hyperlinks, etc. to their absolute counterparts.

jsdom has the ability to run the scripts included in the HTML and fetch remote resources. For security reasons these are disabled by default, and we strongly recommend you keep them that way.

Security

If you're going to use Readability with untrusted input (whether in HTML or DOM form), we strongly recommend you use a sanitizer library like DOMPurify to avoid script injection when you use the output of Readability. We would also recommend using CSP to add further defense-in-depth restrictions to what you allow the resulting content to do. The Firefox integration of reader mode uses both of these techniques itself. Sanitizing unsafe content out of the input is explicitly not something we aim to do as part of Readability itself - there are other good sanitizer libraries out there, use them!

Contributing

Please see our Contributing document.

License

Copyright (c) 2010 Arc90 Inc

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

readability (mozilla/readability)

parser
Searching Database…

11.5K ·GitHub
Readability
What is this?Readability.js is a standalone extraction library that pulls the main article content out of a messy webpage, stripping ads, navigation and sidebars. It is the same library that powers Firefox's Reader View.
Is this a browser extension or a library I call from code?It is a plain JavaScript library, not a packaged browser extension. You create a Readability object from a DOM document and call parse() yourself, either in a webpage or in Node.js.
How do I add it to a project?Install it from npm and require() it, or load Readability.js directly as a script tag on a webpage. In Node.js you pair it with an external DOM library like jsdom since Node has no built-in DOM.
Do I need a DOM library to use it in Node?Yes, since Node.js doesn't ship a DOM implementation, you need jsdom (or similar) to build a document object and pass it to Readability.
Does it sanitize the HTML it extracts?No. Readability only extracts the article content; it does not sanitize it, so if the input is untrusted you must run the output through a sanitizer like DOMPurify yourself.
Who would use this?Developers building reader-mode features, article extractors, RSS/content pipelines, or anything that needs to turn a raw webpage into clean readable text.
Does it run in the browser as well as on a server?Both: it works directly in a web browser using the page's own DOM, and on the server/Node.js side when paired with jsdom.
Is there a quick way to check if a page is worth parsing before running the full extraction?Yes, isProbablyReaderable(document, options) is a lightweight heuristic check that estimates whether parse() is likely to succeed, so you can avoid running the heavier logic on every page.
What does parse() actually give me back?An object with title, content (HTML string), textContent (plain text), length, excerpt, byline, dir, siteName, lang and publishedTime.
Can I keep some CSS classes in the extracted content?Yes, via the keepClasses option you can preserve all classes, or use classesToPreserve to whitelist specific class names while discarding the rest.
Does parsing modify the original page DOM?Yes, parse() mutates the document it's given, so pass a cloned document to the Readability constructor if you need the original DOM to stay intact.
What are similar projects?turndown, ua-parser-js, marked, cheerio, liteparse, markdown-it, PapaParse, protobuf.js
Which topics is it in?parser, node, browser, html, apache-2.0, nodejs, library, javascript, web, open-source
Creator
mozilla
Size
6.9 MB
Stars
11.5K
Updated
CommitJul 09
Age
11y
Commits · 90 Days