You’re signed up for e-mail updates.
News publishers' fact statement quotes OpenAI documents on scraping around rate limits and paywalls, and says LibGen text helped train a data filter
The news publishers' statement of facts, filed in the same consolidated proceeding as the authors' class action, quotes OpenAI's internal documents and witnesses on how it gathered text. It quotes an October 2019 document saying nearly all of OpenAI's datasets came from scraping and that "we're kinda flagrantly going around rate limits." It quotes a 2023 message that crawlers were being blocked by large content providers and that OpenAI was working around this by slowing its scrape, and a message to OpenAI's president about a way around a newspaper's paywall. It says OpenAI told The New York Times in April 2023 that it had not accessed the paper's platforms for training, and quotes a corporate witness later admitting OpenAI had scraped the site before then. It also quotes a July 2021 document saying a filter used to pick high-quality web text was trained on a mix that included LibGen documents. This is the publishers' statement; OpenAI's response is pending and parts are redacted.
Loading the rest of the news…
What the labels mean
Every item says what kind of source it rests on. A court ruling or order is the court's own statement. A company filing is what a defendant says about itself. A government filing is a statement by a government agency that is not a party to the case, such as a statement of interest; it is the government's position, not a ruling. An allegation is what a complaint or motion claims and has not been decided. Reported means press reporting we have not confirmed from the primary document. Summaries are our own; follow the links for the source.
This page is general information, not legal advice. An allegation is not a finding. Whether any of this matters for a particular work or claim is a question for a lawyer.