SurfSense README

The project's README from github.com/MODSetter/SurfSense. As one markdown file: README.md.

SurfSense, the open-source NotebookLM alternative for open web research

Discord Reddit

English | Español | Português | हिन्दी | 简体中文

MODSetter%2FSurfSense | Trendshift

SurfSense: The Open-Source NotebookLM Alternative for Open Web Research

SurfSense is the open-source NotebookLM alternative for AI agents, an open web research platform with live data connectors. Your agents research the live web with structured data from Reddit, YouTube, Instagram, TikTok, Amazon, Walmart, Google Maps, Google Search, Indeed, and any page on the open web, through one REST API or MCP server. Scheduled and event-triggered agents turn what they find into briefs and alerts, and a built-in knowledge base keeps every finding searchable with citations.

[!NOTE] 📢 A note for our NotebookLM-alternative users

For the past couple of months we built SurfSense as the best general research agent for your own knowledge, and that chapter earned us a community we are genuinely proud of. Agentic tools like Claude, OpenCode, Hermes, and OpenClaw have now proven that agents are the future, and reasoning over a static index is becoming something every capable agent does out of the box. What agents still lack is live data from the places where answers actually live, and the workflows around it. That is where we are pointing all of our energy: giving agents the primitives to research the open web.

Nothing you rely on is going away. Your knowledge base, chat with citations, reports, podcasts, presentations, automations, and collaborative chats all keep working, and self-hosting stays free and open source. Read the full announcement on our changelog.

Table of contents

Why agents need SurfSense

Ask any capable agent "what is Reddit saying about this product since launch?" or "what do the reviews of these ten places actually complain about?" and it has nowhere trustworthy to look. Official platform APIs are rate-limited, priced for enterprises, or missing entirely; scraping plumbing is brittle; and driving a browser with an LLM burns minutes and tokens per page. SurfSense gives agents the primitives instead:

Live data connectors

Connector What your agents get Learn more
Reddit Posts, comments, and subreddit streams without the official API's rate limits Reddit Scraper API
YouTube Videos, transcripts, and comment threads at scale YouTube Scraper API
Instagram Public profiles, posts, and reels without the Graph API Instagram Scraper API
TikTok Videos, comments, hashtags, and profiles without Research API approval TikTok Scraper API
Google Maps Places, ratings, and reviews for local business research Google Maps Scraper API
Google Search Live SERPs for search research and monitoring Google Search API
Indeed Public job postings with salaries and full descriptions, by search or company Indeed Scraper API
Amazon Public product data: prices, ratings, offers, sellers, and best-seller ranks Amazon Product API
Walmart Public product data plus the full review history: prices, ratings, sellers, and variants Walmart Product & Review API
Web Crawl Any page on the open web as clean, structured content Web Crawling API
External MCP Connectors Bring any MCP server to your agents, with one-click OAuth for Notion, Slack, Jira, and more External MCP Connectors

The connector catalog is growing beyond social platforms and search; every new source lands as a typed endpoint on the same API and MCP server.

Billing is pay as you go: connectors bill per item actually returned, crawls per page successfully fetched, and failed calls are never billed. Self-hosted installs run with billing off. See pricing.

Quick start

Call a connector from code

Every connector is a REST endpoint you can call from any language with your SurfSense API key:

curl -X POST "$SURFSENSE_API_URL/workspaces/$WORKSPACE_ID/scrapers/reddit/scrape" \
  -H "Authorization: Bearer $SURFSENSE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "search_queries": ["your brand"],
    "community": "SaaS",
    "sort": "top",
    "time_filter": "week"
  }'

Each connector page has copy-paste examples in Python, JavaScript, Go, PHP, Ruby, Java, and C#.

Give the tools to your agents over MCP

Add the SurfSense MCP server to Claude, Cursor, or your own agent framework:

{
  "mcpServers": {
    "surfsense": {
      "url": "https://mcp.surfsense.com/mcp",
      "headers": { "Authorization": "Bearer ${SURFSENSE_API_KEY}" }
    }
  }
}

Your agent can now call every connector as a native tool. See the SurfSense MCP server page for the full tool list, or run the server locally from surfsense_mcp.

Use the cloud

Go to surfsense.com, log in, and ask the agent for live web data in plain English. New accounts start with $5 of free credit and no subscription.

Self-host for free

Run the entire platform, connectors, agents, automations, and the MCP server on your own infrastructure. Self-hosted installs ship with billing off, so scraping, crawling, and agent runs are limited only by your hardware and the model keys you bring.

Prerequisites: Docker Desktop must be installed and running.

For Linux/macOS:

curl -fsSL https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.sh | bash

For Windows:

irm https://raw.githubusercontent.com/MODSetter/SurfSense/main/docker/scripts/install.ps1 | iex

The install script sets up Watchtower automatically for daily auto-updates. To skip it, add the --no-watchtower flag. For Docker Compose, manual installation, and other deployment options, see the docs.

Everything else in the box

The research workspace that made SurfSense the leading open-source NotebookLM alternative is still here, and everything your agents gather lands in it.

Knowledge base

Chat With Your PDFs and Docs

Deliverable studio

AI Report Generator

Automations

Team collaboration

Collaborative AI Chat

Desktop app

Native AI assistance in every application on your computer. Download from the latest release.

Quick Assist

No vendor lock-in

Video Agent Sample

https://github.com/user-attachments/assets/012a7ffa-6f76-4f06-9dda-7632b470057a

Podcast Agent Sample

https://github.com/user-attachments/assets/a0a16566-6967-4374-ac51-9b3e07fbecd7

How to collaborate in real time (Beta)

  1. Go to the Manage Members page and create an invite.

    Invite Members

  2. A teammate joins and that workspace becomes shared.

    Invite Join Flow

  3. Make a chat shared and work in it together in real time, with comments to tag teammates.

    Realtime Comments

How SurfSense compares

SurfSense is the only open-source product that combines a NotebookLM-style research workspace for people with live-data primitives for agents. Here is how that stacks up against each class of tool.

vs browser agents (Browserbase, Browser Use). Browser agents drive a real browser with an LLM in the loop — the right tool when a task needs clicking, logging in, or filling forms. But most research is read-only retrieval, and for retrieval the LLM-in-a-browser loop costs you minutes and thousands of tokens per page. A SurfSense connector call is one HTTP request: seconds, deterministic, and zero tokens spent deciding where to click.

vs scraping APIs (Firecrawl). Scraping APIs are great at turning a generic page into markdown, but a markdown blob still leaves your agent parsing structure out of prose, and they degrade on bot-protected platforms like Reddit, TikTok, and Instagram. SurfSense connectors return platform-native structured items — posts, comments, transcripts, reviews — and bill only for items actually returned; failed calls are never billed.

vs search APIs (Exa, Tavily, Parallel). Search APIs answer from a web index, which is the right tool for "find me pages about X." They cannot pull a Reddit thread's comments, TikTok reactions, YouTube transcripts, or Google Maps reviews — the places where the answer often actually lives.

vs scraper marketplaces (Apify). Marketplaces give you thousands of community actors, each with its own schema, quality, and pricing. SurfSense is one typed API and one MCP server with an agent harness and a research workspace behind it, and it is open source.

SurfSense vs Google NotebookLM

Still comparing us as a NotebookLM alternative? Here is the honest breakdown.

Feature Google NotebookLM SurfSense
Live web data for agents No Reddit, YouTube, Instagram, TikTok, Amazon, Walmart, Google Maps, Google Search, Indeed, and web crawl connectors via REST API and MCP
MCP server No Every connector exposed as a native agent tool, plus bring-your-own MCP servers with one-click OAuth apps
Sources per Notebook 50 (Free) to 600 (Ultra, $249.99/mo) Unlimited
Number of Notebooks 100 (Free) to 500 (paid tiers) Unlimited
Source Size Limit 500,000 words / 200MB per source No limit
Pricing Free tier; Pro $19.99/mo, Ultra $249.99/mo Free and open source to self-host; cloud is pay as you go with $5 free credit
LLM Support Google Gemini only 100+ LLMs via OpenAI spec & LiteLLM
Embedding Models Google only 6,000+ embedding models, all major rerankers
Local / Private LLMs Not available Full support (vLLM, Ollama), your data stays yours
Self Hostable No Yes, Docker one-liner or full Docker Compose
Open Source No Yes
Knowledge Base Sources Google Drive, YouTube, websites File uploads, Google Drive, OneDrive, Dropbox, local folder sync, and crawled pages
File Format Support PDFs, Docs, Slides, Sheets, CSV, Word, EPUB, images, web URLs, YouTube 50+ formats: documents, images, videos via LlamaCloud, Unstructured, or Docling (local)
Search Semantic search Hybrid semantic + full-text with hierarchical indices & reciprocal rank fusion
Cited Answers Yes Yes, Perplexity-style cited responses
Agentic Architecture No Yes, powered by LangChain Deep Agents with planning, subagents, and file system access
AI Automations & Agents No Scheduled workflows, event triggers, and chat-built no-code automations with write-back to Notion, Slack, Linear & Jira
Real-Time Multiplayer Shared notebooks with Viewer/Editor roles (no real-time chat) RBAC with Owner / Admin / Editor / Viewer roles, real-time chat & comment threads
Video Generation Cinematic Video Overviews via Veo 3 (Ultra only) Available (NotebookLM is better here, actively improving)
Presentation Generation Better looking slides but not editable Editable, slide-based presentations
Podcast Generation Audio Overviews with customizable hosts and languages Available with multiple TTS providers (NotebookLM is better here, actively improving)
Desktop App No Native app with General Assist, Quick Assist, Screenshot Assist, and local folder sync

Feature requests and future

SurfSense is actively being developed. While it's not yet production-ready, you can help us speed up the process.

Join the SurfSense Discord and help shape the future of SurfSense!

Roadmap

Stay up to date with our development progress and upcoming features. Check out our public roadmap and contribute your ideas or feedback:

Roadmap Discussion: SurfSense 2026 Roadmap

Kanban Board: SurfSense Project Board

Contribute

All contributions welcome, from stars and bug reports to backend improvements. See CONTRIBUTING.md to get started.

Thanks to all our Surfers:

Star History

Star History Chart

Catalyst Project



SurfSense (MODSetter/SurfSense)

Searching Database…

16.5K ·GitHub
这是什么?一个开源、可自托管的 NotebookLM 替代品:智能体通过连接器从实时网络(Reddit、YouTube、Amazon 等)拉取结构化数据,内置的知识库让每一条发现都可搜索并附带引用来源。
它是云服务、需要我自己部署,还是桌面应用?三种形式都有:按量付费的云端版本 surfsense.com,用 Docker 自行部署的平台(包含网页工作区、REST API 和 MCP 服务器),还有一个可选的原生桌面应用,可以在任何应用里提供 AI 帮助。
自托管版本怎么跑起来?安装并启动 Docker Desktop,然后运行适合你操作系统的安装脚本(Linux/macOS 或 Windows)。它默认会设置 Watchtower 进行每日自动更新,除非你加上 --no-watchtower;文档里也提供了 Docker Compose 和手动安装方式。
它需要哪些 AI 模型——我得自己准备特定的 API 密钥吗?没有固定要求的模型:它通过 OpenAI 规范和 LiteLLM 运行 100 多种 LLM,兼容 6000 多种嵌入模型和所有主流重排序模型,并且通过 vLLM 或 Ollama 完整支持本地模型,所以你有什么密钥或本地接口就能接什么。
哪些方面 NotebookLM 还是更胜一筹?README 承认 NotebookLM 在视频概览和播客音频方面做得更好,不过这两项功能这里也有(双主播播客 20 秒内生成,幻灯片可编辑)。另外实时协作聊天目前仍处于 Beta 阶段。
自托管真的能把所有数据都留在我的电脑上吗?可以——自托管安装完全运行在你自己的基础设施上,计费功能关闭,而且完整支持本地 LLM(vLLM、Ollama),数据完全不必离开你的机器。项目自称是物理隔离、注重隐私的。
它说是开源的——用的是我可以放心依赖的标准许可证吗?GitHub 显示该许可证为 NOASSERTION(是自定义许可证,而非标准开源许可证),而 README 本身只承诺自托管保持免费和开源——商用之前建议先读一下 LICENSE 文件。
它和直接用 Google NotebookLM 有什么不同?NotebookLM 限制来源 50-600 个、笔记本 100-500 个、每个来源最大 200MB,而且只能用 Gemini 系模型。SurfSense 提供不限量的来源和笔记本、没有大小限制、支持 100 多种 LLM 和本地模型,还提供语义与全文混合搜索、MCP 工具以及 Docker 自托管。
智能体能从哪些实时数据源拉取数据?Reddit、YouTube、Instagram、TikTok、Google 地图、Google 搜索、Indeed、Amazon 和 Walmart,以结构化 REST 接口提供;还可以通过爬取抓取开放网络上的任意页面;此外还有外部 MCP 连接器,为 Notion、Slack、Jira 等提供一键 OAuth 授权。
像 Claude 或 Cursor 这样的编程智能体能直接调用它吗?可以——MCP 服务器把每个连接器都暴露成原生的智能体工具,比如 surfsenseredditscrape 或 surfsensegooglesearch。你可以把它添加到 Claude、Cursor 或任何智能体框架,也可以从 surfsensemcp 在本地运行这个服务器。
为什么 README 突然重点讲智能体,而不是 NotebookLM 那个工作区了?作者认为,对静态索引做推理正在变成每个能力强的智能体开箱即有的功能,而智能体还欠缺的是来自答案真正所在之处的实时数据——所以他们的精力放在这里。知识库、聊天、报告、播客和自动化功能都照常可用,自托管也依然免费。
Creator
MODSetter
Size
423.4 MB
Stars
16.5K
Age
2y
Commits · 90 Days