Local Deep Research
lastmod 2025-03-12created 2025-03-121 edits
Run your own research assistant locally
About
- A powerful AI-based research tool that performs deep, iterative analysis using multiple LLMs and web search
- Integrates search across ArXiv, Wikipedia, Google, PubMed, DuckDuckGo, SerpAPI, local RAG, The Guardian, and more
Advanced research features
- Automated deep research: intelligent follow-up question generation
- Source tracking and verification: automatic tracking of citations and sources
- Iterative analysis: comprehensive coverage through multi-step iterative analysis
- Full web page content analysis: analysis based on full content, not snippets
Flexible LLM support
- Local model support: local AI processing based on Ollama
- Cloud model support: supports cloud LLMs such as Claude and GPT
- Langchain model compatibility: supports various Langchain models
- Model selection: configure models according to performance, response speed, etc.
Rich output options
- Detailed research findings: provides detailed reports with citations
- Comprehensive research report: provides comprehensive research results
- Quick summary: summarizes the key points
- Source tracking and verification: supports source tracking and verification
Privacy-focused design
- Can run locally: when using local models, all data stays on the user's device
- Configurable search: stronger privacy
- Transparent data handling: data processing methods are clearly disclosed
Enhanced search integration
- Automatic search engine selection: automatically chooses a search engine based on the query
- Wikipedia integration: reliable fact search
- arXiv integration: search for scientific papers and academic research
- PubMed integration: search for medical and biomedical research
- DuckDuckGo integration: general web search (may be rate-limited)
- SerpAPI integration: provides Google search results (API key required)
- Google Programmable Search: custom search configuration (API key required)
- The Guardian integration: search news and journalism content (API key required)
- Local RAG search: search personal documents (uses vector embeddings)
- Full web page content search: search the entire content of web pages
- Source filtering and verification: filter to trusted sources
- Configurable search parameters: set search scope, time range, etc.
Local document search (RAG)
- Vector embedding-based search: search content in personal documents
- Custom document collections: group documents by topic
- Privacy protection: all documents are processed locally
- Intelligent chunking and search: chunks and searches document content
- Compatible with many document formats: supports PDF, text, Markdown, etc.
- Unified meta search applied automatically: can combine local and web search
Web interface
- Dashboard: intuitive interface
- Real-time progress updates: research progress provided in real time
- Research history management: access and manage past research records
- PDF report export: download research reports as PDF
- Research management: stop or delete research in progress
Supported search engine options
- Auto: automatically selects an engine based on the query
- Wikipedia: suited to general information and fact search
- arXiv: suited to scientific and academic paper search
- PubMed: suited to biomedical and medical research
- DuckDuckGo: privacy-focused general web search
- The Guardian: news and journalism search (API key required)
- SerpAPI: provides Google search results (API key required)
- Google Programmable Search: custom search (API key required)
Hacker News comments
- Local Deep Research – ArXiv, wiki and other searches included | Hacker News
- Applauds the effort toward the local and lo-fi space. But reading the examples in the documentation, the output feels somewhat confusing
- Thinks at least one more intermediate step is needed. For example, a graph database could be used so the LLM stores information, checks interconnections, asks itself questions, and generates the final report
- The final report could be an interactive HTML file that the user can ask questions about or edit directly
- There is a similar open deep research tool called Onyx, which seems to have better UI/UX. The author could consider porting this tool to run locally
- Not saying this project is bad, but worried that many open deep research projects will disappear. It would be better for people to collaborate by focusing on the parts they care about most
- If you want to add embeddings with the internet as a source, recommends trying exa.ai. It includes Wikipedia, thousands of news feeds, Github, and over 70 million papers
- Disclosure: I am one of the founders
- It has ways to do web search and document embedding, but the results are lacking because details are lost in the embeddings. Wonders whether this approach is better
- Recently thinks that local collections preprocessed with curated, structured information for RAG could be a good complement to this dynamic search approach
- Saw that LangChain is used; txtai is worth checking out
- A common failure mode in OpenAI's deep research right now is pulling answers from low-authority sources and citing them as if they were scientific journals. Such sources contain little of value, and even if other sources are high quality, the low-quality sources ruin everything
- Emphasizing content you have already curated (bookmarks) could greatly improve the signal-to-noise ratio (SNR)