Search engine

lastmod 2025-05-20created 2017-10-3115 edits

A search engine is an information retrieval system designed to help find information stored on a computer system. The search results are usually presented as a list. Using a search engine minimizes the time needed to find information.

The most visible form of public search engine is the web search engine, which finds information on the World Wide Web.

Categories

• Programmable Search Engine (PSE)
• typesense - Fast, typo tolerant, fuzzy search engine for building delightful search experiences.

Full-text search (Korean: 전문검색)

• Apache Lucene
  • ElasticSearch
• Apache Solr
• ArangoSearch
• BaseX
• KinoSearch
• Lemur/Indri
• MariaDB
• mnoGoSearch
• MySQL
• OpenSearch
• PostgreSQL
• Searchdaimon
• Sphinx
• Swish-e
• Terrier IR Platform
• Xapian
• SQLite:FTS5 - SQLite's Full-Text Search

Web search engine

• Google
• DuckDuckGo

AI-based web search engines

• Liner | Credible AI Search Engine - Liner
• Perplexity - Perplexity
• Introducing ChatGPT search | OpenAI - SearchGPT
• Genspark - Genspark
• Exa | Web Search API | AI Web Researcher - Embedding-based search engine
  • An embedding-based search engine built to overcome the limits of keyword-based search, returning only results that precisely match the user's intent
  • Works as a multi-step process based on the meaning of the query: vector embedding search across the entire web → LLM-based verification → refined results
  • Provides results in a table format rather than a typical list; you can add columns you want to collect additional information asynchronously
    • Each cell loads individually and asynchronously, designed so the web can be explored like a database
  • Accurately finds even specific, complex criteria such as "open-source developer tool startups based in San Francisco founded in 2021–2025"
  • An LLM evaluates whether each result actually meets the search criteria, and each is provided with evidence (sources) proving it matches the criteria
    • Processing can take from a few minutes to several hours
  • Example search: Math blogs created by math teachers outside the US
  • Demo video: https://youtu.be/Unt8hJmCxd4
  • API documentation: https://docs.exa.ai/websets

Deep learning

• jina - Open-source deep learning-based search engine

Projects

• Marginalia - Independently operated DIY search engine

AI

• ChatGPT
• Liner
• Genspark
• Phind
• Perplexity

See also

• Web crawler
• Web search engine
• TF-IDF
• BM25

Favorite site

• Wikipedia (en) Search engine
• SlideShare - The principles of a very simple search engine
• How do search engines work?
  • Implementing naive text search logic
  • Text search using the database's WHERE .. LIKE and REGEXP statements
  • Implementing search using the database's FULLTEXT index
  • Implementing an inverted index yourself
  • Scoring documents by implementing TF-IDF and BM25 yourself