Skip to content
Search

Sidekick

lastmod 2025-04-08created 2025-04-084 edits

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

Open-source local-first LLM chatbot application for Mac

About

  • Provides information based on the user's files, folders, and websites, and works fully offline
  • Has a built-in, installable model inference engine; if needed, you can connect an OpenAI-compatible API key to use external models
  • Designed so the LLM can give more accurate answers based on contextual information
  • Based on llama.cpp

Key features

  • Local-first chatbot
    • All conversations happen offline, with no risk of data leaks
    • Searches information directly from files, folders, and websites without any extra installation
    • Usable with no knowledge of GPT prompts or configuration at all
    • Runs fast on Apple Silicon with the built-in inference engine
  • Expert-based context management (Expert System)
    • Create experts (Expert) by topic (Computer Science, History, etc.)
    • Register unlimited resources for each expert (RAG-based)
    • Easily add documents by drag and drop
    • Can include web search when needed to reflect up-to-date information
  • Support for various models
    • Can use various local reasoning models such as Alibaba's QwQ-32B and DeepSeek-R1
    • Can call external models when an OpenAI-compatible API key is entered
  • Built-in code interpreter
    • Runs code written by the model directly and reflects the results
    • Improves accuracy on math and logic questions
  • Canvas
    • Visually create and edit websites, code, and text
    • Select text and ask the LLM to edit just that selection
  • Image generation
    • Automatic image generation from text prompts
    • Recognized automatically from the prompt alone, with no separate mode switch
    • Runs on CoreML on macOS 15.2 or later
  • Inline writing assistant
    • Inline assistance in any app with Command + Control + I
    • Can enter an assignment question in Microsoft Word and get an automatic answer
  • Advanced Markdown rendering
    • Supports LaTeX math rendering
    • Data visualization: automatic graph generation from tables (bar charts, line charts, pie charts, etc.)
    • Supports code highlighting and code copy/export
  • Toolbox tools
    • Detector: detects whether text is AI-generated and provides rewriting guidance
    • Diagrammer: generates complex relationship diagrams from prompts
    • Slide Studio: creates slides from a one-line prompt, with export to PDF/PowerPoint
  • Performance and inference optimization
    • Very fast on Apple Silicon thanks to llama.cpp-based inference
    • Further speedup possible with speculative decoding
    • Provides battery life extension and compute distribution on MacBooks

Installation and requirements

  • Mac + Apple Silicon (M1 or later)
  • 8 GB RAM or more
  • Download the .dmg from the Releases page and run it

Development goals

  • A personal local AI assistant that anyone can use easily
  • A fully offline system that works without an internet connection
  • Recognizes folders, files, and websites instantly, with no setup
  • Privacy protection, no tracking of conversations
  • Transparency and extensibility as an open-source project

See also

Favorite site

author Zer0 sitemap