README.md

bigbrowser

A personal search engine for your browsing history.

Ever struggled to find that one article you read weeks ago? bigbrowser indexes the full content of every page you visit.

The browser extension records your browsing activity. The server indexes pages and powers the search.

Self-hosted: No cloud involved, your data stays yours
Full-text search: Search the content of every page you ever visited
Navigation trails: Visualize your browsing as a tree or interactive graph
Bookmarks: Save and search your bookmarked pages
Snapshots: Capture complete page snapshots including HTML, CSS, images, and media

Installation

Build the server

cargo build --release

The binary will be at ./target/release/bigbrowser.

Generate an API key

openssl rand -hex 32

Create a config file

Create bigbrowser.toml:

# Required: API key for the browser extension
api_key = "<your-api-key>"

# Server address (default: 127.0.0.1:3000)
addr = "127.0.0.1:3000"

# Data directory for bigbrowser.db and search index (default: XDG data dir, e.g. ~/.local/share/bigbrowser)
# data_dir = "."   # use current directory

# Web UI authentication (disabled by default, use --hash-admin-password to generate the hash)
# authenticated = true
# password_hash = "<generated-hash>"

Settings can also be set via environment variables with the BIGBROWSER_ prefix (e.g. BIGBROWSER_API_KEY, BIGBROWSER_DATA_DIR). The --data-dir flag overrides the data directory at runtime.

Run the server

Run from the directory containing bigbrowser.toml:

bigbrowser

Visit http://127.0.0.1:3000 to verify the server is running.

Installing the browser extension

A signed extension can be downloaded here: https://bigbrowser.tech/extension/. It is the recommended way to get started.

Configure the extension's preferences:

Instance URL: http://127.0.0.1:3000 (or your server address)
API key: the key from your config
Device name: pick an identifier for this browser, e.g. desktop

Building the extension

Build the extension:

cd bigbrowser_extension
make build

Install in Firefox:

Go to about:addons / Debug Add-ons
Load bigbrowser_extension/web-ext-artifacts/bigbrowser-X.X.X.zip as a temporary addon

Then configure the extension as described above.

Configuration

Web UI authentication

By default, the web UI has no authentication. To enable it:

First, generate a password hash:

bigbrowser --hash-admin-password

Then enable authentication in the config and set the password_hash:

authenticated = true
password_hash = "<generated-hash>"

Documentation

Full documentation is available at bigbrowser.tech

Development

DB migration

Migrations run in-process on startup. To run them manually (e.g. for a custom data dir), pass the database path (default: $XDG_DATA_HOME/bigbrowser/bigbrowser.db or ~/.local/share/bigbrowser/bigbrowser.db):

diesel migration run --config-file bigbrowser/diesel.toml --migration-dir bigbrowser/migrations/ --database-url="$HOME/.local/share/bigbrowser/bigbrowser.db"

Rebuild search index

bigbrowser --rebuild-index [--force]

Test custom scrapers

cargo run -p bigbrowser_scraper -- https://example.tld example.html scrapers/example.lua

License

This project is licensed under the GNU Affero General Public License v3 or later (see the LICENSE file).