How to Build a Zero-Dependency Semantic Search Engine for Your Local Markdown Notes with Claude 3.5 Sonnet and LanceDB

No, you do not need to spin up a Docker container running PostgreSQL and pgvector just to search your personal markdown vault. You also do not need a bloated cloud enterprise search platform to find that half-remembered side-project idea you wrote down six months ago.

If you have a folder full of messy markdown notes, you can build a zero-dependency, serverless semantic search engine on your local machine in under 15 minutes. By combining LanceDB (an embedded vector database that runs entirely in-process) with Claude 3.5 Sonnet for synthesis, you can query your notes in plain English and get direct, cited answers.

Here is how to build a clean, local CLI tool that indexes your markdown files, generates embeddings, and lets you interrogate your personal knowledge base.

Why LanceDB and Claude 3.5 Sonnet?

Most vector search tutorials force you into a convoluted stack: Pinecone (requires API keys, cloud-only), Qdrant (requires Docker), or pgvector (requires a database migration plan). LanceDB is different. It is serverless, stores its data in a single directory on your disk (using the highly optimised Lance columnar format), and scales to millions of vectors without breaking a sweat.

For the reasoning and synthesis layer, we are using Claude 3.5 Sonnet. While smaller local models can handle basic summarisation, Sonnet's ability to maintain structural fidelity and avoid hallucinating details from your personal notes is unmatched. When you ask your CLI a question, Sonnet acts as the ultimate research assistant, extracting the exact context from your vectorized files.

Step 1: The Project Setup

We will use Python for this build. First, create a new directory and set up your virtual environment:

`bash mkdir local-mind-search cd local-mind-search python3 -m venv .venv source .venv/bin/activate `

Next, install the required packages. We need lancedb for vector storage, sentence-transformers for free local embeddings (saving our API budget), anthropic for the Claude integration, and rich to make our CLI look beautiful:

`bash pip install lancedb sentence-transformers anthropic rich pathspec `

Make sure your Anthropic API key is exported in your terminal:

`bash export ANTHROPIC_API_KEY="your-api-key-here" `

Step 2: Extracting and Chunking Markdown Safely

Before we can embed our notes, we need to parse them. Because markdown files can be wildly inconsistent, we will write a parser that respects frontmatter (metadata at the top of the file) and chunks the content by logical boundaries (paragraphs) rather than naive character limits.

Create a file named search.py and start with the imports and the chunking logic:

`python import os import lancedb import pyarrow as pa from pathlib import Path from sentence_transformers import SentenceTransformer from anthropic import Anthropic from rich.console import Console from rich.panel import Panel

console = Console()

def chunk_markdown_file(file_path: Path, chunk_size: int = 500) -> list[dict]: with open(file_path, "r", encoding="utf-8") as f: content = f.read()

Basic frontmatter stripping to clean up indexing if content.startswith("---"): parts = content.split("---", 2) if len(parts) >= 3: content = parts[2].strip()

paragraphs = [p.strip() for p in content.split("\n\n") if p.strip()] chunks = [] current_chunk = [] current_length = 0

for para in paragraphs: para_len = len(para.split()) if current_length + para_len > chunk_size and current_chunk: chunks.append(" ".join(current_chunk)) current_chunk = [para] current_length = para_len else: current_chunk.append(para) current_length += para_len

if current_chunk: chunks.append(" ".join(current_chunk))

return [ { "text": chunk, "file_name": file_path.name, "file_path": str(file_path) } for chunk in chunks ] `

Step 3: Vector Indexing with LanceDB

Now, we need to convert these text chunks into numerical vectors. What makes this workflow tick is our use of a local sentence transformer (all-MiniLM-L6-v2) to generate embeddings on your CPU. This means your personal data is not sent to an external embedding API just to be indexed.

Let's add the indexing logic to search.py:

`python DB_DIR = ".lancedb_data" TABLE_NAME = "notes"

Initialize local embedding model embed_model = SentenceTransformer("all-MiniLM-L6-v2")

def get_embedding(text: str): return embed_model.encode(text).tolist()

def build_index(notes_dir: str): db = lancedb.connect(DB_DIR) # Define our data schema using PyArrow schema = pa.schema([ pa.field("vector", pa.list_(pa.float32(), 384)), # 384 dimensions for MiniLM pa.field("text", pa.string()), pa.field("file_name", pa.string()), pa.field("file_path", pa.string()) ]) table = db.create_table(TABLE_NAME, schema=schema, exist_ok=True) all_chunks = [] for root, _, files in os.walk(notes_dir): for file in files: if file.endswith(".md"): file_path = Path(root) / file all_chunks.extend(chunk_markdown_file(file_path)) if not all_chunks: console.print("[bold red]No markdown files found in the specified directory![/bold red]") return

console.print(f"[yellow]Processing {len(all_chunks)} text chunks...[/yellow]") data_to_insert = [] for chunk in all_chunks: vector = get_embedding(chunk["text"]) data_to_insert.append({ "vector": vector, "text": chunk["text"], "file_name": chunk["file_name"], "file_path": chunk["file_path"] }) table.add(data_to_insert) console.print("[bold green]Local index updated successfully![/bold green]") `

Step 4: Interrogating Your Notes with Claude 3.5 Sonnet

With our notes vectorised and safely stored in LanceDB, we can perform a vector search to find the most relevant chunks. Then, we will pass those chunks to Claude, formatting them as dynamic context. This allows Claude to synthesize an answer based only on your files.

Add this final execution function to search.py:

`python def query_vault(query: str, limit: int = 4): db = lancedb.connect(DB_DIR) if TABLE_NAME not in db.table_names(): console.print("[bold red]Index not found. Please index your vault first using --index.[/bold red]") return table = db.open_table(TABLE_NAME) query_vector = get_embedding(query) # Search local DB results = table.search(query_vector).limit(limit).to_pandas() if results.empty: console.print("[yellow]No matching notes found.[/yellow]") return

Compile context context_blocks = [] for _, row in results.iterrows(): context_blocks.append(f"Source: {row['file_name']}\nContent:\n{row['text']}\n---") context_str = "\n".join(context_blocks) # Construct system prompt following RAG best practices # For details on structuring system messages, see our /glossary system_prompt = ( "You are an expert personal research assistant. Synthesise an answer to the user's query " "using only the provided context retrieved from their personal notes. " "If the answer cannot be found in the context, say so clearly—do not guess or make up information. " "Include brief markdown citations specifying which note files the information came from." ) user_prompt = f"Context from my files:\n{context_str}\n\nQuery: {query}" console.print("[yellow]Interrogating Claude...[/yellow]") client = Anthropic() message = client.messages.create( model="claude-3-5-sonnet-20241022", max_tokens=1000, temperature=0.2, system=system_prompt, messages=[ {"role": "user", "content": user_prompt} ] ) console.print(Panel(message.content[0].text, title="Synthesised Answer", border_style="green")) console.print("\n[bold dim]Sources retrieved for context:[/bold dim]") for _, row in results.iterrows(): console.print(f" - {row['file_name']} (path: {row['file_path']})") ```

Step 5: Gluing It All Together

To make this a genuine tool you actually use, we will expose a simple CLI interface using Python's standard argparse library at the bottom of search.py:

`python import argparse

if __name__ == "__main__": parser = argparse.ArgumentParser(description="Zero-dependency semantic search for markdown files.") parser.add_argument("--index", type=str, help="Path to the directory containing your markdown notes to build/refresh index") parser.add_argument("query", type=str, nargs="?", help="The query or question you want to ask your notes") args = parser.parse_args() if args.index: build_index(args.index) elif args.query: query_vault(args.query) else: parser.print_help() `

Running Your New Local Search

To test your tool, create a test folder of markdown notes or point it directly at your Obsidian vault. First, build the index:

`bash python search.py --index ~/Documents/ObsidianVault `

Once the embedding model completes its indexing, you can query your knowledge base:

`bash python search.py "What were my ideas for the smart home automation project?" `

Within seconds, LanceDB filters millions of possible text coordinates down to the relevant details on your disk, passes them to Claude, and spits out a clean, formatted synthesis with direct file references. If you run into issues with your Anthropic key or client setup, consult our troubleshooting resources at /platforms/claude/articles to get back on track.