Skip to content
Read • Question • ConnectAnthony’s notes & researchReturn with a new question
Anthony’s NotesA personal field guide
From the notebook

Local AI

Below is the practical architecture for your setup:

Synology NAS = PDF storage
Mac Studio = indexing + local RAG server
ChatGPT / Claude = reasoning and synthesis
Obsidian = permanent notes/output

Concise setup steps

1. Organize PDFs on Synology

Create a clean folder on the NAS:

/Synology/PDF-Library/
  Christianity-Western-Morality/
  Philosophy/
  Foreign-Policy/
  AI/
  Health-Fitness/
  Obsidian-PKM/

Rename files clearly:

Holland - Dominion.pdf
Siedentop - Inventing the Individual.pdf
Freeman - Closing of the Western Mind.pdf
Nietzsche - Genealogy of Morals.pdf

2. Mount Synology on your Mac Studio

On Mac:

Finder → Go → Connect to Server

Use:

smb://YOUR-SYNOLOGY-NAME.local

or:

smb://192.168.x.x

Mount the PDF folder so your Mac sees it like a normal drive.

Example path:

/Volumes/PDF-Library/

3. Pick a local RAG tool

For your use case, start with one of these:

ToolBest for
AnythingLLMEasiest local RAG setup
Open WebUI + OllamaMore control, local models
KhojObsidian-friendly personal knowledge search
DifyMore advanced workflows
LlamaIndexDeveloper/custom setup

My recommendation for you:

Start with AnythingLLM.
Add Khoj later if you want tighter Obsidian integration.

4. Install a local model runner

Install Ollama on the Mac Studio.

Then install a model, for example:

ollama pull llama3.1:8b
ollama pull mistral
ollama pull nomic-embed-text

You need two model types:

Embedding model = turns PDFs into searchable vectors
Chat model = answers questions locally

For stronger reasoning, you can use:

Local RAG retrieval
+
ChatGPT/Claude for final synthesis

That hybrid setup is usually better than fully local reasoning.

5. Index your Synology PDFs

In AnythingLLM or your chosen RAG app:

Create Workspace → Add Documents → Select /Volumes/PDF-Library/

Then let it:

read PDFs → chunk text → create embeddings → build searchable index

This is the key step. After indexing, your NAS library becomes searchable by meaning.

6. Connect it to Obsidian

Keep your Obsidian vault local or on iCloud/Drive, for example:

/Users/Anthony/Obsidian/AnthonyVault/

Create folders:

Sources/
Literature Notes/
Permanent Notes/
MOCs/
Reading Plans/
Contradiction Maps/

Your workflow becomes:

Ask local RAG → retrieve passages → send to ChatGPT/Claude → produce Obsidian Markdown → paste/save into Obsidian

Optional later: use an Obsidian plugin or script to automatically save generated notes into your vault.

7. Connect local RAG to ChatGPT

There are two realistic ways.

Easiest way

Use local RAG to retrieve passages, then copy/paste into ChatGPT:

1. Ask local RAG: “Find passages about Christianity and moral equality.”
2. Copy retrieved passages with citations.
3. Paste into ChatGPT.
4. Ask ChatGPT to synthesize, compare, critique, and create Obsidian notes.

This is the simplest and most reliable version.

Advanced way

Expose your local RAG as an API and connect it through:

Custom GPT Action
or
local middleware
or
MCP-style tool workflow

But this requires technical setup: local server, API schema, authentication, possibly tunneling through Cloudflare Tunnel or Tailscale if ChatGPT needs to reach it.

For now, I would not start here.

8. Connect local RAG to Claude

Claude Desktop can connect more naturally to local tools through MCP.

Possible setup:

Claude Desktop
→ MCP filesystem server
→ access Obsidian vault and selected folders
→ optionally access local RAG/search tool

This is probably better than ChatGPT for direct local-folder workflows.

Recommended division:

Claude Desktop + MCP = local file/Obsidian workflows
ChatGPT = higher-level reasoning, research synthesis, polished notes
Local RAG = source retrieval

9. Your actual research workflow

Use this sequence:

1. Store PDFs on Synology.
2. Mount Synology on Mac Studio.
3. Index PDFs with AnythingLLM/Khoj.
4. Ask local RAG for relevant passages.
5. Send retrieved passages to ChatGPT or Claude.
6. Ask for:
   - topic grouping
   - claim extraction
   - author comparison
   - contradictions
   - Obsidian-ready notes
   - reading plan
7. Save outputs into Obsidian.

10. Minimal first version

Do this first, before building anything complex:

1. Install Ollama on Mac Studio.
2. Install AnythingLLM.
3. Mount Synology PDF folder on Mac.
4. Point AnythingLLM at that folder.
5. Index one test folder, not your whole library.
6. Ask: “Find all passages about Christianity and human rights.”
7. Copy the retrieved passages into ChatGPT.
8. Ask ChatGPT to create Obsidian notes and a reading plan.

That proves the ecosystem works.

Final ecosystem map

[Synology NAS PDFs]
        ↓ mounted via SMB
[Mac Studio]
        ↓
[AnythingLLM / Khoj / Local RAG]
        ↓ retrieves passages
[ChatGPT or Claude]
        ↓ synthesizes / critiques / formats
[Obsidian]
        ↓
Permanent notes, MOCs, contradiction maps, reading plans

Best starting stack

Synology NAS
+ Mac Studio
+ Ollama
+ AnythingLLM
+ ChatGPT
+ Obsidian

Add later:

Claude Desktop + MCP
Khoj
Zotero
automated note export
custom API bridge to ChatGPT

Yes, they are different.

AnythingLLM is the RAG application.
Ollama is the local model runner.

Think of it this way:

AnythingLLM = the research workspace / interface / document index
Ollama = the engine that runs AI models on your Mac

What AnythingLLM does

AnythingLLM handles the research-library side:

- creates workspaces
- imports PDFs
- chunks documents into passages
- creates/searches a vector database
- retrieves relevant passages
- gives you a chat interface over your documents
- manages document collections

So AnythingLLM is what lets you say:

Search my PDF library for passages about Christianity and moral equality.

It is the “library brain” or RAG layer.

What Ollama does

Ollama runs local AI models on your Mac Studio.

It handles things like:

- downloading local models
- running Llama, Mistral, Qwen, Gemma, etc.
- running embedding models
- serving those models to apps like AnythingLLM

So Ollama is what provides the model that AnythingLLM uses.

Example:

ollama pull llama3.1:8b
ollama pull nomic-embed-text

Then AnythingLLM can use those models.

Why both are useful together

A local RAG setup usually needs three pieces:

1. Documents
   Your PDFs on Synology
 
2. RAG app
   AnythingLLM
 
3. Models
   Ollama models

The pipeline looks like this:

Synology PDFs
→ AnythingLLM imports and chunks them
→ Ollama embedding model turns chunks into searchable vectors
→ AnythingLLM searches the vector database
→ Ollama chat model answers from retrieved passages

Do you have to use Ollama with AnythingLLM?

No.

AnythingLLM can use different model providers.

You can run it with:

- Ollama for local models
- OpenAI models
- Anthropic/Claude models
- LM Studio
- local or cloud embedding providers

So there are two broad setups.

Setup A: Fully local

AnythingLLM + Ollama

Benefits:

- private
- runs on your Mac
- no need to upload passages to a cloud model
- good for search and basic Q&A

Tradeoff:

- local models are usually weaker than ChatGPT/Claude
- setup is a little more technical
- very large libraries can require cleanup/tuning

Setup B: Hybrid, probably best for you

AnythingLLM for retrieval
+
ChatGPT or Claude for reasoning

In this setup, AnythingLLM finds the relevant passages from your local PDF library. Then you paste or send those passages into ChatGPT or Claude for deeper analysis.

That gives you:

local RAG = finds the right source material
ChatGPT/Claude = does the high-level synthesis
Obsidian = stores the final notes

This is probably the best version for your research workflow.

Simple analogy

Synology = bookshelf
AnythingLLM = librarian and card catalog
Ollama = local reader/assistant
ChatGPT/Claude = expert analyst/editor
Obsidian = your permanent notebook

Bottom line

I recommended AnythingLLM first because it is the user-facing RAG system that organizes and searches your PDFs.

I mentioned Ollama because AnythingLLM needs some model provider. Ollama is the easiest way to run local models on your Mac Studio.

But you do not have to rely on Ollama for the final thinking. The strongest setup is:

AnythingLLM indexes/searches your Synology PDFs locally.
ChatGPT or Claude performs the deeper reasoning.
Obsidian stores the finished notes.

On a Mac Studio M2 Max, local RAG is very feasible. The heavy part is not the “RAG” itself; the heavy part is whichever local AI model you run through Ollama or another model runner.

The basic split

Indexing PDFs / embeddings = moderate workload
Searching your library = light workload
Running local chat models = moderate to heavy workload
Running large local models = heavy workload
Using ChatGPT/Claude for reasoning = very light local workload

1. If you use local RAG only for retrieval

This is the setup I’d recommend for you:

Synology PDFs
→ Mac Studio indexes/searches locally
→ retrieved passages sent to ChatGPT/Claude
→ Obsidian notes created from the answer

In this version, your Mac Studio only handles:

  • reading PDFs
  • OCR if needed
  • splitting documents into chunks
  • creating embeddings
  • searching the vector database

That is very manageable.

Expected impact:

Indexing: noticeable CPU/GPU use while building the index
Searching: very light
Daily use: low

Your Mac may spin up a bit while indexing a big folder, but normal search afterward should feel easy.

2. If you run a small local model

Example models:

llama3.1:8b
mistral:7b
qwen2.5:7b
gemma2:9b

These should run fine on an M2 Max.

Expected behavior:

RAM use: around 6–12 GB depending on model/quantization
Speed: usable
Heat/fans: mild to moderate
Best for: basic Q&A, document chat, extraction, tagging, simple summaries

This is fine for local RAG.

3. If you run medium local models

Example:

qwen2.5:14b
llama3.1:70b quantized
mixtral variants

Expected behavior:

RAM use: 12–45+ GB depending on model
Speed: slower
Heat/fans: more noticeable
Best for: better local reasoning, but still weaker than frontier ChatGPT/Claude

Whether this is practical depends on how much unified memory your Mac Studio has.

Rough memory guide

Model sizePractical on M2 Max?Notes
7B / 8BYesBest starting point
13B / 14BYesBetter quality, still practical
30B / 32BMaybeNeeds more RAM, slower
70BOnly if high RAMSlow; not ideal for daily use
Frontier-level reasoningNoUse ChatGPT/Claude

4. If your Mac Studio has 32 GB RAM

Use:

nomic-embed-text for embeddings
llama3.1:8b or qwen2.5:7b for local chat

Avoid trying to run large 70B models regularly.

Best workflow:

Local RAG retrieval
+
ChatGPT/Claude final reasoning

5. If your Mac Studio has 64 GB RAM

You have more room.

Good options:

qwen2.5:14b
llama3.1:8b
mistral:7b
nomic-embed-text

You can experiment with larger quantized models, but I’d still use ChatGPT/Claude for serious synthesis.

6. If your Mac Studio has 96 GB RAM

You can run larger local models more comfortably.

Possible:

32B models
some 70B quantized models
larger context windows
more simultaneous indexing/search tasks

But even then, frontier cloud models will usually beat local models for:

  • deep synthesis
  • contradiction detection
  • philosophical analysis
  • complex writing
  • multi-document reasoning
  • polished Obsidian note generation

7. What will actually stress the Mac?

Most stressful

Running large local models
OCR on many scanned PDFs
Indexing thousands of documents at once
Using huge context windows
Running multiple models simultaneously

Least stressful

Searching an already-built index
Retrieving passages
Using ChatGPT/Claude for final reasoning
Creating Obsidian Markdown

8. For your setup, I would not overuse local models

Your best setup is hybrid:

Mac Studio:
- index PDFs
- search PDFs
- retrieve source passages
 
ChatGPT/Claude:
- analyze
- compare authors
- find contradictions
- create Obsidian notes
- build reading plans

This gives you the best quality without making your Mac Studio do all the reasoning.

Recommended starting setup

Install:

AnythingLLM
Ollama
nomic-embed-text
llama3.1:8b or qwen2.5:7b

Use local models for:

search
basic Q&A
tagging
first-pass summaries
source retrieval

Use ChatGPT/Claude for:

deep synthesis
argument maps
author comparison
contradiction detection
final Obsidian notes
reading plans

Bottom line

Your Mac Studio M2 Max is strong enough for a local RAG system.

The sensible version will not feel like a huge burden:

Local retrieval/indexing = very doable
Small local models = fine
Medium models = probably fine depending on RAM
Large models = possible but slow/heavy
Frontier-level analysis = still better in ChatGPT/Claude

For your research workflow, you do not need to run a giant local model. You mainly need your Mac to act as a private search/indexing machine, then let ChatGPT or Claude do the harder intellectual work.

The benefit depends on whether AnythingLLM’s Obsidian connection is read-only import or write/export. Most likely, it is mainly useful for making your Obsidian vault searchable inside AnythingLLM, not for automatically writing finished notes back into Obsidian.

AnythingLLM’s changelog says its Obsidian connector for desktop went live in v1.8.4. Source: https://docs.anythingllm.com/changelog/v1.8.4

Main benefit

It lets AnythingLLM treat your Obsidian vault as another knowledge source.

So instead of searching only:

Synology PDFs

you can search:

Synology PDFs
+
your Obsidian notes

That means your AI can compare:

What the PDF says
vs.
What your own notes say

That is the biggest advantage.

How this improves your workflow

1. It can search your existing notes

You can ask:

What have I already written in Obsidian about Christianity and moral equality?

or:

Find my notes related to Siedentop, Holland, Freeman, Nietzsche, natural law, and human rights.

This prevents duplicate notes and helps you reuse your own thinking.

2. It can compare PDFs against your notes

This is probably the highest-value use.

Example:

Compare my Obsidian notes on Christianity and Western morality against the PDFs in this workspace. Where do my notes agree, oversimplify, or contradict the source material?

That is useful because your Obsidian vault becomes part of the evidence base, not just the final storage place.

If AnythingLLM can read your vault, it can suggest links like:

[[Historical Influence vs Philosophical Justification]]
[[Christianity and Moral Equality]]
[[Nietzsche on Slave Morality]]
[[Natural Law]]
[[Human Rights]]

This helps turn AI output into better Obsidian-native notes.

4. It can help build MOCs from what already exists

You can ask:

Using my Obsidian vault, create a Map of Content for Christianity and Western morality. Include existing notes and suggest missing notes.

That is better than asking ChatGPT to invent an MOC from scratch.

5. It can detect gaps in your vault

Example:

Based on my PDFs and my Obsidian notes, what topics have source material but no permanent note yet?

That is exactly the kind of maintenance task a RAG system is good at.

What it probably does not do

Unless the connector specifically says it has write permissions, assume it does not automatically create or edit Obsidian notes.

So the likely limitation is:

Good:
Read/search/index your Obsidian vault
 
Maybe not:
Write new notes directly into your vault

For writing notes into Obsidian, your better options are still:

1. Save Markdown into an AI Inbox folder
2. Use an Obsidian URI
3. Use Claude Desktop + MCP filesystem access
4. Use an Obsidian plugin that talks to AnythingLLM’s API

Best way to use the Obsidian connector

Use it for retrieval, not final storage.

Your workflow becomes:

Synology PDFs
→ AnythingLLM indexes source documents
 
Obsidian vault
→ AnythingLLM indexes your notes
 
AnythingLLM
→ compares sources against your notes
 
ChatGPT/Claude
→ final synthesis and polish
 
Obsidian
→ stores final permanent notes

Best prompt after connecting Obsidian

Use this:

Using both my PDF library and my Obsidian vault, analyze my current research on Christianity and Western morality.
 
Return:
 
1. What my existing notes already cover
2. Which PDFs support those notes
3. Where my notes oversimplify the source material
4. Contradictions between authors
5. Contradictions inside my own notes
6. Missing notes I should create
7. Suggested [[internal links]]
8. A revised MOC structure

Bottom line

Connecting AnythingLLM to Obsidian is useful because it lets the AI reason over both your sources and your existing knowledge system.

For you, the main benefit is:

It can compare your PDF library against your Obsidian notes and tell you what is missing, duplicated, weak, or contradictory.