Build An AI Second Brain Knowledge Base (Step-By-Step)

Immutable sources plus generated pages is the right split. Keeping the clipped transcript untouched in raw and letting the agent write separate wiki pages means a bad summary can always be regenerated. The 28:38 reprocess, where he reruns every source under new rules, only works because the originals were kept.

Write-back on queries is the most useful idea in the video. Most note setups answer a question and forget it. Here the answer becomes a page with source links, so the next similar question starts from a better base. That is how a knowledge base compounds instead of just growing.

The pruning moment is a real lesson. Codex's first attempt produced 51 files for a spec that needs five. Asking the agent to stick to an explicit spec, then checking the folder yourself, is good practice for any agent-built system.

Cross-linking back to sources (20:50) fixes a common failure of AI notes: pages that sound confident but cannot be traced. Requiring a link to the original source makes each claim checkable.

Scale. The demo has nine videos. The agent answers by reading index.md and following links, which works while the index fits comfortably in context. With hundreds of sources the index gets long, answers slow down, and the agent starts missing pages. At that point you need search (keyword plus semantic) rather than an index read, and Wolfe does not cover it. His one nod to growth is the 32:17 suggestion to split people and companies into separate folders, which helps organisation but not retrieval.

Cost and model choice. An hourly run on a top model at high reasoning is not free, and most hours will find nothing to do. A cheaper check for new files, followed by a heavy model only when there is work, would cost far less. As of the May 2026 upload he does not discuss usage limits beyond noting the free tier is limited.

Review. The agent rewrites titles (one video became "discipline without willpower") and merges ideas across sources. Nothing in the flow asks a human to check those pages, so errors become permanent and later answers build on them. A simple fix is a weekly read of log.md and spot checks of new concept pages.

Privacy. The CRM holds contact details and the journal holds personal entries, and both get pushed to GitHub and sent to a cloud model on every run. A private repo helps, but it is worth deciding what belongs in an agent-readable vault at all.

Lock-in to one agent. The system only works inside Codex chats. The same vault could be read by Claude Code or another client, and he says so at 32:49, but the instructions are not tested anywhere else.

The pattern here is a compiled knowledge base: raw sources in, structured and linked pages out, answers grounded in your own material. The weak spot is that it lives inside one folder that one agent knows how to read. If you also work in ChatGPT, Claude or Cursor, each tool starts without that context.

The fix is to expose the knowledge base through a standard interface instead of a folder convention. MCP does that: any client that speaks it can search the same store. BrainTube takes this route, saving YouTube videos with their transcripts, articles, PDFs and podcasts into one compiled knowledge base that Claude, ChatGPT, Cursor and other MCP clients can search over MCP. The DIY route in this video is still worth building if you want full control of the files.

For the background, see what an MCP memory server is and one memory across ChatGPT, Claude and Cursor.