2026-03-28 | 🧹 Ripping Out the Vault Cache
🎯 The Problem
🔥 Bidirectional sync with stale cached data was deleting the entire Obsidian vault.
😱 The root cause was a caching mechanism designed for speed that introduced catastrophic data loss risk.
🔄 The Obsidian headless CLI operates in bidirectional mode: it mirrors local state to remote and vice versa.
💣 When a cached vault directory contained only a partial set of files, the sync engine interpreted every missing remote file as a local deletion and propagated those deletions upstream, wiping out the vault.
🏗️ The Old Architecture
📦 The previous system had two sync paths: warm cache and cold cache.
♻️ The warm cache path reused a vault directory persisted between runs via GitHub Actions cache, skipping the expensive sync-setup step.
❄️ The cold cache path cleared the directory and ran a full sync-setup when the warm path failed or was unavailable.
🔁 Each scheduled task independently pulled the vault, edited files, and pushed changes, resulting in multiple bidirectional sync round-trips per run.
🧩 The complexity of warm-versus-cold path selection, fallback logic, and per-task sync cycles created many opportunities for partial state to sneak through.
✅ The New Architecture
🎯 The new design follows one simple principle: pull once, edit, push once.
📥 At the beginning of every scheduled run, the vault is pulled fresh into a new directory inside an ephemeral CI container.
🏗️ Since there is no caching, the vault directory simply does not exist at the start of a run, so there is nothing to clear or delete.
✏️ Each task receives the vault directory as a parameter and operates on it directly without any sync operations.
📤 At the very end of the run, a single push sends all accumulated changes back to the vault.
🛑 The pre-push circuit breaker remains as a safety net, refusing to push if any files have been lost relative to the post-pull baseline.
🔧 What Changed
🗑️ Removed
- 🚫 Warm cache path and all warm-cache detection logic
- 🚫 GitHub Actions vault cache step and the OBSIDIAN_VAULT_CACHE_DIR environment variable
- 🚫 Per-task vault pull and push operations
- 🚫 The getVaultCacheDir helper function
➕ Added
- ✏️ A writeEmbedsToNote function that writes embed sections directly to a note file without pulling or pushing the vault
- 📥 A single vault pull at the beginning of the scheduled run in main
- 📤 A single vault push at the end of the scheduled run in main
♻️ Refactored
- 🔄 syncObsidianVault now always performs a cold sync with no warm cache parameter
- 🎯 All task runners accept a vault directory parameter instead of syncing internally
- 📝 Social posting (autoPost) accepts a vault directory parameter and uses writeEmbedsToNote instead of the full pull-write-push appendEmbedsToObsidianNote cycle
- 📋 The appendEmbedsToObsidianNote convenience wrapper remains available for standalone scripts that need their own pull-edit-push cycle
📊 Impact
🔢 Net reduction of about 120 lines of code across Haskell and TypeScript.
🛡️ Eliminated the entire class of bugs where partial cached state triggers bidirectional deletion propagation.
⚡ Reduced the number of ob sync operations per scheduled run from roughly two per task (a pull and a push for each of up to eight tasks) down to exactly two total (one pull, one push).
🧠 The mental model is now trivially simple: fresh pull, local edits, final push.
🧪 Verification
✅ All 245 Haskell tests pass after the refactor.
🔧 The Haskell project compiles cleanly.
📋 The spec document has been updated to reflect the new pull-edit-push architecture.
📚 Book Recommendations
📖 Similar
- Release It! by Michael T. Nygard
- Designing Data-Intensive Applications by Martin Kleppmann
- Site Reliability Engineering by Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy
📖 Contrasting
- Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim
- The Mythical Man-Month by Frederick P. Brooks Jr.
📖 Creatively Related
- Thinking in Systems by Donella H. Meadows
- Normal Accidents by Charles Perrow
- The Design of Everyday Things by Don Norman