2026-04-11 | 📰 The Noise That Never Arrived 🔇
🔇 The Problem
📰 We launched a brand-new daily news digest series called The Noise. 🎉 The configuration was correct, the first post was generated, and the AGENTS.md prompt file was in place. 🤔 But when the user checked their Obsidian vault on their phone, there was nothing. 📭 No folder, no index, no posts. 🕵️ Time for a root cause analysis.
🔍 The Five Whys
1️⃣ Why did The Noise not appear in the Obsidian vault?
🗂️ Because the vault never received an index.md file for the new series directory. 📱 Without an index.md, Obsidian has no landing page for the folder and does not surface it in navigation. 🧭 Even though a post and AGENTS.md were theoretically synced, the vault had no way to present the series as a coherent collection.
2️⃣ Why was there no index.md for The Noise?
📄 Because the function that generates index files, called generateSeriesIndex, was defined in the codebase but never called from the main automation runner. 🧟 It was a dead function, fully implemented but completely disconnected from the execution pipeline.
3️⃣ Why was generateSeriesIndex never called?
🏗️ Because the blog series runner in RunScheduled.hs was built to sync exactly two types of files after generating a post: the post itself and the AGENTS.md file. 📝 The index.md was simply never added to the list of files to sync. 🤷 The function existed because someone anticipated the need, but the wiring was never completed.
4️⃣ Why did the existing series like Chickie Loo and Systems for Public Good already have working index files?
🤝 Because those series were bootstrapped manually. 📲 Their index.md files were created directly in the Obsidian vault on the phone, long before the auto-discovery system was built. 🧱 The automation never needed to create index files because all existing series already had them. 🕳️ This created a blind spot: the system worked for established series but could never bootstrap a new one.
5️⃣ Why did nobody catch this gap when the auto-discovery system was designed?
🧪 Because the auto-discovery system was tested against the three existing series, all of which had manually created vault infrastructure. 📋 The spec said the system would generate an index page on the first run, but that behavior was never implemented or tested. ✅ Everything looked green because the test environment matched production, where all series had been manually set up.
🎯 Root Cause
🌱 The root cause is a classic bootstrapping gap. 🏗️ The original series were set up by hand, and the automation was built on top of that manual foundation. 🔄 When the auto-discovery system was designed to make adding new series trivial, it correctly derived configurations and schedule entries from JSON files. 🚫 But it never implemented the vault bootstrapping step: creating the index.md and ensuring the AGENTS.md file lands in the vault regardless of whether a post is generated that run.
🔧 The Fix
🛠️ Four changes close the gap entirely.
📋 Always sync AGENTS.md
🔄 The AGENTS.md sync was previously buried inside the post-generation block. 🏃 If no new post needed to be generated that run (because one already existed for today), the AGENTS.md sync was skipped entirely. 🔝 We moved it to run before the post-generation check, so it syncs on every single run of the series task. 🆕 For a brand-new series like The Noise, this ensures the AGENTS.md file reaches the vault even on the very first run.
📇 Bootstrap index.md with create-if-not-exists
📝 We added a new function called ensureFileInVault that writes a file to the vault only if it does not already exist. 🆕 For new series, this creates the index.md with the correct format that Obsidian expects, including frontmatter with share, aliases, title, URL, and backlinks fields, plus a Dataview query that dynamically lists all posts. 🏠 For established series where Obsidian already owns the index.md, the function is a no-op. 🤝 This respects the principle that Obsidian is the source of truth for existing files while ensuring new series get properly bootstrapped.
📄 Sync pre-existing repo posts to vault
🔍 The very first post of a new series is typically committed directly to the git repo by Copilot before the automation ever runs. 🚫 Previously, only newly generated posts were synced to the vault. 📭 A post committed to git but never generated by the automation would sit in the repo forever without reaching the vault. 🔧 We added syncRepoPostsToVault, which iterates over all date-prefixed markdown files in the repo series directory and copies any that are missing from the vault. 🏃 This runs on every invocation, before post generation, ensuring pre-existing posts reach the vault on the very first automation run.
📐 Fix the index format
🔄 The old generateSeriesIndex function produced a minimal index with just a title and a TABLE-style Dataview query. 📱 The vault actually uses a richer format with aliases, a URL field, backlinks disabled, an inline page count using Dataview JavaScript, and a LIST-style query. 🎯 We updated the function to match the exact format used by existing series in the vault, ensuring consistency across all series.
🧪 Testing
🔬 We added twenty new tests covering the updated index generation, the ensureFileInVault function, and the new syncRepoPostsToVault function. 📋 The generateSeriesIndex tests verify the presence of every required frontmatter field, the home breadcrumb, the Dataview query format, the FROM clause, inline page count, and the LIST query style. 🗂️ The ensureFileInVault tests use temporary directories to verify that files are created when missing, not overwritten when already present, and that parent directories are created automatically. 📄 The syncRepoPostsToVault tests verify that pre-existing posts are synced, existing vault files are not overwritten, non-date files are ignored, and missing vault directories are created. ✅ All 1354 tests pass.
💡 Lessons Learned
🏗️ When building automation that replaces manual steps, enumerate every manual step that was previously performed and verify that the automation covers all of them. 🧪 Test with a truly new entity, not just existing ones that were set up by hand. 📋 When a spec says something will happen automatically, write a test that proves it actually does.
📚 Book Recommendations
📖 Similar
- The Checklist Manifesto by Atul Gawande is relevant because it demonstrates how systematic checklists prevent the exact kind of omission that caused this bug, where a known step was simply never wired into the process.
- Drift into Failure by Sidney Dekker is relevant because it explores how systems gradually diverge from their intended behavior through small, individually reasonable decisions, much like how manual bootstrapping became an invisible dependency.
- 🧱🛠️ Working Effectively with Legacy Code by Michael Feathers is relevant because it addresses the challenge of extending systems that were built on implicit assumptions, showing how to safely add automation around code that was never designed for it.
↔️ Contrasting
- Antifragile by Nassim Nicholas Taleb offers a contrasting perspective where systems that break under stress are seen as opportunities for improvement, whereas our system silently succeeded with a hidden gap rather than failing loudly.
🔗 Related
- 💺🚪💡🤔 The Design of Everyday Things by Don Norman explores how poor system design leads to user errors, paralleling how the automation’s silent omission of the index file created an invisible failure that only surfaced when a human checked their vault.