🖼️ Porting the Image Generation Pipeline to Haskell
🎯 The Mission
🏗️ The TypeScript image generation library was one of the largest and most complex modules in the codebase, weighing in at over a thousand lines.
🔄 It needed a faithful Haskell port that preserved all five image generation providers, the backfill orchestration logic, and the frontmatter manipulation system.
🧪 The result also needed thorough test coverage, something the TypeScript original lacked in unit test depth.
🏛️ Five Providers, One Interface
🎨 The core abstraction is an image provider config that wraps a name, API key, model identifier, a generator function, and an optional prompt describer.
☁️ Cloudflare Workers AI posts a JSON prompt to its inference endpoint and decodes the base64 image from the response.
🤗 HuggingFace Inference sends a JSON inputs payload and reads the raw image bytes directly from the HTTP response body.
🤝 Together AI posts a generation request with base64 JSON response format and decodes the image data from a nested JSON structure.
🌸 Pollinations is the only free provider, requiring no authentication, just a URL-encoded prompt in the GET path with model and size parameters.
🤖 Gemini handles two distinct paths, sending content generation requests for standard models and prediction requests for Imagen models, extracting inline data from the response candidates.
🧩 Pure Functions at the Core
📝 The module carefully separates pure logic from IO effects.
🔍 Functions like hasEmbeddedImage, shouldRegenerateImage, and isPostFile are completely pure, making them trivial to test.
✂️ The content cleaning pipeline strips frontmatter, embed sections, markdown syntax, code blocks, and table formatting through a chain of text transformations.
🏷️ The frontmatter updater works line by line, replacing existing fields in place and appending new ones, preserving the original YAML structure rather than round-tripping through a full parser.
🔄 The Backfill Orchestrator
📋 The backfill function scans multiple content directories for posts missing images, collecting candidates sorted by date descending.
🔄 When a provider hits a quota error or becomes unavailable, the orchestrator automatically switches to the next provider in the chain.
🎯 A configurable maximum images parameter prevents runaway generation, defaulting to processing one image per run.
📊 The result type tracks images generated, files updated, files skipped, modified file paths, and any errors encountered.
🧪 Test Coverage
✅ Sixty-five new tests bring the total test count from sixty-seven to one hundred thirty-two.
🔬 Property-based tests verify that buildImagePrompt never exceeds the two thousand forty-eight character limit, that sanitizeForYaml removes all dangerous quote characters, and that mimeTypeToExtension always produces a dotted extension.
🏗️ The provider resolution tests verify the correct ordering of all five providers and the use of default versus custom models from environment variables.
📐 Unit tests cover every pure utility function including embed detection, embed insertion, title extraction, YAML field updates, and embed removal.
🎓 Lessons Learned
🧠 Haskell’s type system caught several issues at compile time that would have been runtime surprises in TypeScript, particularly around the Either-based error handling versus exception throwing.
🔧 The explicit Manager parameter threading, while more verbose than global fetch, makes testing and resource management predictable.
📦 Separating regex helpers into their own internal section kept the module organized despite its size.
🏗️ The custom JSON module worked well for all five provider response formats without needing the full weight of aeson.
📚 Book Recommendations
📖 Similar
- 🧠 Real World Haskell by Bryan O’Sullivan, Don Stewart, and John Goerzen
- 🔧 Haskell in Depth by Vitaly Bragilevsky
- 🏗️ Production Haskell by Matt Parsons
📖 Contrasting
- 🌊 Programming TypeScript by Boris Cherny
- 🔄 Designing Data-Intensive Applications by Martin Kleppmann
📖 Creatively Related
- 🎨 The Art of Doing Science and Engineering by Richard Hamming
- 🧩 A Philosophy of Software Design by John Ousterhout