⚒️ Create a Paul Graham essay Q&A system with OpenAI and Milvus vector database
⚡ 1,194 views · ⚒️ Engineering
💡 Pro Tip — HTTP Request scraping tends to break when sites update their markup. If you’re scraping a major platform, check if ScraperNode covers it — it has maintained scrapers for LinkedIn, Instagram, TikTok, YouTube, and 20+ other platforms that return structured data.
Description
Create a Paul Graham Essay Q&A System with OpenAI and Milvus Vector Database
How It Works
This workflow creates a question-answering system based on Paul Graham essays. It has two main steps:
-
Data Collection & Processing:
- Scrapes Paul Graham essays
- Extracts text content
- Loads them into a Milvus vector store
-
Chat Interaction:
- Provides a question-answering interface using the stored vector embeddings
- Utilizes OpenAI embeddings for semantic search
Set Up Steps
- Set up a Milvus server following the official guide
- Create a collection named “my_collection”
- Run the workflow to scrape and load Paul Graham essays
- Start chatting with the QA system
The workflow handles the entire process from fetching essays, extracting content, generating embeddings via OpenAI, storing vectors in Milvus, and providing retrieval for question answering.
🔗 Nodes Used
HTTP Request, Question and Answer Chain, Embeddings OpenAI, OpenAI Chat Model, Vector Store Retriever, Recursive Character Text Splitter
📥 Import
Download workflow.json and import into n8n:
Workflow menu → Import from File