🔬 Analyze images, videos, documents & audio with Gemini Tools and Qwen LLM Agent

⚡ 4,379 views · 🔬 Document Extraction & Analysis

Description

📁 Analyze uploaded images, videos, audio, and documents with specialized tools — powered by a lightweight language-only agent.

🧭 What It Does

This workflow enables multimodal file analysis using Google Gemini tools connected to a text-only LLM agent. Users can upload images, videos, audio files, or documents via a chat interface. The workflow will:

• Upload each file to Google Gemini and obtain an accessible URL.
• Dynamically generate contextual prompts based on the file(s) and user message.
• Allow the agent to invoke Gemini tools for specific media types as needed.
• Return a concise, helpful response based on the analysis.

🚀 Use Cases

• Customer support: Let users upload screenshots, documents, or recordings and get helpful insights or summaries.
• Multimedia QA: Review visual, audio, or video content for correctness or compliance.
• Educational agents: Interpret content from PDFs, diagrams, or audio recordings on the fly.
• Low-cost multimodal assistants: Achieve multimodal functionality without relying on large vision-language models.

🎯 Why This Architecture Matters

Unlike end-to-end multimodal LLMs (like Gemini 1.5 or GPT-4o), this template:

• Uses a text-only LLM (Qwen 32B via Groq) for reasoning.
• Delegates media analysis to specialized Gemini tools.

✅ Advantages

🧪 How It Works

🔹 Input via Chat

• Users submit a message and (optionally) files via the chatTrigger.

🔹 File Handling

• If no files: prompt is passed directly to the agent.

• If files are included:

  • Files are split, uploaded to Gemini (to get public URLs).
  • Metadata (name, type, URL) is collected and embedded into the prompt.

🔹 Prompt Construction

• A new chatInput is dynamically generated:

  User message
  Media: [array of file data]

🔹 Agent Reasoning

• The Langchain Agent receives:

  • The enriched prompt

  • File URLs

  • Memory context (15 turns)

  • Access to 4 Gemini tools:

    • IMG: analyze image
    • VIDEO: analyze video
    • AUDIO: analyze audio
    • DOCUMENT: analyze document

The agent autonomously decides whether and how to use tools, then responds with concise output.

🧱 Nodes & Services

⚙️ Setup Instructions

1. 🔑 Required Credentials

• Groq API key (for Qwen 32B model)
• Google Gemini API key (Palm / Gemini 1.5 tools)

2. 🧩 Nodes That Need Setup

• Replace existing credentials on:

  • Upload a file
  • Each GeminiTool (IMG, VIDEO, AUDIO, DOCUMENT)
  • lmChatGroq

3. ⚠️ File Size & Format Considerations

• Some Gemini tools have file size or format restrictions.
• You may add validation nodes before uploading if needed.

🛠️ Optional Improvements

• Add logging and error handling (e.g., for upload failures).
• Add MIME-type filtering to choose the right tool explicitly.
• Extend to include OCR or transcription services pre-analysis.
• Integrate with Slack, Telegram, or WhatsApp for chat delivery.

🧪 Example Use Case

> “Hola, ¿qué dice este PDF?”

Uploads a document → Agent routes it to Gemini DOCUMENT tool → Receives extracted content → LLM summarizes it in Spanish.

🧰 Tags

multimodal, agent, langchain, groq, gemini, image analysis, audio analysis, document parsing, video analysis, file uploader, chat assistant, LLM tools, memory, AI tools

📂 Files

• This template is ready to use as-is in n8n.
• No external webhooks or integrations required.

🔗 Nodes Used

AI Agent, Simple Memory, Chat Trigger, Groq Chat Model, Google Gemini

📥 Import

Download workflow.json and import into n8n: Workflow menu → Import from File

📖 Importing guide · 🔑 Credential setup