Skip to main content

OpenAI

Integrate powerful AI models from OpenAI (GPT-4, GPT-3.5, GPT-5, and specialized models) or Azure OpenAI directly into your TheoBuilder workflows. This node can generate text, answer questions, analyze content, and perform complex reasoning tasks using your business data.

🧭 Quick Summary​

  • Variants: OpenAI and Azure OpenAI

  • Primary Purpose: Generate text, answer questions, analyze content, perform reasoning, and call functions/tools.

  • Key Difference: While both nodes behave the same inside ThreoAI, their authentication differs and so does their usage outside the platform.

    • OpenAI: API Token only
    • Azure OpenAI: Resource Endpoint + API Key + API Version + Deployment Name

    The difference mainly comes down to hosting, compliance, and access. OpenAI’s API is hosted directly by OpenAI, offering the latest models first and using a simple API key for authentication, with billing tied to an OpenAI account. Azure OpenAI, by contrast, runs on Microsoft Azure infrastructure in your chosen region, requires you to create model deployments (referenced by Deployment Name), and integrates with Azure subscription billing. While model availability may lag slightly compared to OpenAI direct, Azure provides stronger enterprise compliance (HIPAA, GDPR, FedRAMP) and regional data residency controls, making it the preferred option for organizations with strict governance requirements.

🔐 Authentication​

OpenAI Authentication​

  • API Token: Required
  • Configured in: ThreoAI Credentials → OpenAI

Azure OpenAI Authentication​

  • Resource Endpoint: e.g., https://<resource>.openai.azure.com/ (required)
  • API Key: (required)
  • API Version: (required)
  • Deployment Name: (required)
  • Configured in: ThreoAI Credentials → Azure OpenAI

📛 Node Purpose​

This node connects to OpenAI/Azure OpenAI to:

  • Generate text
  • Answer questions
  • Analyze and classify content
  • Perform reasoning tasks on business data
  • Call tools/functions as part of multipath workflows

⚙️ Configuration Settings​

The OpenAI & Azure OpenAI node is configured through its settings panel.

Required Settings (for basic use)​

  1. Authentication – OpenAI or Azure OpenAI credentials
  2. Model – The chosen model (OpenAI) or deployment (Azure)
  3. System Message – Sets behavior and role of the assistant
  4. Prompt Editor – Typically holds the user prompt or data from another node

Additional Model Settings​

  • Pass on Previous Messages

    • Default: Off
    • On: AI remembers previous messages for context
    • Off: Each request is processed independently
    • When to Change: Enable for chatbot scenarios where context matters
    • Impact: Better conversation quality but higher token usage/costs

  • Use Connected BotX Chatbot

    • Default: Off
    • On: Uses built-in chatbot conversation history automatically
    • Off: Requires manual setup of history source
    • When to Change: Enable when building chatbots with TheoBuilder’s chat interface
    • Impact: Simplifies chatbot development and ensures flow consistency

  • History Property Source

    • Defines the property carrying conversation history.
    • If an agent is published as a chatbot, selecting Use Connected BotX Chatbot automatically links the property carrier.

GPT-5 Specific Settings​

  • Reasoning Effort

    • Type: Dropdown — Low | Medium (default) | High
    • Description: How much computational effort the AI uses for reasoning
    • When to Change: High for complex problems, Low for simple tasks
    • Impact: Higher improves accuracy but increases response time & costs

  • Verbosity Level

    • Type: Dropdown — Low | Medium (default) | High
    • Description: Controls response length/detail
    • When to Change: Based on whether you need concise summaries or detailed explanations
    • Impact: Affects response length, processing time, and token usage

RAG Training Settings​

  • Training Style

    • Options: Questions & Answers | Text Documents
    • Default: Empty
    • Description: Format of your training data
    • When to Change: Q&A: Structured pairs of questions and answers. Good for training/evaluating direct responses. Clear, precise, but narrow.. Text Documents: Raw unstructured content. Feeds the retriever with context. Rich, flexible, but needs chunking and preprocessing.
    • Impact: Determines how effectively AI can answer from your knowledge base

  • Embedding Model

    • Description: Model used to convert text into vectors
    • Impact: Affects retrieval accuracy and relevance

  • Training Mode

    • Options: Full Training | Rebuild Embeddings | Rebuild Index Only | Fetch Data Only
    • Default: Full Training
    • Description: Defines how much of the training pipeline runs
    • Impact: Partial training saves time/resources for incremental updates

RAG Inference Settings​

  • Distance Function

    • Options: Cosine (default) | Euclidean | Manhattan | Chebyshev
    • Description: Method for similarity comparison
    • Impact: Cosine works best for most text; others may suit niche needs

  • Minimum Confidence Threshold

    • Type: Number (0.0–1.0)
    • Default: 0.0
    • Impact: Higher thresholds reduce irrelevant responses but may miss useful context

  • Top N Contexts

    • Type: Number
    • Default: 0 (automatic)
    • Impact: More contexts = more comprehensive answers, but higher cost/time

Other Options​

  • Model Parameters
    • Temperature, Max Tokens, Response Format (Text | JSON Object)
  • Tools
    • Define tool schemas for multipath workflows
  • Sources
    • Disabled by default; when enabled includes sources in response
  • Output
    • Set output property name (default: gpt_response)
    • Disable Omit to retain full raw response (required for ToolEval)
  • Training Model Button
    • Trains the model using connected Training Data nodes

📤 Outputs​

  • The node produces a JSON object, with the property name defined in the Output settings (default: gpt_response).
  • If Omit is disabled, the full raw response is returned.

🧪 Example Usage​

Example 1 — Function Calling with ToolEval​

  • Define a tool schema in the Tools section (e.g., getWeather)
  • Disable Omit in Output so tool_calls are included
  • Route tool calls using a downstream ToolEval node

Example 2 — RAG Workflow​

  • Configure RAG Training with Training Style + Embedding Model
  • Run inference with Top N Contexts and Confidence Threshold
  • Output enriched text with optional sources

📘 Best Practices​

  • Always configure a clear System Message
  • Use JSON Object response + strict prompting for structured outputs
  • Disable Omit when chaining to ToolEval
  • Pin specific model/deployment for stability
  • Enable Sources if you need traceable answers
  • Keep tool parameter schemas strict and well-documented