RedGX

Architecture

Product Overview

RedGX stores AI requests in a queue and uses independent workers to send them to external inference engines. Results are stored separately, isolating AI workloads from the gateway.

Usage Example

# 1) Submit an embedding job
POST <GPU task endpoint>
API key: <deployment credential>
Content-Type: application/json

{"texts": ["Sentence 1", "Sentence 2"]}
# → 202 Accepted + request ID

# 2) Retrieve status and result
GET <GPU result endpoint>/{request ID}
API key: <deployment credential>
# → completion status + vectors

Requests use deployment-issued API keys and namespace-level read/write access. Endpoint paths and response schemas are supplied for each deployment.

API Interfaces

• Embedding Sentence and document vectorization requests
• Generation Text generation, summarization, and classification requests
• Translation Multilingual translation requests
• Speech-to-Text Audio-to-text requests
• OCR Document-image text recognition requests
• OpenAI Compatibility Compatible chat and embedding interfaces
• Async Controls Status, batch status, cancellation, and wait operations
• Operations Worker, model connection, and service-metric management

RedGX provides GPU task APIs and OpenAI-compatible chat and embedding interfaces, and operates independently from RedGW.

AI Task Brokering

• embedding → TEI
  Sentence/Document Embedding (Search/RAG)

  Sample Model

  bge-m3, etc.

  Interface

  HTTP API

• generation → vLLM
  Text Generation, Summarization, Classification

  Sample Model

  EXAONE, etc.

  Interface

  HTTP API

• translation → NLLB (Custom)
  Multilingual Translation

  Sample Model

  NLLB-200

  Interface

  HTTP API

• stt → faster-whisper
  Speech to Text

  Sample Model

  whisper large-v3

  Interface

  HTTP API

• ocr → PaddleOCR (Custom)
  Document Image to Text

  Sample Model

  PaddleOCR v5

  Interface

  HTTP API

Submit a POST request to receive a request ID, then retrieve the result with GET. Models and resource figures are integration examples; choose the engine required for each environment.

Inference Engine Isolation

RedGX workers do not load AI models directly. They forward requests to external inference engines.

• Fault Isolation — Separates inference errors and GPU memory pressure from the gateway.
• Fast Operations — Applies gateway configuration changes without reloading the model.
• Inference Performance — Uses batching and acceleration provided by dedicated engines.
• Flexible Models — Replaces inference engines without changing the gateway.

RedGX can connect to vLLM, TEI, or a separate inference server. The listed models are integration examples.

Requests Go to a Queue, Answers Come When Ready

Your service servers never sit waiting on a GPU. They hand the request to a queue and collect the result once processing finishes.

Models do not run inside RedGX — inference is delegated to a separate inference server.

Product Specifications

Version

0.x

Product Structure

Independent sibling product to RedGW

Execution Mode

Separated web API and background tasks

Storage

AI task queues, status, and results stored in Redis

AI Engine Integration

External inference engines such as vLLM and TEI

Security & Relay

Nginx-based HTTPS and rate limits

Status Monitoring

Service metrics and queue monitoring

Model Isolation

AI models run in external inference engines

Security & Compliance

Closed Networks

Deploys AI models and runtime images inside the internal network

Authentication & Access

API key authentication and namespace-level read/write access

Workload Isolation

Runs AI workloads in external inference engines

Transport & Limits

Nginx-based HTTPS and rate limits

Contact

info@cubiware.co.kr

Getting Started

1. Define the use case — Select the target AI function and model
2. Configure the environment — Import and connect runtime images and AI models
3. Check operations — Monitor request volume and processing status

Tune the batch size and maximum wait time for the expected usage pattern.