Skip to main content

LibreApps Gateway Overview

The central hub for secure and scalable AI communication.

Overview​

The LibreApps Gateway is a high-performance API gateway built on top of Apache APISIX. It serves as the single entry point for all frontend requests to the backend AI and management services.

Key Responsibilities​

  • Routing: Directs incoming requests to the appropriate service (e.g., LibreChat, LiteLLM).
  • Authentication: Validates JWT tokens from Keycloak to ensure only authorized users can access the AI services.
  • Traffic Control: Provides a centralized layer for rate limiting, CORS, and request transformation.
  • Configuration Management: Uses etcd for dynamic, hot-reloadable configuration.
  • AI Integration: Specialized support for LLM routing and chat interfaces.

Architecture​

Why Use a Gateway?​

  • Simplified Frontend: The frontend only needs to know one URL (the Gateway) instead of multiple service URLs.
  • Centralized Security: Apply security policies in one place rather than in every microservice.
  • Performance: APISIX is built on NGINX and OpenResty, providing extremely low latency.

Best Practices​

  • ✅ Do this: Always route frontend requests through the Gateway.
  • ✅ Do this: Monitor the Gateway logs to identify performance bottlenecks or security threats.
  • ❌ Don't do this: Expose individual microservices directly to the public internet.