LLMStack vs New API
Side-by-side comparison · Updated September 2026
| Description | LLMStack is a source-available builder for AI agents, workflows and chatbots that combine model calls with your own data. Its visual builder can chain multiple models and connect data sources, making it more suitable for assembling an application or process than for simply opening a personal chat app. The project documents deployment on your own infrastructure and points to Promptly as its hosted offering. Supported data inputs include documents, websites and connected sources such as Google Drive and Notion. The builder provides preprocessing and vectorization for retrieval workflows. Apps can be shared publicly or with selected people, and viewer and collaborator permissions control access to shared work. The repository also documents HTTP API access and Slack or Discord triggers. Plan for infrastructure, model-provider usage, credentials and permissions as separate decisions. Installing a self-hosted builder does not make externally hosted models free or keep every data request local. Start with a limited workflow and representative documents, review the generated output, and confirm the permissions required by each connected source before expanding access. Compare AnythingLLM when a document-chat workspace is the main need; LLMStack is oriented toward composing the application and its workflow. | New API is a self-hosted AI gateway and model-management project maintained in the QuantumNous repository. It brings provider channels, access tokens, model restrictions, usage statistics and cost accounting into one interface. Teams supply their own authorized model-provider access and operate the gateway in their chosen environment. The gateway supports several API formats, including OpenAI-compatible requests, Claude Messages and Google Gemini. Compatibility has boundaries: the README marks Gemini-to-OpenAI conversion as text-only without function calling, and OpenAI-compatible to Responses conversion as in development. Test the exact endpoints, streaming behavior and tool calls your application uses before switching traffic. Routing features include weighted channel selection, automatic retry after failures and user-level model rate limits. The dashboard supports request-based, usage-based and cache-hit cost accounting, with token grouping and model-access controls. These controls can help organize usage, but retries and a common API format do not guarantee uninterrupted service or identical behavior across providers. The current repository license is AGPLv3. Docker Compose is the recommended quick-start path in the README, which also documents Docker commands with SQLite or MySQL. Running the software still involves infrastructure, operations and upstream model charges. Review the current license, secure your deployment and validate accounting against provider bills. Compare LiteLLM if you also want a Python SDK or a documented gateway for MCP servers and agents. |
| Category | AI Assistant | Developer Tools |
| Rating | No reviews | No reviews |
| Pricing | Unknown | Open Source |
| Starting Price | N/A | N/A |
| Plans |
| — |
| Use Cases |
|
|
| Tags | Open sourceAI agentsWorkflowsApplicationsData | AI gatewayLLM routingmodel managementrate limitingusage accounting |
| Features | ||
| Visual AI workflow and model-chain builder | ||
| Data imports from documents, websites and connected services | ||
| Document preprocessing and vectorization | ||
| Viewer and collaborator permissions | ||
| Self-hosted deployment instructions | ||
| Hosted offering through Promptly | ||
| HTTP API access for apps and chatbots | ||
| Slack and Discord workflow triggers | ||
| Self-hosted AI gateway and model management | ||
| Provider channels, token groups and model restrictions | ||
| OpenAI-compatible, Claude Messages and Gemini format support | ||
| Documented limits on protocol conversion | ||
| Weighted channel selection and failure retries | ||
| User-level model rate limiting | ||
| Request, usage and cache-hit cost accounting | ||
| Docker Compose deployment documentation | ||
| View LLMStack | View New API | |
Modify This Comparison
Also Compare
Explore more head-to-head comparisons with LLMStack and New API.