New API vs Private LLM
Side-by-side comparison · Updated September 2026
| Description | New API is a self-hosted AI gateway and model-management project maintained in the QuantumNous repository. It brings provider channels, access tokens, model restrictions, usage statistics and cost accounting into one interface. Teams supply their own authorized model-provider access and operate the gateway in their chosen environment. The gateway supports several API formats, including OpenAI-compatible requests, Claude Messages and Google Gemini. Compatibility has boundaries: the README marks Gemini-to-OpenAI conversion as text-only without function calling, and OpenAI-compatible to Responses conversion as in development. Test the exact endpoints, streaming behavior and tool calls your application uses before switching traffic. Routing features include weighted channel selection, automatic retry after failures and user-level model rate limits. The dashboard supports request-based, usage-based and cache-hit cost accounting, with token grouping and model-access controls. These controls can help organize usage, but retries and a common API format do not guarantee uninterrupted service or identical behavior across providers. The current repository license is AGPLv3. Docker Compose is the recommended quick-start path in the README, which also documents Docker commands with SQLite or MySQL. Running the software still involves infrastructure, operations and upstream model charges. Review the current license, secure your deployment and validate accounting against provider bills. Compare LiteLLM if you also want a Python SDK or a documented gateway for MCP servers and agents. | Private LLM runs text-based AI models locally on supported iPhones, iPads and Macs. It is a useful alternative for personal writing, summaries and questions when you want an on-device assistant instead of a hosted team workspace. Download a suitable model first; supported models and performance depend on your device and available memory. The US App Store lists a one-time price of $4.99. The developer describes access across supported Apple platforms with Family Sharing for up to six people and no subscription. Check your own store region for the current price and device requirements. Siri and Apple Shortcuts integration can connect the assistant to your existing workflows. The app privacy policy says conversations stay on-device and are not collected by the developer. This is distinct from the product website, which has analytics and a support form, and from the network requests needed to download models. Local processing is useful for controlling your chat data, but it does not establish that every model answer is correct or that every model will run well on every device. |
| Category | Developer Tools | AI Assistant |
| Rating | No reviews | No reviews |
| Pricing | Open Source | Paid |
| Starting Price | N/A | $4.99 in the US App Store |
| Plans | — |
|
| Use Cases |
|
|
| Tags | AI gatewayLLM routingmodel managementrate limitingusage accounting | local AIoffline chatiPhoneiPadMac |
| Features | ||
| Self-hosted AI gateway and model management | ||
| Provider channels, token groups and model restrictions | ||
| OpenAI-compatible, Claude Messages and Gemini format support | ||
| Documented limits on protocol conversion | ||
| Weighted channel selection and failure retries | ||
| User-level model rate limiting | ||
| Request, usage and cache-hit cost accounting | ||
| Docker Compose deployment documentation | ||
| On-device text-based language models | ||
| iPhone, iPad and Mac support | ||
| Offline chat after model download | ||
| Siri and Apple Shortcuts integration | ||
| Customizable system prompts | ||
| One-time purchase without a subscription | ||
| Family Sharing for up to six people | ||
| Conversation export as PDF or text | ||
| View New API | View Private LLM | |
Modify This Comparison
Also Compare
Explore more head-to-head comparisons with New API and Private LLM.