Orq.ai

Tool Introduction

Orq.ai is a platform designed for teams working on AI products, engineering, and governance; it brings together a multi-model gateway, an MCP tool gateway, agent execution capabilities, prompt management, as well as evaluation and observability tools in a single workspace. Developers can access various models through interfaces that are compatible with OpenAI and Anthropic’s calling methods, and they can apply uniform rules regarding budgeting, privacy, and routing during execution.

It does not present itself as a single large model; rather, it is an infrastructure that connects and manages models, tools, and agents. Currently, more than 500 models and over 30 providers are listed officially, and the specific list as well as prices may change as new providers emerge.

Main functions

AI gateway and multi-model routing

AI Gateway enables applications to call different providers or their own models through a single endpoint, and to switch between models without interrupting the upper-level business code.

• It supports using your own provider key, as well as the model quotas managed by Orq.ai.
• It offers conditional routing, weighted and round-robin load balancing, latency policies, and intelligent routing.
• It supports fallback across providers, retry mechanisms, and response caching, thereby reducing the impact of failures in a single model.
• It can connect to Azure AI Foundry, Vertex AI, or private and fine-tuned models with compatible interfaces.

Agents, RAG, and MCP tools

Teams can configure hosted agents, retrieval-augmented generation processes, and tool calls in AI Studio, and then run them using Agent Runtime.

• The MCP Gateway enables unified connection to remote MCP services, as well as the application of policies and tracking for tool calls.
• The knowledge base and RAG functions are used to import data and provide retrieval context for agents.
• Reusable skills, prompt versions, and deployment configurations help reduce the need to spread maintenance instructions throughout the code.

Evaluation, tracking, and governance

Each model request, agent action, tool invocation, or evaluation creates a queryable span that is used to analyze latency, cost, and output quality.

• It supports dataset experiments, built-in or custom evaluators, prompt comparison, and validation before going live.
• Budgets at the project, user, or key level can be set, along with alerts and fixed limits.
• It supports the detection and masking of sensitive information, audit logging, role-based permissions, and data retention policies.
• It is compatible with OpenTelemetry, facilitating the inclusion of existing agent connections into a unified tracking system.

Usage tutorial

1. Create an Orq.ai account and workspace, and generate an API key in the settings.
2. Choose to use Orq.ai for quota management or employ your own model provider keys, and check the requirements regarding the region where the models are located and data retention.
3. Change the base endpoints of existing clients to the Orq.ai gateway, or install the Python and Node SDKs for development.
4. Configure model routing, retry, fallback, budget, sensitive information rules, and log retention period.
5. Run evaluations using the test dataset, check accuracy, tool calls, costs, and latency before deploying it.
6. After going live, anomalies are monitored through tracking and dashboards, and the model, prompts, and protection rules are continuously adjusted.

Which users are it suitable for

• Engineering teams need to manage multiple LLMs through an interface in order to reduce vendor lock-in.
• Product teams are being established for customer service, internal assistants, RAG, and multi-tool intelligent agents.
• An AI platform team is needed to uniformly track costs, delays, quality, and failed connections.
• Enterprises with requirements for data residency, virtual private networks, auditing, permissions, and on-premises deployment.
• It is hoped that product managers and developers can work in the same environment to manage prompts, experiments, and released versions.

Prices and packages

Orq.ai currently offers two options: pay-as-you-go and enterprise customization. The cost of the models is calculated separately from the fees associated with the platform’s usage. Testing is possible even without providing billing information, but there are daily limits on the number of calls that can be made.

The pay-as-you-go plan also includes a monthly free quota; according to the official specifications, there are a maximum of 50 requests per day when no payment method is assigned. Optional add-ons, annual payment commitments, taxes, and enterprise bulk discounts are not included in the estimated costs, and the final figures will be indicated on the settlement page or in the contract.

Product advantages

• A gateway covers both the model and MCP tools, facilitating centralized management of policies, budgets, and tracking.
• It is compatible with common API protocols; existing applications usually require only minor modifications to start routing.
• Prompt engineering, evaluation, deployment, and production tracking are all available on the same platform, which reduces the costs associated with troubleshooting across different tools.
• The enterprise version supports proprietary VPCs and on-premises deployment, enabling stricter controls over networks and data.

Usage restrictions and precautions

• The fact that there are over 500 models does not mean that each model is available for every account, region, and mode; the model catalog and the console should be consulted for accurate information.
• A gateway adds an additional step to the processing process, and intelligent routing, retry mechanisms, and fallback options can also affect the end-to-end latency and costs.
• A single execution of an agent may involve multiple models, tools, and evaluation steps; therefore, the total cost cannot be estimated solely based on the number of executions.
• Logs may contain prompts, outputs, and metadata; it is necessary to implement masking, set access controls, and define an appropriate retention period.
• The terms of service specify that the platform shall not be used for health information subject to HIPAA regulations, and such data should not be sent directly to the service.
• Model outputs may still be incorrect or biased; high-risk operations require manual review, evaluation sets, and failover mechanisms.

Platform and deployment methods

APIs, SDKs, and open-source status

The platform offers gateway APIs that are compatible with the calling methods of OpenAI and Anthropic, as well as interfaces for deployment, agents, evaluation, and other functions. The official Python and Node SDKs support authentication, streaming responses, retry mechanisms, and deployment tasks; however, the documentation for the Python SDK is still marked as beta, so it is advisable to use a stable version in production environments.

On the official GitHub repository, one can find Python and TypeScript SDKs, examples, as well as some projects under the MIT license; however, the licenses for the two core SDK repositories are not clearly specified. The fact that something is publicly available does not necessarily mean that it can be used under an open-source license, and the Orq.ai hosting platform itself is not designated as open source either. It is necessary to check the license of each repository separately before using them.

Privacy and security

Orq.ai states that its cloud services are built and hosted in the European Union, offering transmission and static encryption, masking of sensitive information, and the possibility of configurable data deletion; it also claims that it does not use customers’ input data, output results, or training data for model development. In reality, these services are also subject to the retention policies of the third-party model providers chosen.

The pay-as-you-go plan offers standard data processing protocols; companies can negotiate terms regarding audit reports, data retention, and other specific conditions. When dealing with sensitive data, it is necessary to examine both Orq.ai’s handling procedures and those of the final model provider.

Frequently Asked Questions

Can Orq.ai be used for free?

A limited number of requests can be made per day even when no payment method is associated; the pay-as-you-go plan also includes monthly limits on gateway requests, spans, data volume, and agent operations. Charges are applied once these limits are exceeded, so it is not a service that offers unlimited use at no cost.

Is there still a charge for using the built-in model key?

The first 1 million key-based requests per month are free; for requests beyond that limit, Orq.ai charges a 4% fee, while the model provider continues to charge the user according to its own rules.

Is Orq.ai an open-source platform?

No. While some examples and tools are available under open-source licenses, the core hosting platform is not designated as open source; moreover, it is necessary to check each individual SDK repository for its license.