Fireworks AI

What is Fireworks AI?

Fireworks AI is a generative AI infrastructure platform provided by Fireworks.ai, Inc., designed for developers and enterprises that need to invoke, fine-tune, and deploy open-weight models. It brings together model catalogs, inference interfaces, training tasks, dedicated GPU deployment, quotas, and monitoring capabilities within a single platform.

Fireworks AI is not a single chatbot, nor is it equivalent to any one specific model. Users need to choose a particular model, as well as the level of data throughput and the method of deployment; the license costs, the length of context allowed, the types of input data, the quality of output, and the price can all vary.

Core product format

Serverless model inference

Serverless offers popular text, visual, and embedded models; the caller pays only for the actual tokens used, without the need to choose a GPU, configure replicas, or deal with cold starts. Standard is the default layer, Priority is enabled through request parameters, while Fast makes use of separate high-performance models.

• Chat and text completion support streaming output, tool invocation, and structured results.
• The visual model receives text and image content; the actual format and size are determined by the constraints of the model.
• Embedding models convert text into vectors, which are suitable for retrieval, clustering, and similarity calculation.
• Prompt caching can reduce the costs associated with repeated contexts, but its effectiveness depends on routing and content consistency.
• Batch inference is charged at 50% of the regular Serverless input and output prices, and it is suitable for large-scale tasks that do not require real-time processing.

Shared Serverless is a best-efforts service that does not offer any guarantees regarding latency or availability. Models may become unavailable, and common models are usually notified at least two weeks in advance; if a fixed version and specific performance levels are required, dedicated deployment should be considered.

Compatible interfaces between OpenAI and Anthropic

Fireworks offers interfaces that are compatible with the OpenAI chat completion format as well as the Anthropic Messages format; existing applications usually only need to have their service addresses, keys, and model identifiers changed. Compatibility does not mean that all parameters, model behaviors, or boundary conditions are identical, so it is still necessary to test each element individually before making the transition.

• It supports synchronous, asynchronous, and streaming responses; the last piece in the streaming response can include information on the remaining token count.
• When the model context overflows, the maximum output may be automatically reduced by default; it can also be set to trigger an error instead.
• The inference key should be stored on the server side; it must not be placed in the web frontend or in public repositories.
• The capabilities for tool invocation, reasoning over fields, and generating structured outputs should be verified based on the specific model.

Model fine-tuning and training

Supervised fine-tuning of SFT

SFT trains models using dialogue examples that include ideal responses; it is suitable for expressing domain-specific knowledge, maintaining a fixed format, ensuring consistency in brand tone, and handling tool calls. The data is presented in an OpenAI-compatible JSON format with each line representing a separate element, and can be uploaded through a user interface, command line, API, or Python SDK.

Preference optimization DPO

DPO utilizes pairs of preferred and non-preferred responses under the same prompt to help the model better align with quality, style, or safety preferences. The current data format and session structure have specific constraints, and data validation must be carried out prior to training.

Enhanced fine-tuning RFT

RFT uses evaluators or reward functions to optimize the performance of models in reasoning, tool usage, and multi-turn Agent tasks. Teams can use managed processes, or they can integrate their own evaluation logic and training loops into the Training API.

LoRA versus full-parameter training

LoRA modifies behavior through smaller adapters, and its deployment cost is related to the strategy used for the base model; full-parameter training is suitable for tasks that require more substantial changes in behavior, but it incurs higher costs in terms of data, computing power, and management. Fine-tuned models as well as those uploaded by users require on-demand, dedicated deployment – they cannot be directly placed on shared Serverless platforms.

Training workflow

1. Define the quality objectives, benchmark set, types of failure, and deployment thresholds.
2. Check the base model’s availability, context length, and supported training methods from the model directory.
3. Clean the data by removing keys, personal sensitive information, and content that is not authorized for use.
4. Generate the training set and independent evaluation set in SFT, DPO, or RFT format.
5. Upload and validate the data; first use small tasks to check the format, loss, and sample quality.
6. Start the training and record the model, hyperparameters, data version, and costs.
7. Compare the quality, security, and latency of the base model and the fine-tuned model on unseen samples.
8. Create a dedicated deployment, carry out phased traffic shifting, and continuously monitor for drift and abnormal outputs.

On-demand dedicated deployment

On-demand pricing is based on GPU usage per second; it offers capacity that is independent of shared traffic and more controllable latency. Users can deploy platform models, their own models, or LoRA models, set minimum and maximum numbers of replicas, and reduce the usage to zero when there are no requests.

• Throughput is primarily determined by the GPU model, quantity, model size, quantization, context, and batching.
• Increasing the number of replicas or the number of GPUs per replica will result in a proportional increase in operating costs.
• You can choose from multiple regions such as GLOBAL, US, EUROPE, and APAC, or apply for a fixed single region.
• Fixed regions cannot be migrated in place; a new deployment must be created and traffic redirection must be carried out.
• Companies can discuss private network connections, direct routing, dedicated capacity, and higher quotas.

API integration process

1. Create an account to obtain a trial amount or link a payment method.
2. In the model directory, select a model with an appropriate input type, context, and price.
3. Create an API key and store it in the server’s environment variables or a key manager.
4. You can choose the official Python SDK, OpenAI SDK, Anthropic SDK, or make direct API calls.
5. First, use non-streaming small requests to verify the model identifier, message format, and usage fields.
6. Add flow-based output, tool calls, structured results, caching, or batch tasks as well.
7. Retry and fallback mechanisms are designed to handle throttling, timeouts, model deactivation, and abnormal content.
8. Set the monthly budget, alerts, and key rotation, then enable production traffic.

Current price example

The model catalogs and Serverless pricing will be updated, and Priority and Fast plans may also have higher costs. When making a selection, it is necessary to verify the details again based on the actual models, service levels, and account billing information; the examples provided in the table cannot be applied to all models.

Training price

The prices listed in the table are calculated per million training tokens, and the number of training tokens depends on factors such as the volume of data, the number of iterations, multi-round processing, as well as images and inference paths. RFT and Dedicated Training API are charged based on GPU hours, while Serverless Training API has separate charges for pre-filling, cached pre-filling, sampling, and training tokens.

On-demand GPU prices

Deployments that are subject to regional restrictions are charged at 1.5 times the standard rate, and it is necessary to contact sales to enable such deployments. There is no additional charge for the start-up time, but setting the minimum number of replicas to a value greater than zero results in ongoing GPU costs.

Account limit and budget

• Accounts without a linked payment method are allowed up to 10 requests per minute.
• The maximum number of account-level requests per minute for accounts with a linked payment method is 6,000, and this limit is still subject to the constraints imposed by the model tokens.
• The valid payment methods correspond to a monthly budget of up to 50 dollars; cumulative top-ups or purchases will raise the budget level.
• Once the monthly budget set by the user is reached, requests for inference, deployment, and fine-tuning will be suspended.
• The prepaid amount is used first; any usage that exceeds the prepaid balance is billed at the end of the month.
• Creating and reading accounts, users, models, datasets, and other management operations are free of charge; however, deployment and training tasks incur costs.

SDK, command line, and open-source boundaries

The public SDKs, cookbooks, startup templates, and certain tools can be used in accordance with their respective licenses, but the hosted inference platform does not thereby become open-source software. Whether the model weights are made available and whether they can be used for commercial purposes depends on the license of each individual model.

Suitable for users and scenarios

• AI application developers: Quick access to chat, reasoning, vision, embedding, and tool invocation.
• Agent team: Utilizes rapid reasoning, structured output, and RFT to improve complex workflows.
• Research and machine learning team: Run SFT, DPO, RFT, or custom training loops.
• High-throughput products: Utilize dedicated GPUs to achieve more stable throughput and model version control.
• Global enterprises: Deployed by region, with an assessment of private network connections and data residency requirements.
• Batch processing team: Performs offline generation, data expansion, and model evaluation at discounted prices.

Advantages

• Serverless, dedicated inference, and training share the same account and resource framework.
• It is compatible with common SDKs, which reduces the effort required to migrate existing applications.
• It supports various open-weight models, preventing the application from being tied to a single model.
• From LoRA to full parameters and custom loops, the training control hierarchy is quite comprehensive.
• Multi-region, private network, and metric export are suitable for managing production infrastructure.

Restrictions and risks

• Model directories, prices, and preview features change frequently; in a production environment, it is necessary to fix the models and prepare alternatives.
• Shared Serverless does not have a unified SLA; latency and availability can vary during peak times.
• Compatibility interfaces still exhibit behavioral differences, so regression testing cannot be skipped.
• Automatic scaling up will increase the costs associated with GPUs, and an incorrect minimum number of replicas may also result in ongoing charges.
• Model outputs may contain hallucinations, inappropriate content, and structural errors; therefore, manual review is necessary depending on the level of risk.
• The quality of fine-tuning depends on the data and the evaluation design; completing the training does not mean that the resulting model is superior to the base model.
• The use, redistribution, and commercialization of third-party models are still subject to their licenses.

Privacy and security

For ordinary reasoning with open models, Fireworks does not record or save prompts or the generated content unless an explicit choice is made to do so; it also does not use customer prompts, training data, or interface inputs to train the model. The proprietary FireFunction allowed for the storage of input and output records for up to 30 days as an exception, and advanced features such as FireOptimizer may also record data when the user explicitly chooses to do so.

• Data in transit is encrypted using TLS 1.2 or higher, while static data is encrypted with AES-256.
• Dedicated workloads benefit from logical isolation, along with access logs and the option to use customer-owned storage buckets.
• The platform lists ISO 27001, ISO 27701, ISO 42001, and SOC 2 Type II.
• For medical workloads, it is necessary to verify the HIPAA agreements, partnership agreements, and the specific scope of services.
• The service has servers located in the United States, and cross-border data processing may occur; the location where the data is stored depends on the deployment area and the terms of the contract.

Copyright, Commercial Use, and Refunds

Users are responsible for the use of the uploaded data, training sets, prompts, and outputs, and must verify the accuracy of the model’s outputs. The terms prohibit reverse engineering, selling or transferring API keys, interfering with the system, violating third-party rights, and any other illegal uses.

The fact that the platform allows businesses to create models does not mean that all model outputs and weights can be used for commercial purposes without any restrictions. Before making a purchase, it is necessary to read the platform’s terms, the license for the underlying models, the license for the datasets, as well as the terms of the corporate order.

Public information indicates that prepaid funds are used first for covering costs, with any excess usage billed on a monthly basis; however, there is no confirmed guarantee of refunds applicable to all accounts. Conditions regarding refunds or deductions for top-ups, incorrect charges, the expiration of trial periods, and corporate contracts must be confirmed with billing support prior to making a payment.

Frequently Asked Questions

Does Fireworks AI offer a permanently free plan?

There is no confirmed permanent, unlimited free plan available. New accounts receive a $1 trial credit, after which charging applies based on the actual usage of models, training, or GPUs.

How should one choose between Serverless and On-demand?

Low to moderate traffic, commonly used models, and quick tests are suitable for Serverless; stable high traffic, custom models, LoRA, or requirements for performance control are better suited for On-demand.

Can I use the OpenAI SDK directly?

Yes, it is possible to use compatible interfaces by modifying the service configuration, keys, and model identifiers; however, the parameter details and behavior related to context overflow still need to be retested.

Can the fine-tuned model run on Serverless?

No. LoRA models that have been fine-tuned by users, as well as custom models, require an on-demand dedicated deployment.

Will the prompt be saved or used for training?

In the open model, ordinary inference is carried out without any data being retained, and data that is not explicitly selected is not used for training; exceptions may apply to proprietary models or advanced features, so it is necessary to check the specific data rules before using them.

Is Fireworks AI an open-source platform?

It is not a recognized open-source hosting platform. While some SDKs, command-line tools, Cookbooks, and example repositories have their code made available publicly, the platform services and third-party models are governed by different terms of service.

How to avoid unexpected overspending?

Set monthly budgets and alerts; start with small-scale load testing to check cache hit rates and output length, and ensure that the dedicated deployment can be scaled down to zero. Once the budget is reached, the service will be paused, but monitoring should still continue.

Are native apps from the Chinese mainland supported?

There are no confirmed native desktop or mobile applications developed in mainland China that are intended for end-users. Fireworks AI is primarily used through a web console, APIs, SDKs, and command-line tools; the availability of these services in terms of networking and compliance must be assessed separately.

Summary

Fireworks AI is suitable for teams that wish to carry out open-model inference, fine-tuning, training, and dedicated deployment on the same platform. Its compatible interfaces and on-demand Serverless capabilities shorten the testing process, while the on-demand and enterprise versions offer greater control over versions, regions, and performance.

Cost evaluation should not rely solely on the price of individual tokens; it is also necessary to take into account factors such as caching, output length, bulk discounts, training tokens, GPU copies, and regional pricing differences. Before going live, end-to-end verification of model quality, licenses, data processing, rate limiting, and budgeting must be completed.