Inference FAQ

How can I access my API keys?
It's simple! You can see it in Settings > API Key.

Is the inference service free?
To view which models require payment, please visit our pricing page

Can you help host my custom models?
Yes, we provide dedicated hosting services along with a team of experts to meet all your needs.
Please kindly email us at [email protected] for more information

Do you have fine-tune service?
Yes, we offer fine-tuning services to ensure optimal performance and customization of AI models.
Interested? Our team of experts is here to help! Contact us at [email protected]

What is BF16/FP8?
BF16 (16-bit Brain Floating Point):

BF16 retains more accuracy, making it suitable for tasks where precision is critical like medical diagnostics or scientific research. With BF16, you get reliable results without compromising speed, though it comes at a slightly higher cost.

FP8 (8-bit Floating Point):

FP8 is all about efficiency. It's fast, lean, and perfect for applications where speed matters more than precision. Think real-time tasks like recommendations or large-scale image processing. FP8 helps you scale at a lower cost, making it ideal for high-throughput needs.

What are the differences between Base vs. Instruct models?
When choosing between base models and instruct models, it’s all about the type of interaction you need and how much guidance is required for your tasks.

Base models are completion models—they’re foundational and versatile, trained on a broad dataset to handle a wide range of tasks. Since they don’t come with specific instructions, they’re highly adaptable but may require a bit more guidance to get the exact results you’re looking for. Think of base models as powerful tools that can complete tasks, but you’ll need to give them more context or direction to get the desired output.

Instruct models are conversational models. They start as base models but are fine-tuned with specific instruction-response pairs. This extra training makes them adept at following direct commands, so when you ask them to do something, they respond precisely. Instruct models are like assistants who already understand how to handle specific tasks right out of the box, making them perfect for when you want quick, accurate responses with minimal input.

In summary, base models offer flexibility and can be molded for various purposes, making them ideal when you need a versatile tool. Instruct models provide precision and are ready to follow instructions right away, which is great for situations requiring conversational or task-specific responses. So, if your task is open-ended and exploratory, base models are a great fit. If you need a model that’s ready to follow your commands with minimal fuss, instruct models are the way to go.