32K
0.00126
0.00126
132B
Chat

DBRX Instruct

DBRX Instruct API: Databricks' state-of-the-art open large language model with advanced features and superior performance on industry benchmarks.
Try it now

AI Playground

Test all API models in the sandbox environment before you integrate. We provide more than 200 models to integrate into your app.
AI Playground image
Ai models list in playground
Testimonials

Our Clients' Voices

DBRX InstructTechflow Logo - Techflow X Webflow Template

DBRX Instruct

Powerful open-source LLM with exceptional performance on a wide range of benchmarks.

Basic Information

  • Model Name: DBRX
  • Developer/Creator: Databricks
  • Release Date: 2023
  • Version: 1.0 Instruct
  • Model Type: Large Language Model (LLM)

Description

Overview

DBRX Instruct is a powerful, open-source large language model (LLM) developed by Databricks. It utilizes a fine-grained mixture-of-experts (MoE) architecture with 132 billion total parameters, of which 36 billion are active for any given input.

Key Features

  • Fine-grained MoE architecture with 16 experts and the ability to select 4, providing 65x more possible expert combinations than other open MoE models
  • Trained on 12 trillion tokens of carefully curated text and code data
  • Exceptional performance on benchmarks for general knowledge, commonsense reasoning, programming, and mathematical reasoning
  • Outperforms leading open models like Mixtral Instruct and Code LLama (70B)

Intended Use

DBRX Instruct is a general-purpose LLM designed for a wide range of natural language processing tasks, including text generation, question answering, code generation, and more. It is particularly well-suited for applications requiring strong programming and mathematical reasoning capabilities.

Language Support

DBRX Instruct is a multilingual model, supporting a wide range of languages.

Technical Details

Architecture

DBRX Instruct is a transformer-based, decoder-only LLM that was trained using next-token prediction. It utilizes a fine-grained MoE architecture, with 16 experts and the ability to select 4 for any given input.

Training Data

The model was pre-trained on 12 trillion tokens of carefully curated text and code data, with a maximum context length of 32,000 tokens. This dataset is estimated to be at least 2x better quality than the data used to train the MPT family of models.

Performance Metrics

On standard benchmarks, DBRX Instruct outperforms leading open models:

  • MMLU (Multiple-choice Model-Linguistic Understanding): 73.7% vs. 71.4% for Mixtral Instruct
  • HellaSwag 10-shot: 89.0% vs. 87.6% for Mixtral Instruct
  • WinoGrande: 81.8% vs. 81.1% for Mixtral Instruct
  • Databricks Gauntlet: 66.8% vs. 60.7% for Mixtral Instruct
  • HumanEval: 70.1% vs. 54.8% for Mixtral Instruct
  • GSM8k: 66.9% vs. 61.1% for Mixtral Instruct

Usage

API Example

License

DBRX Instruct is available for use under the Databricks Open Model License.

Try it now

The Best Growth Choice
for Enterprise

Get API Key