MPT-Chat (30B) Advanced open-source language model with ethical AI practices.
MPT-Chat (30B) is an advanced, open-source language model designed for a broad spectrum of natural language processing tasks, emphasizing efficiency, scalability, and ethical AI practices.
This model is tailored for:
Currently, detailed language support specifics are not provided; however, it typically includes major global languages given its extensive training data.
MPT-Chat (30B) employs a decoder-only transformer architecture, similar to GPT models, enhanced with modern techniques like FlashAttention for efficient attention computation and ALiBi for positional biases, which facilitate better scaling and performance.
The model was trained on a curated dataset of 1 trillion tokens, encompassing a wide range of internet text to ensure relevance and diversity.
The training dataset is significantly large at 1 trillion tokens, selected for its high quality and comprehensive coverage of various domains.
The information incorporated in the model is up-to-date as of its last training cut-off in early 2023.
MPT-Chat (30B) was developed using constitutional AI principles to align closely with human values and minimize biases, supported by rigorous testing to detect and mitigate any unintended biases.
Outlined as part of the development process, focusing on responsible AI use and bias mitigation.
Apache 2.0 license, permitting both commercial and non-commercial use.
MPT-Chat (30B) sets a new benchmark in the realm of open-source language models by combining large-scale machine learning capabilities with a commitment to ethical AI practices, making it highly suitable for developers and researchers in the AI community.