Model Overview
Basic Information
- Model Name: CodeGen2-16B
- Developer/Creator:Salesforce AI Research
- Release Date: May 2023
- Version: 2.0 16B
- Model Type: Autoregressive Language Model
Description
Overview
CodeGen2-16B is a colossal language model developed by the visionaries at Salesforce AI Research. This behemoth of a model is designed to revolutionize the way we approach program synthesis, with the ability to generate and comprehend code across a vast array of programming languages.
Key Features
- Multi-turn program synthesis - a dance between the model and the developer, creating code together
- Infill sampling for code completion - filling in the blanks with precision and elegance
- Instruction tuning for following code generation instructions - a model that listens and learns
Intended Use
CodeGen2-16B is a Swiss Army knife for developers, designed to assist in writing and understanding code. From code generation to code completion, this model is a greate AI tool for those who seek to harness the power of AI in their coding endeavors.
Language Support
Supported languages (and frameworks) are as follows: C, C++, C-Sharp, Dart, Go, Java, Javascript, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Shell, SQL, Swift, Typescript, and Vue.
Technical Details
Architecture
CodeGen2-16B is a Transformer-based model, with a staggering 16 billion parameters. It's amongst the smaller models, capable of processing and generating code with lightning speed, thanks to techniques like Flash Attention.
Training Data
This model is trained on the stricter permissive subset of the deduplicated version of the Stack dataset (v1.1)
Knowledge Cutoff
The model's knowledge is as current as the training data itself, up to June 2022.
Diversity and Bias
The training data is a melting pot of programming languages and domains, but the exact diversity and potential biases are not something we can discuss openly. It's a topic that requires careful consideration and research.
Usage
API Usage Example
const { OpenAI } = require('openai');
const api = new OpenAI({
baseURL: 'https://api.aimlapi.com/v1,
apiKey: '<YOUR_API_KEY>',
});
const main = async () => {
const result = await api.chat.completions.create({
model: 'Salesforce/codegen2-16B',
messages: [
{
role: 'system',
content: 'You are SQL code assistant.',
},
{
role: 'user',
content: 'Could you please provide me with an example of a database structure that I could use for a project in MySQL?'
}
],
});
const message = result.choices[0].message.content;
console.log(\`Assistant: \${message}\`);
};
main();
License Type
The model is a gift to the research community, available for research and non-commercial use under the Salesforce AI Research license.