Basic Information

• Model Name: Llama-3 (8B)
• Developer/Creator: Meta
• Release Date: April 18, 2024
• Version: 1.0
• Model Type: Large Language Model (LLM)

Description

Overview

Llama-3 (8B) is a powerful open-source language model developed by Meta. It is part of the Llama family of models, which includes larger versions like Llama-3 (70B). Llama-3 (8B) is a pretrained and instruction-tuned generative text model optimized for dialogue use cases. It outperforms many available open-source chat models on common industry benchmarks while prioritizing helpfulness and safety.

Key Features

• Improved reasoning and code generation capabilities: Llama-3 (8B) demonstrates enhanced reasoning skills and the ability to generate high-quality code snippets.
• Increased diversity in model responses: The model produces more diverse and engaging responses compared to previous versions.
• Enhanced alignment with human preferences: Llama-3 (8B) is better aligned with human values and preferences, making it more suitable for interactive applications.
• Optimized for assistant-like chat and natural language generation tasks: The model is designed to excel in assistant-like chat scenarios and various natural language generation tasks.

Intended Use

Llama-3 (8B) is intended for commercial and research use in English. The instruction-tuned models are suitable for assistant-like chat, while the pretrained models can be adapted to various natural language generation tasks.

Language Support

Llama-3 (8B) primarily supports the English language. However, since it is an open-source model, it may be possible to fine-tune or adapt it for other languages.

Technical Details

Architecture

Llama-3 (8B) uses an optimized transformer architecture with Grouped-Query Attention (GQA) for improved inference scalability. The model has 8 billion parameters and is designed to be efficient and performant.

Training Data

Llama-3 (8B) was trained on a mix of publicly available online data, with a token count of 15 trillion and a knowledge cutoff of March 2023. The training data covers a wide range of topics and domains, ensuring the model's knowledge is comprehensive and up-to-date.

Performance Metrics

Llama-3 (8B) achieves state-of-the-art results on several benchmarks:

The model also demonstrates impressive speed, with an output speed of 119.6 tokens per second and lower latency compared to the average. It has a context window of 8,000 tokens.

Usage

Ethical Guidelines

Meta has developed ethical guidelines for the use of Llama-3 (8B) to ensure it is used responsibly and ethically. These guidelines cover topics such as data privacy, bias mitigation, and content moderation.

License Type

Llama-3 (8B) is released under a custom commercial license. Developers can use the model for commercial and research purposes, with specific terms and conditions outlined in the license agreement. The company's commitment to open-source and responsible AI development sets a new standard for the industry.