Llama 3 8B Instruct Lite API

Llama 3 8B Instruct Lite Description

Basic Information

• Model Name: Llama 3 8B Instruct Lite
• Developer/Creator: Meta
• Release Date: April 18, 2024
• Version: 1.0
• Model Type: Text Generation

Overview:

Llama 3 8B Instruct Lite is a generative text model optimized for dialogue and instruction-following use cases. It leverages a refined transformer architecture to deliver high performance in text generation tasks.

Key Features:

• Optimized Transformer Architecture: Uses Grouped-Query Attention for scalability.
• Instruction Tuned: Enhanced with supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF).
• High Performance: Outperforms many open-source chat models on industry benchmarks.
• Safety and Helpfulness: Fine-tuned for helpful and safe responses.

Intended Use:

Designed for commercial and research purposes, particularly in creating assistant-like chatbots and other natural language generation tasks.

Language Support:

Supports English primarily, with potential for fine-tuning in other languages under specific licensing terms.

Technical Details

Architecture:

Llama 3 is an auto-regressive language model employing a transformer architecture. The model integrates Grouped-Query Attention (GQA) to enhance inference scalability. Instruction-tuned versions use SFT and RLHF to align outputs with human preferences.

Training Data:

• Source: Publicly available online data.
• Size: Over 15 trillion tokens.
• Knowledge Cutoff: March 2023 for the 8B model.
• Diversity and Bias: Comprehensive training on diverse datasets; ongoing evaluations to minimize biases.

Performance Metrics

Accuracy:

• MMLU (5-shot): 68.4
• CommonSenseQA (7-shot): 72.6
• HumanEval (0-shot): 62.2

Speed:

Optimized for real-time applications with efficient inference capabilities.

Robustness:

Demonstrates strong generalization across various topics and languages, handling diverse inputs effectively.

Usage

Ethical Guidelines:

Meta has implemented a Responsible Use Guide, outlining best practices for ethical model deployment. Developers should integrate safety measures such as Meta Llama Guard 2 and Code Shield safeguards.

License Type:

Custom commercial license details can be found here.

Hardware and Software

Training Factors:

Training utilized Meta's Research SuperCluster and third-party cloud compute for fine-tuning and evaluation.

Carbon Footprint:

• Llama 3 8B: 1.3M GPU hours, 700W, 390 tCO2eq
• Total: 7.7M GPU hours, 2290 tCO2eq (100% offset by Meta’s sustainability program).

Responsibility & Safety

Meta emphasizes an open approach to AI, with a commitment to Responsible AI development. The Llama 3 release includes updated guidelines and resources for developers to implement model safety effectively.

Key Safety Measures:

• Extensive red teaming and adversarial evaluations.
• Refusals mitigation to ensure fewer false refusals.
• Responsible release processes to address misuse and critical risks.

• multilingual conversational assistants for customer support and productivity
• content generation including blogs, reports, and creative writing
• coding assistance and automated code generation
• summarization and complex query answering for research and business
• educational tools and interactive tutoring systems
• multimodal tasks combining text and visual inputs (in applicable versions)
• fine-tuning base for domain-specific applications and specialized NLP tasks