Model Overview Card for Llama 3.1 Lumimaid 70B

Basic Information

• Model Name: Llama-3.1-Lumimaid-70B
• Developer/Creator: NeverSleep
• Release Date: October 22, 2024
• Version: 0.2
• Model Type: Large Language Model (LLM)

Description

Overview:

Llama-3.1-Lumimaid-70B is a fine-tuned variant of the Llama 3 model, specifically designed for enhanced conversational capabilities and role-playing scenarios. It excels in generating coherent and contextually relevant dialogues while maintaining a broad knowledge base.

Key Features:

• 70 billion parameters for robust performance and nuanced understanding.
• Context length of up to 8192 tokens, allowing for extensive interactions.
• Instruction-following capabilities optimized for various tasks.
• Incorporates a diverse dataset with a focus on roleplay and conversational contexts.
• Utilizes advanced tokenization techniques with a vocabulary size of 128,256.

Intended Use:

This model is designed for applications such as chatbots, interactive storytelling, role-playing games, and any scenario requiring dynamic conversational AI.

Language Support:

Llama-3.1-Lumimaid supports multiple languages, making it versatile for global applications.

Technical Details

Architecture:

The model is based on the Transformer architecture, specifically utilizing the LlamaForCausalLM framework. Key architectural details include:

• Layers: 40
• Hidden Dimension: 14,336
• Number of Heads: 32
• Activation Function: GELU
• Data Type: bfloat16 for efficient computation.

Training Data:

Llama-3.1-Lumimaid was trained on a curated dataset that emphasizes roleplay scenarios while incorporating non-roleplay data to enhance its general knowledge.

• Data Source and Size: The training dataset includes extensive roleplay dialogues and general knowledge texts, ensuring a well-rounded understanding of language and context.
• Knowledge Cutoff: The model's knowledge is current as of December 2023.
• Diversity and Bias: The training data was selected to minimize bias while maximizing diversity in topics and dialogue styles. This approach enhances the model's ability to handle varied inputs effectively.

Performance Metrics:

The model has shown competitive performance across various benchmarks:

• High accuracy in generating contextually appropriate responses.
• Strong performance in maintaining coherence over long dialogues.

Usage

Code Samples:

The model is available on the AI/ML API platform as "Llama 3.1 Lumimaid 70b" .

API Documentation

Detailed API Documentation is available here.

Ethical Guidelines

NeverSleep emphasizes ethical considerations in AI development by promoting transparency regarding the model's capabilities and limitations. They encourage users to adhere to responsible usage guidelines to prevent misuse or harmful applications.

Licensing

Llama-3.1-Lumimaid is licensed under CC BY-NC 4.0, allowing non-commercial usage while restricting commercial applications without permission.

‍

Get Llama 3.1 Lumimaid 70b API here.