Llama 3.2 90B Vision Instruct Turbo API

Llama 3.2 90B Vision Instruct Turbo Description

• Model Name: Llama 3.2 90B Vision Instruct Turbo
• Developer/Creator: Meta
• Release Date: September 25, 2024
• Version: 3.2
• Model Type: Multimodal (Text and Image)

Overview

Llama 3.2 90B Vision Instruct Turbo is a large-scale multimodal AI model capable of processing both text and images. It represents Meta's first foray into multimodal AI, offering advanced visual reasoning capabilities alongside powerful language processing.

Key Features

• Multimodal processing of text and images
• 90 billion parameters
• Long context length support (up to 128k tokens)
• Optimized transformer architecture
• Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF)
• High-resolution image processing (up to 1120x1120 pixels)

Intended Use

The model is designed for a wide range of applications, including:

• Document-level understanding
• Interpretation of charts and graphs
• Image captioning
• Visual question answering
• Data extraction and processing
• Image comparison
• Personal visual assistance

Language Support

The model supports multiple languages, making it suitable for multilingual tasks and applications.

Technical Details

Architecture

Llama 3.2 90B Vision Instruct Turbo utilizes an optimized transformer architecture. For image processing, it employs separately trained image reasoning adaptor weights that are integrated with the core LLM weights through cross-attention.

Training Data

• Data Source and Size: 6 billion (image, text) pairs
• Knowledge Cutoff: December 2023

Performance Metrics

The model demonstrates strong performance across various benchmarks:

• Matches OpenAI's GPT-4o on chart understanding (ChartQA)
• Outperforms Anthropic's Claude 3 Opus and Google's Gemini 1.5 Pro on interpreting scientific diagrams (AI2D)

Comparison to Other Models

Llama 3.2 90B Vision Instruct Turbo competes with leading models like Claude 3 Haiku and GPT-4o-mini in image recognition and visual understanding tasks.

Usage

Code Samples

Ethical Guidelines

The model includes a new Llama guard safety model to ensure responsible and ethical use.

Licensing

The Llama 3.2 models, including all associated multimodal capabilities, are governed by a specific licensing agreement that restricts commercial use within Europe. According to the Llama 3.2 Acceptable Use Policy, individuals or organizations based in the European Union are not granted rights to utilize these models for commercial purposes. This restriction is crucial for developers and organizations considering the deployment of Llama 3.2 models in their applications within the EU.

For more detailed information on the acceptable use and licensing terms, please refer to the Llama 3.2 Use Policy.

• Visual Q&A & Captioning — answer questions about images and generate accurate captions
• Document/Diagram Analysis — extract info from charts, schematics, diagrams
• Multimodal Assistants — power chatbots that understand text + images
• Enterprise Visual Reasoning — financial report extraction, legal visualization tasks
• Educational Tools — image-based tutoring and interactive learning aids

‍