M3SGG Documentation

Welcome to the documentation for M3SGG (Modular, multi-modal Scene Graph Generation), a modular framework for video scene graph generation and analysis.

Overview

M3SGG builds on established SGG research and extends it with modular components, dataset support, and training/evaluation tooling. It supports multiple approaches and provides utilities for training, evaluation, and analysis of video scene graphs.

Key Features

• Multiple SGG Models: STTran, DSG-DETR, STKET, Tempura, SceneLLM, OED, VLM

• Dataset Support: Action Genome, EASG, and Visual Genome datasets

• Language Integration: Summarization and language modeling capabilities

• GUI Application: Interactive demo application for visualization and testing

• Comprehensive Evaluation: Multiple evaluation modes (PredCLS, SGCLS, SGDET)

Quick Start

To get started quickly, see the Installation guide and then check out the Usage Guide examples.

User Guide:

• Installation
  • Requirements
  • Installation Methods
  • Installation Steps
  • Dataset Setup
  • Verification
  • Troubleshooting
  • Next Steps
• Usage Guide
  • Quick Start
  • Configuration System
  • Command Line Interface
  • Applications
  • Jupyter Notebook Examples
  • Python API
  • Training Modes
  • Model-Specific Usage
  • Configuration Presets
  • Advanced Usage
  • Performance Optimization
  • Troubleshooting
  • Next Steps
• Datasets
  • Supported Datasets
  • Dataset Processing
  • Dataset Statistics
  • Quality Assurance
  • Performance Considerations
  • Custom Datasets
  • Next Steps
• Models
  • Model Overview
  • STTran Model
  • Tempura Model
  • SceneLLM Model
  • STKET Model
  • Model Comparison
  • Model Selection Guidelines
  • Training Considerations
  • Model Customization
  • Deployment
  • Troubleshooting
  • Next Steps
• Training Guide
  • Training Overview
  • Basic Training
  • Training Parameters
  • Model-Specific Training
  • Training Strategies
  • Monitoring Training
  • Optimization Techniques
  • Hyperparameter Tuning
  • Checkpointing
  • Troubleshooting
  • Best Practices
  • Next Steps
• Evaluation Guide
  • Evaluation Overview
  • Basic Evaluation
  • Evaluation Metrics
  • Evaluation Procedures
  • Mode-Specific Evaluation
  • Advanced Evaluation
  • Evaluation Analysis
  • Benchmark Comparison
  • Custom Evaluation
  • Evaluation Best Practices
  • Troubleshooting
  • Evaluation Reports
  • Next Steps

API Reference:

• API Reference
  • Datasets Module
  • Core Library
  • Models
  • CLI Module
  • Language Module
  • Utils Module
  • Core Components

Additional Information:

• Contributing
• Changelog
• License

Indices and Tables

• Index

• Module Index

• Search Page