xgen

XGen is a family of 7 billion parameter large language models developed by Salesforce AI Research, specifically designed for long sequence modeling with support for input sequences of up to 8,000 tokens. The models offer various configurations, including a base model and an instruction-finetuned version for research purposes, and are built to function as auto-regressive samplers. Key features include implementation using the OpenAI Tiktoken tokenizer and integration with the Hugging Face Transformers library for ease of use.