Supported by the UAE government Institute of Technological Innovation (TII) announced the launch of Falcon 3, a family of open-source small language models (SLMs) designed to run efficiently on lightweight, single-GPU-based infrastructures.

Falcon 3 includes four model sizes – 1B, 3B, 7B and 10B – with base and instructional options, promising to democratize access to advanced AI capabilities for developers, researchers and businesses. According to the Hugging Face leaderboard, models have surpassed or are close to popular open source counterparts in their size class, including Meta's Llama and category leader Qwen-2.5.

Development comes at a time Demand for SLMsWith fewer parameters and a simpler design than LLM, they are growing rapidly due to their efficiency, availability, and ability to deploy on resource-constrained devices. They are suitable for a number of applications in industries such as customer service, healthcare, mobile applications, and IoT, where typical LLMs may be too computationally expensive to operate efficiently. according to Evaluates reportsthe market for these models is expected to grow at a CAGR of nearly 18% over the next five years.

What does the Falcon 3 bring to the table?

Trained for 14 trillion tokens—twice as many as the previous Falcon 2—the Falcon 3 family uses a decoder-only architecture that focuses on batched queries to share parameters and reduce memory usage for the key-value (KV) cache during inference. It provides faster and more efficient operations for various text-based tasks.

Basically, the models support four main languages ​​- English, French, Spanish and Portuguese - and are equipped with a 32K context window, which allows them to process long inputs such as heavy text documents.

“Falcon 3 is designed for universal, general and special tasks and provides users with enormous flexibility. Its base model is ideal for generative applications, while the instructional variant is preferred in conversational tasks such as customer service or virtual assistants,” notes TII. website.

according to leader board According to Hugging Face, all four models of the Falcon 3 perform well, with the 10B and 7B versions being the stars of the show, achieving state-of-the-art results on reasoning, language comprehension, following instructions, code and math tasks.

Among the models in the 13B-parameter size class, the 10B and 7B versions of the Falcon 3 outperform competitors, including Google's Gemma 2-9BMeta's Llama 3.1-8B, Mistral-7Band Yi 1.5-9B. They outperformed Alibaba's category leader Qwen 2.5-7B – as well as MUSR, MATH, GPQA and IFEval – except for MMLU, a test to assess how well language models understand and process human language.

network distribution

Available now with Falcon 3 models Hugging faceTII aims to serve a wide range of users, enabling cost-effective AI deployment without computational bottlenecks. With the ability to perform specific, domain-focused tasks with fast processing times, models are used in edge and privacy-sensitive environments, including customer service chatbots, personalized recommendation systems, data analytics, fraud detection, healthcare diagnostics, supply chain optimization, and education.

The institute also plans to further expand the Falcon family by introducing models with multimodal capabilities. These models are expected to be released in January 2025.

Notably, all models support the responsible development and deployment of AI with the TII Falcon License 2.0, a permissive license based on Apache 2.0, with an acceptable use policy. To help users get started, TII has also launched the Falcon Playground, a testing environment where researchers and developers can test Falcon 3 models before integrating them into their applications.

Daily insight into business use cases with VB Daily

If you want to impress your boss, VB Daily has got you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory changes to practical deployments, so you can share insights for maximum ROI.

Read ours Privacy Policy

Thank you for subscribing. See more VB news is here.

An error occurred.