One interface, many LLMs

November 30, 2022, marked a new era in AI with the launch of ChatGPT. The app reached 100 million monthly active users in just two months — a new record. Since then, every software business, big or small, has defined its AI Strategy. Countless tweets have been written about the most awesome prompts. And doomers have predicted the end of the world thanks to AGI.

But let's assume it doesn't result in an apocalypse. Today's LLMs are impressive in their capabilities. There's a vast amount of different models, each unique in its own way. As of today, Raycast AI combines a dozen of those models in a single interface, making them easily accessible anywhere on your Mac. This blog post describes what’s the fuzz about these models, when to use each one, and what to expect next from us.

Why support different models?

Let’s first take a step back. Why support multiple models in the first place? At the end of last year, we actually decided to stick only with OpenAI models. When GPT-4 Turbo came out, it had everything: speed, intelligence, a long context window, and even supports vision, a.k.a. sending images, not just text.

Fast forward to today, the world is different. New models are released almost every week, each with unique benefits. Whenever we are uncertain about a decision, we turn to our community, and it was the same this time. We asked and received a clear answer: Give us more models! So we got to work and looked at which models make actual sense for us to support.

The good, the bad, the ugly

There are broadly three categories to look at when comparing models: Speed, intelligence, and context window. Let’s talk about them one by one, because as always, the devil is in the details:

• Speed: The faster, the better, right? The problem is that the more intelligent a model is, the more parameters it has, and the more resources it needs to run at scale. So, you can't really look at speed in isolation.
• Intelligence: Well, first of all, those models aren't really intelligent. Neither can they reason about your coding problems or have general knowledge. All they do is predict the next word, but you probably know this by now. Nevertheless, some models are more capable. These models are usually trained on a larger dataset and therefore have better capabilities.
• Context Window: This is a weird one. The way those models work is by always looking at all the input text to generate content. This means that the longer the input, the more the model needs to take into consideration. There are now models that can read hundreds of pages, but this often leads to slower response times.

As you can see, we are going in circles, and things depend on each other. So, it's not really about there being one perfect model, but more of an "it depends.”

One interface, many models

While looking at various models, we tried to figure out what the "superpower" of each was. This helped us decide which models are worth adding besides OpenAI’s. We also wanted to ensure that users have an easy way of selecting the right model for their use case. Therefore, we redesigned our model picker to display more details, enabling them to choose the appropriate model for the task at hand.

Perplexity - Where knowledge meets productivity

Our community has long desired access to Perplexity in Raycast. Adding it was a natural choice, and we collaborated with the Perplexity team to bring their online Sonar models to Raycast. These models are designed for question-answer scenarios and are enriched with current internet knowledge, making them ideal for Quick AI. Now, you can simply ask a question and receive an answer powered by Perplexity.

To celebrate this partnership, we're offering free trials of Perplexity Pro to all new annual Raycast Pro subscribers. New annual Pro subscribers receive 3 months of Perplexity Pro at no additional cost. Additionally, subscribers to our paid add-on that unlocks the most powerful models get 6 months of Perplexity Pro.

Access to knowledge should be available to everyone. With Perplexity integrated into Raycast, you can access it anytime, anywhere on your Mac.

Anthropic - Bigger, better, faster

Anthropic's latest models, the Claude 3 series, have taken 𝕏 by storm. This series includes three models: Haiku, Sonnet, and Opus. Each model has an impressive context window of 200k tokens, handling an entire book. Additionally, Opus has topped the Arena leaderboard, making it the most intelligent model available that can solve highly complex tasks with remarkable fluency.

Groq - Speed up mundane tasks

Open-source models have accelerated rapidly, and many see them as the future of LLMs. Raycast stands for speed, and so does Groq, which is the fastest model inference engine out there. Both Llama 2 70b and Mixtral 8x7b are powered by Groq, making those models the perfect choice for AI commands to speed up your mundane tasks.

What’s next?

AI Commands are perfect for reusing prompts. We plan to bring similar abstractions to your AI Chats, allowing you to reuse models, system instructions, and more. This way, you can build a personal library of assistants for your unique use cases.

Consolidating multiple models in a single interface is also a key aspect of becoming the UI for AI. All models are already available via our API to build your own smart productivity tools. Bringing extensions closer to AI will unlock the next wave of productivity gains.

We hope you enjoy the new models! Did we miss a model with a "superpower" that you would like to see in Raycast? Ping us!