Grok’s latest version of xAI can process images

xAI, the OpenAI competitor founded by Elon Musk, presented the first version of Grok capable of processing visual information. Grok-1.5V is the company’s first-generation multimodal AI model, which can not only process text, but also “documents, diagrams, graphs, screenshots, and photographs.” In the xAI announcement, he gave some examples of how his abilities can be used in the real world. You can, for example, show him a photo of a flowchart and ask Grok to translate it into Python code, have him write a story based on a drawing, and even ask him to explain a meme you don’t understand. Hey, not everyone can keep up with everything the internet spits out.

The new version comes just weeks after the company unveiled Grok-1.5. This model was designed to be better at coding and math than its predecessor, as well as being able to handle longer contexts so it can check data from more sources to better understand certain requests. xAI said its early testers and existing users will soon be able to take advantage of Grok-1.5V’s capabilities, although it did not give an exact timeline for its deployment.

In addition to introducing Grok-1.5V, the company also released a benchmark dataset it calls RealWorldQA. You can use any of RealWorldQA’s 700 images to evaluate AI models: each element comes with questions and answers that you can easily check, but which can confuse multimodal models like Grok. xAI claimed that its technology received the highest score when the company tested it with RealWorldQA against competitors, such as OpenAI’s GPT-4V and Google Gemini Pro 1.5.