Every New iOS 18 AI Feature for the New Siri Just Leaked

We’ve heard plenty of rumors about Apple’s alleged plans to supercharge its digital assistant Siri at the 2024 Worldwide Developers Conference (WWDC). Today, via Apple Insider, we have the most comprehensive leak yet. Sourced from “people familiar with Apple’s AI initiative,” the leak contains just about everything Siri will be able to do across more than a dozen first-party iPhone apps.

The full leak is worth checking out, especially if you’re an iPhone user. However, we’ll give you the gist of Apple’s goals with the “new” Siri and share a few highlights that we think will most affect iPhone users’ daily lives.

What Apple expects from the “new” Siri

The overall goal of Siri appears to be to make it more powerful and better understand voice commands delivered in natural language. According to the leak, Apple trained Siri for this purpose by having Apple technicians provide intentionally obtuse commands. For example, instead of asking something like “Hey Siri, show me pictures of my cat,” it tests more vague commands like “I want to start a blog” or “I’m feeling nostalgic right now.” These are not specific commands telling Siri to do a particular thing, but rather commands that Siri will have to first interpret and then decide how best to deliver what it thinks the user might want/ need.

The benefit of this is obvious: it trains Siri to be better for users who don’t know (or don’t want to use) the proper syntax needed to execute a command. For example, a user saying “Hey Siri, I want some coffee” may or may not turn on the smart coffee machine, while saying “Hey Siri, turn on the coffee machine” probably would. The first is a natural statement, while the second is a direct order. Apple wants to reduce this locked syntax, making Siri much easier to use.

THE Apple Insider However, the leak doesn’t mention how this works. For example, are these Siri features powered by “Ajax”, which is the code name for Apple’s internal wide language model (LLM)? Or are they based on ChatGPT, since Apple has reportedly partnered with OpenAI for some of its AI-based systems? It might be a bit of both, but we’re not sure yet.

New Siri Features: A List of Highlights

As mentioned, the full leak is exhaustive, reviewing no less than 18 first-party iPhone apps and how Siri will work with each of them. Here are a few that seem really interesting to us:

• Camera: Siri will be able to control the camera via voice commands. You’ll be able to turn video recording on or off, open the camera in a specific mode (photo, portrait, video, etc.), then start a shutter self-timer and switch to the front or rear camera. Theoretically, this could allow you to set up your iPhone for a group photo, walk away, and then use voice commands to capture the photo remotely.
• Email: The Mail app is undergoing a complete overhaul. It will apparently be able to automatically categorize emails using machine learning, something Gmail users are likely already familiar with. In addition to this, Siri will also be able to perform detailed functions only through voice commands. This includes things like writing an email, sending it, scheduling it, marking an email as spam, and setting a reminder to read an email later. It will also be able to summarize emails and create “smart replies”, a feature arguably on par with Smart Reply on Android and Help Me Write in Gmail.
• Pictures: Apple is likely to introduce many photo editing features based on generative AI. Pixel users will probably recognize many of them, as we’ve only heard so far about features you can already use on Pixels with Magic Editor and Google Photos, such as moving/removing an object from the photo and filling in spaces with generative elements. AI, find specific photos with specific people/animals and apply generative AI filters.
• Safari: Apple’s web browser will use Siri for web page summaries, something Google has already brought to Android through Gemini. Safari will also be able to create new tab groups or open a new Private tab via voice commands.
• Voice Memos: You will be able to use Siri hands-free using voice memos. For example, you can ask Siri to create a new voice recording and then start speaking. You can then stop recording, save it under a specific name, and then even move it to a specific folder, all without putting a finger on your iPhone.

When will they be launched?

According to Apple serial leaker Mark Gurman, at least some Siri features won’t actually land at WWDC. Apple will almost certainly announce a few, but not all of them will be available in 2024. Gurman says it will be 2025 before the majority of these features will be available via a software update.

Of course, that doesn’t mean Apple won’t be rolling out at least a few of these at or around WWDC. However, it’s probably best not to expect iOS 18 to come with all of the features in this Siri leak, as it’s much more likely that these will leak out over the next few months.