Apple Intelligence

As we get closer to June each year, developers and tech enthusiasts around the world begin to look forward to Apple's Worldwide Developers Conference (WWDC). This year, however, felt extra special.

Let’s set the scene: In what felt like a blink-and-you'll-miss-it past few years, we entered the COVID-19 pandemic in early 2020 and came out of it in 2023 with a vastly different world and a lieu of advanced AI tools at our disposal. In was as if almost every major tech giant, including OpenAI, Google, Meta, Microsoft, and others, entered this new gold rush, trying to capitalize on the hype. There one notable player missing from the table, though. Apple was quiet — eerily quiet.

Throughout this whole process of innovation, excitement (and for some, fear), those of us paying attention kept asking ourselves, “How is Apple going to partake in this feast?” I personally found myself thinking about this more often than I'd like to admit.

Well, 2024 came and we started getting some hints. By February, Apple CEO Tim Cook started to publicly talking about AI, assuring that Apple was putting “tremendous time and effort” into leveraging the power of AI into its ecosystem. By the time June came around, it was no longer a matter of whether or not Apple would mention AI in its developer conference… but a matter of how.

On June 10th, WWDC 2024 was finally here. We would finally have an answer to the famous “what has Tim been cooking?” question. That answer was… Apple Intelligence.

And it is beautiful.

But how should we talk about this new Apple Intelligence? The best way to approach this is to divide it into its different core technologies: language, images, a new Siri, and privacy.

Language

Since the introduction of OpenAI’s GPT model, other large language model (LLMs) have been popping up in the mainstream very rapidly. Leading up to WWDC, it was almost certain that Apple was going to announce some sort of LLM features in its operating systems.

It turns out Apple had been very busy this whole time, and our first new set of features came, unsurprisingly, as a collection of language features.

Writing Tools

We all saw this one coming, but having LLM-powered writing tools at the operating system level is a very welcome addition. For any text, we'll now be able to open a quick menu with tools to proofread, rewrite, or even summarize it very easily.

I'm curious to see how services like Grammarly adjust to this change.

Notification Priorities and Reduced Interruptions

We're still not sure if this is an iOS- and iPadOS-only feature, but it looks like Apple Intelligence will be able to prioritize different notifications at different times, depending on what matters most to you.

Taking this a step further, there's a new Reduce Interruptions feature that only shows you notifications that might need immediate attention, so it seems to extend the Focus modes we've had for a few years.

Mail Summarization and Smart Replies

The Mail app, which continues to be my email client of choice, now gains the ability to summarize each email and provide you with helpful smart reply suggestions — another welcome addition.

Images

Something slightly less expected, however, was Apple's introduction of a series of stable diffusion-based image tools as part of their Apple Intelligence suite.

Image Playground

It all starts with their new Image Playground app, which contains a prompt-based UX where users can generate new images using a prompt of their choosing. It seems that Apple is expecting users to really embrace this new experience, as they’re baking it in across the operating system, adding it to Messages and Notes, at least in the beginning.

The generated images they showed off at WWDC are… well… interesting. I have a strong feeling these will get better over time, but we already have other models that greatly surpass the capabilities of Image Playground. Time will tell, but I'm confident Apple will pull this off eventually.

Genmoji

Going a step further from Image Playground, we're now getting what Apple calls Genmoji, which are new emoji generated by their image generation model. I can see this becoming extremely popular, so we definitely need to keep an eye on this one.

Image Search and Memory Movies

The Photos app also gets its own suite of new AI-enabled features, including an advanced search and the ability to automatically create custom memories with a simple prompt. Like everything else in this article, I've yet to test this one out, but I can see these being very useful.

Image Cleanup

We wanted it. It's here. Sure, Google had it first, but now the iPhone has it too.

We can now remove unwanted clutter from our images in the Photos app. I have my opinions on this, but overall I think this will be pretty useful to those who want to use it.

I’ll be sure to explore this topic in a bit more detail in the future.

Siri Is Now Smart Again?

Over the years, Siri seems to have been getting worse. At the same time, other tools have gotten much smarter quicker than Siri… and it shows. During WWDC this year, however, Apple gave us some much-needed hope.

New Siri Design and Type to Siri

The most noticeable difference to Siri seems to be its new design, where a new glow element is beautifully wrapped around the edge of the screen. We’re also getting a new type-to-Siri feature that will come in quite handy with this new smarter version of Siri.

Richer Language Understanding

One of the main updates to Siri is a new enhanced experience when users talk to it, where conversations are supposed to flow much more naturally. This is extremely welcome, as in some instances, Siri has become comically bad.

Onscreen Awareness and Personal Context

Now, for the icing on the cake, Siri will know what is currently on your screen and will have context of the personal information that resides on your device. This includes notes, calendar events, photos, news, files, and more.

This has the potential to single-handedly become the most important feature of this new era for Apple. We have so much valuable information on our devices, that being able to see them all in context as we talk to Siri can lead to some serious user experience improvements. I'll definitely keep an eye on this one.

Apple's Commitment to Privacy

Now, unless you're running a custom LLM on your computer, commercial LLMs have historically sent your prompt to company servers to retrieve the information. However, most Apple Intelligence features actually seem to run on your device, which is a great step for privacy.

For whatever cannot run on the devices themselves, Apple also introduced Private Cloud Compute. To quote their website:

“Apple Intelligence is designed to protect your privacy at every step. It’s integrated into the core of your iPhone, iPad, and Mac through on-device processing. So it’s aware of your personal information without collecting your personal information. And with groundbreaking Private Cloud Compute, Apple Intelligence can draw on larger server-based models, running on Apple silicon, to handle more complex requests for you while protecting your privacy.”

This is truly a game-changer, and I'm sure we're bound to see other companies follow this approach as we continue to see AI integrated in more and more technologies.

Oh Yeah, ChatGPT is Here, Too

Also, for whatever Siri can't do itself, it will outsource to ChatGPT with a new partnership Apple has with OpenAI. It seems that we can opt in to have ChatGPT answer questions that Siri struggles with.

So, in all, we get the best of both worlds… not bad at all.

Parting Thoughts

So, what do I think of all this? I believe this is just the beginning, in more ways than one.

First, this is all definitely not coming out immediately. Apple will be rolling out these features slowly throughout the iOS 18, iPadOS 18, and macOS Sequoia update cycle, so we will likely see these features come out throughout the fall and winter, and well into 2025.

That being said, first-generation software products are never perfect, so I'm eager to see how the world of Apple Intelligence evolves over time. We're at a point where AI is the least effective it's ever going to be, and we've already seen how quickly things have progressed over these past few years.

We also have a new Apple event coming this week on September 9th, with new iPhones and Apple Watches expected to be unveiled. I wouldn't be surprised if they announce some exclusive AI features for the new iPhones, or if the Apple Watch gets some AI love as well.

I encourage you to go ahead and read through the official Apple Intelligence announcement page to dive deeper into all this. It’s pretty interesting stuff.

However, the most important takeaway from all this is the fact Apple is finally entering this space head-on — that sure is going to change things.