AI for the rest of us.
Apple Intelligence is an artificial intelligence platform developed by Apple Inc.[1] Relying on a combination of on-device and server processing, it was announced on June 10, 2024 at WWDC 2024 as an integration for Apple’s iOS 18, iPadOS 18, and macOS Sequoia operating systems, which were announced alongside Apple Intelligence. Apple Intelligence is free for all users with supported devices, and GPT-4o integration is also free for all users without needing to sign in.[2] It is not currently in the developer betas for iOS 18, iPadOS 18, and macOS Sequoia, however it is scheduled to be released as a developer beta sometime during the summer.
Functionality and features
Apple Intelligence features writing tools that are powered by AI, including Rewrite and Proofread. This feature helps enhances a users’ writing to make it more friendly, concise and professional, similar to Grammarly’s AI writing features. It can also be used to generate summaries, key points, tables, and lists from an article or piece of writing.[3][4] ChatGPT will be available as part of Writing Tools, which help users generate content for anything they are writing about. With Compose, users can access ChatGPT image tools to generate images to complement what they are writing.[5]
Image Playground
Apple Intelligence can also be used to generate images on-device by using the Image Playground app. Similarly to OpenAI’s DALL-E, it can be used to generate images using AI, using phrases and descriptions to create an image with customizable styles such as Animation and Sketch. In Notes, users can access Image Playground on iPad through the new Image Wand tool in the Apple Pencil palette without having to open the Image Playground app. Rough sketches can be transformed into images, and users can also select empty space to create an image using context from the surrounding area.
Image Playground is also available in apps like Keynote, Freeform, and Pages, as well as in third-party apps that use the Image Playground API. [6]
Genmoji[edit]
Using Apple Intelligence, users can create an original Genmoji by simply typing a description. Users can pick someone from their photos and create a Genmoji that resembles them. Similar to emoji, Genmoji can be added inline to text messages and can also be shared as a sticker or reaction in a Tapback on iMessages and other supported third-party apps.
Siri overhaul[edit]
See also: Siri
Siri, Apple’s virtual assistant, has been updated with enhanced capabilities made possible by Apple Intelligence. The latest iteration features an updated user interface, improved natural language processing, and the option to interact via text by double tapping the home bar without enabling the feature in the Accessibility menu. In addition, Apple Intelligence adds the ability for Siri to use personal context from device activities to make conversations feel more natural and fluid. Siri can also help users navigate their device more easily, give users device support everywhere they go, and will have a larger app support via the Siri App Intents API.[7][8] Siri will be able to deliver intelligence that’s tailored to the user and their on-device information. For example, a user can say, “Play that podcast that Jamie recommended,” and Siri will locate and play the episode, without the user having to remember whether it was mentioned in a text or an email. They could also ask, “When is Mom’s flight landing?” and Siri will find the flight details and cross-reference them with real-time flight tracking to give an arrival time.[5]
Mail[edit]
Apple Intelligence adds a feature called Priority Messages to the Mail app. A new section at the top of the inbox shows the most urgent emails. Across a user’s inbox, instead of previewing the first few lines of each email, they can see summaries of the email without needing to open it. Smart Reply provides suggestions for a quick response, and will identify and outline questions in an email.[5]
Photos[edit]
Apple’s Photos app includes a feature to create custom memory movies and enhanced search capabilities. Users can describe the story they want to see, and using Apple Intelligence, selects matching photos and videos. It organizes these into a movie with a narrative arc based on identified themes. Additionally, users can search for specific photos or videos by description and/or keyword, and Apple Intelligence can pinpoint particular moments within video clips.[9]
ChatGPT integration[edit]
As a result of the company’s partnership with OpenAI, Apple Intelligence also includes a system-wide integration with ChatGPT, allowing Siri to determine when to send certain complex user requests to ChatGPT. This system-wide integration is powered by GPT-4o.[5] Users are asked before any questions are sent to ChatGPT, along with any documents or photos, and Siri then presents the answer directly.[5][10] Using ChatGPT features are free for all users without needing to sign in, however paid subscribers can sign in to gain access to paid features systemwide. All requests made through Apple Intelligence are not stored by OpenAI, and users’ IP addresses are obscured; however if users can also choose to connect their ChatGPT account, their data preferences will apply under ChatGPT’s policies.[11]
Private Cloud Compute[edit]
Although Apple Intelligence largely utilises on-device processing, for more complex tasks Apple Intelligence can scale accordingly its computational capacity and draw on server-based models for more complex requests. These models run on servers powered by Apple silicon, which allows Apple to ensure that data is never retained or exposed. Private Cloud Compute cryptographically ensures that Apple Intelligence does not talk to a server and independent experts can inspect the code that runs on these Apple silicon servers to check and verify user privacy.[12][13][2][14]
Development[edit]
Background[edit]
In October 2015, Apple Inc. acquired Perceptio, an on-device artificial intelligence modeling company.[15] Following the acquisition, Apple engaged in efforts to ensure its artificial intelligence operations remained covert; according to University of California, Berkeley professor Trevor Darrell, the company’s secrecy deterred graduate students.[16]
The Private Cloud Compute platform powering Apple Intelligence is designed heavily with user privacy and end-to-end encryption in mind. Independent experts can inspect the code that runs on Private Cloud Compute servers to verify user privacy, and Private Cloud Compute cryptographically ensures that Apple Intelligence does not talk to a server unless its software has been logged for inspection.[13]