Jarvis in your pocket. My notes from the Apple Intelligence WWDC demo
Watched the full Apple Intelligence segment at WWDC today, some details may be off, corrections welcome.
The argument I've made for years is that the models were ready but the interfaces weren't. That changed today.
ARCHITECTURE
Three tiers. A lightweight on-device model handles quick intents. Apple Foundation Models run on Private Cloud Compute, Apple silicon servers where data is not stored or used for training, with independent auditors able to inspect the code. Gemini handles heavier queries in the cloud. Not fully local, despite what pre-keynote reporting said.
THE CONCERT DEMO
The presenter hears about a concert. Asks Siri to find tickets. Finds out it's a lottery. Siri sets a reminder for when the lottery opens, then plays one of the artists singles on Apple Music after request.
Output quality sits around 70-120B parameter range. Nothing new there. The advantage is data access. Siri has your emails, messages, calendar, camera, screenshots, conversation history and spatial environment as native context. Every other assistant works with whatever you paste in.
WHAT SHIPPED
Screenshot analysis with connectors: The demo showed a photograph a paper schedule or event poster and the events go into your calendar, structure preserved.
Screenshot editing: not just reading a screenshot but generating modifications to it in real time.
Inconsistency detection: Like autocorrect Apple Intelligence now rephrases writing for grammatical mistakes and factual inaccuracies in real time.
Safari Notify Me: watch any webpage for changes. A page that says "Registration Opening Soon" triggers a notification when it changes to "Registration Open." No third-party service required.
Mail and Messages: unread message threads collapse into one summarized notification. A confirmation email from an airline adds the flight to your calendar automatically and becomes Siri context. You never have to brief it about the meeting.
Phone call widget: when an airline asks for your confirmation code mid-call, a widget surfaces it from your email. No app switching, no dropped calls while you search.
HomeKit cameras: events get natural language descriptions, collated into coherent summaries, with dynamic notifications that update as a situation develops. Searchable by plain language: "package delivery by UPS."
Shortcuts builder: describe a multi-step automation in plain language, it builds it. Maps triggers Shortcuts triggers Messages with your ETA when you leave work. No visual editor, no node graph.
Extension builder: describe a Safari extension and it builds and deploys it.
Image Playground: on-device image generation with access to your full photo library. Style transfer, wallpapers, retroactive recomposition of badly framed shots. An API ships today so third-party apps can call the same stack.
Reframe: point at a photo with bad framing and the model infers what the scene looked like outside the frame, generates it, and lets you recompose. Works on old photos.
Vision Pro: point at a physical object and ask a spatial question. The demo showed checking whether one object would fit inside another. This is the first Vision Pro feature that requires the Vision Pro form factor to work.
Passwords: scans for credential exfiltration.
THE DEVELOPER STORY
Swift API for Apple Foundation Models. Any developer calls Apple's full intelligence stack without building infrastructure or handling privacy liability. App Intents replaces SiriKit as the framework for Siri integration across all apps. Apple Intelligence is now in Xcode, with code suggestions baked into the dev environment. Developer betas available today.
WHY THIS IS DIFFERENT
Apple wired an intelligence layer into every existing Apple product at once. The memory persists across Siri conversations and syncs via iCloud across Watch, iPhone, iPad, Mac, and Vision Pro. It heard a name mentioned once in a conversation and used it later without prompting. No setup.
The trust advantage is structural. The memory lives in the same place as your photos and iMessages. Apple did not have to ask for permission because users granted it years ago.
None of the individual features are technically new. All of them win on integration and distribution. That gap is not closeable from inside a chat app.
Public release this fall alongside iPhone 18. Betas available today.
