iPhone Live Text

The new iOS15 feature Live Text finds words in the tiniest of corners. A story on how I searched for SOUL and found it in a little Amsterdam bookstore.

iPhone Live Text
A second hand bookshop.

As someone who loves words and technology, I just had to share this functionality. It more or less freaked me out. Since the latest update of iOS, a simple Spotlight Search goes through my photo’s and finds texts that are in the photos. But the precision and depth of this is just spectacular. Let me explain.

If you are a bit like me, you probably have many photos on the iPhone. It is my hobby to shoot some quick snapshots everywhere I go, almost like an extended memory. I have more than 30,000 pictures in the photo app. So I got excited when I read about an update to Spotlight. That is a search that searches through your collection of content and apps.

And after the update I played around with the Spotlight search, I searched for the word SOUL and to my surprise I see in the results a completely non-related photo of a second hand bookshop:

iOS spotlight example on iPhone showing results from Visual Look up
Spotlight Search on iOS 15 on the word SOUL

Out of curiosity I open the result and on the photo is an orange dot. I zoom in on the photo and the dot remains on a certain spot on the photo:

Example screenshot of a search with Visual Look Up on iPhone.
The result of searching for SOUL

I zoom in closer and closer, and at some point the text becomes highlighted and readable. On the back of the book, all the way in the corner of the store and the photo, the title says: the global SOUL. Literally, my jaw dropped. What a power, what a precision and speed. Scary and exciting at the same moment.

Zooming on the result to find the word SOUL in the left corner.

Live Text through the camera

In the iOS 15 update, there are many more features around text. You can copy and paste the texts out of the stored photo’s, but you can do the same with Live Photo’s. That is also called Live Text. You point at a text, for instance a phone number, and dial it straight from here. Really handy.

iPhone Live text, click on web link in a live picture