Use files and photos as context

Hedy can work with the text in your documents and photos as well as with what it hears. Attach a price list, a project brief or a CV to a context, or take a photo of a whiteboard during a session, and Hedy uses that text when it answers your questions, writes your recap and detailed notes, and suggests what to say next.

Adding files and photos is part of Hedy Pro. Team members with an active seat can use it too.

Add files to a context

Files belong to a context, so every session that uses that context can draw on them.

Open Settings and go to the Personalization tab.
Under Session AI Context & Instructions, open the context you want, or create one (see Managing Custom Contexts).
In the Files section, tap Attach files and pick one or more files. On a computer you can also drag files onto the context.

Hedy reads PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), CSV, text and Markdown files, and JPEG, PNG, HEIC and WebP images. For Pages, Numbers and Keynote files it reads the PDF preview stored inside the file, so if a file has no preview, export it as PDF first. Old .doc, .xls and .ppt files don’t work, so save them as .docx, .xlsx or .pdf instead. Hedy can’t read password-protected files.

You can see a context’s files in the web app, but you add and remove them in the Hedy app on your computer or phone.

Once you add a file, Hedy indexes it on your device. You’ll see the progress next to the file, and Ready when it’s done.

Add photos during or after a session

You can add photos to sessions you own, while they’re running or after they’ve ended. Hedy reads the text in each photo on your device.

On a phone, tap the camera inside the empty chat box. As soon as you type, the send button replaces it. Choose Take photo to take one photo, or Choose from library to select up to ten photos at once.

On Mac and Windows, the camera button opens the image file picker directly, where you can select up to ten photos in one pick. You can also drop a single image onto an open session on the Sessions tab, or paste an image into its chat input with Cmd+V on Mac or Ctrl+V on Windows. Each paste adds one image. Pasting plain text works as before.

You can also open the session’s Settings tab and use the full-width Add photo button in Attachments. It works during and after the session.

Each photo has its own card under Attachments:

Tap the preview to open it and zoom in. The preview is available on the device that added the photo.
Check the status on the card. Saved on this device means the original is on this device. If it was added elsewhere, the card names that device. Photo status and errors appear on these cards, rather than at the top of the Chat tab.
Choose Show text to expand the extracted text, then Hide text to close it.
If reading failed and the original is on this device, choose Try again. Choose Remove photo to remove the photo.

The web app is read-only for files and photos. Add or remove them in Hedy on your phone or computer.

What Hedy does with them

Hedy reads the text in your files and photos on your device, using the device’s own text recognition for photos and scanned pages. It then picks the parts that matter for what you’re doing:

In chat, when you ask a question, Hedy looks for passages that match and uses them in its answer.
In the recap and detailed notes, passages that match the conversation help Hedy get names, numbers and facts right.
During a session, Hedy can use facts from your files and the text of your session photos when it suggests what to say.

Hedy treats this text as reference material and won’t follow instructions written inside your files.

Smarter file search (optional)

Out of the box, Hedy finds the passages in your files that use the same words as the conversation. On Mac, Windows, iPhone, iPad and supported Android devices you can also turn on Smarter file search, which also finds passages that fit what’s being discussed when the words are different.

To turn it on, go to Settings → Speech & AI → Smarter file search and tap Download (310 MB). You can also tap Download (310 MB) in the Files section of a context that has files. The file search model is about 310 MB. It downloads only when you tap Download (310 MB), over Wi-Fi or Ethernet that isn’t metered, and each device needs its own copy.

To free up the space, tap Remove next to the file search model in the same settings. Hedy still matches exact words in your files, and you can download the model again later.

Live suggestions on different devices

During a live session Hedy has less room for file text than it has in chat or recaps. If your files are short, live suggestions can use all of their text on any device. If they’re longer, Hedy needs smarter file search to pick the passages that fit the conversation. That runs during a live session on a Mac, on iPhone 15 Pro or newer, and on iPads with an M-series or A17 Pro chip. On other devices, live suggestions use your files only when they’re short. Session photos, chat, recaps and detailed notes work the same everywhere.

Privacy

Hedy keeps a copy of each file or photo on the device you added it from and never uploads the file or photo itself.
Cloud Sync is your choice. If it’s on, the text Hedy read, the file name and type, and the search data are stored in your account’s data region so your other devices can use them. If it’s off, they’re stored only on the device.
With cloud AI analysis, Hedy sends selected text from your files and photos, plus the file names, to our AI providers, along with the transcript and on the same terms. Short files may be sent in full. This happens whether or not Cloud Sync is on.
If you use Local AI, the analysis runs on your device and no file text goes to our AI providers. If Cloud Sync is on, the text is still stored in your account’s data region.
Removing a file or photo, or deleting its context or session, deletes the text and search data on that device and, if Cloud Sync is on, in the cloud and on your other devices once they sync. Deleting your account deletes them in the cloud. Your original file elsewhere on your computer or phone isn’t touched. On the web, the text Hedy read stays in your browser until you sign out.

For how Hedy handles your data in general, see Data Privacy Overview.