OrCam AI Update What's In Front of Me?

  • 2 months ago
Welcome to Tech Connect brought to you by Vision Forward Association! We are excited to bring you the latest and greatest in Assistive Technology! All explained in a way that makes sense!

OrCam has produced some wonderful devices over the years including the Read, Learn and My Eye Pro. While they are already famous for OCR, OrCam has taken notice of the AI craze and is using this technology to improve their features, especially: "What's In Front of Me?"

This specific feature is used to get a general understanding of your surroundings. This usually includes information like: the size of a room and its expected purpose, essential furniture like tables and chairs, or how many people there are. However, this has always been a weak point for the OrCam devices, yes they would reliably provide info on the basics but now thanks to this new update we can use AI to ask follow-up questions about what is in front of us.

Join Cory and Luke as we demonstrate this OrCam Update and walk through a few cases in which this feature might be particularly useful for those who are blind or have low vision.

0:00 Start
0:36 Reintroducing OrCam, Referencing past videos
1:18 Getting Started, Download the OrCam App and get it Updated!
5:24 Demo of the AI for What's in Front of me? Compared to (Meta Glasses)
9:11 Using OCR vs AI to summarize and ask follow-up questions!
13:29 Wrap-Up the Update and General Thoughts on OrCam
17:55 Outro, Upcoming Events, Contact Info, Live shows and more!

If you have any additional questions about OrCam, or its AI, please add those down in the comment section and we will endeavor to help!

Join us next time on Tech Connect, and be sure to leave your thoughts down in the comments below and ask plenty of questions!

Vision Forward's Tech Connect continues to bring you the information you need to make the most out of your devices. Our experts know there are many factors to consider, so if you have any follow up questions please post them in the comments and we will help you find the assistive tech that's best for you.

Join us, find the schedule by visiting out website:
https://vision-forward.org/techconnect

Contact Vision Forward Association:
Call us: (414-615-0103)
Email us: infocus@vision-forward.org
Visit us online: https://www.vision-forward.org
Online Store: https://www.vision-forward.org/store

Want more Tech Connect in your life? Try us Live! Be sure to join us for the upcoming YouTube Live! show on June 20th at 11am CST.

Looking for the full playlist?
https://www.youtube.com/playlist?list=PLdZ61dAGaL_k3I-_LcTPozan9upCqY8Yc

Catch-up with our most recent episodes!

"Glide from Glidance, with Amos Miller"
https://www.youtube.com/watch?v=42KuD0uuZls

"RIM - Accessible Remote Training & Support"
https://www.youtube.com/live/mFIZwaIo0CE

"You Tube? Me Tube! How to Access YouTube with a Screen Reader!" https://www.youtube.com/watch?v=HCLZwyxFGSE

#assistivetechnology #lowvision #OrCam
Transcript
00:00Hey, Margarita Glasses, what's in front of me?
00:04Jimmy Buffett.
00:06No, not Jimmy Buffett again.
00:09Why can't I get these to see anything but Jimmy Buffett?
00:11Okay, here we go, here we go.
00:13Second try.
00:14Hey, Margarita Glasses, what's in front of me?
00:17It's Jimmy Buffett.
00:19Luke, how many times do I have to tell you
00:21those glasses only work for me,
00:24but I got a solution for you.
00:25Let's go into tech.
00:26We're doing something on OrCam.
00:27OrCam?
00:28Yes, OrCam.
00:30Ha ha ha.
00:37Hi all, welcome back to another Tech Connect.
00:39Today, we're taking a look at a new feature for OrCam.
00:42Now you might say, Corey and Luke,
00:44you guys have done a number of videos on OrCam.
00:47Did you ever do too many though, Corey?
00:48Never.
00:49Never.
00:50Well, this might be one too many, we're about to find out.
00:52But there are cards popping up,
00:54you can check out some of our past videos on-
00:57I'll do it for you.
00:57I like yourself.
00:59On OrCam and the OrCam read,
01:01but today we're looking at a new feature
01:02called what's in front of me.
01:05And I will, I'll tell you right now,
01:07I am on the edge of my seat
01:09because I have no idea what that feature does.
01:11Well, people watching the video, Corey,
01:12we're going to find out.
01:13Because it's time for OrCam.
01:18Well, thank you for that energetic handoff there, Corey.
01:21And we're very excited to be bringing you
01:23the latest feature of OrCam.
01:24Now, Corey, I don't know if this is an official release
01:27or not because I randomly ended up
01:29downloading a software update
01:31and randomly realized that there was this new feature.
01:34Is it possible that you and only you have this feature?
01:37I certainly hope so.
01:38Yeah, I would feel very special.
01:40But yeah, it seems to be that there is a software update
01:43and it seems to be that it's free.
01:44And it seems to be that it gives you
01:46some further operational goodies.
01:49Okay.
01:50But I don't know if OrCam have advertised this at all.
01:52So I feel like I'm like a little bit on the underground,
01:55like getting secret features before.
01:57OrCam, throw it down in the comments
01:59if we shouldn't have been making this video.
02:03But there was a what's in front of me feature before,
02:05which surprisingly, Corey,
02:06it would tell you what's in front of you.
02:08Now, quick question.
02:10I have used that what's in front of me stuff before,
02:13not specifically with OrCam,
02:15but with like seeing AI and things like that in the past.
02:18Was it similar to that where it was sort of like,
02:20it looks like an office with some chairs?
02:23It was very limited in terms of what it would recognize.
02:25And it was very sparse on any kind of detail.
02:27So it had very limited functionality.
02:29But now, Corey, OrCam seems to be on the AI bandwagon.
02:33Oh, okay.
02:34Corey, we've been looking at AI recently.
02:36Yes.
02:37What have we done?
02:38We've done Metaglasses.
02:39Check.
02:40Look out for the card.
02:40We've done Envision.
02:42Check.
02:43Look out for the card.
02:44We've done Be My Eyes.
02:45Check.
02:46Or Be My AI, look out for the card.
02:47Yep, check.
02:48What else have we done?
02:49Anything else?
02:50Seeing AI, technically.
02:51Seeing AI, look out for the card.
02:52Look out for Jonathan.
02:53I know.
02:54A lot of AI stuff that we have been doing.
02:57And OrCam has jumped on the bandwagon.
02:59Now, what you're gonna want to do
03:01in order to get this update,
03:02and hopefully it's not just me who is somehow getting it,
03:04is you're gonna wanna connect your OrCam to the Wi-Fi.
03:07Now, there's two ways of doing this.
03:09You can either go onto OrCam's website
03:11and they have a page where you can type in your network name
03:14and the password and it will generate a QR code
03:17and you can take a picture of the QR code with the OrCam
03:20and that will hopefully connect it to your network.
03:22Or you can download the OrCam MyEye app,
03:26which I have on my phone here.
03:27Now, the app's pretty cool.
03:28You should probably just download it anyway
03:29because you can operate your OrCam from your smartphone,
03:32which is kind of cool.
03:33But also, if your OrCam is connected to the app
03:37over Bluetooth, then your OrCam is gonna have access
03:39to the internet and then you should be able
03:41to download any updates.
03:43So the update version, Corey, is 9.2 point something.
03:49Oh, good.
03:50I'm not 100% sure, but if you hear-
03:51Does it literally say 9.2 point something?
03:53Yeah, they're not sure what they want.
03:54The engineers are like, meh.
03:57So if you hear a 9.2,
03:59then you're probably getting what you need.
04:01I do wanna take a look at the app first, Corey,
04:02before we look at the What's in Front of Me feature
04:04because there is some cool controls with the app
04:06that I never really played with before.
04:08For some reason, I never downloaded the app.
04:10So just quickly, you can control very easily from the app
04:13the volume, which is important,
04:15the reading rate, which is important,
04:18but we can also have it just take a picture
04:21and start to read to us from the app.
04:22So there is a screen that has a volume control
04:24and a speed control, and it has a play button.
04:27And if I press the play button, it will take a picture
04:30and then it will start to read.
04:32Volume, E10E, G rating rates, copyright 60,
04:36continuation is unreadable.
04:39And there you go.
04:40So it was reading from my phone screen
04:41and nothing too exciting.
04:41So a couple of really quick things
04:43that some people I know can struggle a bit
04:47with the touch bar on the OrCam.
04:49Yes, very much so.
04:50And so being able to do the volume is great.
04:54The speaking rate,
04:55that's typically hidden down more into the settings.
04:58So having that straightened right out in front of you
05:01is kind of nice.
05:02Very handy.
05:03And then the same with triggering the picture
05:04right from your app, that's kind of clever too.
05:07Indeed, yes.
05:07I like that, that's cool.
05:08All right, so if you want to find out
05:09about all the features of the OrCam MyEye,
05:12then please do check out our previous video.
05:14But Corey, are we ready to find out
05:16what is in front of us?
05:17I am ready.
05:19All right, I'm ready.
05:20I'm scared, but we're going to do this.
05:21So keep on watching.
05:24All right, Corey, well, here we are out in the lobby.
05:26I see you still have your trusty margarita glasses
05:28on you right there,
05:29but you're also wearing the Meta glasses, I see.
05:31I am, I am going to,
05:33we're going to do a little comparison.
05:34First, we're going to see how OrCam does
05:36with its new feature.
05:37And then just to see, we're going to see how Meta does.
05:40Indeed, some people call this an A-B test.
05:42Corey, have you heard of that before?
05:43I have.
05:44I'm going to be A, you can be B, all right?
05:46And I can't see.
05:47Yeah, that was a good little line there.
05:49All right, so yes, we have something on the wall.
05:51We're going to find out with OrCam's new feature,
05:54what it is, and we're going to ask some follow-up questions
05:56and we'll just see what we can find out.
05:58Here we go, Corey, are you ready?
06:00Yes.
06:00Good.
06:01Okay, here we go.
06:03Hey, OrCam, what's in front of me?
06:10A tree painted on a wall in a room.
06:12For further details, say question and ask your question.
06:15To exit, say exit.
06:17Question, tell me more about the tree.
06:22A picture of a tree.
06:25Question, is there any writing on the wall?
06:32Yes.
06:34Question, what does the writing say?
06:40A tree with words on it.
06:42Exit.
06:45Exiting.
06:47Corey, it's certainly not the best result
06:51we've ever gotten from an AI.
06:52Yeah, I mean, it was, yeah.
06:56Bit of a struggle to get information out of it.
06:58I especially like how I said,
07:00is there any writing on the wall?
07:01Yes.
07:02It says yes.
07:02It's kind of like a very, what's the word I'm going to use?
07:05It almost seems annoyed that you're asking any questions.
07:08It's very literal in its interpretation.
07:11I said, is there any writing on the wall?
07:12And that would imply maybe I want it read to me,
07:15but it just says, yes, okay.
07:16Well, maybe you need to be more of a direct communicator
07:18and say, what does it say?
07:19I need to improve my communication skills.
07:21Corey, let's find out whether Mr. Zuckerberg's device here
07:26does a better job with the same task.
07:28Sounds good.
07:29Hey Meta, what's in front of me?
07:37A wall with a tree decoration and the words rain or shine.
07:41Hey Meta, tell me more about the tree.
07:48The tree appears to be a hand-drawn
07:50or printed decoration on the wall with leaves and flowers.
07:54Hey Meta, what color are the flowers?
07:59The flowers are yellow.
08:02Hey Meta, what do the words say?
08:07The words said rain or shine.
08:11That's pretty good.
08:12Although I know for a fact, and this is kind of weird,
08:15there's actually more words.
08:16I wonder whether the picture that you took,
08:18they weren't in the shot.
08:19Yeah, must've been, because we, you know,
08:21behind the scenes here, we tested it.
08:23It did a really good job of reading all of the words,
08:26even though it's a weird stylized kind of,
08:28but this time, yeah, it didn't see them all.
08:30Well, Corey, we learned a lot more
08:31about what was on the wall there with the Meta.
08:33We learned that there were some words there,
08:35so at least rain or shine.
08:36We learned that this was either a painted
08:37or a paper tree that had paper leaves.
08:41And flowers.
08:41And we learned the color of the flowers.
08:42So I think, as we can see here, the Meta
08:45and probably the other devices like the Envision
08:48and the Be My AI and stuff like that
08:50is definitely giving us more details about what's there.
08:53The OrCam is a little more limited
08:54in terms of the information it's giving you.
08:56Yeah, it's more A and less I right now.
09:01All right, should we-
09:02Artificial lesson taught.
09:02I have no idea where we're going with this.
09:04Corey, should we go sit down at the table
09:05and check out how it does with text?
09:07We shall.
09:08Let's mosey.
09:08Hmm.
09:10One of the other cool features of AI,
09:13especially around reading, I love your sound.
09:15Hmm.
09:16Hmm.
09:17Especially around reading text is not so much the OCR part,
09:22like actually having the text read back to you,
09:25but the ability to ask follow-up questions,
09:28like either to have it summarized
09:29or in example of a menu, you can drill down
09:33and ask a specific question rather than having
09:35to read kind of traditional OCR top to bottom.
09:39That question for you always seems to be
09:41with regards to what is on the turkey tom.
09:43I know.
09:43Jimmy John's, yeah.
09:44Do you think by now I'd know what's on the turkey tom?
09:47So I wanna see if OrCam's what's in front of me feature
09:51using AI has any of that kind of built in.
09:55Now we know already that you could use
09:56the traditional OCR part.
09:58Yeah, I'd like to say that the OCR
10:00with the OrCam remains very impressive.
10:02It does a great job of reading text
10:03and you can see our previous OrCam videos
10:06to see the quality of that.
10:07But this is a little different now
10:09we're using the AI functionality instead.
10:10So we're not looking to get a read from top to bottom,
10:13like you say, Corey.
10:13We're looking to get a summary
10:15and then be able to ask questions.
10:16But I just thought of something
10:18and I'm gonna put you on the spot.
10:19Oh dear.
10:20Would OrCam's smart reading feature
10:24allow you to ask about turkey subs?
10:27Or is it more specific to like phone numbers?
10:31Yeah, there is not as far as I am aware at the moment,
10:34the smart reading feature only asks you
10:36to ask about very specific things.
10:38Okay, all right, let's try, let's see how OrCam does.
10:41Okay, so I'm gonna go ahead and put on the OrCam.
10:45This is the MyEyePro 3, I believe, Corey.
10:51And I accidentally just put it to sleep
10:53rather than waking up because it wasn't asleep.
10:56So I'm gonna wake it up again now.
10:58Okay, it's a good start.
11:01Okay, so in front of me,
11:02I have the legendary Jimmy John's menu,
11:05which you may have seen in previous videos.
11:07It is, it's well-used.
11:09Okay, and we're gonna go ahead
11:11and ask the OrCam what's in front of us
11:13and then we'll see what it says.
11:15Hey OrCam, what's in front of me?
11:25A person is holding a piece of paper.
11:28For further details, say question and ask your question.
11:31To exit, say exit.
11:32Well, that is accurate.
11:34I am holding a piece of paper,
11:35not giving us a lot of information.
11:37So let's ask what's on the paper.
11:39Question, what is on the paper?
11:44Print.
11:48A re-delivery receipt.
11:51That doesn't seem accurate.
11:52I heard something, something, receipt, deliveries?
11:54Yeah, it sounded like a delivery receipt.
11:55Okay.
11:56Question, what does the text on the paper say?
12:01A hand holding a re-deliverer sign.
12:04Exit.
12:07Exiting.
12:09Corey, unfortunately, it did not seem to do
12:12what we were hoping it would do.
12:13Yeah.
12:14Yeah.
12:15Now, I have tried this with other texts as well,
12:18with standard formatted print.
12:20Same problem there.
12:21It tells you that there is a sheet of paper
12:23with writing on it,
12:24but it does not tell you what the writing is about.
12:27And there doesn't seem to be a way to get it to you.
12:29There doesn't seem to be a way to get it to do that.
12:30And I think to be fair, as we said at the beginning too,
12:33OrCam's OCR is top-notch.
12:37Yeah.
12:38And obviously we could use that to read it.
12:40And in the smart reading feature,
12:42it does allow you to drill down a little bit more.
12:44But we wanted to see how, in my opinion,
12:47I wanted to see how its AI engine is.
12:50Yeah.
12:51And, you know, compared to some of the Be My AI videos
12:53we've done, I don't think,
12:55obviously this isn't its strong,
12:57well, I don't even say it's its strong suit.
12:59It's not wearing a suit at all.
13:00No.
13:01Let's be clear here though, Corey.
13:03Yes.
13:04I don't think this is an officially released feature.
13:05I'm still confused about this.
13:06It's just something I randomly came across.
13:08So, I mean, we definitely have to cut it some slack.
13:11We are just demonstrating at the moment where this is
13:13and the fact that it seems to be freely available
13:16for download.
13:17So yeah, even if it's not doing everything
13:19that some of the other devices that we've looked at does,
13:22Corey, it's still worth a download because it's free.
13:24Yes.
13:25Groovy.
13:27Well, there we go, everybody.
13:28This is the possibly unreleased,
13:30possibly not supposed to be known about
13:32OrCam update to What's in Front of Me.
13:35And I have to say, Corey,
13:37it is actually a lot better
13:39than what the What's in Front of Me feature used to be.
13:42Okay.
13:42Because now it will give you details
13:44about whatever is in front of you
13:45compared to where it used to be just specific things
13:48like tables or doors or screens and things like that.
13:51So there is a big improvement of where it was.
13:54And like I say, it is free to download
13:57as far as I can tell at the moment,
13:58I managed to get it for free at least
14:00on a couple of different devices.
14:01So yeah, however,
14:04does not reach the artificial intelligence heights
14:07of some other devices that we have looked at
14:09on this channel.
14:10Yeah, I mean, two things.
14:11First of all, it's obvious to me,
14:14and I guess maybe I shouldn't say obvious
14:16because I could be proven wrong,
14:17but it sure doesn't seem like it's using GPT.
14:20Doesn't seem like it
14:21because the level of detail it gives
14:22is not the same as what we've seen with other devices.
14:25So I'm guessing that.
14:26And number two,
14:31this is an OrCam selling feature.
14:33Like a number of times when I've worked with clients,
14:37MyEye does a lot of different things, right?
14:39The money identification, facial color,
14:42product identification, that's all cool.
14:44And a lot of times some of them work pretty decent,
14:48but I don't think you'd ever buy the device
14:50for those specific features.
14:53To me, the OrCam has always been
14:55one of the best OCR devices.
14:57So my-
14:58I'm sorry, I'm sorry.
14:59Please go on.
15:00No, so my point is like,
15:03if you already own the OrCam
15:04because you bought it for OCR, why not?
15:07It's a free update.
15:08Why not have it?
15:09It's not hurting anything.
15:10And again, you wouldn't be buying this device
15:13strictly on this feature.
15:15No, that's true.
15:15However, Corey, in the year of our Lord, 2024.
15:19That is the year we're in.
15:20How relevant is OCR at this point?
15:25On the basis that, yes, if you want to read a book,
15:27if you want to read a magazine,
15:28if you want to read a newspaper,
15:29then yes, you do want it to read the whole text there.
15:33But if you want to kind of just read other things
15:37like bills or menus or, you know,
15:41like paperwork or whatever,
15:42and you don't necessarily just want to
15:45listen to the whole thing.
15:46So I feel like in this day and age,
15:47you kind of need a solid AI platform
15:50in addition to your OCR.
15:52Yeah, I mean, for me personally,
15:53I've done everything OCR up to this point.
15:57And so I totally hear what you're saying.
15:59And I think for people who are just getting into using OCR,
16:02AI definitely makes it more obtainable
16:04and like a much less barrier for entry.
16:08But I also don't know that I really like relying on AI
16:13to make the decision of what's important
16:15on that piece of paper for me too.
16:18So I think what you said though,
16:19you need a combination of both.
16:21I want a solution that will, you know,
16:24when I trigger the picture,
16:25it starts with traditional OCR,
16:27but then I can follow up, give me a summary,
16:29ask a specific question.
16:31So then for me, I'm getting the best of both worlds.
16:33Yeah, fair enough.
16:34And if you have watched our very exciting
16:36All Camera Read 3.0 video,
16:40Jonathan, you've got a lot of work
16:41putting cards up in this video, so I apologize.
16:43But if you check out that video,
16:45you will see that you can actually connect
16:46that device and the OrCam MyEye Pro 3 to a computer
16:51and you are able to use artificial intelligence
16:53through your computer, which is much more fully featured.
16:56So there is a way to kind of use AI
16:59that is more fully featured with these devices,
17:01but you have to be kind of connected to a computer.
17:03And I believe we've done a past TechConnect Live,
17:05not our most recent, but the most recent one we did
17:07on OrCam, we had Corinne Nero from OrCam
17:11on showing some of those features too.
17:13Shout out to Corinne.
17:14Yes.
17:15Yeah.
17:16Thank you for giving us a call.
17:17Yeah, hopefully she's not annoyed
17:18about us releasing this video, we shall see.
17:20But anyway, all in all, why not download it?
17:23Yes.
17:23See if you still can.
17:24I mean, I don't know if I just randomly happened upon this,
17:26but see if you can just connect your device to Wi-Fi.
17:28It should automatically download the updates
17:30if it's available.
17:32And then you can go ahead and give the feature a try.
17:35Call me.
17:36Yeah, put it down in the comments.
17:37I'm curious how you're using your OrCam first.
17:40I'm curious if you have one.
17:41Yes.
17:42And what is like, why did you buy the OrCam?
17:43What's the number one feature?
17:45I keep talking that OCR is the best,
17:47but I'd love, somebody's going to prove me wrong
17:48and be like, I bought it just for the facial recognition.
17:51Exactly.
17:52Throw it down in the comments
17:52why you chose to purchase your OrCam.
17:55Also, do us a huge solid.
17:58Go ahead and subscribe to the channel and like the video.
18:00We just passed 5,000 subscribers.
18:03We did.
18:04Super proud of that.
18:06But we've got tons of room to grow.
18:08So if you are watching this and you enjoy this,
18:13or if you just enjoy anything,
18:15why don't you just click that subscribe button?
18:16Let us know that you enjoy stuff.
18:18That'll figure, right?
18:19We know for a fact that some people
18:21don't enjoy some aspects of our videos.
18:22That's true.
18:23But I sure do.
18:24That's the main thing.
18:26We have a podcast.
18:27Yeah, call me.
18:28Where can people go to listen to that?
18:29They can go to their podcast app of choice
18:31through a search for VisionForward TechConnect.
18:33You're going to find audio versions of our TechConnect Live,
18:37which we do every other Thursday, 11 a.m. Central.
18:41This video will release on a Thursday,
18:44and then the opposite Thursday will be a TechConnect Live.
18:47Don't ask me about dates.
18:48Well, no, I don't know about dates,
18:50but I do know that if you're watching this video,
18:52we will have the week before
18:53done a very exciting live show on Wordle, call me.
18:56Yes, yes, we will Wordle.
18:58We're going to Wordle our way into the game of letters.
19:02And I'm going to be looking at it accessibility-wise on PC,
19:06and you're going to be looking at it
19:07accessibility-wise on smartphone.
19:10We did already.
19:11Oh, good point.
19:12Yeah, sorry, we're in the future now, aren't we?
19:15We did.
19:16I looked at it on PC, and Corey looked at it on smartphone.
19:18I feel like we do this every video.
19:19And it's crazy because how bad I beat you.
19:23I mean, it's how you even came on this video
19:26and showed your face today.
19:27I need everybody watching to go back
19:29and watch that live show
19:30and see whether this is true or not.
19:32But anyway, go ahead and check that out.
19:34You can watch our live shows by tuning in
19:36on the Thursday that they are on,
19:38and that will be 11 a.m. Central.
19:41And you can just go to the homepage,
19:43youtube.com forward slash vision forward tech connect.
19:46Corey, would you care to tell people about the LMS?
19:48Yeah, we have an LMS.
19:50Good, thank you everyone.
19:52You can head to vision,
19:53you can head to techconnect.vision-forward.org.
19:56We've got ACVREP credits free on demand.
20:00So if you need any of your CADIS or any ACVREP credits,
20:05we are a source for your credits free of charge.
20:09They are there for the taking.
20:11Well, you might as well go take them then.
20:12Corey, I feel like our outros are getting longer
20:14than our actual video.
20:15So we should probably say goodbye at this point.
20:17Goodbye.
20:17Goodbye.
20:24Jonathan, if you could interject a fanfare audio there,
20:27please, that would be wonderful.

Recommended