Word Lens — Augmented Reality Translation 203
Barence writes "PC Pro has a review of a new augmented reality iPhone app that translates from Spanish to English on the fly. 'Point the camera at a decent-sized chunk of Spanish text and within a couple of seconds you'll get a rough and ready translation,' said the reviewer. 'And most magnificently of all, the translation is overlaid, at the correct size, on the original object.' The team behind the project has produced a video of Word Lens in action."
Not going to lie (Score:4, Interesting)
Monty Pythons Hungarian Translation Book (Score:5, Funny)
Okay let's see, we combine the terror of OCR with mangled language translation and the pit fall of cropped or intersecting text patches and variable fonts and multiple contexts? My hovercraft is indeed full of eels.
Re:Monty Pythons Hungarian Translation Book (Score:4, Interesting)
Indeed. Sometimes a bad translation is worse than no translation since it might convince you that you do actually understand the foreign text.
Re:Monty Pythons Hungarian Translation Book (Score:4, Funny)
Will you please fondle my bum?
Re: (Score:2)
True. However, most oddities are going to translate as just that, oddities. If I'm pointing at a sign at a train station next to a yellow line, and it says "please not to mock the lizard", then I'm going to assume it's garbage. If it says "please to stand backward of yellow line" then I get the gist of what it is saying.
I can see this as really useful for things like menus. Even the literal translation of each word gives me SOME idea of what I'm about to order. OK, so maybe the marmaset comes out medium
Re: (Score:3)
If I'm pointing at a sign at a train station next to a yellow line, and it says "please not to mock the lizard", then I'm going to assume it's garbage. If it says "please to stand backward of yellow line" then I get the gist of what it is saying.
When the 4:45 Express Lizard comes barreling through the station, you'll understand "not to mock the lizard" as you get sucked off the platform and chopped into messy Lizard Chow beneath its steel talons.
Although, I suppose "stand backward of yellow line" would pr
Re: (Score:3)
Hell, even in Houston last time I was there, I saw more and more signs and postings I could not read, I won't even get into how many radio stations I had to scan to get one that spoke English.
It is a shame that in parts of the US mainland, that you are starting to have to take spanish lessons in order to function.
Re: (Score:2)
Re:Monty Pythons Hungarian Translation Book (Score:4, Insightful)
Google Goggles already does translations of pictures. Google Translate already turns voice English to voice Spanish. Word Lens sounds pretty cool, but it's really not that far ahead of other similar projects.
It doesn't require Internet connection.
Re: (Score:2)
Re: (Score:3)
Re:Not going to lie (Score:5, Insightful)
Sorry, the iPhone is the easiest platform to monetize right now. It'll almost always be the first choice for apps.
Being an android user is a lot like being a Mac user waiting for Windows games back in the day. It sucks, but that's how it is.
Re:Not going to lie (Score:5, Insightful)
Certainly a slick and quick way to get an app to market.
Re:Not going to lie (Score:5, Interesting)
Also, the API is basically C (Objective C++). I had no trouble pulling in a ton of my existing C++ imaging code and just compiling it right up, layering a nice UIKit UI over it.
I have an Android phone, but I'll have to learn Java and *PORT* my C++ code to it.
Re: (Score:2)
What's your beef with the Android NDK then?
http://developer.android.com/sdk/ndk/index.html [android.com]
Re: (Score:2)
I was thrilled with my Android phone because I was able to use most of my java libraries without modification. C++ is an awesome language with tons of amazing libraries, but sometimes I just want to write code without worrying about unreleased memory.
Re: (Score:2)
Hah! I like Android for exactly the same reason. I'll be able to easily port my Java code to it. But it would be a pain to have to port to Objective C...
That's probably a good thing (Score:2)
to wait for the bugs to be worked out over a few releases. It's cool and awesome, but I don't know how useful it'd be in its present state of word-only translation.
Re: (Score:2)
Sorry, the iPhone is the easiest platform to monetize right now. It'll almost always be the first choice for apps.
On the contrary, I personally know of two new companies who are developing over 100 apps between them, and chose to aim for Android only (for the first rev).
Both companies' rationale: less competition, higher growth rate, and more investment dollars (Google plus MANY others, vs. just Apple) in the Android market. Note, it was NOT learning curve, compatibility of Objective C/Cocoa vs Java, and other technical factors.
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
That's because, unlike the PC market, apps are typically 99 cents or 1.99.
My risk is much lower than $30.00 (or even $70.00) for a pc or console applicaiton.
Re:Not going to lie (Score:5, Informative)
Here, try this: http://www.appbrain.com/search?q=translation [appbrain.com]
There are many to choose from. None have the cutsy, but useless superimposition upon the original, but digging around in there will find page after page of ocr and translation apps.
Lets be perfectly frank. This is an app you will use three times then forget you even have it. It is simply not useful. By the time you run around shooting pictures of signs and finally find one that says "El baño" you will have already peed your pants in Mexico.
Far more useful is Google Translate [appbrain.com], which uses voice recognition allowing you to speak your sentence, and will then speak it back in the language of choice. (You can use text input and copy and paste from dozens of free scanner apps as well).
Re: (Score:3)
2) unlike you, I believe apps like these will become very useful. There are many instances where I've wondered what a particular sign meant... "danger, rockfall ahead" would have been useful to know. Or thin
Didn't miss (Score:3)
The app does two things out of the gate that show it works - it has a mode that reverses words randomly, and another mode that erases random words. Both are damn impressive and show it can do what it says - so I went ahead and bought the spanish->English pack knowing I might have need of it in the future when travelling.
If you follow app sales closely (and being an iPhone developer, I take every chance I get to review trends) by far the way to make money in an app is with a free version that has in-app
Re: (Score:2)
Sure, there are other apps that do this, but the "cutesy" text overlay fits in perfect with Apple's current line of applications.
Re: (Score:2)
Wow factor wears off quickly when functionality fails.
And it does fail. This thing can not do large blocks of text like Google Goggles.
I don't need the restaurant menu overlaid on the menu. I already have a menu in hand and know what it looks like. That is simply not important. I need the translation, and that's all. Seriously, I can't imagine even YOU believe this is a real world example that would justify the cost of this app.
Like I say, you buy this app, you will wow your friends a few times and move o
Re: (Score:2)
Yes, except for the whole part where this works offline, which is what many people are likely to be when traveling.
If this actually works on a menu, it would have been *hugely* useful on the last trip I made. Whether it super-imposes text or not is just icing.
Re: (Score:2)
But no android app.
But, does it run on Linux?
Re: (Score:2)
The augmented reality part is indeed very impressive. But no so much the translation. I found this youtube video that shows more real world uses: http://www.youtube.com/watch?v=45SeN7TWcbE [youtube.com]
It seems that it translates word by word, which is not the very best approach.
Apple Confusion (Score:2)
Saw this yesterday (Score:2)
Someone was showing this off at TechShop last evening. Very nice.
the cool thing is that it's so cheap (Score:5, Interesting)
i remember the days when all the new cool tech was only seen in the government and large corporations first and then trickled down to us peons. these days with our rampant consumerism it's the opposite. we see cool stuff like this first and it's cheap and the big boys are now playing catch up because things move so fast
if it wasn't for our vane consumerism this would be a government project costing tens of millions of $$$ in R&D and the devices would be single use devices that also cost some ridiculous amount of money
Re: (Score:2, Funny)
Yeah I know! GPS was totally created by the consumer market! It's a good thing too, otherwise it would have been limited to military and would be hugely expensive and unreliable! Amirite!
And jet engines! Man, I'm glad those hit consumers first, and the internet is a prime example! Oh, and don't forget digital photography! Imagine how far back in the stoneage we'd be if all those things had belonged to the military industrial complex first! Their strict specifications, demand for reliability and track
Re: (Score:2)
Smart people realize that you need a healthy mix of both consumer-driven and government-funded R&D. Although the result sometimes intersect, they have different motivations and often different useful results.
Re: (Score:2)
GPS might have been military first but what about directions? google maps and even the old mapquest were a lot better than the directions i would get with my old military GPS. and what about modern encryption? yes we had cool crypto tech in the military, but there are a lot of R&D dollars going into consumer encryption that will be better than what the military has.
ARM CPU development is being driven by consumer spending and those are better than anything in mobile military devices. same with modern dis
Re: (Score:2)
Low piracy -> Cheap prices.
Cheap prices + Easy -> Low piracy.
The amazing thing Apple did is putting their market into this sweet spot.
Re: (Score:2)
i remember the days when all the new cool tech was only seen in the government and large corporations first and then trickled down to us peons. these days with our rampant consumerism it's the opposite. we see cool stuff like this first and it's cheap and the big boys are now playing catch up because things move so fast
if it wasn't for our vane consumerism this would be a government project costing tens of millions of $$$ in R&D and the devices would be single use devices that also cost some ridiculous amount of money
Funny you say that as I remember big government financed project for English/Farsi translator costing million of dollars.
Too bad that it does not work very well (Score:4, Informative)
Too bad that this system only works with limited amount of texts. I installed this app on my iPod Touch and tried the default text reversing filter. If I used a Serif font, this could not read the words realiably. Fonts needed to be Sans Serif. Also this uses some dictionaries so if the word is not in dictionary (eg. deemed offensive) or some random gibberish, this could not recognize it. And all this I did with large black text on white background so viewing conditions are definitely not the issue.
Re: (Score:2)
You wanted good reliable OCR on 960x720 images? Uhm.
Re: (Score:2)
The camera of an iPhone has a lot higher resolution than 960x720. The screen of an iPhone 4 is 960x720, but the OCR algorithm wouldn't run on what the app eventually outputs to the screen.
Re: (Score:2)
If a human can read it, an OCR program should be able to read it as well. If not, that's a bug.
Re: (Score:2)
He said "iPod touch".
Re: (Score:2)
That is a good first step... (Score:2)
Of course, I tried to use a similar argument decades ago in school when everyone told me I needed to learn Spanish (while living in a state that was dramatically closer to French Canada than to Mexico, but oh well), and I still ended up taking three years of a language that I almost never en
Re: (Score:3)
Now they just need to do the same for economically relevant languages. The top developing countries currently are Brazil, India, and China (in no particular order) and none of them speak Spanish as a primary language.
Querying wolfram alpha [wolframalpha.com], the most spoken languages in the world are:
So languages #1,3,and 5 all have a completely different character set (esp. Mandarin), while #2 and #4 share the basic roman character set (with a few symbols outstanding). I can see why they went Spanish. Also, many popular travel destinations (i.e., Central and South America, Spain, etc) have spanish signs where this would be useful.
I ex
Re: (Score:2)
If we also count people who know two or more languages, I'm betting English would be far more than 760 million. A lot of surveys only ask what your primary language is. English is easy to pick up, Hollywood movies and the USA/UK/Canadian music industries also help spread english everywhere.
Re: (Score:2)
If we also count people who know two or more languages, I'm betting English would be far more than 760 million.
I suspect you're probably counting a few tens of millions of people who believe they speak or write English, but really, REALLY don't. [engrish.com]
Re: (Score:2)
Most of the "original" Spanish text on the demo video is Spanglish anyway. I bet if they were translating signs written in actual, correct Spanish, their automatic English translation would be even worse than it already is.
Re: (Score:2)
I'm afraid I'll have to call shenanigans on those stats, it looks like the same boring numbers I've been hearing for 15+ years and they don't make sense, there are easily 1+ billion people in the world who speak english. It may not be their native language but they can speak it anyway, there aren't a whole lot of people out there who speak mandarin as a second language (compared to english, it's the lingua franca of the post-WW2 world, at least so far).
Re: (Score:2)
Now they just need to do the same for economically relevant languages. The top developing countries currently are Brazil, India, and China (in no particular order) and none of them speak Spanish as a primary language.
Querying wolfram alpha, the most spoken languages in the world are:
Indeed a lot of people speak Spanish, I don't doubt that. However it is not as economically relevant as other languages, as it is not spoken much at all in the top emerging economies.
I can see why they went Spanish.
As can I. Spanish is pretty easy as far as languages for text translation go. And from Spanish it is a very short hop to Portuguese; and only a slightly longer hop to French or Italian.
I am not looking to discredit their work or say it is of no value. I'm just saying I hope that this was a good learning exercise for the
The word "Awesome" applies sometimes (Score:3)
I remember back in the early 90s when a guy showed me an 8Gb backup tape he had in his shirt pocket and I thought, Holy Crap, 8 GIGA-bytes fits in your pocket now? That's Awesome! And now, years later, you can carry many times that much data on a keychain. Equally Awesome. And this, this translator thing... totally and completely Awesome and Amazing. If you picture yourself as someone from say 100 years ago looking at today's world, some things we take for granted are pretty much like magic.
Re: (Score:2)
The killer app for augmented reality (Score:4, Interesting)
These guys just opened a gold mine.
I'm sure there will be a ton of cynical and jaded comments here, but this is a working prototype of augmented reality that is actually immediately and unquestionably useful, even in its infant state. Even non-technical people can see the promise of this, and graspable promise equals investment.
Bravo, and congratulations to the developers!
Re: (Score:2)
I'm sure there will be a ton of cynical and jaded comments here
You've cracked the Slashdot code.
Re: (Score:2)
I agree! *Precisely* because this is so impressive, will it also see many complaints about bad translations and so on. The better an application is, and the more hyped it is as a consequence, the more complaints will it see due to expectations so high that they are even impossible to meet given today's technology.
I also agree that the translations offered will probably not be the best, but so aren't the Google Translations either, with their online huge database of statistics on how to translate words. And
Another story about how badly it works (Score:5, Interesting)
Overall, not worth the money until it gets heavily reworked.
Re:Another story about how badly it works (Score:5, Insightful)
Overall, not worth the money...
What did you expect for 5 bucks?
MOD PARENT UP (Score:2)
$5 and it's not good enough!?!
On the other hand. On iTunes there's one review complaining that it should be free and ad supported. Sigh
Re: (Score:2)
Just think of the possibilities for ad placement, though...
Re:Another story about how badly it works (Score:4, Interesting)
In the future it would be nice if sign makers were required to include a data matrix [wikipedia.org] in some corner, and put various translations into an online database. That way, any device with the most rudimentary camera and data service could translate it accurately, and it would be a lot cleaner than putting multilingual signs everywhere.
Re: (Score:2)
Using existing technology and not requiring that everyone post new signs, you could have signs in Google Street View automagically OCRed and post translations as map data. Then your GPS/phone could just tell you what the nearest sign says.
This shows how full of shit Steve Jobs is (Score:2, Insightful)
... when he says that one of the reasons the iPhone won't run Flash is because it doesn't have the processing power.
You're telling me it can have the power needed to do something like this - analyze an image for text, decode it, put sentences together, translate, match the most appropriate font and colours, scrub the original text, render the new text at the appropriate angle and position - but not to play Flash movies. I call bullshit.
Re:This shows how full of shit Steve Jobs is (Score:4, Informative)
Much as I hate apple.. I hate flash more. And it wouldn't surprise me if what you described _is_ actually less resource intensive than flash.
I have a higher-end quad core i7 and 12GB ram .. playing a flash video makes all 4 cores work and uses a rediculous amount of ram.
Re: (Score:2)
Re: (Score:2)
Don't be a diculous again.
Re:This shows how full of shit Steve Jobs is (Score:5, Insightful)
I call your bullshit. I don't ever recall Steve Jobs saying that the iPhone does not have the processing power to run Flash. He has said that Flash is buggy, crashes often, and does not have good performance in mobile devices. (All of which I agree with.)
Basically, his position has been that Flash is too crappy for the iPhone, irrespective of whether the iPhone can run it or not.
-dZ.
Re: (Score:3)
You really don't understand how crappy Flash is [macrumors.com] to begin with.
Re: (Score:3)
The fact that it is POSSIBLE to have a "ten fold improvement" is pretty much PROOF of how crappy it is/was.
Apple's "jihad" against Flash has pushed Adobe to improve Flash significantly. Adobe always claimed that there were never any performance issues, no battery-hog-issues, yet they are capable of this ten fold improvement?
I mean, in the demo, the Adobe guy makes fun of Apple's MacBook Air for being "underpowered". The latest Air is *NOT* underpowered. It *SHOULD* be more than capable of playing back 10
This shows how great optimized code is (Score:2)
No, this demonstrates what can be done when people write code using
Re: (Score:3)
First of all he didn't say that at all. Basically, what he said is that he doesn't want crappy software sucking the battery life.
Second, if Adobe themselves can make up to tenfold improvements [macrumors.com], that says a lot about how crappy Flash really is.
Re: (Score:2)
Except in this case.
Actually the iPhone has the horsepower to run flash and to the best of my knowledge Steve Jobs never said that it didn't. Battery life and usability (plus even more really crappy applications) were always the concerns.
Re: (Score:3)
Re: (Score:3)
No network connection required.
http://gizmodo.com/5714462/translate-signs-and-menus-instantly-using-word-lens-crazy+amazing-iphone-app [gizmodo.com]
Re: (Score:2)
They are cheating from the start (Score:5, Informative)
"LO TRADUCE EL TEXTO" is not a Spanish phrase, unless you want to say "Text translates it". The Spanish phrase would have been "TRADUCE TEXTO" but I think the result with that tool was so bad they changed the Spanish text until the bad translation rendered a good English phrase.
The same happens with other examples from that video such as "ROPAS OPCIONAL EN ESTA PLAYA". The only way you are going to read that sign is if you ask an English speaker to write it.
What they did was write the English phrases, translate them to Spanish and then translate them back for the video.
Re: (Score:2)
As far as I can see it's translating each word separately without considering the context or re-ordering when appropriate. Of course this is going to result in terrible translations, so they have to cheat.
Re: (Score:2)
This is of course of no credit to the writers of this "translation" app. It seems to be looping over the words, looking them up in the dictionary and spitting out the top translation it gets for them, with no attempt to actually take care of word order or to use the context in which a word appears to choose the best translation.
Call it what it is: it’s just a Spanish-English dictionary. The hard part was getting the OCR to work right, and the visual effects are basically frosting. But Spanish-English dictionaries are still useful, if you use them like a Spanish-English dictionary and not like a full-fledged translation tool.
Re: (Score:3)
"Lo traduce el texto" is definitely not grammatical in standard Spanish, because it has two direct objects [wikipedia.org]: "lo" and "el texto". It might be acceptable in some dialect I haven't come across.
No, it's not generally ungrammatical. (Score:3)
"Lo traduce el texto" is definitely not grammatical in standard Spanish, because it has two direct objects [wikipedia.org]: "lo" and "el texto". It might be acceptable in some dialect I haven't come across.
Oh boy. This was one of the things I studied in my first year of grad school in Linguistics, so it was a long time ago. This is called "clitic doubling." I remember things being as follows:
Re: (Score:3)
It is not magic, but they are lying in the ad to make it look much better than it actually is.
The examples they show of it working are written in a way that no spanish person would have used.
Here's for instance my attempt to translate spanish to english in the same way: "For favour: Not throw the cigarettes passionate on the ground".
That's the kind of thing they're showing on the signs in the video.
dragon speak (Score:2)
I bought it (Score:2)
The app is free but the Spanish-to-English functionality costs $5.
It is totally helpless vs. handwriting.
When viewing nice clean computer text on my screen, and when the phone is held very still, it produces the usual clunky translation.
It would be very much better than nothing when attempting rapid translations in a foreign culture.
Re: (Score:2)
For $5, I think that's the point. As they say, use it to translate signs, menus, etc.
Re: (Score:2)
I bought it; mostly to support the developer. I hope they make a boatload, hire new devs, and keep doing cool stuff :)
Now then. (Score:2)
Where are all the people who feel it's not really necessary apply correct spelling ?
I like it! (Score:3)
I've been researching this area for my masters. Just getting the basic text localisation (i.e. recognising an area as containing text) working reliably is very difficult - there are some good algorithms out there but in the real world, with 1000s of fonts, font sizes, angles, lighting conditions etc, I've yet to see a 1 size fits all approach. And even if you do find an area of text, throwing that into an OCR engine is going to produce garbage for the most part. In short, its quite easy to show something off in controlled conditions but I wouldn't expect anything like the performance seen in the video in the real world.
The above said, very impressed to see that on an iphone and for it to be so responsive; these things can only get better and once some form of viable HMD makes it onto the scene these types of application are going to be massive.
Re: (Score:2)
Re: (Score:2)
Way better than signing up for eharmony and having them sell your name and address to magazine publishers.
Re: (Score:3)
That's easy. If she doesn't have a ring on her finger, she's looking. If she's not gay, you have a shot.
It is up to you to approach her. Simple as that. Walk up, and say hello...or try something like asking her the time, chat a little, then tell her "actually, I didn't need to know the time, but I saw you over here and just had to come talk to you"...and take
Re: (Score:2)
In all seriuosness, I think augmented reality beer goggles might be a best selling app.
You could go on dates with people who virtually look like their dating site profile photo, for a change.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
No, you need a small fish for that.
Re: (Score:2)
Then how do you know it's in spanish and not, say, in french? /duck
Re: (Score:3)
La traduction de anglais à espagnol est belle merdique.