Forgot your password?
typodupeerror
Iphone Software Technology Apple

Word Lens — Augmented Reality Translation 203

Posted by Soulskill
from the no-tengo-los-pantalones dept.
Barence writes "PC Pro has a review of a new augmented reality iPhone app that translates from Spanish to English on the fly. 'Point the camera at a decent-sized chunk of Spanish text and within a couple of seconds you'll get a rough and ready translation,' said the reviewer. 'And most magnificently of all, the translation is overlaid, at the correct size, on the original object.' The team behind the project has produced a video of Word Lens in action."

This discussion has been archived. No new comments can be posted.

Word Lens — Augmented Reality Translation

Comments Filter:
  • Not going to lie (Score:4, Interesting)

    by FredFredrickson (1177871) * on Friday December 17, 2010 @01:57PM (#34589862) Homepage Journal
    This is pretty damn cool. But no android app. No news if they plan on releasing one. In fact, their site is pretty void of any information at all. I would buy this just to play with it, but I'll never be an iphone guy.
    • by goombah99 (560566) on Friday December 17, 2010 @02:01PM (#34589906)

      Okay let's see, we combine the terror of OCR with mangled language translation and the pit fall of cropped or intersecting text patches and variable fonts and multiple contexts? My hovercraft is indeed full of eels.

      • by wjousts (1529427) on Friday December 17, 2010 @02:04PM (#34589954)

        Indeed. Sometimes a bad translation is worse than no translation since it might convince you that you do actually understand the foreign text.

        • by goombah99 (560566) on Friday December 17, 2010 @02:06PM (#34589976)

          Will you please fondle my bum?

        • by natehoy (1608657)

          True. However, most oddities are going to translate as just that, oddities. If I'm pointing at a sign at a train station next to a yellow line, and it says "please not to mock the lizard", then I'm going to assume it's garbage. If it says "please to stand backward of yellow line" then I get the gist of what it is saying.

          I can see this as really useful for things like menus. Even the literal translation of each word gives me SOME idea of what I'm about to order. OK, so maybe the marmaset comes out medium

          • by idontgno (624372)

            If I'm pointing at a sign at a train station next to a yellow line, and it says "please not to mock the lizard", then I'm going to assume it's garbage. If it says "please to stand backward of yellow line" then I get the gist of what it is saying.

            When the 4:45 Express Lizard comes barreling through the station, you'll understand "not to mock the lizard" as you get sucked off the platform and chopped into messy Lizard Chow beneath its steel talons.

            Although, I suppose "stand backward of yellow line" would pr

          • by cayenne8 (626475)
            I'm guessing this app will come in handy in the southern border states.

            Hell, even in Houston last time I was there, I saw more and more signs and postings I could not read, I won't even get into how many radio stations I had to scan to get one that spoke English.

            It is a shame that in parts of the US mainland, that you are starting to have to take spanish lessons in order to function.

      • by oodaloop (1229816)
        Google Goggles already does translations of pictures. Google Translate already turns voice English to voice Spanish. Word Lens sounds pretty cool, but it's really not that far ahead of other similar projects.
      • by TeknoHog (164938)
        I've had it with these motherfucking eels in this motherfucking hovercraft!
    • by mveloso (325617) on Friday December 17, 2010 @02:03PM (#34589936)

      Sorry, the iPhone is the easiest platform to monetize right now. It'll almost always be the first choice for apps.

      Being an android user is a lot like being a Mac user waiting for Windows games back in the day. It sucks, but that's how it is.

      • by mark72005 (1233572) on Friday December 17, 2010 @02:13PM (#34590060)
        I'm guessing it's probably the easiest for a corporation to develop for, since the hardware is a known quantity and for the most part the OS versioning is too, you will get consistent function across devices. It will also be tested by Apple before they allow it on the app store.

        Certainly a slick and quick way to get an app to market.
        • Re:Not going to lie (Score:5, Interesting)

          by Stele (9443) on Friday December 17, 2010 @02:29PM (#34590302) Homepage

          Also, the API is basically C (Objective C++). I had no trouble pulling in a ton of my existing C++ imaging code and just compiling it right up, layering a nice UIKit UI over it.

          I have an Android phone, but I'll have to learn Java and *PORT* my C++ code to it.

          • by Troed (102527)

            What's your beef with the Android NDK then?

            http://developer.android.com/sdk/ndk/index.html [android.com]

          • LOL I feel the same way - in the opposite direction.
            I was thrilled with my Android phone because I was able to use most of my java libraries without modification. C++ is an awesome language with tons of amazing libraries, but sometimes I just want to write code without worrying about unreleased memory. :)
          • by Atzanteol (99067)

            Hah! I like Android for exactly the same reason. I'll be able to easily port my Java code to it. But it would be a pain to have to port to Objective C...

      • to wait for the bugs to be worked out over a few releases. It's cool and awesome, but I don't know how useful it'd be in its present state of word-only translation.

      • by Chapter80 (926879)

        Sorry, the iPhone is the easiest platform to monetize right now. It'll almost always be the first choice for apps.

        On the contrary, I personally know of two new companies who are developing over 100 apps between them, and chose to aim for Android only (for the first rev).

        Both companies' rationale: less competition, higher growth rate, and more investment dollars (Google plus MANY others, vs. just Apple) in the Android market. Note, it was NOT learning curve, compatibility of Objective C/Cocoa vs Java, and other technical factors.

      • by Stregano (1285764)
        You said back in the day like Mac users are still not waiting for ports on good games, lol
      • Yes, for now that may be true. But considering more Android devices are selling every day than iPhones, it shouldn't be long before Android catches up, and probably passes Apple. Especially considering there are 3 official app markets for Android, and you can install apps from anywhere and are not limited to just the official markets.
      • That's because, unlike the PC market, apps are typically 99 cents or 1.99.

        My risk is much lower than $30.00 (or even $70.00) for a pc or console applicaiton.

    • Re:Not going to lie (Score:5, Informative)

      by icebike (68054) on Friday December 17, 2010 @02:28PM (#34590280)

      Here, try this: http://www.appbrain.com/search?q=translation [appbrain.com]

      There are many to choose from. None have the cutsy, but useless superimposition upon the original, but digging around in there will find page after page of ocr and translation apps.

      Lets be perfectly frank. This is an app you will use three times then forget you even have it. It is simply not useful. By the time you run around shooting pictures of signs and finally find one that says "El baño" you will have already peed your pants in Mexico.

      Far more useful is Google Translate [appbrain.com], which uses voice recognition allowing you to speak your sentence, and will then speak it back in the language of choice. (You can use text input and copy and paste from dozens of free scanner apps as well).

      • Actually, I think this is an interesting app, for two reasons: 1) The "wow" factor: even if the app doesn't work all that well and the only translation available is (crappy) Spanish, this app actually demonstrates one of those things that pops into everybody's head when they think of the future of AR.
        2) unlike you, I believe apps like these will become very useful. There are many instances where I've wondered what a particular sign meant... "danger, rockfall ahead" would have been useful to know. Or thin
        • The app does two things out of the gate that show it works - it has a mode that reverses words randomly, and another mode that erases random words. Both are damn impressive and show it can do what it says - so I went ahead and bought the spanish->English pack knowing I might have need of it in the future when travelling.

          If you follow app sales closely (and being an iPhone developer, I take every chance I get to review trends) by far the way to make money in an app is with a free version that has in-app

        • by Stregano (1285764)
          Please don't capitalize the word app and then try to make it an Apple thing. That is like me saying, "Please wash your Windows (in a Windows sense)". Sorry bro, but let's try this:

          Sure, there are other apps that do this, but the "cutesy" text overlay fits in perfect with Apple's current line of applications.

        • by icebike (68054)

          Wow factor wears off quickly when functionality fails.

          And it does fail. This thing can not do large blocks of text like Google Goggles.

          I don't need the restaurant menu overlaid on the menu. I already have a menu in hand and know what it looks like. That is simply not important. I need the translation, and that's all. Seriously, I can't imagine even YOU believe this is a real world example that would justify the cost of this app.

          Like I say, you buy this app, you will wow your friends a few times and move o

          • Yes, except for the whole part where this works offline, which is what many people are likely to be when traveling.

            If this actually works on a menu, it would have been *hugely* useful on the last trip I made. Whether it super-imposes text or not is just icing.

    • by MrMarket (983874)

      But no android app.

      But, does it run on Linux?

    • by RemyBR (1158435)

      The augmented reality part is indeed very impressive. But no so much the translation. I found this youtube video that shows more real world uses: http://www.youtube.com/watch?v=45SeN7TWcbE [youtube.com]

      It seems that it translates word by word, which is not the very best approach.

  • I expect mistakes of Newton hand writing recognition like proportions. Let the laughs begin.
  • Someone was showing this off at TechShop last evening. Very nice.

  • by alen (225700) on Friday December 17, 2010 @02:03PM (#34589932)

    i remember the days when all the new cool tech was only seen in the government and large corporations first and then trickled down to us peons. these days with our rampant consumerism it's the opposite. we see cool stuff like this first and it's cheap and the big boys are now playing catch up because things move so fast

    if it wasn't for our vane consumerism this would be a government project costing tens of millions of $$$ in R&D and the devices would be single use devices that also cost some ridiculous amount of money

    • Re: (Score:2, Funny)

      by Anonymous Coward

      Yeah I know! GPS was totally created by the consumer market! It's a good thing too, otherwise it would have been limited to military and would be hugely expensive and unreliable! Amirite!

      And jet engines! Man, I'm glad those hit consumers first, and the internet is a prime example! Oh, and don't forget digital photography! Imagine how far back in the stoneage we'd be if all those things had belonged to the military industrial complex first! Their strict specifications, demand for reliability and track

      • Smart people realize that you need a healthy mix of both consumer-driven and government-funded R&D. Although the result sometimes intersect, they have different motivations and often different useful results.

      • by alen (225700)

        GPS might have been military first but what about directions? google maps and even the old mapquest were a lot better than the directions i would get with my old military GPS. and what about modern encryption? yes we had cool crypto tech in the military, but there are a lot of R&D dollars going into consumer encryption that will be better than what the military has.

        ARM CPU development is being driven by consumer spending and those are better than anything in mobile military devices. same with modern dis

    • by ceeam (39911)

      Low piracy -> Cheap prices.

      Cheap prices + Easy -> Low piracy.

      The amazing thing Apple did is putting their market into this sweet spot.

    • by citizenr (871508)

      i remember the days when all the new cool tech was only seen in the government and large corporations first and then trickled down to us peons. these days with our rampant consumerism it's the opposite. we see cool stuff like this first and it's cheap and the big boys are now playing catch up because things move so fast

      if it wasn't for our vane consumerism this would be a government project costing tens of millions of $$$ in R&D and the devices would be single use devices that also cost some ridiculous amount of money

      Funny you say that as I remember big government financed project for English/Farsi translator costing million of dollars.

  • by Anonymous Coward on Friday December 17, 2010 @02:06PM (#34589978)

    Too bad that this system only works with limited amount of texts. I installed this app on my iPod Touch and tried the default text reversing filter. If I used a Serif font, this could not read the words realiably. Fonts needed to be Sans Serif. Also this uses some dictionaries so if the word is not in dictionary (eg. deemed offensive) or some random gibberish, this could not recognize it. And all this I did with large black text on white background so viewing conditions are definitely not the issue.

    • by ceeam (39911)

      You wanted good reliable OCR on 960x720 images? Uhm.

      • by zn0k (1082797)

        The camera of an iPhone has a lot higher resolution than 960x720. The screen of an iPhone 4 is 960x720, but the OCR algorithm wouldn't run on what the app eventually outputs to the screen.

      • by MrEricSir (398214)

        If a human can read it, an OCR program should be able to read it as well. If not, that's a bug.

    • You're disappointed it couldn't translate "random gibberish"? I mean, really? And it would figure that something that is sold through Apple for an Apple product would censor 'teh swears' to protect all its poor stupid sheep from anything sharp with which they might hurt themselves. That's not a technical failing of the App, hell, it was probably deliberate to mesh with Apple policy.
  • Now they just need to do the same for economically relevant languages. The top developing countries currently are Brazil, India, and China (in no particular order) and none of them speak Spanish as a primary language.

    Of course, I tried to use a similar argument decades ago in school when everyone told me I needed to learn Spanish (while living in a state that was dramatically closer to French Canada than to Mexico, but oh well), and I still ended up taking three years of a language that I almost never en
    • by rsborg (111459)

      Now they just need to do the same for economically relevant languages. The top developing countries currently are Brazil, India, and China (in no particular order) and none of them speak Spanish as a primary language.

      Querying wolfram alpha [wolframalpha.com], the most spoken languages in the world are:

      1. Madarin - 1 Billion
      2. English - 760 Million
      3. Hindi - 490 Million
      4. Spanish - 417 Million
      5. Russian - 277 Million

      So languages #1,3,and 5 all have a completely different character set (esp. Mandarin), while #2 and #4 share the basic roman character set (with a few symbols outstanding). I can see why they went Spanish. Also, many popular travel destinations (i.e., Central and South America, Spain, etc) have spanish signs where this would be useful.

      I ex

      • by Yvan256 (722131)

        If we also count people who know two or more languages, I'm betting English would be far more than 760 million. A lot of surveys only ask what your primary language is. English is easy to pick up, Hollywood movies and the USA/UK/Canadian music industries also help spread english everywhere.

        • by idontgno (624372)

          If we also count people who know two or more languages, I'm betting English would be far more than 760 million.

          I suspect you're probably counting a few tens of millions of people who believe they speak or write English, but really, REALLY don't. [engrish.com]

          • Most of the "original" Spanish text on the demo video is Spanglish anyway. I bet if they were translating signs written in actual, correct Spanish, their automatic English translation would be even worse than it already is.

      • by mikael_j (106439)

        I'm afraid I'll have to call shenanigans on those stats, it looks like the same boring numbers I've been hearing for 15+ years and they don't make sense, there are easily 1+ billion people in the world who speak english. It may not be their native language but they can speak it anyway, there aren't a whole lot of people out there who speak mandarin as a second language (compared to english, it's the lingua franca of the post-WW2 world, at least so far).

      • Now they just need to do the same for economically relevant languages. The top developing countries currently are Brazil, India, and China (in no particular order) and none of them speak Spanish as a primary language.

        Querying wolfram alpha, the most spoken languages in the world are:

        Indeed a lot of people speak Spanish, I don't doubt that. However it is not as economically relevant as other languages, as it is not spoken much at all in the top emerging economies.

        I can see why they went Spanish.

        As can I. Spanish is pretty easy as far as languages for text translation go. And from Spanish it is a very short hop to Portuguese; and only a slightly longer hop to French or Italian.

        I am not looking to discredit their work or say it is of no value. I'm just saying I hope that this was a good learning exercise for the

  • I remember back in the early 90s when a guy showed me an 8Gb backup tape he had in his shirt pocket and I thought, Holy Crap, 8 GIGA-bytes fits in your pocket now? That's Awesome! And now, years later, you can carry many times that much data on a keychain. Equally Awesome. And this, this translator thing... totally and completely Awesome and Amazing. If you picture yourself as someone from say 100 years ago looking at today's world, some things we take for granted are pretty much like magic.

  • by psydeshow (154300) on Friday December 17, 2010 @02:23PM (#34590202) Homepage

    These guys just opened a gold mine.

    I'm sure there will be a ton of cynical and jaded comments here, but this is a working prototype of augmented reality that is actually immediately and unquestionably useful, even in its infant state. Even non-technical people can see the promise of this, and graspable promise equals investment.

    Bravo, and congratulations to the developers!

    • by node 3 (115640)

      I'm sure there will be a ton of cynical and jaded comments here

      You've cracked the Slashdot code.

    • by Jugalator (259273)

      I agree! *Precisely* because this is so impressive, will it also see many complaints about bad translations and so on. The better an application is, and the more hyped it is as a consequence, the more complaints will it see due to expectations so high that they are even impossible to meet given today's technology.

      I also agree that the translations offered will probably not be the best, but so aren't the Google Translations either, with their online huge database of statistics on how to translate words. And

  • by smooth wombat (796938) on Friday December 17, 2010 @02:27PM (#34590274) Homepage Journal
    Just read this on MSNBC. The author shows what happens when trying it on basic Spanish [msn.com].

    Overall, not worth the money until it gets heavily reworked.
  • ... when he says that one of the reasons the iPhone won't run Flash is because it doesn't have the processing power.

    You're telling me it can have the power needed to do something like this - analyze an image for text, decode it, put sentences together, translate, match the most appropriate font and colours, scrub the original text, render the new text at the appropriate angle and position - but not to play Flash movies. I call bullshit.

    • by Anrego (830717) * on Friday December 17, 2010 @02:43PM (#34590504)

      Much as I hate apple.. I hate flash more. And it wouldn't surprise me if what you described _is_ actually less resource intensive than flash.

      I have a higher-end quad core i7 and 12GB ram .. playing a flash video makes all 4 cores work and uses a rediculous amount of ram.

    • by dzfoo (772245) on Friday December 17, 2010 @02:43PM (#34590516)

      I call your bullshit. I don't ever recall Steve Jobs saying that the iPhone does not have the processing power to run Flash. He has said that Flash is buggy, crashes often, and does not have good performance in mobile devices. (All of which I agree with.)

      Basically, his position has been that Flash is too crappy for the iPhone, irrespective of whether the iPhone can run it or not.

                  -dZ.

    • by Yvan256 (722131)

      You really don't understand how crappy Flash is [macrumors.com] to begin with.

      • The fact that it is POSSIBLE to have a "ten fold improvement" is pretty much PROOF of how crappy it is/was.

        Apple's "jihad" against Flash has pushed Adobe to improve Flash significantly. Adobe always claimed that there were never any performance issues, no battery-hog-issues, yet they are capable of this ten fold improvement?

        I mean, in the demo, the Adobe guy makes fun of Apple's MacBook Air for being "underpowered". The latest Air is *NOT* underpowered. It *SHOULD* be more than capable of playing back 10

    • ... when he says that one of the reasons the iPhone won't run Flash is because it doesn't have the processing power.

      You're telling me it can have the power needed to do something like this - analyze an image for text, decode it, put sentences together, translate, match the most appropriate font and colours, scrub the original text, render the new text at the appropriate angle and position - but not to play Flash movies. I call bullshit.

      No, this demonstrates what can be done when people write code using

    • by Yvan256 (722131)

      First of all he didn't say that at all. Basically, what he said is that he doesn't want crappy software sucking the battery life.

      Second, if Adobe themselves can make up to tenfold improvements [macrumors.com], that says a lot about how crappy Flash really is.

  • by FedeTXF (456407) on Friday December 17, 2010 @02:42PM (#34590484)

    "LO TRADUCE EL TEXTO" is not a Spanish phrase, unless you want to say "Text translates it". The Spanish phrase would have been "TRADUCE TEXTO" but I think the result with that tool was so bad they changed the Spanish text until the bad translation rendered a good English phrase.
    The same happens with other examples from that video such as "ROPAS OPCIONAL EN ESTA PLAYA". The only way you are going to read that sign is if you ask an English speaker to write it.
    What they did was write the English phrases, translate them to Spanish and then translate them back for the video.

    • by ElMiguel (117685)

      As far as I can see it's translating each word separately without considering the context or re-ordering when appropriate. Of course this is going to result in terrible translations, so they have to cheat.

  • reminds me of a friend of mine who's hospital forced him to replace the typists that transcribed dictations with dragon speak software. They quickly discovered that it was pretty accurate except for small words like "not", as in "the tumor does (not) appear to be malignant"
  • The app is free but the Spanish-to-English functionality costs $5.

    It is totally helpless vs. handwriting.

    When viewing nice clean computer text on my screen, and when the phone is held very still, it produces the usual clunky translation.

    It would be very much better than nothing when attempting rapid translations in a foreign culture.

    • For $5, I think that's the point. As they say, use it to translate signs, menus, etc.

    • by lwsimon (724555)

      I bought it; mostly to support the developer. I hope they make a boatload, hire new devs, and keep doing cool stuff :)

  • Where are all the people who feel it's not really necessary apply correct spelling ?

  • by MrBandersnatch (544818) on Friday December 17, 2010 @07:41PM (#34594730)

    I've been researching this area for my masters. Just getting the basic text localisation (i.e. recognising an area as containing text) working reliably is very difficult - there are some good algorithms out there but in the real world, with 1000s of fonts, font sizes, angles, lighting conditions etc, I've yet to see a 1 size fits all approach. And even if you do find an area of text, throwing that into an OCR engine is going to produce garbage for the most part. In short, its quite easy to show something off in controlled conditions but I wouldn't expect anything like the performance seen in the video in the real world.

    The above said, very impressed to see that on an iphone and for it to be so responsive; these things can only get better and once some form of viable HMD makes it onto the scene these types of application are going to be massive.

Forty two.

Working...