Computers and Visual Data

Regular readers will know I am skeptical self-driving cars are just around the corner, in part due to the limitations on computers being able to interpret visual data. As I said before:

The recognition of objects by computers is still only in its infancy, and nowhere near robust enough to deploy in any safety-critical system.  Given the pace of development of other areas of computing abilities, such as sound recognition in apps like Shazam, object recognition is seriously lagging behind and I suspect for very good reasons: software, being made up of pre-programmed algorithms, simply isn’t very good at it.

Computers are unable to do this for anything other than the most basic of pre-determined objects and scenarios, while the environment in which humans operate their cars is fiendishly complex.

Via Tim Worstall, I found this story:

Amazon’s facial recognition technology falsely identified 28 members of Congress as people who have been arrested for crimes, according to the American Civil Liberties Union (ACLU).

There are clearly instances where facial recognition technology works, but it’s still not very good and I suspect will remain that way for any conditions which are not perfect (i.e. good light, looking face-on, and of an expected skin colour) for some time yet. I’d be interested to know if the Chinese system which was deployed to fine jaywalkers works as claimed; it’s hard to believe anything which comes out of that country.

Liked it? Take a second to support Tim Newman on Patreon!
Share

28 thoughts on “Computers and Visual Data

  1. The fact that it doesn’t work hasn’t stopped the Plod from rolling it out across the country in legally dubious ways, see Big Brother Watch.

  2. I think the reason it doesn’t work is thatcwe don’t see, we perceive. The brain filters so much information and processes it with data from other senses with protoception which AI dosen’t.

    I am not sure that we recognise people per say, but see people in a moving context.

  3. I think the reason it doesn’t work is thatcwe don’t see, we perceive.

    Yup.

  4. It seems that AI is just the next tool to assert peoples’ dreams for central planning. I don’t intend to be a Luddite when saying this, but the passion that goes into promoting AI wreaks of the desire to control,people.

  5. Argh. More neo-Luddism.

    Two-dimensional image recognition is difficult for computers, yes. The problem isn’t there’s something special about people, it’s that they take literally years training the neural network between their ears to learn how to interpret visual signals, and tolerate a lot of false positives (cf. “the face on Mars”). We’re trying to prefab computers with all that knowledge.

    Wanking on about self-driving cars being unfeasible because AI doesn’t handle image processing very well ignores the fact that self-driving cars use a lot more than the human-visible spectrum to detect the world around them: GPS, LIDAR and short range RF beacons in cities.

    Amazon’s facial recognition technology falsely identified 28 members of Congress as people who have been arrested for crimes

    Get a dozen random people off the street and see if they do any better. Get any dozen cops and see if they do any better. This notion that machine intelligence has to be perfect or it’s bogus reeks of more neo-Luddism. It just has to be good enough.

  6. I agree that self driving cars are a lot further away than thought. However, I don’t think their absence will be due to inadequacies in technology.
    People take a lot of pride in their vehicles. They are a reflection of one’s persona. I remember doing a job somewhere, and for the whole two hours I was there, this man was cleaning his metallic blue VW Transporter across the road. It looked superb.
    A vehicle is a real symbol of liberty. I can get in it whenever I want, go wherever I want and drive in a manner according to my feelings at the time.
    We hear about driverless cars, and our first reaction is, Yeah, I can see the point of that, safer etc, but we don’t immediately think about what we’re losing. Currently, you are the controller of that vehicle and how it performs is mainly down to you. You decide your route, the speed you go at, the distance you are away from the next vehicle, when to change lanes, etc etc. These are all examples of us determining our own destiny to a certain extent, and they are pleasures.
    Now, imagine getting in a driverless car, stuck three feet behind another one going at 60 mph, having absolutely no control over what happens. Yes, a bit like being in a bus or train except scarier. So, what’s the point of having that beautiful VW Transporter now?
    When people realise what driverless cars mean in reality, they will rebel big time. In fact they’ll probably vent the anger they should have released over things like grooming gangs, remoaners and why Lineker gets paid so much taxpayer money.

  7. “It just has to be good enough.”

    I think you will find it will need to be much better than that. People will forgive the flaws in people, they will not be anywhere near as forgiving in the flaws of AI.

  8. I’m quite certain that driverless tech will work (though I have no idea when) but the real issue is social. As in –

    1. Will it be considered reasonable to pop your seven year old daughter in one and have it take her off to school?
    2. Could you get one to take you to the ‘Queens Head’, where you’ll have an absolute skinful, and then get it to take you home?

    I fancy that society’s answer to both would be a resounding NO – if you have to be responsible for the driverless car you may as well drive the bugger yourself!

    Engineering is satisfyingly clean and precise right up to the point the end users climb aboard 😉

    -Phil

  9. It works quite well in reverse, though. An automated immigration system that flags people who don’t resemble their (on file) passport details would make life easier for those of us who aren’t up to no good.

  10. I’ve always thought that rather than attempting to use a stereoscopic visual image to detect obstacles, cars would be far better using a form of shortwave radar like bats do.

  11. I share the scepticism of self-driving cars, although I see the principal difficulties as non-technical. Nothing is perfect, there will always be some crashes, even if the car/driver is blameless. Child runs out into road, for example. So liability for crashes will be a legal morass: not the passengers obviously (precedent: a bus). It will have to be the car saleman or retailer or manufacturer. Or maybe just the owner. Expect huge business in empty legal shells as defences. Then draconian attempt to bypass such shell constructs. Ultimately resulting in arbitrary confiscation from alleged associates.
    Never happen? Already does.

    On the computer vision thing, it’s a hard problem, but no reason to assume it’s impossible, given effort and incentive. Humans aren’t perfect at it, and some cannot do face recognition at all – I am one.
    The trick is to see it as an aid, not a definitive “computer says you’re the perp!”, for example by screening thousands/millions of possibles into a short list for investigation. Even witness statements of “It was him what done it” are unreliable and need supporting evidence; an effective computer system can help target getting it.
    Actually the whole AI thing is existential, dare I say, a religious topic. Are we or are we not, just walking biological computers, no different from what is theoretically achievable by a mechanical equivalent? Discuss.
    Which brings me back to self-driving car technology. It doesn’t have to be perfect. Have you ever ridden in a mini-cab? Beating the average mini-cab driver’s safety ability would be only a 2-pipe problem.
    But still got all the legal stuff to solve.

  12. @Daniel Ream on July 27, 2018 at 8:14 pm said:

    Argh. More neo-Luddism.

    Two-dimensional image recognition is difficult for computers, yes. The problem isn’t there’s something special about people, it’s that they take literally years training the neural network between their ears to learn how to interpret visual signals, and tolerate a lot of false positives (cf. “the face on Mars”). We’re trying to prefab computers with all that knowledge.

    More computer fuckwittery. I’m 66, riding motorcycles since I was 16 and still in one piece. I can read the road ahead, watch other drivers and anticipate their actions. From this luddite, your neural network requires people to be re-progammed to interpret visual signals. Really?

  13. Tim the Coder: “Actually the whole AI thing is existential, dare I say, a religious topic. Are we or are we not, just walking biological computers, no different from what is theoretically achievable by a mechanical equivalent? Discuss.”
    Is there some ‘spirit’, some essence of being human beyond reach of understanding? The true-believing religious may consider it so and that there particular faith is it. The more agnostic might view the religious impulse as just a best effort to grasp what is and shall be forever beyond reach. An illusion with possible side benefits (or not) in regulating how we live here and now. Natural Philosophy, the scientific method, is not so different in this regard, save it puts the horse before the cart and starts with trying to understand the everyday here-and-now.
    As for AI; the hard things are easy and the easy things are hard. The early AI researchers looked at things like proving theorems and playing chess as examples of intelligence. But they are considered intelligent because for most people generally they are unnatural and therefore difficult. Walking down the street and not tripping over kerbs on the other hand is for most so easy as to not be worth conscious thought.
    Visual processing: the eye has a lens, an iris, a sensor, but it is not a camera; what the eye tells the brain is not the equivalent of a photograph. It is a scan of edges and trajectories, integrated with context from every other sense, matched to expectations and best estimates. [The ‘prolepsis’ of Epicurus grasps this as well as any since.]
    So, self-driving cars? Possible, but in practice you have just reinvented the personal tram. It won’t go off-road or indeed anywhere if the computer says no.

  14. I can read the road ahead, watch other drivers and anticipate their actions.

    This is exactly it: it’s about *interpretation* of visual information and *anticipation*, not raw data collection and measurement.

  15. I’ve always thought that rather than attempting to use a stereoscopic visual image to detect obstacles, cars would be far better using a form of shortwave radar like bats do.

    See comment above. It’s not a question of a car being able to detect an object, but its ability to understand what that object is in the context of the environment, and anticipate what actions the object *might* take and what its response will be.

  16. Yeah, I only worked on missile systems for EADS, so what would I know…

    Probably quite a lot about missile systems.

  17. It’s strange that you should cite computer failings in pattern recognition as a reason against self driving cars. Pattern recognition is a large part of how humans perceive the world. For instance, we’re lousy at directly perceiving distance. Because the base-line of binocular,distance between eyes, is small. Doesn’t work out beyond a few feet. So we actually estimate distance by working on the assumed size of objects relative to perceived size. Not helped by the high def field of view in human sight being not much better than a coin held at arms length. And it’s from that scant information we derive things like velocity & vector.
    It’s getting this wrong that’s the cause of most car accidents.
    “It’s not a question of a car being able to detect an object, but its ability to understand what that object is in the context of the environment, and anticipate what actions the object *might* take and what its response will be.”
    Yes. But there’s a presumption there that we get things right. Most of the time, we don’t. It’s just, in most cases, there’s no consequences to getting it wrong. We congratulate ourselves on “seeing the kid might run across the road” ignoring all the kids who might have run across the road we missed seeing & the one did run across the road, wasn’t spotted & got run over.
    I

  18. “I’ve always thought that rather than attempting to use a stereoscopic visual image to detect obstacles, cars would be far better using a form of shortwave radar like bats do.”

    Well, I was under the impression that bats used ultra sound for echo location, but I suppose you could call it shortwave radar*.

    Anyway; I’d be a bit careful about making the jump from problems with facial recognition to “self-driving cars aren’t going to work”, and God knows that I’m not generally convinced about the whole thing.

    Main reason being, cars don’t have to recognise people (unless flashing the headlights at your mates and neighbours is a hitherto unknown design goal) as people, but only as objects to be avoided. At the moment, they do have to recognise and interpret road signage and act on them. And road signs have standard designs, even if the UK highways agencies are generally shit at keeping them actually visible and readable.

    *Given the number of people who start leaping up and down about emissions from mobile phone masts and what-have-you, the resulting fuss about self-driving cars equipped with radar should be hilarious.

  19. Anyway; I’d be a bit careful about making the jump from problems with facial recognition to “self-driving cars aren’t going to work”,

    There are several major obstacles in the way of self-driving cars becoming a reality, and one I believe is they are dependent on computers interpreting visual data – something they are not good at, and I suspect never will be in the same way computers won’t be writing novels or composing songs.

    But there’s something else fueling my skepticism: the technologies we see around us (e.g. smartphones, the internet, the internal combustion engine, the jet engine, etc.) appeared on the market and quickly established themselves. Conversely, there are other technologies (e.g. cold fusion, personal jet-packs, wind and wave energy) which get talked about as if they’re just around the corner but never seem to arrive. And we have been talking about self-driving cars being a year or two away for several years now.

  20. Tim Newman,

    “But there’s something else fueling my skepticism: the technologies we see around us (e.g. smartphones, the internet, the internal combustion engine, the jet engine, etc.) appeared on the market and quickly established themselves. Conversely, there are other technologies (e.g. cold fusion, personal jet-packs, wind and wave energy) which get talked about as if they’re just around the corner but never seem to arrive. And we have been talking about self-driving cars being a year or two away for several years now.”

    The thing with most of these inventions is that the love of them is driven by an ignorant media that simply wants them, sees them as highly beneficial, but has no clue about whether they are achievable. This is then fuelled by hucksters and charlatans promising they’ll be along in a few years.

    If you invent or engineer things, you have a different perspective. You know the raw materials, whether it’s software, electronics, polymers, motors, hydraulics or engines, and you find ways to use them to make a better thing. You want to cut the amount people drive? Build some software in the remote-working area. Something that means people need to come into the office less. That’s an even better solution and more realistic.

  21. I was thinking about self-driving cars while driving to work this week. I think it’s far more than a visual processing issue; a self-driving car would also need to be able to recognise, interpret and act on the intentions of other road users (including other drivers and pedestrians). There are all sorts of edge cases that we all deal with every day, which require us to make judgement calls based on these interpretations.

    That guy waiting to pull out of the junction – has he seen me? Should I slow down a little just in case he pulls out and collides?

    That teenage girl with her nose in her mobile phone – will she step onto the road without looking?

    The car in front that seems to be all over the road – do I keep a safe distance or do I take the first opportunity to overtake?

    That car in the left lane – are they really turning left? Or are they just queue jumping and will they cut me up at the lights?

    Those blue lights behind me – how do I safely pull into the side to allow the emergency services to pass me?

    I suggest that a self-driving car would need to have a “theory of mind” about the intentions of other road users.

  22. @CC

    “I suggest that a self-driving car would need to have a “theory of mind” about the intentions of other road users.”

    This is an interesting point and I agree absolutely – I’m often looking for cues about a vehicle’s intent (e.g. if you know the local routes well, you often know at which junction a lorry is going to turn off) but also the kind of behaviour I can expect from it.

    I gauge quite a lot from the make/model of a car, its age and what state it is in, the appearance of the driver …. I try to use them to gauge risks of erratic or selfish driving. Youngster in a cheap beat-up hot hatch? Best just get out of the way. Same for stressed-looking middle-aged executives in their Audi, all white vans, or anyone else who looks too eager on the gas pedal.

    On the other hand, someone driving at a reasonable speed but who you notice glancing nervously at road signs, who makes late turns or fails to indicate or seems to brake for no clear reason, you’re best to overtake as soon as it’s safe to do so because you don’t want to be behind them. Sometimes I’ll see a car that’s a few decades old but still looks absolutely pristine – one that lots of pride and work’s gone into. They’re usually being driven calmly and at a reasonable speed, so not being a speed freak I’ll often just tuck in behind them and follow them. Family cars with kids in them are also often good to follow, provided it doesn’t look like the kids are causing chaos to break out inside – the driver’s likely following a route they know and is unlikely to take a lot of risks with their kids inside.

    I can imagine an AI also learning these patterns for itself, but perhaps it won’t need one – if its processing speed and reaction times are so much better than mine, maybe it doesn’t need to prioritise “risky” cars in the same way I do.

    One thing I can see being a bugger for self-driving cars in busy city centres – once pedestrians, especially kids, have figured out they can just walk out in the road and the traffic is guaranteed to stop (particularly if they’re in a 20 mph zone), that is exactly what they are going to do. On a busy street there can be hundreds of people in visible range. Any one of whom has a fair chance of sauntering across the road. I imagine an AI can keep track of them all and their behaviour rather better than I can do as a human, but once they know it is safe to cross at will, road discipline is going to fall apart.

  23. “once pedestrians, especially kids, have figured out they can just walk out in the road and the traffic is guaranteed to stop (particularly if they’re in a 20 mph zone), that is exactly what they are going to do.”

    Thats a very good point – we’re assuming that computers will replace what drivers do now, and people outside the car will still react to it as they do now. Whereas the fact a computer is now the driver will affect how the humans behave around it.

  24. I think that we are already reaching the automation culmination point.

    Computer stuff became so complicated that if you are not full time programmer, you don,t just have the time to familiarize yourself with every last hyper complicated IT solutions.

    I bought recently new Alfa Romeo. It,s climate automatic has 28 pages manual. My Serbian friend flies Boeing 777 and he claims that this plane has 6 page passenger area climate regulation manual.

    I wanted to print out my contacts from my phone. My IT friend spent 23 minutes for that. I would write them on the paper faster.

    I think that those self drivers will be also so sophisticated that when there is distance less than 100 miles, then programming this car takes more time than driving yourself there and back one or more times.

    Data mining. I do not use password for any of my electronic devices because entire Big Tech has access, probably the law enforcement too because I never heard that any of the ISP providers was ever harassed by IRS or some of the data protection guy was accused for child porn or any other dirty trick LE does to get backdoor . So, when entire world have access to my data, then one single phone thief doesn,t make things worse anymore.

    When you are really too lazy to drive yourself then thanks to free market capitalism, there are plenty of unemployed young people ready to drive you around for less than minimum wage. And this guy will wash the car, does all other jobs and will fuck your wife also if needed.

    Tell me, what kind of super robot can do all those things and in the very same time costs less than hiring somebody from the unemployed army ?

  25. Yes He can. But I am more afraid that liberals planning something like 1917 and it is only matter of the time when Soros transforms Antifa full Cheka.

    Then every anti communist person has bigger problems than 300 USD phone bills.

  26. @CC
    “I suggest that a self-driving car would need to have a “theory of mind” about the intentions of other road users.”

    Yes. Look at any of those dash-cam videos on Youtube, especially the Russian ones, the lack of situational awareness is staggering. The is no evidence of anticipation or defensive driving. A self-driving car is that but more so, relying entirely on fast braking and not skidding out of control.

Comments are closed.