View Topic

http://www.iflscience.com/technology/should-self-driving-car-be-programmed-kill-its-passengers-greater-good-scenario

This combines a very old moral dilemma with some fancy new AI tech, and thus makes for some insane debates. What are your thoughts on the matter?

I must say I'm conflicted. My outlook is inherently altruistic when I am just a bystander, believing that the life of one person in the car is completely outweighed by the lives of ten people saved by the car swerving, but my outlook completely changes when I consider myself as the driver, thus proving my own natural instinct for self-preservation completely overrides any moral considerations I would have in that situation.

The classic moral dilemma is the one where you have to consider yourself standing next to a train track, and right next to you is a lever to change the lane of an oncoming train to either left or right. If the train continues on its intended course to the left lane, it will kill seven people who stand no chance of getting out of the way. If you switch the lane, then only one person on the right lane will die, meaning you've saved seven lives at the cost of one.

Now to make it more personal, consider that you are not the only bystander next to that track, and there are no alternate lanes to which you could switch the train to. The train will still kill seven people who stand no chance of getting out of the way if no one intervenes, but this time, your intervention can only take the form of pushing a very large man in front of you onto the track. This hypothetical person somehow weighs enough to stop the train before colliding with the seven individuals, and you are somehow strong enough to push him. But here's the catch: this hypothetical man is staring straight at you when you have to decide whether you'll push him or not.

Yet another scenario is one in which the only two choices you have are to throw yourself in front of the train, leading to certain death but saving the seven people further along the track, or to let the train continue and kill the aforementioned seven people. What would you do?

The technology presents a new layer in that the designers can decide to intervene well before such an accident happens. But it is then a conscious choice that they make without the same urgency of being in a scenario such as the one with the trains.

To be honest, I'm not sure there is any morally justifiable answer to this question.

In my opinion, it cant be the fault of the owner of the car because he wasnt driving. It cant be the fault of the car, because its a car, you cant prosecute them. So, it must be the fault of the pedestrian that got hit. Its the only logical explanation :]

You're forgetting that the car's designer can exert control over such a dilemma, because they can decide to pre-program the car to swerve into a wall, thus sacrificing the driver to save the pedestrians.

I'm sure any designer would be conflicted with the choice he faces, because since the car is not controlled by the driver (who really becomes more of a passenger with this tech) it is the designer's code and technology which drives this car, also indirectly implicating him in any situation like this. Either way, his choice of programming is going to cost people their lives, and it can well be argued that the responsible thing to do would be to minimise mortality.

Also, why did you post this topic twice?

I posted it twice? I'll go correct that ASAP.

EDIT: I've locked the other one. Now a mod just has to delete it.

I posted it twice? I'll go correct that ASAP.

EDIT: I've locked the other one. Now a mod just has to delete it.

I have deleted the other topic.
I think.

I posted it twice? I'll go correct that ASAP.

EDIT: I've locked the other one. Now a mod just has to delete it.

I have deleted the other topic.
I think.

You did. Well done. :]

As a designer I would thoroughly log everything then take every action necessary to avoid collisions with pedestrians, prioritizing the continued safety of my passengers, with the sure knowledge that if I, from the car's perspective, cannot control what happens after a high speed impact, only what happens before.

I would not, for legal, moral, practical and economic reasons, decide to kill my passengers to save pedestrians.

Legality: The moment I make a decision to crash the car instead of killing the pedestrians, I am making the decision to end human lives and become legally responsible for their death.

Morality: See legality, I also become morally responsible for their death. This is the weakest of the arguments.

Practicality: There's no way for a car designer to really differentiate, with certainty, that what it's going to run into is a single human or group of humans, and not something similar. Infrared sensors could detect body heat I guess, but two humans walking probably looks pretty similar to an elk.

Economics: In order to promote acceptance of the car I'm designing, the purchaser of the car needs to know that their safety and not anyone else's would be the priority. If it's a choice between purchasing a car that will kill me and my family to save strangers and one that doesn't, I know what I'd choose. This one probably trumps all others, because whichever car becomes successful will probably have this property, unless the designers of the cars are legally required to choose one way or the other.

There is some precedent. The airborne collision avoidance system (ACAS) in aircraft ALWAYS decides to avoid collision, and to my knowledge has always been successful, with the system being capable of redirecting up to 255 aircraft that are at the same positions in Latitude, Longitude, and Time to different altitudes to avoid any collision (Hasn't always worked for reasons interesting to the discussion: Human pilots don't always listen to their computers).  A car has different constraints, not being able to just fly up or down to avoid collisions, but has the same moral constraints. ACAS will NEVER decide to cause a collision for any reason, not even to save another aircraft with potentially more passengers. I think this is correct, the car should simply do whatever it can to avoid collision. If it doesn't work out, hopefully the logging system is in overdrive during near collisions in order to prove that the car did all it could to avoid the collision.

 https://en.wikipedia.org/wiki/Airborne_collision_avoidance_system

Also keep in mind that a computer has a lot more potential to avoid collisions than a human does, with a lot more ability to actually perform the required maneuvers in a lot less time.

And yet 75% of people who responded to the study indicated their preference for the car to swerve, killing the passenger but saving the pedestrians.

The way I see it: and this is all just my opinion ion by the way, I couldn't properly back any of it up so take it as you will.

Anyway, while I would personally want to try everything in my power to avoid the other potential victims, I wouldn't even mind having an automated system online to allow this. However I firmly believe that even if every car was automated, that this ability should be voluntary to the occupant of the vehicle because while I would prefer this, as Laurentus just stated ~25% of respondents to a study wouldn't.

For me, personally, the idea of a machine potentially killing it's possibly unwilling user in order to protect the also potentially disagreeing others seems wrong to me. I would not hold anything against any driver who disabled such an ability in their vehicle, and I severely dislike the idea of it potentially being mandatory.

To relate back to the metaphors earlier, in my opinion, I would stop the train personally, but I would never push someone else in front of it, that person themselves should decide whether or not they take that action in that strange scenario, I am not willing to kill someone for the "greater good" I believe that's a bad way of thinking, but as I say it is personally mine.

What do you think?

I honestly don't know what I'd do. It's easy to say that I would sacrifice myself, or throw someone else in front of the train, but I really don't know.

This actually relates well to traditional models of moral philosophy, and that of utilitarianism.

Here is a well-known case in the Common Law world, which set a precedent that the ends don't justify the means ("the greater good," so to speak).

https://en.m.wikipedia.org/wiki/R_v_Dudley_and_Stephens

These were obviously men of character, and yet that didn't stop them from doing something this horrible for the sake of surviving and continuing to care for their families. And I honestly can't say that I might not do the same in their shoes. I don't know, and I hope I never find out. And of course, if I were the one on the brink of death, and heard them discussing something like that, I'd most definitely fight for my continued survival.

And yet 75% of people who responded to the study indicated their preference for the car to swerve, killing the passenger but saving the pedestrians.

The 75% (for swerving to save one human vs kill 10 pedestrians) is somewhat misleading, as about 25% of respondents to this said the algorithm should randomly choose whether or not to swerve, making about ~30% of respondents who said the car should just plow over the pedestrians and ~25% who would accept either outcome. (Yeah, those don't even total to 75%, it's an approximation - the 75% that is, the true stat is about ~65% if you look at their actual results)

Additionally, every situation given to the polled involved the tradeoff of 1 passenger for 1 or 10 pedestrians. This isn't my use case. If I'm driving I usually have myself, my ex-wife, my sister in law, and my 2 sons in the car. Fuck no, I'm not going to sacrifice them for strangers. I will swerve to avoid if I can, which might result in my death, I wouldn't intentionally kill the pedestrians, but I'd prioritize my own life and the life of my passengers.

In any case, as I alluded to before, the entire situation is hypothetical. It is entirely reasonable to believe that the car would be able to save both the passenger(s) and the pedestrian(s) in practically all circumstances.

This is an engineering problem, not a moral one. The question I would ask is "What can the car do to either prevent this situation or fix it if it occurs?". As an engineer, I would not decide to just plow over the pedestrians OR slam the car into a wall. It's a false dichotomy.

For example, the pedestrians didn't magically appear in front of a moving car. They walked there. Have the car slow down if it thinks pedestrians are going to get on the road. Self-driving cars have this already - they take into account the projected intentions of pedestrians that might enter the roadway and slow down, so they can stop to avoid a collision if necessary.

The AV will also be able to brake much faster - to the tune of at least 200 ms, than any human would. They might also be provided with braking systems that would be unsafe for a human to control - maybe front-side airbags of a sort, explosive braking.

The cars behind the AV also need to be taken into account - car debris is a much bigger road hazard than splattered pedestrians.

I'd also be worried about abuse of the system - what if you warmed up mannikins and put them in front of the car, killing its passenger by causing it to swerve? Hacking of this sort is probably going to happen.

Anyway, I don't think the question is as simple as the survey presented.

Evelynx, you're twisting the parameters of the situation, though. To remove all other considerations, say that you are on a narrow road deep in some city. Ten pedestrians could well come around a corner, a short enough distance away from you that your only choice is to swerve the car or run them over, and thus the original question stands, if there are no other cars behind the driver to consider. And don't add your family into the equation, because that is a violation of the original parameters that were set and put to you in this hypothetical situation. A hypothetical that stands every chance of becoming a real situation. Remember, bringing that in asks a completely different question to the original, namely: would I sacrifice a family to save pedestrians as the designer. That is fundamentally different from the original scenario.

Evelynx, you're twisting the parameters of the situation, though. To remove all other considerations, say that you are on a narrow road deep in some city. Ten pedestrians could well come around a corner, a short enough distance away from you that your only choice is to swerve the car or run them over, and thus the original question stands, if there are no other cars behind the driver to consider. And don't add your family into the equation, because that is a violation of the original parameters that were set and put to you in this hypothetical situation. A hypothetical that stands every chance of becoming a real situation. Remember, bringing that in asks a completely different question to the original, namely: would I sacrifice a family to save pedestrians as the designer. That is fundamentally different from the original scenario.

Purely hypothetical. If the car cannot reasonably avoid pedestrians given the state of the road, it can slow down until it can. This is why we have speed limits under the current human powered system.
My point is that the situation in question doesn't exist in real life (How many fat men have you seen leaning over balconies get pushed off to save the lives of pedestrians about to get hit by a trolley?). And I'm aware that it asks a different question, the survey is intended to determine if the utilitarian AV would be generally acceptable, my argument would be that it fails to determine this by omitting valid and frankly extremely common situations.

Engineers have to work in real life.

The one with the trolley was nonsensical, designed to simply test the moral argument. It is usually used to test the philosophy in isolation before using a more complex real world example, that actually happened.

This one, while being a hypothetical now, stands every chance of becoming a reality. With how many cars are out there, and how many the companies would eventually sell, things like these are certain to happen.

Fair point about the designers and lawmakers having to try everything in their power to prevent this from happening (through various types of braking technology, speed limits and a multitude of other things I'm not considering because of my lack of knowledge with engineering and lawmaking), this is still bound to happen, even in fringe cases. It's an inevitability.

Example: buildings in many parts of the world are designed to withstand an earthquake, even though it may never even pay off in many of the designers' lifetimes. Contingency planning is absolutely something designers do.

The fact that we're even having this discussion based on a study done by actual engineers and scientists, means someone somewhere feels pretty certain that they need to consider something like this.