simont | Wanting to be an ideal intelligence again

I've mentioned before that I often find myself noticing fundamental bugs in the way the human brain works and wishing my brain was better designed.

Here's another one: my brain is often very bad at predicting how it would behave in dangerous or scary situations. It's annoyingly common for me to evaluate several courses of action in advance of an event, decide which one I like best, and then when the time comes to actually commit myself then I discover that the one I'd chosen is terribly scary now that it's actually physically staring me in the face rather than being considered as an abstract strategic puzzle.

If I were designing an ideal intelligence, I would give it a properly working imagination. It would be able to set up a hypothetical situation, put itself into that situation, and then reason exactly as if it were real. It would either be able to temporarily completely suppress the knowledge that the situation wasn't real, or alternatively it would just be able to reliably inhibit that knowledge from impinging on its reasoning processes. In fact, now I've written that either/or, I'm not entirely sure I can robustly define the difference between those two possibilities; but either way, the fundamental architecture of my intelligence would be designed in such a way that if it decided it would react a certain way in a scary situation, you could depend on it being right.

Flat | Top-Level Comments Only

I don't think you're addressing the right problem here. All you'll end up with is an intelligence that can reliably predict when it'll do the wrong thing because the right thing is just too scary.

Wouldn't it be better to work on the root cause of the discrepancy: i.e. close the gap between what you would do and what you should do?

Fear exists for a reason. Given that not only are people not perfectly rational, they are also not perfectly intelligent, nor do they have perfect information, it seems reasonable for species survival for individuals to be able to later chicken out of bold plans made to impress mate and friends when tired or drunk.

Well, yes. But I think it's wrong to presume that the fear-influenced course of action is always the best one, which is what Simon seems to be suggesting.

Consider the case where Simon is arachnophobic and being chased by an escaped circus lion. There are two paths to safety. One is difficult and has a small risk of him being eaten, the second is quick, but has a spider sitting in it.

What he *should* do is ignore the irrational fear of spiders and take the path with least likelihood of being eaten.

What he *would* do is weigh up the two fears (or spiders, or the risk of being eaten), and depending how intense the phobia is, may choose the path that risks being eaten.

What he's asking for is for his ideal intelligence to correctly predict the outcome of the weighing-up in the second instance when considering the hypothetical situation. It strikes me that that's not necessarily the optimal solution.

An intelligence that correctly predicts you'll do something really dumb when afraid is not as ideal as an intelligence that chooses the optimal course of action *despite* the fear.

Obviously if the fear is a rational one (say one of the escape paths from the lion leads through a wasp nest, and you're allergic to wasp stings), the outcome for both types of intelligence would be the same.

I certainly didn't intend to imply that the fear-influenced course was always best! But neither am I convinced it's never the best.

In your example with the lion and the spider, I might find it useful to evaluate such a scenario purely in my imagination as a part of the process by which I was attempting to deal with arachnophobia; a willingness to endure a spider in the course of escaping from a lion might be milestone #1 on my therapy progress chart. #2 might be willingness to endure a spider in the course of escaping from something painful but non-lethal (enraged housecat?), #3 willingness to endure a spider instead of facing a verbal bollocking, etc :-)

I don't disagree that a truly ideal intelligence would not have any fears which weren't rational and proportionate to the real danger, and thus might not need this ability as badly as I do. However, I'm a cautious and conservative engineer type, so I plan to design my truly ideal intelligence with multiply redundant safety mechanisms.

I'm not so sure.

Wouldn't that lead to a fear of thinking, if thinking about a situation meant you would have experience the fear of the worst possible consequence?

It would be great for literature, although I'm not sure how much danger money they would have to pay librarians. Read any Lovecraft?

"Just imagine what it would be like to face a monster so hideously wrongly twisted that the very thought of its continued existance is so unbearable that you are driven permanently insane"
*fx: reader drops book and proceeds to drive pencil through eye*

This is why I am now looking at my lab book and saying "Why the hell did you do that instead of investigating this perfectly obvious cause?" - because there were people in the room who kept looking over my shoulder, and it was winter! Grahhh. Must make decisions outside work and do what I told myself when I get there, or possibly stop being afraid of my colleagues.

Come to think of it, I had a dream once where I was pregnant and my thoughts about whether I wanted to keep the foetus were completely different to the ones I had when I was a) awake and b) not pregnant (or thinking I was). It's made me rather more reluctant to make plans for what will happen in the rest of my life, in case I'm really in that situation and my feelings about what I want to do are different to the ones I had when I made the plan, and also more keen to have a lot of money saved up.

Mmm. Impending parenthood is certainly something that can surprisingly change one's attitude to all sorts of things. The phenomenon isn't limited to the person actually carrying the foetus and experiencing all the biology first-hand, either; the purely psychological effects on the father-to-be can also be noticeable.

Er, what's wrong with learning to compensate for your known mental inaccuracies..?

Working around a bug is often necessary, but doesn't mean it shouldn't also be fixed in the next release :-)

Are you thinking of a specific thing you did?

But of course, I think we're all familiar with this behaviour. I can think of two common causes:

(a) We're overly optimistic about how well we will act
(b) We act apparently illogically in a crisis when we allow an immediate want (eg. getting further from a mugger) overrule a longer term want (eg. escaping from a mugger) because we don't have time to think, and revert to something instinctive.

We can:

(a) Consider objectively what we're likely to do, taking into account how often we and other people are right about that
(b) Practice the behaviour we want until it becomes automatic.

LOL. I was actually thinking Vulcan. I remember wondering what the point of Vulcans was -- they obviously had emotions (love, curiousity, etc) or why would they bother to go to strange new planets and rescue their friends? The point being, they didn't let their emotions overrule them, they were rational about how to achieve their emotional aims. Which is what I hadn't realised I ever wasn't when I was young :)

Wanting to be an ideal intelligence again

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject

no subject