simont: A picture of me in 2016 (Default)
simont ([personal profile] simont) wrote2006-06-08 10:31 am

Wanting to be an ideal intelligence again

I've mentioned before that I often find myself noticing fundamental bugs in the way the human brain works and wishing my brain was better designed.

Here's another one: my brain is often very bad at predicting how it would behave in dangerous or scary situations. It's annoyingly common for me to evaluate several courses of action in advance of an event, decide which one I like best, and then when the time comes to actually commit myself then I discover that the one I'd chosen is terribly scary now that it's actually physically staring me in the face rather than being considered as an abstract strategic puzzle.

If I were designing an ideal intelligence, I would give it a properly working imagination. It would be able to set up a hypothetical situation, put itself into that situation, and then reason exactly as if it were real. It would either be able to temporarily completely suppress the knowledge that the situation wasn't real, or alternatively it would just be able to reliably inhibit that knowledge from impinging on its reasoning processes. In fact, now I've written that either/or, I'm not entirely sure I can robustly define the difference between those two possibilities; but either way, the fundamental architecture of my intelligence would be designed in such a way that if it decided it would react a certain way in a scary situation, you could depend on it being right.

[identity profile] feanelwa.livejournal.com 2006-06-08 09:53 am (UTC)(link)
This is why I am now looking at my lab book and saying "Why the hell did you do that instead of investigating this perfectly obvious cause?" - because there were people in the room who kept looking over my shoulder, and it was winter! Grahhh. Must make decisions outside work and do what I told myself when I get there, or possibly stop being afraid of my colleagues.

[identity profile] feanelwa.livejournal.com 2006-06-08 10:56 am (UTC)(link)
Come to think of it, I had a dream once where I was pregnant and my thoughts about whether I wanted to keep the foetus were completely different to the ones I had when I was a) awake and b) not pregnant (or thinking I was). It's made me rather more reluctant to make plans for what will happen in the rest of my life, in case I'm really in that situation and my feelings about what I want to do are different to the ones I had when I made the plan, and also more keen to have a lot of money saved up.