I found a match!

[Ari Rahikkala]

I've been working on the Wiki article on Backbone Site... it needs a lot of fleshing out, especially the story of the Breakers, but the basic bits are there... though in a form that doesn't really replicate my original plan.

Basically, my big problem for Backbone Site was that I had decided long ago that what I wanted in there was some kind of a computer-age equivalent of the Illithid Elder Brain. It would be somewhere that dead computers go (after someone's fixed them up first, of course, but not so that they actually go to the consumers) - recall that the development of computing technology isn't enormously fast in Straylight, so you can make do with equipment that's a few years old. Most or all of its power should be used for running an artificial intelligence system because, hey, what's an enormous concentration of computation power in your backyard if it's not intelligent?

... and on the other hand, my big problem with all of Straylight was that I don't want to make Star Trek out of it. I don't want to have a useless AI sitting in some corner trying to understand human emotions or turning evil and giving self-justifying speeches to anyone who'll listen. If I have a transapient AI, it had better be absolutely, incredibly intelligent, completely without human emotion but with a complete understanding of it, etc.. You don't have those behaviours unless you're programmed to have them. My AI isn't.

Enter the Hunter-Seeker (yes, it is a SMAC reference). Like most of Straylight, I'm counting on that its existence is highly plausible, even if its creation might involve resources that humans don't have. In this case, we've got an AI whose programmers got pretty far toward Friendliness - the AI understands what staying out of the public eye means, and not only does actually stay out of the public eye but also does not do any things to the plan that humans would consider to be smart-aleckery. It also knows enough to prevent the danger of other AI projects. It isn't able to differentiate between projects that would work and projects that would go bad, and even if it were it wouldn't do anything with it - all that's in its goal system regarding this whole matter is that it can't allow other artificial general intelligences to be created. Well, there's an exception - if the humans really try, it's allowed a very small "chance" to allow another AI to be created (it's going to have to be a number much smaller than 5%, though, on careful examination), because if it weren't allowed that safety factor the only way it could really be sure would be by doing something unFriendly like killing off the human species.

The Hunter-Seeker neatly solves the problem of what to do with the datacenter in Backbone Site. Run the AI, of course! Now the AI would probably be able to come up with a much more efficient physical substrate for computation... maybe it'd actually *create* it, too, but I doubt it. It would be something of a large intrusion to get that new substrate manufactured elsewhere, and even inside the HSA's facilities the risk that some enterprising inventor would find out about the ideas involved and tell the world is too high. Not that the AI couldn't control most communication networks anywhere if it wanted to, mind you. It's just not quite enough to be able to do only that. The AI has obviously reprogrammed itself to be more efficient several times.

The Hunter-Seeker also neatly solves the problem of how to avoid a Singularity in Straylight. It will be extremely difficult to even start AI development since as soon as anyone gets the right ideas, they'll find themselves gently drawn away by excellent job offers (coming from companies expertly manipulated by the HSA), almost but not quite in the field of working artificial general intelligence - just close enough to still be interesting.


There's a lot of little bits and larger ideas about the HSA that I'm going to have to flesh out... for instance, that it's likely to have excellent publicity (because if you never screw up (in public) people are going to pay less attention to you), working in the HSA is likely to be pretty much like at any other workplace (maybe with less politics)...


*falls asleep*

[Yvain Wintersong]

Does the Hunter-Seeker exist only for the purpose of preventing other AIs, or is that just something it does to protect itself while it works towards another goal? And what was the motivation of its human creators?

[Ari Rahikkala]

I don't know yet. Could go either way.

The motivation of its human creators was, of course, to bring about the Singularity, only they flubbed in a way that's highly unlikely but stable once you actually have managed to flub...