These notes won’t be much use without listening to the episode, but if you do listen, one or two of you might find one or two of them helpful.

Lyle Cantor‘s comment (excerpt) in the Concerning AI Facebook group:

Regarding the OpenAI strategy of making sure we have more than one superhuman AI, this is not as good an idea as it might seem. For whatever reason, humans find analogies to nature highly appealing, and have positive associations to terms like “ecosystem” and “bio-memetic.” Open AI seems to think something like this: the solution to Friendly AI is to have an ecosystem of competing AIs, so no AI gets too much power.

…

Eric Saumur‘s response (excerpt):

Natural Selection works the same way. Replication is all that matters. But you still end up with peacocks that spend huge amounts of energy on tails and fast sword fish and slow snails and 30 million other variations. Why? Because success at replication in an ecosystem isn’t all about reproduction rate. It is also about long term survival and not becoming too easy a food source for something else. Long term survival is about diversity and that was why sex was invented. Sexually replicating species are more diverse than asexually reproducing species and so they don’t end up being a big buffet for something else. Some of them always have the right defense against against the predator or disease.

…

For reference: 0007: Cosmists, Terrans, and Cyborgists, 0018: The One True Doomsday, 0005: Paths to Superintelligence, 0004: Prioritization of Risks

If 99.999% of humans alive today were killed by a disease, how many would survive? More than 7 million.