Here Be Dragons: Science, Technology and the Future of Humanity
The title, "Here Be Dragons", refers to the uncharted territories of knowledge and technological abilities, where we might discover unknown threats. The mapmakers of old just hinted at possible dangers, but the message of this book is that we should do our best to prepare for situations where we might hold tools or knowledge that could destroy us.
You may have heard how famous scientists lately have written open letters and op-ed pieces with warnings about artificial intelligence or advice against sending signals to possible extraterrestrial intelligences. Are they serious? What are the risks, really, and how have these people come to be worried about such things?
If you want to dive into the current discourse about these possible threats to human existence, this could be the book to seek out. The mathematician Olle Häggström gives an introduction to the reasoning about some of these issues, and responds with his own arguments.
As the starting example Olle Häggström discusses the possible geoengineering techniques that might be employed to counter the global warming, and what risks would be associated with them. He then goes on to various schemes for improving humans, for example by extending our life spans, and what risks could be associated with that. Hostile artificial intelligences and rampant nanomachines also get their treatment, before Häggström goes on to deeper questions of the nature of science and probabilities. Two fairly abstract chapters with statistical arguments about expectations of the end of our civilisation are followed by a discussion about the search for space aliens. At the end, the author discusses some of the philosophy of how to weigh various future risk scenarios, balancing facts and moral values.
Here Be Dragons could perhaps be described as suitable for "advanced beginners". It seems to be intended mostly for fellow academics, but anyone who would like a more than superficial overview of what the various arguments look like could start with a chapter or two of this book.