Information Hazards - LessWrong
An Information Hazard is some true information that could harm people, or other sentient beings, if known. It is tricky to determine policies on information hazards. Some information might genuinely be dangerous, but excessive controls on information has its own perils.
This tag is for discussing the phenomenon of Information Hazards and what to do with them. Not for actual Information Hazards themselves.
An example might be a formula for easily creating cold fusion in your garage, which would be very dangerous. Alternatively, it might be an idea that causes great mental harm to people.
Bostrom's Typology of Information Hazards
Nick Bostrom coined the term information hazard in a 2011 paper [1] for Review of Contemporary Philosophy. He defines it as follows:
Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.
Bostrom points out that this is in contrast to the generally accepted principle of information freedom and that, while rare, the possibility of information hazards needs to be considered when making information policies. He proceeds to categorize and define a large number of sub-types of information hazards. For example, he defines artificial intelligence hazard as:
Artificial intelligence hazard: There could be computer-related risks in which the threat would derive primarily from the cognitive sophistication of the program rather than the specific properties of any actuators to which the system initially has access.
The following table is reproduced from Bostrom 2011 [1].
TYPOLOGY OF INFORMATION HAZARDSI. By information transfer mode Data hazard Idea hazardAttention hazardTemplate hazardSignaling hazardEvocation hazardII. By effect TYPESUBTYPEADVERSARIAL RISKSCompetiveness hazardEnemy HazardIntellectual property hazardCommitment hazardKnowing-too-much hazardRISKS TO SOCIAL ORGANIZATION AND MARKETSNorm hazardInformation asymmetry
