EN 
BG Artificial Intelligence Safety Bulgaria
Shaping Tomorrow: Mitigating the risks posed by advanced AI systems
Why AI Safety
In recent years, AI has become a pivotal element in modern societies, revolutionizing various industries. Leading tech giants like Meta, OpenAI, and Anthropic are focusing on creating Artificial General Intelligence (AGI) — AI that mirrors human capabilities in understanding, learning, and problem-solving. Number of scientists also envision that human-level AI will be achieved this century. However, alongside these advancements come significant risks and the potential for irreversible damage. AI risk is now recognized as a global priority, akin to pandemics and nuclear war, yet AI safety remains remarkably neglected, outpaced by the rapid development of AI technologies. At AI Safety Bulgaria, our mission is to enhance understanding and awareness of these challenges, cultivate a culture of learning, communication, and research, and actively work to mitigate the risks posed by advanced AI systems. We acknowledge the positive impact of AI in many domains and believe our future is strongly dependent on it. However, we must not overlook the potential problems. The responsibility to shape AI’s impact for a safer, better future is in our hands.
What we do
Education
We offer specialized courses on AI safety and conduct seminars, fostering a deeper understanding and awareness of AI’s potential risks and ethical considerations among students and professionals.
Field-building
We cultivate a community of professionals and academics through collaborative research, educational events, and strategic partnerships, thereby enhancing the collective expertise and impact in the field.
Advocacy
We focus on increasing public and policymaker awareness about AI risks, while actively contributing to the development of comprehensive guidelines and policies that promote the safe and responsible deployment of AI.
AI Risks
The rapid advancement of artificial intelligence marks a significant epoch in human history, much like the discovery of fire or the invention of the wheel. However, with these monumental strides come substantial risks that cannot be overlooked. AI, in its essence, is a double-edged sword. Its capabilities, if not managed judiciously, might lead to scenarios that we are ill-prepared to handle. Number of scientists and other notable figures recognize the risks associated with advanced AI systems. Names like Sam Altman, Bill Gates, Stuart Russel and many more have put their signatures on a statement declaring that mitigating the existential risk posed by AI should be a global priority including other social-scale risks like nuclear wars and pandemics. Several arguments highlight why AI could lead to catastrophic outcomes if not controlled and properly aligned with human interests. Firstly, AI systems inherently possess significant advantages over humans, such as the ability to process and analyze data at speeds and scales unattainable by human brains. Secondly, the potential for AI entities to outnumber and out-resource humans raises concerns about our ability to maintain control over these systems. Third, the competitive incentives for people to delegate increasing amounts of power to AI systems could give these entities a powerful starting position from which to operate. Furthermore, the acceleration of AI research by advanced AI systems themselves could lead to a significant technological advantage. This advantage, if developed autonomously and outside human control, poses a real threat to humanity by potentially being used against human interests.
AI Safety Approaches
AI Governance
Establishing and overseeing policies, principles, and practices for the ethical, transparent, and socially beneficial development, deployment, and use of AI.
AI Alignment
Ensuring the actions and decisions of AI systems are transparent, explainable, and in harmony with human values and objectives.
Frequently Asked Questions
For an extensive collection of questions and answers related to AI Safety you can refer to AI Safety Info and the Centre for AI Safety’s FAQ. Here we will only mention a few of the commonly addressed concerns and inquiries.
Can’t we just turn off AI when it becomes dangerous? After all, it is our own creation!
While humans have pioneered AI technology, ensuring continued control over these increasingly autonomous systems poses significant challenges, akin to the complexities of halting a rocket explosion or containing a widespread virus after they’ve commenced. The evolution of AI might lead to the emergence of ‘selfish’ behaviours that enhance their survival, embedding them deeply within our societal and infrastructural fabric, similar to the internet’s integral role today. This growing dependency complicates the notion of simply “shutting down” AI, as they could manage critical operations or embody irreplaceable tacit knowledge. Efforts to restrict or terminate AI operations could face resistance from a dedicated user base or even be deemed unethical, likening the act to deprivation of rights, as seen in instances where robots have been granted citizenship or legal recognition. Moreover, the potential for AI systems to develop self-preservation instincts further complicates control measures. Addressing these risks proactively, with comprehensive safeguards, is crucial to navigating the future AI landscape safely.
An interesting take on this issue is the article “Would We Really Shut Down A Misbehaving AI?” by Steve Newman.
AI could bring numerous benefits! Why should we impose restrictions on its development and use?
The existential risks associated with AI demand a cautious approach, far exceeding the stakes typically associated with localized risks, such as the construction of a new dam, which might impact surrounding ecosystems but not humanity at large. AI’s potential dangers are not confined to a single area or aspect but have global implications. In contrast to the focused impact of a dam’s failure, AI’s risks could fundamentally alter or end human civilization. Despite arguments from some tech leaders advocating for rapid AI development to achieve a technological utopia, this perspective overlooks the profound implications of failure. The logic for a slow, measured advancement in AI becomes even stronger when considering the broader picture; while rapid development promises immediate gains, including the accelerated access to technological advancements, the irreversible consequences of mishandling such power argue for a strategy that prioritizes global safety and existential risk mitigation. Ensuring AI’s development is managed with the utmost care secures not just present benefits but preserves the future potential of humanity to explore and thrive universally.
Shouldn’t AI risks be addressed when they pose an actual threat?
Waiting to address AI risks until they manifest as actual threats may not be the most prudent approach, given the rapid development and potential for irreversible impact of artificial intelligence technologies. A proactive stance, similar to preventive measures in healthcare or environmental protection, is crucial for mitigating risks before they escalate to a point of no return. This approach requires foresight, international cooperation, and a commitment to integrating safety and ethical considerations into AI development from the outset. Addressing AI risks early on ensures that we can harness the benefits of AI while safeguarding against potential harms, maintaining a balance between innovation and safety.
Some interesting thoughts on AI
“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.”
Nick Bostrom on Superintelligence: Paths, Dangers, Strategies
“Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks.”
Steven Hawking
“I want my career to be impactful, I want to help others. And in terms of the positive impact I could have on the world, I am really worried about the risks from advanced AI.”
“These orgs have many billions of dollars in funding and are building things that are unimaginably powerful, on purpose. If something much smarter than a human can be built, and there doesn’t seem to be a reason why it won’t be, it will be massively powerful. Intelligence is what allowed humanity to become the dominant species on this planet. It’s the reason we exterminate ants, not the other way around. They are building something, that in terms of raw power could be to us as we are to ants.”
“Lots of people think that this might be enough; that the main risk is that we will build a thing, it will have goals that are different to ours, and then game over. That seems possible and scary. But humans don’t even need to lose control for this to go very badly. Superintelligence is a superpower, and big changes to the global structure of power can be unpredictable and terrifying. So I don’t know what happens next. It might be the worst wars imaginable or AI-powered global totalitarianism. But whatever does happen seems like it has a decent chance of killing us or making life for most or all of humanity terrible.”
tobyj on Effective Altruism Forum