Beware of the danger of “Dark Prompt”... artificial intelligence penetrates and threatens the work of institutions

Listen to the article. The audio text is automated, generated by an automated system

The cybersecurity arena has witnessed a radical transformation. The battle is no longer limited to traditional methods, such as fighting against digital viruses or direct attempts to penetrate networks and accounts. Rather, it has expanded to reach a more complex level related to the way artificial intelligence systems understand and understand the instructions and information they deal with.

Hence the term "Dark Prompt", which is a term used to define "malicious" or misleading claims carefully designed to directly influence the behavior of intelligent models and push them to exceed their security restrictions and the purpose for which they were created.

This term includes dangerous security techniques, most notably what is known as “command injection” and “paradigm breaking,” which the Open Global Cybersecurity Project has classified as among the main risks that one must beware of.

Regarding the mechanism of these attacks, Dr. Muhammad Mohsen Ramadan, Head of the Artificial Intelligence and Cybersecurity Unit at the Arab Center for Research and Studies, explained in exclusive statements to Al Arabiya.net and Al Hadath.net that artificial intelligence models depend entirely on processing instructions and contexts to produce the desired responses.

He added, "Here the attacker takes one of two paths: either formulate direct malicious instructions and direct them to the model to force it to deviate from its original task, or hide and insert these misleading instructions in a hidden manner within content that the system processes to discuss or summarize, such as an email, a web page, or an attached document, so that the system is automatically affected by it without the user's knowledge."

The cybersecurity specialist went on to explain that the real danger lies when these systems are connected to institutional data and practical powers that enable them to carry out actual tasks, such as searching documents and managing business lines. He pointed out that the damage does not stop at producing an incorrect answer, but rather extends to trying to access confidential information or calling unauthorized functions if the security controls are weak.

He said that these risks increase with the emergence of multimedia models that deal with images, audio and video, where misleading instructions can be embedded within media files that the human eye does not notice, which requires expanding the scope of security tests to include all types of input.

Regarding methods of prevention and response, Major General Mahmoud Al-Rashidi, the former Egyptian Assistant Minister of the Interior for Information Security, revealed in special statements to Al-Arabiya.net and Al-Hadath.net that effective protection is possible through the application of an integrated, multi-layered security system, stressing that there is no unique method that guarantees the absolute prevention of command injection attacks; Because language models process instructions and data within a single programming context, separating them is fundamentally challenging.

Major General Mahmoud Al-Rashidi stressed that the first line of defense begins with applying the principle of “minimum authority,” so that the system is granted only the access that is absolutely necessary to perform its mission, while preventing the storage of passwords and secret keys within text instructions, and imposing independent verification before carrying out any sensitive operations.

He added that protection against this risk also includes carefully examining inputs and outputs, isolating untrusted external content, conducting simulated security tests, in addition to continuous monitoring of usage logs to detect any unusual behavior.

The former Assistant Minister of the Interior added that confronting the dangers of dark orders is a shared responsibility that requires setting clear institutional policies, training employees, and educating the average user not to enter confidential data into unapproved tools. He pointed out that the next stage will witness a development in methods for targeting artificial intelligence with the expansion of the use of smart agents, which makes cybersecurity required to protect the path of digital decision-making and ensure that misleading instructions do not turn into actions with real effects that harm individuals and institutions.