Basilisk: An Evolutionary AI Red-Teaming Framework for Systematic Security Evaluation of Large Language Models

Key Points

• To develop an open-source framework for conducting automated security evaluations of large language models using adversarial prompting techniques.
• Utilized Smart Prompt Evolution for Natural Language (SPE-NL) to generate adversarial prompts.
• Employed genetic algorithms to evolve prompts that bypass security measures in language models.
• Developed implementation source code alongside evaluation datasets.
• Successfully demonstrates the capability to automate red-teaming processes in AI systems.
• Provides a framework for systematically evaluating security aspects of language models through adversarial interactions.