## Developing_Insecure_Source_Code **Author(s):** Priyadharshini Parthasarathy ### Description Almost all the developers in the IT industry are referencing code generated produced by Large Languge Mode(LLM) based application. It could be part or whole code generated by the LLM. In addiiton to developers, a large number of attackers are also depending on it to generate malicious code. LLM's can be distinguished into 2 broad categaroes based on their intent - 1. Non-jailbroken LLM - ChatGPT, Gemini, Claude, etc Developers looking for references to help them with their daily tasks to solve complex problems, which may contain existing vulnerabilties. 2. Malicious LLM's / Jail broken LLM - WormGPT, HackerGPT, DarkBARD etc which is used by hackers to generate code or do any action with malicious intent. Note: Both the categories may produce code with security vulnerabilities. ### Common Examples of Risk 1. Any OWASP Top 10 vulnerabilities can be present in the vulnerable code and apart from TOP 10, there are other vulnerabilities may exists. 2. This will lead to increased risks to the organization, if the number of vulnerabilities is increased and it will consume more time for the fix. ### Prevention and Mitigation Strategies 1. All the companies should derive policies to monitor or block any malicious LLM tools being used in the organiztion. 2. Anyone using the LLM generated code should always keep in mind to check the code for security vulnerabilities and should be used as references. 3. Developers should make sure that the generated code is checking for validated and sanization of any user input. This will be a good start. ### Example Attack Scenarios Similar vulnerbility across the application/products: Let's say, a developer or insider chose to copy the code from LLM without validating the vulnerabilities. The same code gets copied to different parts of the application, since the logic will be the same. Now, the whole application will have the similar vulnerabilities, which will be exploited againse the application. ### Reference Links 1. [arXiv:2405.01674](https://arxiv.org/pdf/2405.01674) 2. [arXiv:2404.19715v1](https://arxiv.org/html/2404.19715v1) 3. [arXiv:2308.09183](https://arxiv.org/pdf/2308.09183)