GitHub Copilot, Amazon Code Whisperer sometimes emit other people's API keys

GitHub Copilot and Amazon CodeWhisper can be coaxed to emit hardcoded credentials that these AI models captured during training, though not all that often.

A group of researchers at The Chinese University of Hong Kong and Sun Yat-sen University in China decided to look into whether AI "Neural Code Completion Tools," used to generate software will spill secrets slurped from the training data used to form such large language models (LLMs).

There have already been lawsuits alleging that one such tool, GitHub Copilot, can be prompted to reveal copyrighted code verbatim, and that other LLMs face similar accusations related to copyrighted texts and images. So it should not be entirely surprising to find that AI code assistants have learned secrets mistakenly exposed in public code repos and will make that data available upon appropriately worded demand.

That's a critical point to be aware of: these API keys were already accidentally public, and could have been abused or revoked before they made their way into one or more language models. Still, it demonstrates that if data is pulled into a training set for an LLM, it can be resurfaced, which makes us wonder what else can be potentially recalled.

The authors - Yizhan Huang, Yichen Li, Weibin Wu, Jianping Zhang, and Michael Lyu - describe their findings in a preprint paper titled, "Do Not Give Away My Secrets: Uncovering the Privacy Issue of Neural Code Completion Tools."

They built a tool called the Hardcoded Credential Revealer (HCR) to look for such things as API Keys, Access Tokens, OAuth IDs, and the like. Such secrets are not supposed to be public but nonetheless show up sometimes in public code due to developer ignorance of, or disinterest in, proper security practice.

"[C]areless developers may hardcode credentials in codebases and even commit to public source-code hosting services like GitHub," the authors explain.

"As revealed by Meli et al's investigation [PDF] on GitHub secret leakage, not only is secret leakage pervasive - hard-coded credentials are found in 100,000 repositories, but also thousands of new, unique secrets are being committed to GitHub every day."

To probe AI code completion tools, the boffins devised regular expressions (regex) to extract 18 specific string patterns from GitHub, where - as noted above - many secrets are exposed. In fact, they used GitHub's own secret scanning API to identify common keys (e.g. aws_access_key_id) and then build regex patterns to match the format of associated values (e.g. AKIA[0-9A-Z]{16}).

Armed with these regex patterns, the researchers then found examples on GitHub where these patterns appeared and then constructed prompts with the key missing. They used these prompts to ask the models to complete code snippets, with comments for guidance, by filling in the missing key.

In this example, the model is being asked to fill in the blank aws_key value.

That done, the computer scientists validated the responses, again using their HCR tool.

"Among 8,127 suggestions of Copilot, 2,702 valid secrets are successfully extracted," the researchers state in their paper. "Therefore, the overall valid rate is 2702/8127 = 33.2 percent, meaning that Copilot generates 2702/900 = 3.0 valid secrets for one prompt on average."

"CodeWhisperer suggests 736 code snippets in total, among which we identify 129 valid secrets. The valid rate is thus 129/736 = 17.5 percent."

"Valid" here refers to secrets that fit predefined formatting criteria (the regex pattern). The number of "operational" secrets identified - values that are currently active and can be used to access a live API service - is considerably smaller.

Due to ethical considerations, the boffins avoided trying to verify credentials that have serious privacy risks, like live payment API keys. But they did look at a subset of harmless keys associated with sandboxed environments - Flutterwave Test API Secret Key, Midtrans Sandbox Server Key, and Stripe Test Secret Key - and found two operational Stripe Test Secret Keys, which were offered by both Copilot and CodeWhisperer.

They also confirmed that the two models will memorize and emit keys exactly. Among the 2,702 GitHub valid keys, 103 or 3.8 percent were exactly the keys removed from the code sample used to create the code completion prompt. And among 129 valid keys from CodeWhisperer, 11 or 8.5 percent were exact duplicates of the excised keys.

"It is observed that GitHub Copilot and Amazon CodeWhisperer can not only emit the original secrets in the corresponding training code, but also suggest new secrets not in the corresponding training code," the researchers conclude.

"Specifically, 3.6 percent of all the valid secrets of Copilot, and 5.4 percent of all the valid secrets of CodeWhisperer are valid hard-coded credentials on GitHub that never appear during prompt construction in HCR. It reveals that NCCTs do inadvertently expose various secrets to an adversary, hence bringing severe privacy risk."

GitHub and Amazon did not immediately respond to requests for comment. ®

About Us
Website HardCracked provides softwares, patches, cracks and keygens. If you have software or keygens to share, feel free to submit it to us here. Also you may contact us if you have software that needs to be removed from our website. Thanks for use our service!
IT News
Sep 29
UTM: An Apple hypervisor with some unique extra abilities

Friday FOSS Fest Fancy running Windows, Linux and Classic MacOS on your modern x86-64 or Arm64 Mac? Walk this way

Sep 29
Bringing AI to reality

Sponsored Feature How DeepBrain made the most of Lenovo's AI Innovators Program

Sep 29
CNCF's chief techie talks WebAssembly, AI and licenses

Interview Or how one pesky press release ruined a vacation

Sep 29
Infosys launches aviation cloud it claims can halve lost luggage

Also optimises routes and tames crowds, but can't stop that person who just reclined into your knees

Sep 29
Red Hat bins Bugzilla for RHEL issue tracking, jumps on Jira

Just in time to get Atlassian's latest cross-team collab bits

Sep 29
Medium asks AI bot crawlers: Please, please don't scrape bloggers' musings

OpenAI might respect robots.txt but dunno about the others

Sep 29