The ability to distinguish and identify what’s real from AI generated images, videos, audio files and written content is becoming more important than ever. The large language models that are able to produce various types of digital content are getting tricky to cypher out.
This month, Google announced the beta test of its SynthID watermarking technology that embeds a digital watermark — that’s imperceptible to the naked eye —simultaneously into any type of content produced using its Gemini AI models.
The SynthID toolkit not only tags content, it can also be used to identify if any content or a portion of the content scanned into its system as a sanity check was produced using the technology.
How Does SynthID Watermarking Work?
Nature’s October 2024 issue features a deep technical dive into how the watermarking system works. At a simplified level, when any LLM responds to a user query or prompt the model has to predict what the “next best” output should be to fulfill the user request.
While the model builds the response character by character, the SynthID tool is instantly assigning an unseen probability value for each text token the model can choose to insert. For example, if you ask the LLM to complete the sentence “I have a pet __”, the text token “dog” would have a high probability value of perhaps 0.9 while the text token “gila monster” would logically be much lower at something like 0.01.
This predictive process continues to build until the requirements of the original query are completed.
“The final pattern of scores for both the model’s word choices combined with the adjusted probability scores are considered the watermark. This technique can be used for as few as three sentences. And as the text increases in length, SynthID’s robustness and accuracy increases,” according to a company blog post on the topic.
The company’s statement went on to note that its technology is now freely available for developers to weave into their own AI generative large language models.
“We also open-sourced it through the Google Responsible Generative AI Toolkit, which provides guidance and essential tools for creating safer AI applications. We have been working with Hugging Face to make the technology available on their platform, so developers can build with this technology and incorporate it into their models,” Google stated.
Tools Such As SynthID Are Necessary
Peter Slattery, Ph.D., MIT FutureTech is a leading researcher in the field of AI risks. He wrote in an email exchange that this type of technology is critically important.
“We urgently need technologies like SynthID to help safeguard the integrity of online information and protect our communication ecosystem. We know from the MIT AI Risk Repository and incident trackers that misuse of AI generated content for misinformation and deepfakes is widely referenced and increasingly prevalent, so this seem like an essential mitigation to explore,” wrote Slattery.
SynthID Is Not A Silver Bullet Regarding AI Content
However, Slattery noted at least one study found that researchers were able to tamper or steal a digital watermark pattern that was similar in concept to SynthID.
“I think we need to be very careful to ensure that watermarks are robust against tampering and that we do not have scenarios where they can be faked. The ability to fake watermarks could make things worse than having no watermarks as it would give the illusion of credibility,” Slattery explained.
The Google researchers in the published Nature study acknowledged that despite SynthID’s strong detection capabilities it is not a panacea.
“Another limitation of generative watermarks is their vulnerability to stealing, spoofing and scrubbing attacks, which is an area of ongoing research. In particular, generative watermarks are weakened by edits to the text, such as through LLM paraphrasing—although this usually does change the text significantly,” as stated in the published study.
