According to an FBI warning released in March, malicious actors "almost certainly will leverage synthetic content for cyber and foreign influence operations in the next 12-18 months". Foreign actors are already using synthetic content in their influence campaigns, and the FBI anticipates it will be increasingly used by foreign and criminal cyber actors for spearphishing and social engineering, representing an evolution of cyber operational tradecraft.
The FBI defines synthetic content as the broad spectrum of generated or manipulated digital content, including images, video, audio, and text. While commonplace tools like Photoshop can be used to create synthetic content, this report highlights techniques based on artificial intelligence (AI) or machine learning (ML) technologies. These techniques are known popularly as deepfakes or GANs (generative adversarial networks). Although, generally speaking, synthetic content is considered protected speech under the First Amendment, the FBI has stated that it may investigate malicious synthetic content attributed to foreign actors or otherwise associated with criminal activities.
Former "fraud czar" of Google, Shuman Ghosemajumder, told BI last year that deepfakes were likely to evolve and spread further, with "perfectly realistic" deepfakes in our near future.
Synthetic content may also be used in a newly defined cyber-attack vector referred to as Business Identity Compromise (BIC). BIC will represent an evolution in Business Email Compromise (BEC) tradecraft by leveraging advanced techniques and new tools. While BEC primarily includes the compromise of corporate email accounts to conduct fraudulent financial activities, BIC will involve the use of content generation and manipulation tools to develop synthetic corporate personas or to create a sophisticated emulation of an existing employee. This emerging attack vector will likely have very significant financial and reputational impacts on victim businesses and organizations. The reality and imminence of this new threat are highlighted in Constella's recently published articles on synthetic media and the risks its proliferation poses for brands and executives.
The FBI has issued a stark warning saying "malicious actors almost certainly will leverage synthetic content for cyber and foreign influence operations in the next 12-18 months." This emerging attack vector will likely have very significant financial and reputational impacts on victim businesses and organizations.