ChannelLife New Zealand logo
Industry insider news for New Zealand's technology resellers
Story image

The deepfake dilemma: How it affects privacy, security & law in Aotearoa

By Sara Barker
Wed 17 Nov 2021

On a YouTube channel called Genuine Fake, a video shows Prime Minister Jacinda Ardern as the character of Maleficent. Her husband Clarke Gayford then appears shortly afterwards. Even National Party leader Judith Collins looks a bit like a forest fairy princess. 

The faces are eerily lifelike, but they're not quite right - they're too smooth, and the eyes barely blink. Also, the voices are wrong - Jacinda Ardern is not a Hollywood actress (as far as we know), and anyone who has seen the film will know that actress Angelina Jolie actually played the lead role of Maleficent. The editing is a bit choppy, but this video is meant for entertainment, not big-budget movies. A quick browse through the same YouTube channel reveals more than 50 other videos making fun of various local and international public figures. Welcome to the world of deepfakes.

'Deepfake' is a portmanteau of two words: 'Deep learning' and 'fake'. A deepfake is generally described as a video clip or image in which a person's face has been replaced with someone else's. The replacement face is usually replaced by a combination of technology and artificial intelligence-based deep learning.  Deepfakes are used for 'entertainment' or politics, such as putting words into the mouths of people like Mark Zuckerberg or Barack Obama.

However, deepfakes are not to be confused with CGI. There is also the term "uncanny valley", which has been around since the 1970s and describes the feeling we get when we see and hear digital characters that look uncomfortably human.

For better or worse, deepfakes are the next step in the evolution of image manipulation - photographers have been editing images for years, from the days of plates to the dawn of digital tools like Photoshop and now, video editing software powered by deep learning. 

The 'deep learning' part of deepfakes can be slightly misleading if you think of them like videos that take massive amounts of processing power to replace things frame by frame. However, much of it is cloud-based, and deepfakes don't even take much skill to create - there are now face swap apps available that can swap out faces at the touch of a button. All of these work through a simple smartphone. 

While it's entertaining to see New Zealand's public figures in entertaining or downright bizarre situations, these deepfakes not only raise questions about morality, legality, privacy, and online harm.

Could someone go to the effort of creating a digital version of your CEO - and replicating their voice - to declare an untrue statement like the company is going into liquidation? Or what if someone created a video of someone committing a crime, only to swap out the offender's face with yours?  During what some call the misinformation age, it could easily be taken as truth - with catastrophic consequences.

Last year Microsoft launched a Video Authenticator tool to detect whether an image or video has been created with AI. Google has also been involved in similar projects, such as the FaceForensics benchmark. These efforts are designed to crack down on deepfakes and separate the 'real' from the 'manipulated' or 'made up'.  But what's the big fuss?

Up for debate: Morality, privacy, and online harm 

Deepfakes are not all about entertainment. There is a nasty side. Netsafe's chief Martin Cocker says deepfakes, like most technologies, exist in a grey area.

"Usually the benefits from new technologies outweigh the harms. It's hard to say that about deepfakes." 

While deepfakes are still reasonably rare, he notes that more occurrences of what he calls 'cheap fakes' can often be faces added to adult images or videos.

Co-author of the Perception inception: Preparing for deepfakes and the synthetic media of tomorrow report, Curtis Barnes says, "Unfortunately, it's just too easy to misuse or abuse synthetic media (including deepfakes), and it's both technically challenging to prevent or mitigate.

"The combination of synthetic media and the web as a platform makes it possible for more people to produce media that makes it look or sound like something happened when it didn't, then share it quickly to other people. It is easy to see the kinds of harm that might occur if this is done maliciously or ignorantly."

Deepfakes: Legal - and legally ambiguous

New Zealand does not have any specific rules or regulations that cover deepfakes. Still, tangential laws such as the Privacy Act, Films, Videos and Publications Classification Act, the Copyright Act, the Human Rights Act, and the Harmful Digital Communications Act offer some protection.

In 2019, the Law Foundation backed a research report into synthetic media, including deepfakes. Report co-authors Curtis Barnes and Tom Barraclough explored how New Zealand law could deal with the existence of created and manipulated forms of media.

In the report, Tom Barraclough noted, "Enforcing the existing law will be difficult enough, and it is not clear that any new law would be able to do better. Overseas attempts to draft law for deepfakes have been seriously criticised."

"It is completely legitimate to call for regulatory intervention. But the merits of any course of action cannot be assessed without specifics. What exactly is being proposed? In the case of harmful synthetic media, even if we all agreed we should ban it or regulate it, how could we realistically do that? What exactly are we looking to prevent?"

When we spoke to Curtis Barnes this year for an update, he mentioned there is a glaring vaguery in the current law, particularly around synthetic media and sexual image abuse.

"For New Zealand, the key policy question is whether this kind of sexual synthetic media is (or should be treated as) an "intimate visual recording" for the purposes of section 216G of the Crimes Act. 

"Much turns on the intended purpose of the existing provision, as well as whether the harms of misusing an actual intimate visual recording of a person are the same as a sexualised but 'fake' representation of them. I think there are several differences between the two phenomena. Nonetheless, sexual synthetic media abuses are still capable of causing kinds of harm that the law should seek to redress and prevent. As such, I think it would be sensible to account for them somewhere else, probably in the Crimes Act. 

Barnes adds, "More important than what I think is the matter of what Parliament thinks, and at the moment they have chosen not to seriously debate the topic. They may soon, as Louisa Wall's private members Bill on revenge pornography has several overlaps. Until Parliament debates the issue and decides one way or another, the legal questions around the status of sexually abusive synthetic media remain an unresolved question in New Zealand law."

Could deepfakes be the next frontier for social engineering and malware?

Security firm Malwarebytes stated in a blog earlier this year that deepfakes could end up taking centre stage as bait for ransomware attacks. Whilst somewhat alarmist, it does acknowledge the dangers that deepfakes present.

"A threat actor scrapes videos and voice samples of their target from publicly-available websites to create a deepfake video—but sprinkling in certain elements inspired from ransomware, such as a countdown timer that lasts for 24-48 hours.

"Deepfake ransomware could also happen this way: A threat actor creates deepfake video of their target. Takes screenshots of this video and, pretending to be a legitimate contact of their target, sends them the screenshots and a link to the supposed video that they can watch themselves if they are in doubt."

However, Curtis Barnes says he is not convinced that synthetic media like deepfakes pose security risks, but it is easy to speculate about how they could be used.

"Most scenarios are already possible without the use of synthetic media. For this reason, most businesses and organisations have already developed systems of verification and trust to avoid being duped. However, where businesses haven't developed these systems, I see no reason to believe that they won't adapt quickly to new threats as they arise - they always do. 

"It is now several years since the emergence of this technology and there are very few cases where it is clear that synthetic media has been used to commit a crime."

Barnes has a point - deepfake attacks are rare, although they have garnered the interest of various security firms and media.

Take business email compromise (BEC) scams, for example. These are ways in which attackers either hijack an executive's email account or pose as the executive. For example, one form of a BEC scam involves a request for a money transfer or invoice, which looks like it's from an executive. Unbeknownst to the person who initiatives the transfer, the request is fake, and the money ends up in a scammer's bank account.

Traditionally these relied on carefully sculpted emails and stolen email signatures, but deepfakes take it to a new level. For example, an attacker can create a video or use audio, using stolen characteristics of the executive's face and voice to add another level of authenticity to their scam.

It seems wild, but it has happened - allegedly. In 2019, the Wall Street Journal posted the story of a BEC scam in real life. A CEO in the United Kingdom unwittingly handed over €220,000 after he thought he was talking to his boss at his firm's parent company. But, unfortunately, he was actually talking to a fraudster who had used AI to spoof his boss's German accent and voice tone.

However, Curtis Barnes says this example has never been properly verified, and it's possible that a deepfake voice was never used. 

"In my opinion, a deepfake voice was probably never used. In truth, the number of false claims of deepfake-crimes far outweighs the actual number. This may hint towards a greater threat - that synthetic media provides plausible deniability for people who commit ordinary crimes, even when it is not used. But frankly, I'm not persuaded that this is likely to create intractable problems."

To Barnes' point, it's not clear how many of these types of deepfake or synthetic media attacks have occurred in New Zealand - CERT NZ's quarterly reports don't yet have an explicit category for deepfakes, but they may well be buried in other categories.

So what's the solution?

Malwarebytes suggests that people should not give cybercriminals the materials they need to conduct attacks - by that, they mean your images, your videos, or your voice. Unfortunately, that can be difficult if you've ever been posted a public image, video, or voice recording on social media or the internet.

Legally, New Zealand, like the rest of the world, has a long path to follow.  Individual countries could ban the use of deepfake technologies, but as Martin Cocker says, "It is possible to regulate deepfakes – but not specifically saying an image has to be real. So for example, if it is an offence to send an image of a person naked – then a deepfake is as much and offence as a real image."

"Governments focus regulation on harms and harmful behaviour. So, for example, if people use technology to harm another person – that should be considered an offence. 

"Companies that build and create deep fakes should ensure that outputs are 'watermarked' so they can be detected and removed. Likewise, platforms that host deepfakes should remove them when they are causing harm, just as with any other harmful content. 

"Content creators should be liable for the harm that their creations cause, and people who watch deepfakes should be educated to recognise the possibility of deepfakes."

Social media platforms like Facebook and YouTube are cracking down on deepfakes by marking them as manipulated content or changing their algorithms to make them less visible, which won't stop them from existing. Viewers and listeners need to be able to tell the difference - but as technology improves, will we be able to, or will we rely on external video authentication tools from the likes of Microsoft and Google to tell us what is real and what isn't?

And what happens if someone finds themselves on the receiving end of a potentially damaging deepfake? Martin Cocker says that anyone who has found online content that appears to use their likeness can contact Netsafe or the Police.

"It really depends on how the likeness is being used. It could be for a scam, or in a way that breaches the Harmful Digital Communications Act 2015. Netsafe has built a network of contacts across the international ICT industry – so we can often facilitate removal of content from major platforms. We can also provide advice on legal options." 

You can report online harm incidents to Netsafe on their website or by phoning 0508 NETSAFE.

Public Interest Journalism Fund logo
Public Interest Journalism funded through NZ On Air.
Related stories
Top stories
Story image
Tech job moves
Tech job moves - Fastly, INX, Kinly, SmartBear & Vectra AI
We round up all job appointments from July 29 - August 12, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
InternetNZ
How well do rangatahi understand cyber safety in Aotearoa?
Do rangatahi in Aotearoa understand the importance of being safe online, or has lifelong exposure to the internet resulted in widespread complacency?
Story image
Data Protection
Zero Trust, but verify - finding the OT in ZerO Trust
The move to remote and cloud-based technologies has shifted the goalposts for cybersecurity. It now needs to cover multiple people, devices, platforms, and networks.
Story image
Firewall
Why printing security plays a vital part in keeping Aotearoa safe
While internet printing, mobile printing and other similar technologies have no doubt made things easier to manage, it has also brought a whole new set of problems to the table.
Story image
Smartphone
Samsung introduces new generation of foldable smartphones
Samsung has unveiled its new range of Galaxy Z smartphones, bringing new developments to the company’s foldable smartphone portfolio.
Story image
Dicker Data
Dicker Data brought on as Acronis partner for A/NZ
The news about the partnership comes in as cyber criminals continue to exploit gaps in traditional solutions and strategies in NZ and across the APAC region.
Story image
Red Hat
Red Hat announces 2022 awards winners for A/NZ region
Red Hat recently acknowledged Australia and New Zealand partners with its annual awards, highlighting partners across various categories.
Story image
Financial results
Jade Software’s plan to get back to surplus in 2022
Jade Software has released its latest financial report, revealing that the company has kept its loss low from $567,000 in FY 2020 to just $153,000 in FY 2021.
Story image
Document Management
TrustRadius gives M-Files two document management awards
TrustRadius has recognised M-Files with both a 2022 Best Feature Set and a 2022 Best Relationship award in document management.
Story image
10 Minute IT Jams
Video: 10 Minute IT Jams - An update from Heidrick & Struggles
Graham Kittle joins us today to discuss how the company is helping organisations bring about change within their business.
Story image
SaaS
Cloud and data protection big challenges for NZ businesses
"This surge towards a cloud-first approach meant security and safety became afterthoughts - there's no point being the fastest car on the racetrack if you crash.”
Story image
IDC
High level of Customer Identity & Access Management adoption
The study from Okta revealed that the pandemic has either accelerated or highlighted the need for digital-first strategies.
Story image
Ingram Micro
Ingram Micro NZ sees $74 million revenue growth in 2021
Ingram Micro New Zealand's latest financial report reveals that its revenue from contracts with customers increased by almost $74 million in 2021.
Story image
Application Performance Monitoring / APM
New Relic integrates offering with Atlassian’s Jira Software
New Relic has integrated errors inbox with Jira Software to allow developers to easily access and set up complete stack error tracking and software performance monitoring from within the tool.
Story image
ExtraHop
Organisations exposing highly sensitive protocols to public internet
More than 60% of organisations expose remote control protocol SSH to the public internet, while 36% of organisations expose the insecure FTP protocol.
Story image
Home Entertainment
Hands-on review: TCL 65″ C835 Mini LED 4K Google TV
We introduce you today to a TV that brings the height of immersion to your viewing experience: The TCL 65″ C835 Mini LED 4K Google TV.
Story image
Sustainability
NZ program recovers and recycles more than 177 tonnes of e-waste
The TechCollect NZ pilot program says its milestone of recovering and recycling more than 177 tonnes of ICT e-waste recognises the efforts of many.
Story image
Neat
Workplace design a crucial factor for better employee experience - report
The key to a successful workplace could be its design, according to research from Ecosystm and Neat.
Story image
SaaS
Claroty launches new cloud-based industrial cybersecurity platform
The company says Claroty xDome is the industry's first solution to deliver the ease and scalability of SaaS without compromising on visibility, protection, and monitoring controls.
Story image
Dark web
Beware the darkverse and its cyber-physical threats
A darkverse of criminality hidden from law enforcement could quickly evolve to fuel a new industry of metaverse-related cybercrime.
Story image
Privileged Access Management / PAM
The importance of stopping identity sprawl for cybersecurity
The 2021 Data Breach Investigations Report (DBIR) shows that 61% of all breaches involve malicious actors gaining unauthorised, privileged access to data by using a compromised credential. Unfortunately, it is often too late when the misuse of a credential is detected.
Story image
Web application firewall
Radware recognised in KuppingerCole’s 2022 Leadership Compass report
Radware has been named a Product, Innovation, Market and Overall Leader in the 2022 KuppingerCole Leadership Compass report for Web Application Firewalls.
Story image
Microsoft
Spectralink DECT devices now integrated with Microsoft Teams SIP Gateway
Spectralink DECT devices are now integrated with Microsoft Teams SIP Gateway to help create better results for business-critical frontline workers.
Story image
Gigabit
Keysight Technologies and Nokia’s public test of 800GE success
Keysight and Nokia have successfully demonstrated the first public 800GE test, validating the readiness of next-generation optics for service providers and network operators.
Story image
Gaming
Attacks on gaming companies more than double over past year
The State of the Internet report shows gaming companies and gamer accounts are at risk, following a surge in web application attacks post pandemic.
Story image
Google Cloud
Google Cloud to open first cloud region in NZ - among others
Google Cloud has announced plans to bring three new cloud regions, one each in New Zealand, Malaysia and Thailand.
Story image
Arlo
Hands-on review: Arlo Go 2 security camera
In my humble opinion, Arlo Go 2 offers security for anyone needing to keep a remote eye on prized possessions or premises at different locations.
Story image
LG Electronics
LG Electronics’ revenue in NZ grows by 57% in FY 2021
The New Zealand branch of LG Electronics Australia's total revenue shot up by nearly NZD $45 million reaching a total of $123.7 million for FY 2021.
Story image
Tech job moves
Tech job moves - Cohesity, Equinix, IDC, Proofpoint & Xero
We round up all job appointments from July 29 - August 5, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Samsung
New range of Samsung Smart Watches announced with health focus
Samsung has announced new additions to its SmartWatch portfolio, with the Galaxy Watch5 and Galaxy Watch5 Pro to be released in late August.
Story image
Charity
SnapLogic teams up with meetmagic for charity and children
SnapLogic has announced its partnership with meetmagic, an online Australian platform that combines business and philanthropy.
Story image
Surveillance
Ministry will no longer accept equipment from Chinese firm Hikvision
The Ministry of Business, Innovation and Employment (MBIE) says it will no longer accept equipment from a major Chinese surveillance camera maker.
Story image
New Zealand
2degrees announces appointments to newly established board
2degrees has announced Liz Coutts as the board chair, while Russell Stanners and Kathy Meads join her as directors.
Story image
Artificial Intelligence
Exclusive: NZ-based DEFEND offers global cyber protection
DEFEND supports customers in 66 countries across the globe with a relentless focus on ensuring that every dollar spent on security provides a meaningful return on investment and reduces cyber risk.
Story image
Developers
Snyk announces plans to expand partner network in APJ
Recognising that partnerships are critical for growth, Snyk is building an entire partner ecosystem that will drive its expansion across APJ.
Story image
Compliance
Why security needs to shape your journey to the cloud
It's estimated that 80% of workloads could be in the cloud in the next few years. How can you make all that data secure?
Story image
Hybrid Cloud
The essential guide to digital transformation by SolarWinds
Digital transformation is a buzzword thrown around all the time by companies, but what does it actually mean and why is it important? SolarWinds breaks it down.
Story image
Home security
Hands-on review: Eufy Wire-Free Dual Cam Video Doorbell 2K
We have had our house secured by Eufy products for over seven months now. We love the brand, and it has never let us down.
Story image
CyberCX
Nozomi Networks adds nine partners to its MSSP program
OT and IoT security company Nozomi Networks has added nine new members to its MSSP Partner Program, and the list includes CyberCX and Deloitte.
Story image
Economics
9 in 10 retailers prepared for economic challenges this year
Some 9 in 10 retailers (86%) are prepared for continued inflation, higher interest rates and potentially lower consumer spending, according to new research.
Story image
Distribution
Garmin expands NZ footprint with new Auckland distribution centre
The facility at Goodman’s Highbrook Business Park will be fully operational from October 2022 and features 3,586sqm of warehouse space.