Charles Gutjahr

I'm pretty sure that stealing all the books and research papers from the US nuclear weapons programme would be illegal. I'm also confident that giving those books to, say, Iran would break a whole raft of other laws too.

What if instead you used the stolen books to train an AI model, transforming the books into an LLM which doesn't contain those original sources verbatim? Surely still illegal to give to Iran because nuclear secrets can be gleaned from the LLM. However what if transformation is so extreme—the LLMs parameters so small—that all the nuclear secrets drop out leaving (probably) only basic physics knowledge which isn't illegal to distribute?

It's a moot point. I'm sure the US authorities would lock you up for stealing the books, and you're not going to be allowed to distribute the LLM.

That's not what happened in this week's US court victory for Anthropic. The judgement found that transforming books into an LLM was fair use, but that stealing books was not—so only the stealing part can be taken to trial. Anthropic's LLM was not immediately shut down.

If instead the books had been stolen from the US nuclear weapons programme rather than from Libgen and PiLiMi it surely would have been shut down immediately. Obviously that's because stealing nuclear secrets is a more serious crime than stealing copyrighted books, but they're both illegal. Anthropic hasn't yet been convicted of stealing books so it is reasonable to wait to take action, but that's a delay not an abandonment.

A trial on Anthropic's stealing of books could—and in my opinion should—order that Anthropic's LLMs get shut down. Not just Anthropic; journalists report it is likely that all major LLMs are trained on stolen books. It is possible that in time all major LLMs get shut down by the courts.

The industry should be responding now by training LLMs legally, not waiting for the law to catch up. I'd like to start using AI but I don't want to be handling stolen goods.

Children under 16 will be banned from major social media platforms after the Australian parliament passed the Online Safety Amendment bill last week. I think this a bad law which is not actually going to make kids much safer and could make things worse for everyone¹.

However I do not blame the government for this misguided law, I put the blame squarely on the big social media platforms. Tech companies should have dealt with the problems their platforms are causing, but instead it feels like they have been shirking their responsibilities. Australians are overwhelmingly angry with them: 77% want kids banned and 87% want greater penalties on social media companies that do not comply with Australian laws. The government was clearly pressured to step in and do something, anything. The result is a law that is bad for those platforms, a law that they could and should have prevented.

I'll give you a personal anecdote about how Facebook has failed me. This is something I reckon Facebook should have solved, and their lack of action has made it inevitable the government would step in.

My Facebook feed is flooded with scam advertisements. There are often times when most advertising I see on Facebook is for scams. They are the same scams that Facebook has been showing me for years and has failed to stop. Not only does Facebook keep showing me those scams, but Facebook also routinely rejects my reports about them.

In my last blog post I wrote about on particular fake Paul Hogan scam ad which I reported to Facebook but they didn't remove. Since then I escalated that report by requesting another review of the ad. The second review came back and yet again Facebook did not remove that ad:

This is not an isolated incident. The last 30 scam ads that I've reported to Facebook have not been taken down. The last time Facebook took action on a scam I reported was so long ago that Facebook has since deleted the message from my Support Inbox.

Sometimes instead of showing me scams Facebook shows me advertisements featuring hardcore pornography. Here's an example of one such ad, censored by me so that I am not distributing porn on my own website:

The advertisement that I have pixelated showed a man and woman having sex, with an erect penis and copious amounts of semen. It was a clear breach of Facebook's Advertising Standards, which have specific prohibitions on adult nudity and sexual activity. Facebook displaying this pornographic ad was made even worse by the fact that they placed it next to an ad for a private girl's school.

I reported the ad to Facebook but they did not remove it:

Facebook has failed here.

I understand that occasionally things slip through the cracks, but Facebook should take action when such problems are reported. Unfortunately Facebook routinely fails to take action when I report a scam. That is unacceptable to me, and surely to most Australians as well.

What would be acceptable? I can't speak for all Australians but I think a reasonable expectation is that Meta should not allow scam ads or pornographic ads to appear on Facebook in the first place. I reckon Facebook could step up and actually achieve that: simply have a person review each ad before it is published, looking for the obvious scams and porn. Perhaps AI will be helpful to detect scams because Facebook is an AI leader, but if they're using AI to detect scams now it is clearly failing given the prevalence of scams. So I think Facebook really just needs a lot of people to do the reviews. It would be expensive but surely that is better than alienating most of the population and forcing government to ban social media?

Facebook scams are not the only problem on social media and not the main reason behind the government ban on social media—just the one I encounter the most. But those scams are a reasonable illustration of the kinds of problem that social media platforms should be making more of an effort to fix. If they don't fix these things then the government will... and probably badly.

btw. I will be taking my own advice on my new project neighbo.au. You can read about that on the Neighbo blog.

Footnotes

Facebook has been showing me advertisements for investment scams for many years. When I report the scams to Facebook they mostly reject my reports despite the ads being clear breaches of Facebook's advertising standards. Their response is typically "we didn't remove the ad".

Why? The scam ads should be easy to detect because the scams have been following the same pattern for years.

I'd expect Facebook to be able to use their AI technology to automatically detect the scams. Facebook and its parent company Meta make some of the best AI technology and are spending billions on AI, but perhaps their AI simply isn't up to the task. Humans certainly are though; a person would notice the scam in seconds. So why doesn't Facebook notice the scam when I report it?

I suspect humans aren't even looking at the reports I send in. I reckon they should be.

A boring section where I demonstrate why these are scams

I say these are obvious scams but here is a detailed example so you can judge for yourself.

Take a look at this typical ad that Facebook showed me in early September. The ad has all the hallmarks of this particular scam:

• Mentions a famous Australian, in this case Paul Hogan.
• Implies that they are either dead or in trouble: "Paul Hogan's tragic end!"
• Pretends to be a legitimate news website though often the URL doesn't make sense or match the image, like Czech website wn24.cz here for an Australian celebrity.

This pattern is so common and so consistent it should immediately raise red flags. It isn't proven to be a scam at this point... but every ad like this that I've looked into turns out to be a scam. I'm sure you'll see the pattern yourself after just a few ads. Here's a selection of the hundreds of scam ads I've seen on Facebook in the last few years to demonstrate:

They're not using exactly the same text and images, but they are similar enough that I reckon a human reviewer should immediately recognise ads that fit this pattern.

Clicking on any of those advertisements would take me a fake news article about the celebrity pictured in the ad. For this Paul Hogan ad it is a fake news.com.au website with a headline "Bank of Australia is suing Paul Hogan for what he said on live TV" which looks like this:

This is not a legimate news article. The easiest way to determine that is by the URL on the address bar:

https://qiyudz.top/products/w9587r-real-bra-tank-candy-orange?source_id=YHVt2vvbmMjQSE

• Uses an obscure domain: qiyudz.top instead of news.com.au
• The domain doesn't match the one shown on the ad: this is qiyudz.top but the ad showed wn24.cz
• The slug doesn't relate to the story: 'real-bra-tank-candy-orange' clearly isn't referring to a news story

Those are basic checks which anyone could do even if they're not Australian and don't know who Paul Hogan is. That should be sufficient evidence that this is a fake news story, but someone with a little knowledge and a little time to do some more checks would find several more problems:

• Wrong design: this doesn't look like the legitimate news.com.au site
• Links don't work: clicking the National, World, Lifestyle etc links doesn't do anything
• No advertising: real news sites are desperate for money so they cover their pages in ads
• Non-existent bank: there is no such thing as the 'Bank of Australia'
• Unlikely author: Alan Kohler and 7.30 appear on ABC, which is a competitor to news.com.au
• Undated article: says 'Posted 20h ago' rather than the actual date, and that '20h ago' never changes

At this point we've established that this is not a legitimate news article but we have not yet confirmed that it is a scam. The evidence that this is a scam is in the text of the article. It's a very long page so I haven't included everything here but you may view the entire page in these screenshots:

The page has several of the investment scam warning signs from ScamWatch:

• Fake news stories or ads that claim a celebrity recommends this scheme to make big money.
• Emails, websites or ads with testimonials and over-the-top promises of big returns.
• High pressure tactics designed to rush you to act so you don't 'miss out'.

We've already established that this is a fake news story about a celebrity with a fake ad, so that's the first warning sign.

The second warning sign is the over-the-top promise of big returns. The fake story makes several claims of unrealistic returns, for example:

Paul Hogan: If you don't believe me, I'll prove it to you. Give me 375 A$, and with the help of X, I'll earn you a million in less than six months!

where X is the name of the scam product, and:

Paul Hogan: Try to imagine how much money will be in your balance in 2 weeks. If you invest at least 375 $ right now, then in just 2 weeks, you will have tens of thousands!

For legitimate investment products a good investment might earn you between 5–10% per annum. This scam is promising between 1600–12000% per annum which is completely unrealistic.

The over-the-top claims continue in the testimonials below. These testimonials are made to look like Facebook comments but a quick check of the HTML shows that they have not come from Facebook and are unlikely to be genuine.

The third warning sign are the high pressure tactics. The page attributes this fake quote to Hogan:

I'm not sure how long it will remain free. I've heard that registration on the platform will become paid after a few days, so I recommend hurrying. I hope you'll be able to use the platform before it becomes paid or registration closes for new users.

Then the signup form below says:

Registration will be free until the end of X

where X is always tomorrow's date. No matter when you look at the site the date is always tomorrow.

There is a final warning sign that ScamWatch doesn't mention: when looking at several ads the product name quoted on the article keeps changing. I clicked on four different ads in a space of a few minutes and got four completely different product names in exactly the same article:

Even if you think this was a genuine transcript of a real conversation with Paul Hogan (which you shouldn't!) then why would he have had exactly the same conversation four times with just the product name changed? Of course he would not have... the conversation must be fake.

Reporting the scam to Facebook

This advertisement is against Facebook's Advertising Standards. Three relevant parts are:

• Fraud, scams and deceptive practices: Ads must comply with the Community Standard on fraud, scams and deceptive practices.
• Unacceptable business practices: Ads must not promote products, services, schemes or offers using identified deceptive or misleading practices, including those meant to scam people out of money or personal information.
• Circumventing Systems: Ads must not use tactics that are intended to circumvent our ad review process. This includes techniques that attempt to disguise the ad's content or destination (landing) page.

I reckon I've already established that the ad breaches those first two parts.

The third part I haven't yet mentioned but is also in breach here. The ad mostly takes me to the fake story, but in certain circumstances it will disguise it. For example if I use a VPN to make it look like I'm in the USA or if I manipulate the URL to remove some parameters the ad will instead take me to a real news website—actually 9news.com.au rather than news.com.au, but the point here is that it shows me a real news site rather than a scam site. This is likely a technique used by the scammers to make it harder for Facebook to detect the scam. This circumvention means that perhaps Facebook can't see this breach of their rule... but I can, so I report it.

Facebook provides a link on the ad to allow me to report it:

After reporting this ad I didn't hear anything for a week. Then one week later—exactly one week, down to the minute—I received the response "We didn't remove the ad". That they rejected the report exactly one week later despite it being so obviously a scam makes me highly suspicious that no-one looked at my report and so an automated response was generated instead. The response says "We use a combination of technology and human reviewers to process reports" so perhaps in some cases a human reviewer looks at the report... but in this case I suspect they did not.

Here is the full response from Facebook:

So what next?

One could argue that I don't need to do anything. It's not like I'm the only person to notice these scam ads. ABC have been covering this problem for years in articles such as "Fake celebrity scam ads hijack Facebook accounts to target Australians" and Media Watch's coverage of Facebook scams. Media Watch had the same problem as me: they reported a scam ad and were told "it does not go against our Ad Policies" despite it being an obvious scam. And that was five years ago!

Australian businessman Andrew Forrest has been depicted in similar scam ads and so has taken Meta to court over them. If he can successfully demonstrate Facebook has been negligent in publishing these scam ads then perhaps that will finally cause Facebook to stop the ads. The case hasn't yet reached trial though.

But it feels wrong wait and to do nothing; it has been five or six years now so it feels like I've ignored the scam for too long. Meta hasn't stopped it so it is up to us to do it instead. Writing this blog post is a small start for me.

I have also taken to reporting the scam ads to the Australian Government's new ScamWatch Report a scam ad website. I don't know what action the Government will be taking but I hope they can rectify Facebook's failure to take action on my scam reports.

Finally, I intend to request a review of this decision from Facebook to give them another chance to take responsibility. If possible I will provide the URL of this blog post so that they can see detailed evidence of the scam. I'll follow up with another blog post on their response...