Not with a Bug but with a Sticker

The authors show us just how dumb AI systems are and how overblown the claims of computer vision investors, and LLM producers really are. Don't believe the AI hype, the systems are brittle and not that smart, everyone invested in them just wants you to think the opposite.

Purchase Not with a Bug but with a Sticker on Amazon

# Not with a Bug, But with a Sticker

- [[Ai]] systems provide confident answers with no [[self-doubt]] in situations that would cost lives. Pg XX
- like when giving medical diganosis or evaluating someone’s fitness for financial support

- adversarial machine learning Pg XX, 20
- ugh the sources for the book are on a website and at the time of reading the site page was gone because they moved it and didn’t deal with the old URL. Pg XX
- further, what happens if the domain expires or the author dies? Sources in the back of the book always exist if the book exists but online stuff is very ephemeral
- www.ram-shankar.com/notwithabug
- I emailed them when I found the dead URL and looking back a few weeks later it’s not fixed, the actual sources are here: [https://www.ram-shankar.com](https://www.ram-shankar.com/new-page)
- the sources contain no page references or anything to connect them back to specific statements so it’s on the reader to do the hard work to figure out which source was meant to go with which statement.
- LAZY Authors

### 1 - Do you want to be part of the future?

- we have deployed LLM systems all over in our haste to not be seen as “behind the times” before fully understand the impact to health care, or wherever they are deployed. Pg 4
- we don’t want to be called a [[luddite]] but Luddite Shouldn't Be a Dirty Word

- out of 415 [[Ai]]-based tools tested to diagnose COVID none were found fit for clinical use. Pg 7
- we put more trust though in the hyped capabilities of [[Ai]] than in people. Far more than it’s accuracy warrants. A person with that low a hit rate would be sued into oblivion for bad diagnosis

- The Book of Why - Judea Pearl Dana Mackenzie Pg 9
- #tagnote

National Security Commission on Artificial Intelligence

United States, [[Ai]]
- Pg 14

- You Look Like a Thing and I Love You - How Artificial Intelligence Works and Why It's Making the World a Weirder Place 210620211726 Pg 19

### 2 - Subtle Specific & Ever-Present

- adversarial machine learning examples of images don’t look any different to human perception but ML models get them totally wrong. Pg 60
- this goes for audio as well. Adversarial examples sound like static to humans but purchase products or divulge private information over a smart speaker Pg 61


- [[OpenAI]] Pg 64
- the building blocks used for [[Ai]] promote the conditions for being easily tricked. Pg 66
- a model only knows the specific data you feed it. Anything omitted will fool it
- anything that is over-represented will skew the results
- think more white people in photos making it hard for a system to recognize black people
- think the white supremist talk on [[Twitter]] that had it’s AI thing derail into racism easily

### 4 - Here’s Something I Found on the Web

- data poison attacks deliberately feed an [[Ai]] model bad data to corrupt it’s function. Pg 87
- think of the [[Reddit]] groups that got bad stuff used by Ai on purpose

- Ghost Work - Mary L Gray Siddharth Suri Pg 89

- researchers from [[MIT]] found that the top 10 datasets used to benchmark [[Ai]] are riddled with errors. Pg 90 ^b9a1d3
- like images that are identified wrong so they train the model on an incorrect identification
- cheap labour is paid by the piece thus they are incentivised for speed not accuracy.

- we are told not to trust what we read on the internet yet we often trust some [[Ai]] voice or [[ChatGPT]] output, which is fed by the internet. Pg 96
- we give it undeserved trust that we wouldn’t give random sources on the internet that it pulls from

- you can poison an ML model by messing with just 0.1% of it’s data. Pg 102

- the more complex the [[Ai]] model is the more vulnerable it is to both poisoning attacks and adversarial machine learning attacks. Pg 104


### 5 - Can You Keep a Secret

- another example of Elon Musk sounding smart but what he says falls apart as soon as someone that knows the field he’s talking about listens to his words. Pg 108
- it’s all English, and computery, but it means nothing and vastly over-simplifies the entire idea
- in this case he’s saying it’s easy to defend against adversarial machine learning but experts in the field say it’s actually impossible to defend against them
- this relates to The Death of Expertise

- #tagnote

Defenders of [[Ai]] systems only know what they intended the system to do. Attackers know how the system functions in the real world, because they don’t come with assumptions.

This is similar to when you can’t see an easy spelling mistake in your own writing, but it jumps out to you in the writing of someone else.
- Pg 109,110

- so give it a bit of time and people break the latest defences for [[Ai]] every time. Pg 118
- yes that means [[ChatGPT]] will be broken
- I wonder if this ML search crap in tools like [[Notion]] could be abused to steal data from other workspaces?
- could AI tools in WordPress divulge data from the database like passwords and usernames that shouldn’t be given out? Based on this book and the state of bad security the answer is likely yes


- CSAM Pg 120
- specifically [[Apple]]’s ill fated CSAM scanning attempt that wes later dropped
- # a16z Funded AI Platform Generated Images That “Could Be Categorized as Child Pornography,” Leaked Documents Show
#Omnivore

[Read on Omnivore](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f)
[Read Original](https://www.404media.co/a16z-funded-ai-platform-generated-images-that-could-be-categorized-as-child-pornography-leaked-documents-show/)

## Highlights

> Andreessen Horowitz [⤴️](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f#50b939d2-2c7d-4775-a922-895834f009df)

[[Andreesen Horowitz]]

> child pornography [⤴️](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f#fa04c9ee-a98e-4f39-8180-b6b818e5863d)

#tagnote

Child sexal abuse material

> After discovering Civitai was being used to generate what some OctoML employees thought could qualify as explicit images of children, OctoML ultimately decided to keep working with the company, but not advertise the relationship like it had previously. [⤴️](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f#7c322e40-9148-4bd2-ab4c-bbedf4cde710)

See it’s not about the offensive material, it’s about the marketing that you learn what they’re doing.

> Civitai’s [terms of service](https://civitai.com/content/tos?ref=404media.co) allow users to share AI models that are designed to create better AI generated pornographic images, and it allows users to share AI models designed to create better AI generated images of real people, but it forbids users from sharing images or models of nonconsensual pornography. [⤴️](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f#56097ad3-9bd6-4c7b-ab91-40305253ea0e)

This is a big tech get out of jail card, they hand wave at the TOS and then say it’s not their fault even though they designed the tool, and in this case knew it was being used for stuff that was against their TOS.

They’ll say they can’t help how people use their tool, it’s just a tool. Yet later on we see examples of other companies NOT allowing CSAM or pornographic images at all.

> Other popular AI tools on the market simply refuse to produce adult content. Leonardo AI, for example, which offers a very similar image generating service, has a moderation filter that will refuse to generate prompts like “woman nude.” ChatGPT will refuse to generate a short erotica story. When trying to generate nude images with DALL-E, we received a “content warning”: “This prompt has been blocked. Our system automatically flagged this prompt because it may conflict with our [content policy](https://www.bing.com/new/termsofuse?ref=404media.co#content-policy). More policy violations may lead to automatic suspension of your access.” [⤴️](https://omnivore.app/me/a-16-z-funded-ai-platform-generated-images-that-could-be-categor-18c6ebb651f#4e276ad2-651d-40bb-88ac-67de46f79b04)

So it is possible to vet the material, they just didn’t bother.


- this is related to the CSAM scanning or in that case the lack of scanning

- if you have an [[Ai]] model in your product you can safely assume it can be compromised regardless of the protections you added. Pg 124
- so we can safely assume that [[Notion]] AI has some way to trick it into doing stuff Notion doesn't want it to do

- normal use of [[Ai]] models and adversarial machine learning use of models are not distinguishable. Pg 131
- they usage patterns look the same so heuristics won't stop adversarial attacks

### 6 - Sailing for Adventure on the Deep Blue Sea

- when the DOD asked their special advisory board about the ability to protect [[Ai]] from foreign attack they were told the job is impossible. Pg 135
- there are no repeatable verifiable tests for Ai model security. Don't we want that before we start trusting our personal data to them? Before we start trusting parts of our lives (like medical diagnosis) to them?

- [[Microsoft]] Windows has bugs not because programmers don't know about security but because [[Microsoft]] is in a feature race with [[Apple]] and can't stop shipping new things to go back and fix old things. Pg 138
- this same scenario goes for current Ai companies. They're in a race for user adoption because of the virtuous cycle of customer capture I referred to earlier.

- [[Ai]] is used in many pretrail risk assessments helping decide if someone should get [[bail]] Pg 146
- but what about the strong racial profiling bias that has already been shown to exist in the book?
- Pg 147 shows how harmful the profiling can be

- [[Ai]] protections, protect different levels of data in a different manner. Existing methods may protect a white male face from adversarial machine learning attacks, but not protect a black woman, or women at all, from the same type of attack. Pg 150
- the example showed protection for a white male at 70% and for a black woman at 17%, and the model in question was considered "good"

- Between Truth and Power - Julie E Cohen Pg 153

- tech companies who have a bad view/experience of how to secure [[Ai]] get more say in the laws that police the technology they want to further. The people that the Ai will predominantly be used on don't get a say because they don't have the time or money to lobby for the rules they'd like to see. Pg 153
- see Scarcity 180920201044 for more an lobbying, laws and money
### 7 - The Big One

- companies are not incentivized to protect [[Ai]] systems as that doesn't make money. They are incentivized to push features out. Pg 161

- Industry Unbound - Ari Ezra Waldman Pg 165
- about how big tech undermines our privacy and changes our perception about what should be private


**Thoughts**

After reading this I don't want to put anything personal in Ai. Yet it is likely that some service I use is already putting my data in an Ai model that I'm not aware of and have no say in.

If big corporations don't have the money or incentive to protect Ai then what are medium companies doing...literally nothing because they likely don't have domain specific experts that understand the problem and could come up with anything coherent to protect the Ai models in use in the company.

Would an Ai model given the company HR manual give different information to different users based on how they asked the question? Could the boss see who is making the queries and then take action against them if they don't like the questions being asked? Based on this book I think the question is not if, but when will we get a story like this?