dekhn 3 months ago

I believe it was already known that anything trained on The Pile contained references to copyrighted material from scihub. It seems unlikely that folks who chose to use these sources were completely unaware of the nature of the data. Presumably, given the urgency in the last 2-3 years to be a leader in this space, a number of shortcuts were taken.

  • nobrains 3 months ago

    Zuck did a calculation: "Does the risk of lawsuits and bad PR outweigh the benefits of being early?".

    If u remove morals from the equation, nearly every CEO would have made that same decision if in that position.

    • throw5959 3 months ago

      You talk about morals, but did you consider that they are releasing the model as open source, and given that OpenAI and others do the same, Zuck is really the only current option to have a reasonably comparable open source model? Also, did you consider that it might be more moral to create an AI model than to uphold copyright law, which actually many on this site deem immoral?

      IMHO this is a moral win on Zuck side.

Havoc 3 months ago

Stripping out the copyrights is quite damning.

There is wrongdoing and there is obvious evidence that you known what you’re doing is wrong. That really limits options on Defence

dudek1337 3 months ago

When I read about this I immediately thought about Aaron Swartz. He was persecuted for downloading copyrighted stuff, while Meta and other Corps will get a slap on a wrist. Such a sad story. And it's his death's anniversary today... RIP

asdefghyk 3 months ago

Its also reported elsewhere ( in media articles linked to by Hacker News ) they torrented copyright material. AMAZING

vivzkestrel 3 months ago

Stupid question: I have 400000 ebooks (yup pirated ones) what happens if I build an LLM with this?

  • rcakebread 3 months ago

    You'd still ask stupid questions?

  • blitzar 3 months ago

    You would have a net worth of 1bn

  • fooker 3 months ago

    Depends on the parameter count.

    Too high? Straight to jail.

    Too low? Believe it or not, straight to jail.

  • anothername12 3 months ago

    You’ll be fine. It’s like laundering money.

  • covofeee 3 months ago

    You also need $100m to train it

  • wil421 3 months ago

    Build Chappie.

  • gooosle 3 months ago

    You go to jail forever.

  • solumunus 3 months ago

    What do you imagine could happen?

palata 3 months ago

Same old story: Meta is too big to care. What will happen? A fine? Sure, they can pay.

ungreased0675 3 months ago

I would speculate this is true of all the leading commercial LLM models. Don’t have enough training data? Just steal some!

  • Havoc 3 months ago

    On true for all - you’d need to split it by era I think

    During the early Llama 1 days The Pile dataset was in heavy use by many. Bit later people figured out that a subset of it - Books 3 - was especially problematic.

    I’m guessing all the big houses threw that piece out in later models since it’s extra radioactive

    • archerx 3 months ago

      What was problematic about it?

      • Havoc 3 months ago

        Thousands of pirated copyrighted books

  • gooosle 3 months ago

    Copy some*

    • pera 3 months ago

      It depends who you are:

      - if you are an individual then it's called "pirating copyrighted work"

      - if you are a multi-billion dollar corporation then it's called "use of uncleared material for training"

      • throw5959 3 months ago

        Intent matters. I'm very happy we don't live in a world where it doesn't matter.

        • taskforcegemini 3 months ago

          one is personal use, the other is making money off of it?

          • throw5959 3 months ago

            Meta open sourced the result.

    • BSDobelix 3 months ago

      That's exactly the difference, one does not steal in the digital world. If i could download/copy a car i would do it ;)

    • dehrmann 3 months ago

      Courts have yet to decide on which it is, and it might depend on how well the model can transform vs. recite.

      • vidarh 3 months ago

        The point is that whatever courts decide, it is not theft. It may or may not be copyright infringement, but copyright infringement is not theft.

        • exe34 3 months ago

          but muh shareholders!

  • bodiekane 3 months ago

    If we're going to use absurd hyperbole like "steal", I think we should just keep going further.

    Zuckerberg murdered some old library books to train a model. Zuckerberg genocided training data!

    Heck, everyone who read your comment here stole it. I'm so sorry for your loss.

    • aprilthird2021 3 months ago

      It's not absurd hyperbole. If I took the text of, say the NYT Bestseller this week, stored the text along with various Projects Gutenberg books, then created a program to randomly deliver you a chapter from any such book. That would probably get me a lawsuit.

      This is just that with lots of levels of indirection.

rurban 3 months ago

Jail time? Or just multi-million fines.

Will he be allowed to lead Meta if convicted as criminal?

  • covofeee 3 months ago

    You saw the WP cartoon right?

  • exe34 3 months ago

    he should run for election!

cma 3 months ago

A book's copyright is no more valid than a website's

pointedAt 3 months ago

ugh, that stuff from piratebay is full of errors and "pranks"

#dataPoisoning

htrp 3 months ago

i was under the impression that almost everyone trained on books3

alightsoul 3 months ago

Given how things are going, maybe it will be ruled as "fair use" whereas something like controlled digital lending at the internet archive was ruled as "infringing" disgusting. So AI might become the only "legal" way to access a lot of knowledge for free you otherwise wouldn't have access to.

musicale 3 months ago

"I'm shocked, shocked to find out that piracy is going on here!"

"Your LLM, Captain Zuckerberg."

"Oh, thank you very much!"

udev4096 3 months ago

Everyone knows that LLMs are trained on shit ton of pirated content

atulvi 3 months ago

Good. These laws are anti progress.

  • idiotsecant 3 months ago

    We're literally extracting, refining, and re-using the information, art, and thoughts of fellow humans to make billionaires money.

    This isn't the 90s. Computing isn't about discovery, not in the big leagues. Its about grinding up authenticity and feeding it into a machine to convert it into shareholder value.

    If they want the value, let them pay for it or release the models open source for all to benefit.

    • archerx 3 months ago

      They have released all the models for free so far unlike other companies like OpenAI who are most likely doing the same but keeping it private and proprietary.

  • ulfw 3 months ago

    What "progress"?

    • pizza 3 months ago

      Exfiltration of information from the economy

      • exe34 3 months ago

        does the economy lose this information? are pages now missing from the books on your bookshelf?

        • pizza 3 months ago

          I didn't say it was correct, I was just saying what the position is

          • exe34 3 months ago

            it's okay, you can say it's wrong!

  • covofeee 3 months ago

    Copyright is your friend.

    • horsawlarway 3 months ago

      No.

      There is a theoretical implementation of copyright that is your friend.

      The realities of the laws as implemented today are abusive and hostile.

      • palata 3 months ago

        Does it mean that they should be removed entirely? Surely we can agree on the fact that I should not be allowed to make a copy of a book, put my name on it instead of the real author, and sell it? Or even claim that I wrote it and put it on my resume?

        • horsawlarway 3 months ago

          > Does it mean that they should be removed entirely?

          Maybe. I think it means we're at a spot where I'm not reasonably convinced that existing copyright laws are actually better than the free-for-all you're describing.

          I'm definitely with you that it'd be ideal if we had a way to handle direct plagiarism like described in your comment (Although if you dropped the "put my name on it" part, I don't really see much issue).

          But we also have all the fun today of companies using copyright to silence critique, shut out competitors, take educational information offline, demonetize videos they don't like, and otherwise absolutely abuse the hammers copyright law has given them (often - with no reciprocal hammer to stop this type of abuse).

          And that's not even getting into the discussion of whether or not 70+ year old characters and stories should be available for modern authors to reuse and reinterpret. (even more egregious when you consider the vast majority of those tales are direct reinterpretations of older stories themselves...).

          Or we can discuss whether it should really be legal to sell electronics hardware that has digital locks inside it that not only am I (the legal owner) not given the key to, but for which it is literally illegal for me to even attempt to open.

          ----

          So basically - If I had to pick between "You'll own nothing and rent everything, and all public discourse is subject to DMCA strikes or other removal" vs "no copyright"... My vote is for "no copyright".

          But the reality is I think we can strike a much better balance than those extremes, we just can't do it without upsetting large profit streams for existing, very wealthy and entrenched, entities... and usually that doesn't happen without tearing things down first.

    • bodiekane 3 months ago

      Copyright is the friend to the 1% and the enemy of the everyone else.

      (Of course, I'm using "the 1%" rhetorically, it's really more like 0.01%)

      As a society, we all clearly benefit from fair use far more than we benefit from members of the copyright cartel buying another mansion or private jet.