Brought to you by the Department of Erasing History.

      • girsaysdoom@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        8
        arrow-down
        1
        ·
        4 months ago

        It might be that someone wanted to change something that was on a website before the archive could get to it too.

        • KairuByte@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          8
          ·
          4 months ago

          That’s a ridiculous amount of effort to go through to slow down a scraper for one site, especially when that site could just be… turned off.

            • KairuByte@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              4 months ago

              Then it still makes no sense, as you being unable to take down the content means you also very likely can’t edit the content. I can’t think of a situation where you:

              1. Need content to not be scraped
              2. Need time to remove/edit that content
              3. Have access to do the above
              4. Don’t have access to pull the content immediately
              5. Have control of a large enough botnet to take down Internet Archive
              6. Don’t have a big enough botnet to take down the aforementioned content