archive.is is compromised. We need a new archiving strategy (Git repository)

136

Archived archive.is is compromised. We need a new archiving strategy (Git repository) (pizzagate)

submitted 1.7 years ago by newworldahead

This is infuriating: https://voat.co/v/pizzagate/1569388

We need a new strategy to ensure that the evidence we share never disappear. I suggest we create a GiT repository for pizzagate.

What is GiT? GiT is a decentralized open source version control system. You can add as many files as you want in your GiT "repository". Any change to the files in the repository is kept. Nothing is ever destroyed unless you go out of your way to destroy it. Even if something gets removed from the repository, the history of the original file is kept.

Why would it help us? Thanks to its decentralized nature, all users can have their own up to date copy of the "main" repository. This means that if the main repository is taken down, the data is not lost.

How could we use it?

archive.is allow us to download a zip archive of any page we save (see this comment https://voat.co/v/pizzagate/1569388/7648430))
You should continue saving to archive.is.
Then you download the zip file
Then you put the zip file in your local pizzagate repository
And you "push" (this is a GiT term) your changes to the main remote repository.
A group of tech saavy people should be in charge of the main repository. (to moderate it and to keep the content well organized)

Of course we would need a nicely wrtiten tutorial to help non techy users to understand how to use GiT properly.

Your thoughts?

EDIT:

Just an idea on how to organize the repository:

In the root folder, there could be a folder called "voat". This folder would contain Voat discussions (one sub folder per voat thread). The sub folder name could be the Voat post id. Inside each of these folders, you would find a file containing the voat thread in zip format (each voat thread should be arcihved on archive.is). Another sub folder called "sources" would then contain all archived evidence for this particular voat discussion (images, external web pages, etc).

EDIT 2: Automate the whole thing

As someone in the comment pointed it out, there has already been git repositories created for that purpose but they have been forgotten about.

This is true. It's because GiT is a technology not really user-friendly that is used mainly by programmers. We cannot expect that everyone will know how to use it.

This means we must find a way to automatically archive everything that gets posted to the pizzagate voat subverse. We can probably do this using the voat API. Each new thread should be automatically archived in a compressed format (like .zip) and added to the repository (the repository should contain files, NOT urls). The content of all external links in the thread should also be automatically archived.

EDIT 3: A GiT repository that automatically archive the PG subverse already exists

https://voat.co/v/pizzagate/1569702/7650190. However the problem is that only the Voat threads are archived and not the external URLs contained in the threads.

Archiving the images and external web pages that are linked in the voat threads will obviously take more disk space. Finding a reliable and trustable server that gives us the disk space and bandwidth we need might become a cost issue. We have to find a solution about that.

This is infuriating: https://voat.co/v/pizzagate/1569388

We need a new strategy to ensure that the evidence we share never disappear. I suggest we create a GiT repository for pizzagate.

What is GiT? GiT is a decentralized open source version control system. You can add as many files as you want in your GiT "repository". Any change to the files in the repository is kept. Nothing is ever destroyed unless you go out of your way to destroy it. Even if something gets removed from the repository, the history of the original file is kept.

Why would it help us? Thanks to its decentralized nature, all users can have their own up to date copy  of the "main" repository. This means that if the main repository is taken down, the data is not lost.

How could we use it?

1. archive.is allow us to download a zip archive of any page we save (see this comment https://voat.co/v/pizzagate/1569388/7648430)
2. You should continue saving to archive.is.
3. Then you download the zip file
4. Then you put the zip file in your local pizzagate repository
5. And you "push" (this is a GiT term) your changes to the main remote repository.
6. A group of tech saavy people should be in charge of the main repository. (to moderate it and to keep the content well organized)

Of course we would need a nicely wrtiten tutorial to help non techy users to understand how to use GiT properly.

Your thoughts?

EDIT:

Just an idea on how to organize the repository:

In the root folder, there could be a folder called "voat". This folder would contain Voat discussions (one sub folder per voat thread). The sub folder name could be the Voat post id. Inside each of these folders, you would find a file containing the voat thread in zip format (each voat thread should be arcihved on archive.is). Another sub folder called "sources" would then contain all archived evidence for this particular voat discussion (images, external web pages, etc).

EDIT 2: Automate the whole thing

As someone in the comment pointed it out, there has already been git repositories created for that purpose but they have been forgotten about.

This is true. It's because GiT is a technology not really user-friendly that is used mainly by programmers. We cannot expect that everyone will know how to use it.

This means we must find a way to automatically archive **everything** that gets posted to the pizzagate voat subverse. We can probably do this using the voat API. Each new thread should be automatically archived in a compressed format (like .zip) and added to the repository (the repository should contain files, NOT urls). The content of all external links in the thread should also be automatically archived.

EDIT 3: A GiT repository that automatically archive the PG subverse already exists

https://voat.co/v/pizzagate/1569702/7650190. However the problem is that only the Voat threads are archived and not the external URLs contained in the threads.

Archiving the images and external web pages that are linked in the voat threads will obviously take more disk space. Finding a reliable and trustable server that gives us the disk space and bandwidth we need might become a cost issue. We have to find a solution about that.

48 comments

want to join the discussion? or register in seconds.

Sort: Top

[–] voatsucksreally 2 points 21 points 23 points (+23|-2) 1.7 years ago

This is another reason why I built citizeninvestigators.org evidence is being censored everywhere, and we have already lost so much information in the last couple of months. The site archives information people post by tags so that anyone can go back and find information related to any topic that's been discussed quickly and easily. Check it out if you haven't seen it yet.

This is another reason why I built [citizeninvestigators.org](https://citizeninvestigators.org) evidence is being censored everywhere, and we have already lost so much information in the last couple of months. The site archives information people post by tags so that anyone can go back and find information related to any topic that's been discussed quickly and easily. Check it out if you haven't seen it yet.

permalink

[–] TRUTHWARRIOR 0 points 4 points 4 points (+4|-0) 1.7 years ago

The sandyhookjusticereport.com no longer exists due to threats made to wolfgang halbigs family. If your site is taken down, where goes the data?

The sandyhookjusticereport.com no longer exists due to threats made to wolfgang halbigs family.  If your site is taken down, where goes the data?

[–] gittttttttttttttttt 0 points 13 points 13 points (+13|-0) 1.7 years ago

git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git

Archive.is brought in cloudflare to block all external URL's from being archived which my bot was doing. I don't have the space or funds to backup all external URL's just voat URL's. All external URL's are going into a txt file though currently but that isn't good enough. Some person said they were going to send $100 bitcoin and never did. If someone wants to help me financially I will rent a VPS with more space to backup all external URL's as well. Just DM me.

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know. A lot of what was mentioned in this post is already being done.

I tried telling the community archive.is was comprimised and one of the shill mods deleted it! I don't think the mod is still here. I already have a git repo going every day on a cron job of all voat posts:  https://gitlab.com/pizzascraper/pizzagate-scraper

~~~
git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git
~~~

Archive.is brought in cloudflare to block all external URL's from being archived which my bot was doing.  I don't have the space or funds to backup all external URL's just voat URL's.  All external URL's are going into a txt file though currently but that isn't good enough.  Some person said they were going to send $100 bitcoin and never did.  If someone wants to help me financially I will rent a VPS with more space to backup all external URL's as well.  Just DM me.

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know.  A lot of what was mentioned in this post is already being done.

I tried telling the community archive.is was comprimised and one of the shill mods deleted it! I don't think the mod is still here. I already have a git repo going every day on a cron job of all voat posts:  https://gitlab.com/pizzascraper/pizzagate-scraper

~~~
git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git
~~~

Archive.is brought in cloudflare to block all external URL's from being archived which my bot was doing.  I don't have the space or funds to backup all external URL's just voat URL's.  All external URL's are going into a txt file though currently but that isn't good enough.  Some person said they were going to send $100 bitcoin and never did.  If someone wants to help me financially I will rent a VPS with more space to backup all external URL's as well.  Just DM me.

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know.  A lot of what was mentioned in this post is already being done.

4 replies

[–] voatsucksreally 0 points 0 points 0 points (+0|-0) 1.7 years ago

I never heard of that site or heard about threats made to them, when did that happen? I backup the entire database multiple times a day. If the site gets taken down, the data won't be lost. The only instance where data will be lost is if users post it as a link via a third party site (like archive.is) and it is then taken from the 3rd party. So users will need to make sure they are uploading the content to the site, and not just linking to it externally. Other than that, we will also need to rely on everyone making backups of their information externally/locally.

I never heard of that site or heard about threats made to them, when did that happen?  I backup the entire database multiple times a day. If the site gets taken down, the data won't be lost. The only instance where data will be lost is if users post it as a link via a third party site (like archive.is) and it is then taken from the 3rd party. So users will need to make sure they are uploading the content to the site, and not just linking to it externally. Other than that, we will also need to rely on everyone making backups of their information externally/locally.

1 reply

[–] 4745764567 0 points 7 points 7 points (+7|-0) 1.7 years ago

This is all pointless. You need to store it in a decentralized manner.

IPFS can host files\pics. Zeronet can host websites. Torrents can host data and bitchute can stream video torrents.

Stop relying on other peoples computers to host this stuff. Use decentralized tech or don't whine when it disappears.

This is all pointless. You need to store it in a decentralized manner. 

IPFS can host files\pics. Zeronet can host websites. Torrents can host data and bitchute can stream video torrents.

Stop relying on other peoples computers to host this stuff. Use decentralized tech or don't whine when it disappears.

permalink

[–] 144SansRetour 0 points 0 points 0 points (+0|-0) 1.5 years ago

Does anyone know how to create a Zeronet site? This could be used as a decentralized p2p of ALL pizzagate related files. Then could we have the mods of /v/pizzagate link to it from the sidebar and executive summary?

Does anyone know how to create a Zeronet site? This could be used as a decentralized p2p of ALL pizzagate related files. Then could we have the mods of /v/pizzagate link to it from the sidebar and executive summary?

[–] 144SansRetour 0 points 0 points 0 points (+0|-0) 1.5 years ago

Also don't forget Bitmessage! it's a VERY easy to use messenger that has chans-- each participant of a chan receives redundant copies of all documents, so it's a highly decentralized method of disseminating and backing up information. Also, images can be encoded in html Base64 format that is recognized within the Bitmessage client and displayable as an image.

Also don't forget Bitmessage! it's a VERY easy to use messenger that has chans-- each participant of a chan receives redundant copies of all documents, so it's a highly decentralized method of disseminating and backing up information. Also, images can be encoded in html Base64 format that is recognized within the Bitmessage client and displayable as an image.

[–] 0000000001 0 points 0 points 0 points (+0|-0) 1.6 years ago

IPFS can do video, images and markdown sites. I vote IPFS.

IPFS can do video, images and markdown sites. I vote IPFS.

[–] pizzagate4real 0 points 5 points 5 points (+5|-0) 1.7 years ago

There have already been several git repositories created for this purpose...they always seem to get lost/forgotten about...

There have already been several git repositories created for this purpose...they always seem to get lost/forgotten about...

permalink

[–] newworldahead [S] 0 points 2 points 2 points (+2|-0) 1.7 years ago

You're right. This needs to be automated then. Voat has an API. I haven't check it, but perhaps it would be possible to use it to archive threads as well as the links it contains and put it all on the GiT repository... Hmm, I'll do some research on this.

You're right. This needs to be automated then. Voat has an API. I haven't check it, but perhaps it would be possible to use it to archive threads as well as the links it contains and put it all on the GiT repository... Hmm, I'll do some research on this.

[–] gittttttttttttttttt 0 points 1 points 1 points (+1|-0) 1.7 years ago

Voat API won't work. They aren't giving out keys last time I checked. Check my comment. You are speaking about what I already did.

Voat API won't work.  They aren't giving out keys last time I checked.  Check my comment.  You are speaking about what I already did.

[–] Paranoidsbible 0 points 5 points 5 points (+5|-0) 1.7 years ago

I wouldn't say compromised. More like receiving multiple DMCA requests and possible legal threats. If a site is hosted in the US, they have to take great pains in following the law or face a lot of fine. It's a good idea, though, to make multiple backups. Including the age old CTRL+S to save copies to your computer itself.

I wouldn't say compromised. More like receiving multiple DMCA requests and possible legal threats. If a site is hosted in the US, they have to take great pains in following the law or face a lot of fine. It's a good idea, though, to make multiple backups. Including the age old CTRL+S to save copies to your computer itself.

permalink

[–] newworldahead [S] 0 points 0 points 0 points (+0|-0) 1.7 years ago

You're right. I should not have written "compromised". Should have written something along the lines of "not trustable" instead (even if they are not directly to blame). Too bad I cannot edit my post title.

You're right. I should not have written "compromised". Should have written something along the lines of "not trustable" instead (even if they are not directly to blame). Too bad I cannot edit my post title.

[–] tjarco 0 points 4 points 4 points (+4|-0) 1.7 years ago

please read my comment to the 'infuriating' thread https://voat.co/v/pizzagate/1569388/7648766

please read my comment to the 'infuriating' thread https://voat.co/v/pizzagate/1569388/7648766

permalink

[–] newworldahead [S] 0 points 2 points 2 points (+2|-0) 1.7 years ago

I read your comment and still think that this is infuriating (without the quotes).

I read your comment and still think that this is infuriating (without the quotes).

[–] tjarco 0 points 1 points 1 points (+1|-0) 1.7 years ago

haha, I too want to believe this is happening, just lot of question marks. The archived content of OP doesn't by far seem as incriminating as lots of other things archived which made me question the validity of this entire thread...

haha, I too want to believe this is happening, just lot of question marks. The archived content of OP doesn't by far seem as incriminating as lots of other things archived which made me question the validity of this entire thread...

1 reply

[–] gittttttttttttttttt 0 points 3 points 3 points (+3|-0) 1.7 years ago (edited 1.7 years ago)

git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know. A lot of what was mentioned in this post is already being done.

~~~
git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git
~~~

Archive.is brought in cloudflare to block all external URL's from being archived which my bot was doing.  I don't have the space or funds to backup all external URL's just voat URL's.  All external URL's are going into a txt file though currently but that isn't good enough.  Some person said they were going to send $100 bitcoin and never did.  If someone wants to help me financially I will rent a VPS with more space to backup all external URL's as well.  Just DM me.

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know.  A lot of what was mentioned in this post is already being done.

I tried telling the community archive.is was comprimised and one of the shill mods deleted it! I don't think the mod is still here. I already have a git repo going every day on a cron job of all voat posts:  https://gitlab.com/pizzascraper/pizzagate-scraper

~~~
git clone https://gitlab.com/pizzascraper/pizzagate-scraper.git
~~~

Archive.is brought in cloudflare to block all external URL's from being archived which my bot was doing.  I don't have the space or funds to backup all external URL's just voat URL's.  All external URL's are going into a txt file though currently but that isn't good enough.  Some person said they were going to send $100 bitcoin and never did.  If someone wants to help me financially I will rent a VPS with more space to backup all external URL's as well.  Just DM me.

Non tech users can download a full zip without cloning

https://gitlab.com/pizzascraper/pizzagate-scraper/repository/archive.zip?ref=master

If you have questions or suggestions let me know.  A lot of what was mentioned in this post is already being done.

permalink

[–] newworldahead [S] 0 points 0 points 0 points (+0|-0) 1.7 years ago

This is awesome! I think I stumbled into your repository in the past but I didn't know it was still active. The only very important things missing are the external URLs, it needs to be done, but I understand the cost issue. GitLab has a limit of 10 gig, not so bad. If we forget about archiving youtube videos, I wonder how long would it take to reach this amount.

This is awesome! I think I stumbled into your repository in the past but I didn't know it was still active. The only very important things missing are the external URLs, it needs to be done, but I understand the cost issue. GitLab has a limit of 10 gig, not so bad. If we forget about archiving youtube videos, I wonder how long would it take to reach this amount.

[–] gittttttttttttttttt 0 points 2 points 2 points (+2|-0) 1.7 years ago

Thanks. It was on the sidebar but sidebar has been cleaned up. It is still in the resources link https://voat.co/v/pizzagate/1517819 .

Yes gitlab has a 10gb limit. We are currently at 376M in archives/ folder. I am not sure if they count the .git objects which brings us to 677mb. I am zipping the post which is reducing file size by 90%. If we do external links we will increase size by a lot since there is a lot more external links in comments etc then there is voat posts. I can set up a git server as well. I just need the funds. I have the expertise. We may even need to use something other than git. Git starts lagging hard once it hits gigabytes. I encourage everyone to backup locally everything you see that is important.

Also do not forget this repo. Thousands of important files are in it.

https://gitlab.com/lurduyor/pizzagate

git clone https://gitlab.com/lurduyor/pizzagate

download link for non tech users or people who want faster downloads:

https://gitlab.com/lurduyor/pizzagate/repository/archive.zip?ref=master

Thanks.  It was on the sidebar but sidebar has been cleaned up.  It is still in the resources link https://voat.co/v/pizzagate/1517819 .

Yes gitlab has a 10gb limit.  We are currently at 376M in archives/ folder.  I am not sure if they count the .git objects which brings us to 677mb.  I am zipping the post which is reducing file size by 90%.  If we do external links we will increase size by a lot since there is a lot more external links in comments etc then there is voat posts.  I can set up a git server as well.  I just need the funds.  I have the expertise.  We may even need to use something other than git.  Git starts lagging hard once it hits gigabytes.  I encourage everyone to backup locally everything you see that is important.

Also do not forget this repo.  Thousands of important files are in it.

https://gitlab.com/lurduyor/pizzagate

~~~
git clone https://gitlab.com/lurduyor/pizzagate
~~~

download link for non tech users or people who want faster downloads:

https://gitlab.com/lurduyor/pizzagate/repository/archive.zip?ref=master

Thanks.  It was on the sidebar but sidebar has been cleaned up.  It is still in the resources link https://voat.co/v/pizzagate/1517819 .

Yes gitlab has a 10gb limit.  We are currently at 376M in archives/ folder.  I am not sure if they count the .git objects which brings us to 677mb.  I am zipping the post which is reducing file size by 90%.  If we do external links we will increase size by a lot since there is a lot more external links in comments etc then there is voat posts.  I can set up a git server as well.  I just need the funds.  I have the expertise.  We may even need to use something other than git.  Git starts lagging hard once it hits gigabytes.  I encourage everyone to backup locally everything you see that is important.

Also do not forget this repo.  Thousands of important files are in it.

https://gitlab.com/lurduyor/pizzagate

~~~
git clone https://gitlab.com/lurduyor/pizzagate
~~~

download link for non tech users or people who want faster downloads:

https://gitlab.com/lurduyor/pizzagate/repository/archive.zip?ref=master

1 reply

[–] lawexaminer 0 points 3 points 3 points (+3|-0) 1.7 years ago

Look at how wikileaks deals with this problem: http://wikileaks.openanthropology.org/Mirrors.html

The investigation needs to do the same - employ a mirrored backup system.

Look at how wikileaks deals with this problem: http://wikileaks.openanthropology.org/Mirrors.html

The investigation needs to do the same -  employ a mirrored backup system.

permalink

[–] newworldahead [S] 0 points 0 points 0 points (+0|-0) 1.7 years ago

Yes, this is great. Thanks for sharing that. I think in our case it's the GiT repository itself that should be mirrored on different servers. This would be much better than asking Voat users to keep an up-to-date copy of the repository on their own computer.

Yes, this is great. Thanks for sharing that. I think in our case it's the GiT repository itself that should be mirrored on different servers. This would be much better than asking Voat users to keep an up-to-date copy of the repository on their own computer.

[–] lawexaminer 0 points 1 points 1 points (+1|-0) 1.7 years ago

Seafile is a GIT based file syncing application https://www.seafile.com/

Seafile is a GIT based file syncing application  https://www.seafile.com/

[–] EQJ 0 points 2 points 2 points (+2|-0) 1.7 years ago

This is a comment in the original thread. Looks like someone was able to access successfully [–] humongadunga 0 points (+0|-0) 1.4 hours ago (edited 59 minutes ago) The archives might be down but the original URLs are still up. I just made image captures of the contents on these web pages, will be uploading shortly.

Edit: Now Uploaded

https://sli.mg/Y3pa1b -- Tony Podesta | Facebook

https://sli.mg/f0vCRD

https://sli.mg/uu5rOD -- Page Not Found | Facebook

https://sli.mg/dAwdf7 -- David Halperin - A new generation of leadership....

https://sli.mg/soa6z9 -- Chilling eating a cucumber from the garden. - Oliver Friendly

This is a comment in the original thread.  Looks like someone was able to access successfully
[–] humongadunga 0 points (+0|-0) 1.4 hours ago  (edited 59 minutes ago)
The archives might be down but the original URLs are still up. I just made image captures of the contents on these web pages, will be uploading shortly.

Edit: Now Uploaded

https://sli.mg/Y3pa1b -- Tony Podesta | Facebook

https://sli.mg/f0vCRD

https://sli.mg/uu5rOD -- Page Not Found | Facebook

https://sli.mg/dAwdf7 -- David Halperin - A new generation of leadership....

https://sli.mg/soa6z9 -- Chilling eating a cucumber from the garden. - Oliver Friendly

permalink

[–] birthdaysuit11 0 points 1 points 1 points (+1|-0) 1.7 years ago

WTF, these people are screwing these babies. The FINDERS never died, they groom these children and use them for sex purposes. I mean look at some of the freaking comments, it's obvious.

WTF, these people are screwing these babies. The FINDERS never died, they groom these children and use them for sex purposes. I mean look at some of the freaking comments, it's obvious.

load more comments ▼ (10 remaining)

Menu

Sets

Subverses

pizzagate

SEARCH

Pizzagate Subverse Network

For Newcomers

Submission Rules

Advertisement

MODERATORS

Subverse Rules

ABOUT

Preview

Sort: Top

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

share a link

pizzagate

SEARCH

Pizzagate Subverse Network

For Newcomers

Submission Rules

Advertisement

MODERATORS

Subverse Rules

ABOUT

Preview

Sort: Top

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview

Preview