0
7

[–] thoughtpolice [S] 0 points 7 points (+7|-0) ago 

@Crensch @Hypnogoat @kevdude @NeedleStack @thoughtpolice

post bot for politicalnews is now posting. For now, only Associated Press links. The bot also generates the archive link and posts the comment with the link.

Any suggestions or ideas are welcomed. will try to add Reuters and other sources during the week.

[–] [deleted] 0 points 2 points (+2|-0) ago 

[Deleted]

0
3

[–] thoughtpolice [S] 0 points 3 points (+3|-0) ago 

Yes please! I added the words 'MILEY, CYRUS, KANYE, RIHANA' to the filter, no kidding.

Maybe I should add 'MTV' too.

0
2

[–] kevdude 0 points 2 points (+2|-0) ago 

Wow. You are the man.

[–] [deleted] 0 points 1 points (+1|-0) ago 

[Deleted]

0
3

[–] thoughtpolice [S] 0 points 3 points (+3|-0) ago  (edited ago)

EDIT: changed screenshot: http://i.imgur.com/CN6EIdD.png (deleted personal info)

  1. List of items to post, can be videos or news articles.

  2. A web-browser control to check out the items on list (1). You just double-click an item and it opens in here.

  3. A web-browser control to steal cloud fare cookies from. Also does double duty displaying the last posted item, as in the picture.

Those text-boxes at the bottom are to post any YouTube video. You paste the link in the first text-box, press the "Get title" button and you get the YouTube video title formatted for posting, all you have to do is copy paste into VOAT to post manually.

The toolbar at the top has a lot of buttons to do stuff. The first button starts posting every 60 seconds or the interval you set it to.

The check-boxes top right are to post to /v/botcamp when you are testing stuff (boot camp for bots) and to use a proxy server. (I use Fiddler to analyze web traffic)

0
2

[–] thoughtpolice [S] 0 points 2 points (+2|-0) ago  (edited ago)

The VOAT api? No, I am not using the voat api to post.

Basically I started the bot to scan youtube channels for news videos (using the youtube API) and then posting them in /v/politicalnewsvideos.

To post, i was basically simulating mouse clicks and keyboard activity, since the cloudfare check makes things complicated. Of course it was an ugly hack.

Now I managed to work around the cloudflare check, and I am using direct winhttp calls to post submissions and comments to VOAT, no more hacks.

To get the videos from youtube, as I said, youtube api and winhttp.

To get news from AP, winhttp and I just crawl the site http://bigstory.ap.org/ . I tried using their RSS feed but ended up just using their home page.

I parse the source and extract the news articles programatically.

Now I plan to do Reuters and others in the same way, unless you have better ideas.

Regarding the cloudflare check, I am using a webbrowser control to do the javascript work for me, when the webbrowser control is done, I steal the http only cookies from it using a call to wininet. Then I use that cookie data in my own calls to winhttp from there on. The cookies have the javascript validation data that cloudflare wants.

Before every request I just reload the webbrowser control and steal his cookies again. (I am a cookie thief)

Kind of a hack, but it works and it was easy and I can treat the cloudflare validation as a black box. There is some sample code on the web to do the parsing and calculations to do the cloudflare checks programatically, but I was lazy and went for the lowest effort in this case.

What I learned is why they still have the cloudflare checks still up, but that is another story, let's just say that interesting things can be done.

0
6

[–] thoughtpolice [S] 0 points 6 points (+6|-0) ago 

@Crensch @Hypnogoat @kevdude @NeedleStack

Guys if you see any article posted by the bot that you judge not worthy just go ahead and mod delete it.

You can recognize the bot posts from the comment with the archive link, that says at the bottom:

This comment has been posted by the thoughpolice post bot

I am sure the post bot will not get offended.

0
1

[–] kevdude 0 points 1 points (+1|-0) ago 

Please don't. I don't want to give anyone the sense that we are deleting things. How often will the bot post? Is there a way to slow it down as we get more users?

0
4

[–] thoughtpolice [S] 0 points 4 points (+4|-0) ago 

Yes, of course. The whole idea of the bot is just to help get content until humans take over.

Right now I set it up to post every 5 minutes, but everything can be regulated.

No problem with the deletions, I will do it myself, just send me a pm or mention me in a post comment and I will delete it. that way it will not go to the delete log.

0
5

[–] thoughtpolice [S] 0 points 5 points (+5|-0) ago 

@Crensch @Hypnogoat @kevdude @NeedleStack @thoughtpolice

The bot now crawls the following:

  • ASSOCIATED PRESS
  • BBC
  • REUTERS

I am approving manually what gets posted from the list that the bot maintains.

0
1

[–] kevdude 0 points 1 points (+1|-0) ago 

Sounds good. Let us know if we can do anything to help.

0
4

[–] thoughtpolice [S] 0 points 4 points (+4|-0) ago 

@Crensch @Hypnogoat @kevdude @NeedleStack

The bot is posting again, I have been sick for the last couple of days so basically I was in bed 95% of the time

The site changed some settings for posting so the bot broke. I don't know why they did that, the name of some of the variables changed, for example "user" to "username". Are they trying to break the bots on purpose? I will code around this by getting the variable names dinamically from the post page, but when I feel better. For now I just changed the code to use the new names.

I added a word and url filter, it filters by words in the title and words in the url. It has a white list and a black list For example, black listed:

any url that contains any of those strings will not get posted. there are more, those are just examples.

Feel free to suggest more words and url parts for the filters.

0
4

[–] thoughtpolice [S] 0 points 4 points (+4|-0) ago 

@Crensch @Hypnogoat @kevdude

Subverse news for 2015-08-25:

Today we had a 50% increase in subscribers, from 20 to 30.

In the last two days, we had a 100% increase in subscribers, from 15 to 30.

Yay!

It would be a good idea to spend five minutes a day in /v/introductions recommending the sub to newcomers who may be interested.

We are doing great! ;-)

0
3

[–] kevdude 0 points 3 points (+3|-0) ago 

@scrotums @Dereliction @ChillyHellion @Mickgoestojail @England4 @Imapopulistnow

Hello you have been pinged based on comments you made in one (or more) of these threads: https://voat.co/v/news/comments/500177 , https://voat.co/v/news/comments/469378 , https://voat.co/v/news/comments/476023 . Based on your opinions regarding rules we thought you might be interested in this sub, v/politicalnews. We do not have excessive moderation. If it is topical, and related to news or politics, we want you to post it. We have a ton of content as, in addition to user posts, we make use of a bot to aggregate news sources. The only stuff we delete is spam, dox, and illegal stuff. If you have any questions or comments regarding the bot (posts under u/thoughtpolice) please share them. This isn’t reddit, so no safe spaces. Use your votes because you are basically in charge. We hope you will subscribe and help us grow this alternative to reddit-style censorship! (If you are already subscribed we apologize for bugging you).

[–] [deleted] 0 points 3 points (+3|-0) ago  (edited ago)

[Deleted]

0
1

[–] thoughtpolice [S] 0 points 1 points (+1|-0) ago 

Just found a bug in the filtering routine too. I forgot to do anything when a listed string is found in the url. Sorry!

0
1

[–] thoughtpolice [S] 0 points 1 points (+1|-0) ago 

Delete everything. If you want, just to make things clear, maybe leave a message stating something like:

DELETED DUE TO BOT FILTER ERROR

Or something like that.

0
1

[–] thoughtpolice [S] 0 points 1 points (+1|-0) ago 

Yes I noticed the same thing too. And there is no easy way to filter them by url, since there is no /sports/ in the url or anything that gives you a hint.

I will configure AP as non default so the articles have to be approved by me to get posted.

[–] [deleted] 0 points 1 points (+1|-0) ago 

[Deleted]

[–] [deleted] 0 points 0 points (+0|-0) ago 

[Deleted]

0
3

[–] thoughtpolice [S] 0 points 3 points (+3|-0) ago 

@Crensch @Hypnogoat @kevdude @NeedleStack

These kind of comments that we get in some of the posts:

  • Politicalnews
  • this isn't political OR news.
  • How the hell is this political news?
  • So.... This is politically relevant?

How do we handle this? Ignore? Take the chance to explain what the sub is about? Other ideas?

My first impulse of course is to reply "FUCK YOU!" but we all know that is not polite.

Links for the interested:

I am not judging, but can't these people just move on when something does not interest them?

0
1

[–] kevdude 0 points 1 points (+1|-0) ago 

They might be trying to start a conversation/ Or just ignore.

[–] [deleted] 0 points 1 points (+1|-0) ago 

[Deleted]

0
3

[–] thoughtpolice [S] 0 points 3 points (+3|-0) ago 

Thanks, well, maybe I will just ignore for now.

load more comments ▼ (26 remaining)