Complete website scraping tools (user-data)?

Cristian Level X³

August 27, 2016, 12:09 pm

Write the reason you're deleting this FAQ

Complete website scraping tools (user-data)?

I want to gather as much information as possible from one specific website. I'm not going to share the actual website because, well, it doesn't matter and I don't want to promote unnecessary stuff here.

So, my main goal is to figure out which periods of the day that website is getting more traffic, which specific write is driving more clicks to their articles as well as other general traffic related information. But the user's details and weekly/daily traffic periods are the most important.

Complete website scraping tools (user-data)?

I don't really want an SEO analysis because I can already do that pretty well, I want more of a user interaction approach analysis and just so you know we are talking about a Wordpress based website here and I have virtually no access to their Google analytics or anything on the inside.

Any good tools I can use? Paid or free, doesn't matter, I just want to scrap some good useful data.

Comments

Please login or sign up to leave a comment

Join

Lynne

Oh I am following this with great interest! I would love to do this with my websites!

Great topic Cristian.

8 years ago

Are you sure you want to delete this post?

TommyCarey

Hmmmm I know of scrapers that do a ton of stuff, but I've never heard of one that will scrape user data to figure out traffic patterns of a persons website.

I wonder if you could just use something like SEMRush to do this. You could even use Spyfu but I'm not 100% sure if they would have what you need. Both of those are paid services and are well known for what they do.

You might have to use a few different services to get exactly what you want since I don't know of any data scrapers to figure out a websites traffic

I hope that someone here has something they can share with you whether it be a third party scraper or something they custom coded

Also, have you ever through of posting a WTB to get this type of scraper to be designed? I'm sure if you hired the right person they could get this done is a day or two. In the past I've hired people to build custom scrapers for a specific website and it was fairly cheap. I think I paid $90 and it scraped together emails, business names, phone numbers, etc. But that's different from what you need

I wish I could have been more help.

8 years ago

Are you sure you want to delete this post?

Cristian

A WTB may be a good solution though I would prefer a professional tool rather than explain everything to a coder.

I can calculate the organic traffic a website may get with Ahrefs, I can use SimilarWeb to figure out overall potential traffic, but I need specific periods, like a Google Analytics chart that clearly shows me the more popular periods as well as potential patterns.

8 years ago

Are you sure you want to delete this post?

overcast

If you want to crawl the backlinks then scrapy is one tool that runs on python. And it can scrape a large number of links. And it can be adding a lot of value. You can also use this for SEO value. And then it can be then exported into Excel and other tools if possible.

You can also check out import.io. That site seems to be allowing different type of scraping. And it can be good enough for many options. I have learned that it may require a bit of effort on our part to scrape though. Because none of them are not push button scraping solutions.

7 years ago

Are you sure you want to delete this post?

Seocheckout

Complete website scraping tools (user-data)?

Complete website scraping tools (user-data)?

Comments

Please login or sign up to leave a comment