Go Back   Computer Forums > General Computing > Programming
Join Computer forums Today

Thread Tools Search this Thread Display Modes
Old 09-04-2013, 05:56 AM   #1
Baseband Member
Join Date: Mar 2011
Posts: 45
Question Website visit automation script

The script needs to visit a series of game retailer websites and input some keywords into their search boxes. Then it yields the product results sorted by price in an HTML table in a separate page. The resulting HTML table is a list of the accumulated results.

The goal is to obtain a sorted overview of the total result set, sorted by price so that the cheapest version of the game (lowest price) is visible on the top of the list.

How would I begin coding this?
Which implementation languages are recommended (Javascript, AHK, Selenium, Java, C#, etc.)?

CFu is offline   Reply With Quote
Old 09-05-2013, 10:13 AM   #2
Site Team
root's Avatar
Join Date: Mar 2004
Posts: 7,872
Default Re: Website visit automation script

personally I'd use PHP.

what you want to do is run the script that should have logic as such

start script.
start data stream,
use open page function
send search variables to page. (e.g. site.com/search?keywords=search+terms)
(or your might have to send a post statement rather than using the get method -depends how their search page works)
set up a loop, search through the returned page line by line (for $line0, line<=sizeof($page), line++)
use regex to search regular expressions, (like <tr><td class="product">Product name)
use the split function to split out all the redundant stuff around the line
store your returned results in a database (with other information you require, such as data and time which can be generated locally)

destroy variables
close script

schedule a task

task = "C:\php\php.exe d:\scripts\price_scraping.php"
time = midnight each night.
I didn’t fight my way to the top of the food chain to be a vegetarian…
Im sick of people saying 'dont waste paper'. If trees wanted to live, they'd all carry guns.
"The inherent vice of capitalism is the unequal sharing of blessings; The inherent vice of socialism is the equal sharing of miseries."
root is offline   Reply With Quote

automation, visit

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off

All times are GMT -5. The time now is 08:35 AM.

Powered by vBulletin® Version 3.8.8 Beta 4
Copyright ©2000 - 2016, vBulletin Solutions, Inc.
Search Engine Friendly URLs by vBSEO 3.6.0