Question

On a coupon site someone posted a shell script for finding Godaddy discount codes.

1 - Could someone explain how this script works?

Specifically, I'm confused about the syntax:

links url -dump | grep AI

2 - Does shell scripting allow you to spider a site just as perl/python/ruby would?

3 - Is the most efficient way to accomplish the desired goal or would perl/python/ruby be a more effective technology to use for this task?

4 - Is this ethical/legal?

#!/bin/sh

gdaddy=600
while [ "$gdaddy" -lt "700" ]
do

for i in a b c d e f g h i j k l m n o p q r s t u v w x y z
do
echo "The results for gdr0$gdaddy"a"$i" >> output
links http://www.godaddy.com/default.aspx?isc=gdr0$gdaddy"a"$i -dump | grep -A1 "SPECIAL OFFER" >> output
echo >> output
echo >> output
done

gdaddy=`expr $gdaddy + 1`
done
Was it helpful?

Solution

1. links is a text-based web browser. The -dump command makes links output the text of the web page to the terminal, and the following grep command outputs any line that contains the words "SPECIAL OFFER" and the following line (-A1 means "and 1 line After that").

2. You can spider a site using shell scripting, by using links or similar to fetch the web pages and output their URLs. (I've done this, for a website spell checker script.)

3. Use whatever tools you're happiest with. Personally I prefer Python for this kind of thing, but as I say, I've used shell scripting to do it.

4. Legal? Ask a lawyer. Ethical? Ask your conscience.

OTHER TIPS

Legal and Ethical

  • Assuming you are in the U.S., there aren't any laws restricting the access of a website by a script such as yours.
  • Those pages are not referenced in robots.txt.
  • And for godaddy in particular, it's not an ethical problem... When I swapped my registration service over to them I called their sales number, told them what I wanted to do, and they told me on the phone the best code to use.
  1. Dump the content returned for URLs, where the last letter is substituted for a-z, and find a line in it containing "SPECIAL OFFER". Pad it with newline.

  2. Yes, with utilities such as links, wget, telnet.

  3. It is good enough for not demanding things such as this (traversing a small set of URLs)

  4. That's up to site's terms of service and your legislation.

Legality relates to where you live. Consult a legal professional.

Ethical - if you have to ask, it's not. =)

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top