Answers:
Question 1: the class of the
span
is a list, you need to check ifexpensiver
is inside this list. In other words, replace:if percent_difference_title == "expensiver"
with:
if "expensiver" in percent_difference.span['class']
- Questions 2 and 3: you need to pass a list of column values to
writerow()
, not string. And, since you want only one record per city, callwriterow()
outside of the loop (over thetr
s).
Other issues:
- open
csv
file for writing before the loop - use
with
context managers while working with files - try to follow
PEP8
style guide
Here's the code with modifications:
import requests
import csv
from bs4 import BeautifulSoup
BASE_URL = 'http://www.expatistan.com/cost-of-living/comparison/{home_city}/{city}'
home_city = 'Phoenix'
with open('City.txt') as input_file:
with open("Expatistan.csv", "w") as output_file:
writer = csv.writer(output_file)
writer.writerow(["City", "Food", "Housing", "Clothes", "Transportation", "Personal Care", "Entertainment"])
for line in input_file:
city = line.strip()
url = BASE_URL.format(home_city=home_city, city=city)
soup = BeautifulSoup(requests.get(url).text)
table = soup.find("table", class_="comparison")
differences = []
for title in table.find_all("tr", class_="expandable"):
percent_difference = title.find("th", class_="percent")
if "expensiver" in percent_difference.span['class']:
differences.append('+' + percent_difference.span.string)
else:
differences.append('-' + percent_difference.span.string)
writer.writerow([city] + differences)
For the City.txt
containing just one new-york-city
line, it produces Expatistan.csv
with the following content:
City,Food,Housing,Clothes,Transportation,Personal Care,Entertainment
new-york-city,+48%,+129%,+63%,+43%,+42%,+42%
Make sure you understand what changes have I made. Let me know if you need further help.