beautifulsoup - Python: Writing multiple variables to a file -
i'm new python , i've written scraper prints data scrap exact way need it, i'm having trouble writing data file. need exact same way , in same order when prints in idle
import requests import re bs4 import beautifulsoup year_entry = raw_input("enter year: ") week_entry = raw_input("enter week number: ") week_link = requests.get("http://sports.yahoo.com/nfl/scoreboard/?week=" + week_entry + "&phase=2&season=" + year_entry) page_content = beautifulsoup(week_link.content) a_links = page_content.find_all('tr', {'class': 'game link'}) link in a_links: r = 'http://www.sports.yahoo.com' + str(link.attrs['data-url']) r_get = requests.get(r) soup = beautifulsoup(r_get.content) stats = soup.find_all("td", {'class':'stat-value'}) teams = soup.find_all("th", {'class':'stat-value'}) scores = soup.find_all('dd', {"class": 'score'}) try: game_score = scores[-1] game_score = game_score.text x = game_score.split(" ") away_score = x[1] home_score = x[4] home_team = teams[1] away_team = teams[0] away_team_stats = stats[0::2] home_team_stats = stats[1::2] print away_team.text + ',' + away_score + ',', stats in away_team_stats: print stats.text + ',', print '\n' print home_team.text + ',' + home_score +',', stats in home_team_stats: print stats.text + ',', print '\n' except: pass
i totally confused on how print txt file same way prints in idle. code built run on completed weeks of nfl season. if test code, recommend year = 2014 , week = 12 (or before)
thanks,
jt
to write file need build line string, write line file.
you'd use like:
# open/create file output open('my_output_file.csv', 'wb') csv_out: ... # beautifulsoup code , parsing goes here ... # build output strings link in a_links: away_line = ",".join([away_team.text, away_score]) stats in away_team_stats: away_line += [stats.text] home_line = ",".join(home_team.text, home_score]) stats in home_team_stats: home_line += [stats.text] # write output strings file csv_out.write(away_line + '\n') csv_out.write(home_line + '\n')
this quick , dirty fix. want csv
module (docs)
Comments
Post a Comment