python 3.x - How to get certain text from a url links -

-

so im trying statistics in statistics box page on url page each team. example of page looks on hyperlink put below. im trying have if prints out;

month : win % month : win % time: win%

but not sure how write code, since last piece of code wrote in main giving me error.

http://www.gosugamers.net/counterstrike/teams/16448-nasty-gravy-runners

    import time     import requests     bs4 import beautifulsoup       def get_all(url, base):  # called print team links         r = requests.get(url)         page = r.text          soup = beautifulsoup(page, 'html.parser')          team_links in soup.select('div.details h3 a'):             members = int(team_links.find_next('th', text='members:').find_next_sibling('td').text.strip().split()[0])             if members < 5:                 continue             yield base + team_links['href']          next_page = soup.find('div', {'class': 'pages'}).find('span', text='next')           while next_page:             # gives server break             time.sleep(0.2)              r = requests.get(base_url + next_page.find_previous('a')['href'])             page = r.text             soup = beautifulsoup(page)             team_links in soup.select('div.details h3 a'):                 yield base_url + team_links['href']             next_page = soup.find('div', {'class': 'pages'}).find('span', text='next')       if __name__ == '__main__':          base_url = 'http://www.gosugamers.net'         url = 'http://www.gosugamers.net/counterstrike/teams'          links in get_all(url, base_url): # when run generate links teams            r = requests.get(links)            page = r.content            soup = beautifulsoup(page)             statistics in soup.select('div.statistics tr'):                win_rate = int(statistics.find('th', text='winrate:').find_next_sibling('td'))                print(win_rate)

not sure want team stats:

from bs4 import beautifulsoup, tag import requests  soup = beautifulsoup(requests.get("http://www.gosugamers.net/counterstrike/teams/16448-nasty-gravy-runners").content)  table = soup.select_one("table.stats-table") head1 = [th.text.strip() th in table.select("tr.header th") if th.text] head2 = [th.text.strip() th in table.select_one("tr + tr") if isinstance(th, tag)] scores = [th.text.strip() th in table.select_one("tr + tr + tr") if isinstance(th, tag)]  print(head1, head2, scores)

output:

([u'jun', u'may', u'all time'], [u'winrate:', u'0%', u'0%', u'0%'], [u'matches played:', u'0 / 0 / 0', u'0 / 0 / 0', u'0 / 0 / 0'])

Comments

Post a Comment

Popular posts from this blog

wordpress - (T_ENDFOREACH) php error -

-
creating wordpress website , i'm getting following php error: parse error: syntax error, unexpected 'endforeach' (t_endforeach) in /home/soralmedia/public_html/wp-content/themes/soral media/woocommerce/myaccount/my-orders.php on line 98 here's code file: <?php /** * orders * * shows recent orders on account page. * * template can overridden copying yourtheme/woocommerce/myaccount/my-orders.php. * * however, on occasion woocommerce need update template files , (the theme developer). * need copy new files theme maintain compatibility. try this. * little possible, happen. when occurs version of template file will. * bumped , readme list important changes. * * @see http://docs.woothemes.com/document/template-structure/ * @author woothemes * @package woocommerce/templates * @version 2.5.0 */ if ( ! defined( 'abspath' ) ) { exit; } $my_orders_columns = apply_filters( 'woocommerce_my_account_my_orders_columns', array( ...
Read more

Export Excel workseet into txt file using vba - (text and numbers with formulas) -

-
i new vba have developed code use in excel move text/numbers txt file. issue have 3 functions need go same txt file. when run first function has set fs = createobject("scripting.filesystemobject") set = fs.createtextfile(module1.nys45uploadfilename, true) a.writeline (str) a.close. the other 2 functions had save. set fs = createobject("scripting.filesystemobject") set = fs.savetextfile(module1.nys45uploadfilename, true) a.writeline (str) a.close. the first function go txt file, other 2 error object doesn't support property or method. when changed 3 create last function goes txt file. can not figure out word use in order make other 2 functions follow txt file in order keyed. the entire code follows private sub addrecord_click() on error goto errhandler module1.nys45uploadfilename = application.getsaveasfilename(filefilter:="textfiles (*.txt), *.txt") if module1.nys45uploadfilename = "false" exit sub ...
Read more

Using django-mptt to get only the categories that have items -

-
sample structure: all uppercase: category mixed case: item root ├── books │   ├── fiction │   │   └── classics │   └── non-fiction ├── clothing └── electronics ├── laptops ├── phones │   ├── apple │   │   ├── iphone 6 │   │   ├── iphone 6 plus │   │   ├── iphone 6s │   │   └── iphone 6s plus │   ├── motorola │   │   ├── moto g4 │   │   ├── moto g4 play │   │   ├── moto g4 plus │   │   └── moto x │   └── samsung └── tablets └── apple i'm trying have index page show "electronics" category. "clothing" category shouldn't show since it's empty , "books" category shouldn't show either since, although has child categories, child categories don't have items. similarly, "electronics" page should show "phones" category. "laptops" category shouldn't show since it's empty , "tablets" category shouldn't show either since...
Read more