2017-01-02 13 views
-1

Кто-нибудь знает, как я могу очистить веб-сайты, прочитав IE списка URL-адресов из .txt, а затем напишу каждый результат url в .txt с использованием имени a из .txt. Таким образом, будут отображаться URL-адрес и файлы имен, из которых код считывает и записывает тело с соответствующей строкой из файла .txt. Самое близкое, что я нашел, находится ниже кода, однако это сохраняет все это в один .txt-файл, который является фиксированным именем, а не переменной; и чтение URL-адресов из списка. Я предполагаю, что цикл будет лучшим, но я не видел кода или много помощи для этого типа задач.python website-scraping с использованием переменных из текстовых файлов

import requests 
from bs4 import BeautifulSoup 
from collections import Counter 
urls = ["http://en.wikipedia.org/wiki/Wolfgang_Amadeus_Mozart","http://en.wikipedia.org/wiki/Golf"] 

with open('thisisanew.txt', 'w', encoding='utf-8') as outfile: 
    for url in urls: 
    website = requests.get(url) 
    soup = BeautifulSoup(website.content) 
    text = [''.join(s.findAll(text=True))for s in soup.findAll('p')] 
    for item in text: 
      print(item ,file=outfile,) 

Спасибо за вашу помощь заранее!

+0

Таким образом, вы не пробовали писать код самостоятельно, а просто ищут кто-то сделает это за вас? –

+0

Перейти на google python читать txt файл по строкам; python записывает данные в файл. – Bobby

+0

Читать 1. [ask] 2. [mcve] – MYGz

ответ

0
import requests 
from bs4 import BeautifulSoup 
from pathlib import Path 
urls = ["http://en.wikipedia.org/wiki/Wolfgang_Amadeus_Mozart","http://en.wikipedia.org/wiki/Golf"] 
for url in urls: 
    # make requests 
    response = requests.get(url) 
    soup = BeautifulSoup(response.text, 'lxml') 
    text = '\n'.join(s.get_text() for s in soup.findAll('p')) # add '\n' betweent each p 
    # write to file 
    filename = Path(url).name # get the last part of url 
    out_file = Path(filename).with_suffix('.txt') # add.txt to filename 
    with out_file.open(mode='w') as f: 
     f.writelines(text) 

из:

Golf.txt:

Golf is a club and ball sport in which players use various clubs to hit balls into a series of holes on a course in as few strokes as possible. 
Golf, unlike most ball games, does not require a standardized playing area. The game is played on a course with an arranged progression of either nine or 18 holes. Each hole on the course must contain a tee box to start from, and a putting green containing the actual hole or cup (4.25 inches in width). There are other standard forms of terrain in between, such as the fairway, rough (long grass), sand traps, and hazards (water, rocks, fescue) but each hole on a course is unique in its specific layout and arrangement. 
Golf is played for the lowest number of strokes by an individual, known as stroke play, or the lowest score on the most individual holes in a complete round by an individual or team, known as match play. Stroke play is the most commonly seen format at all levels. 

Wolfgang_Amadeus_Mozart.txt

Wolfgang Amadeus Mozart (/ˈwʊlfɡæŋ æməˈdeɪəs ˈmoʊtsɑːrt/; MOHT-sart;[1] German: [ˈvɔlfɡaŋ amaˈdeːʊs ˈmoːtsaʁt]; 27 January 1756 – 5 December 1791), baptised as Johannes Chrysostomus Wolfgangus Theophilus Mozart,[2] was a prolific and influential composer of the Classical era. 
Born in Salzburg, he showed prodigious ability from his earliest childhood. Already competent on keyboard and violin, he composed from the age of five and performed before European royalty. At 17, Mozart was engaged as a musician at the Salzburg court, but grew restless and traveled in search of a better position. While visiting Vienna in 1781, he was dismissed from his Salzburg position. He chose to stay in the capital, where he achieved fame but little financial security. During his final years in Vienna, he composed many of his best-known symphonies, concertos, and operas, and portions of the Requiem, which was largely unfinished at the time of his death. The circumstances of his early death have been much mythologized. He was survived by his wife Constanze and two sons.