Получение более зернистые дифференциалы от difflib (или способ последующей обработки в диф для достижения той же вещи)

Question

Загрузка this страницы и делая незначительные редактировать его, изменяя первый в этом абзаце :

Я тогда разобрать оба источника с BeauifulSoup и дифф их difflib.

url = 'https://secure.ssa.gov/apps10/reference.nsf/links/02092016062645AM' 
response = urllib2.urlopen(url) 
content = response.read() # get response as list of lines 

url2 = 'file:///Users/Pyderman/projects/temp/02092016062645AM-modified.html' 
response2 = urllib2.urlopen(url2) 
content2 = response2.read() # get response as list of lines 
import difflib 
d = difflib.Differ() 

diffed = d.compare(content, content) 

soup = bs4.BeautifulSoup(content, "lxml") 
soup2= bs4.BeautifulSoup(content2, "lxml") 
diff = d.compare(list(soup.stripped_strings), list(soup2.stripped_strings)) 
changes = [change for change in diff if change.startswith('-') or change.startswith('+')] 
for change in changes: 
    print change 

Печать изменений дает:

- The Achieving a Better Life Experience (ABLE) Act, H.R. 5771, legislation passed on December 19, 2014. It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 65 to full retirement age (FRA). This provision will apply to any individual who attains age 65 on or after December 19, 2015 (the one year anniversary of enactment of this bill). Two new Universal Text Identifiers (UTIs), UTI WCP060 and WCP061 were created to comply with this change. 
+ The Achieving a Better Life Experience (ABLE) Act, H.R. 5771, legislation passed on December 19, 2014. It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 68 to full retirement age (FRA). This provision will apply to any individual who attains age 65 on or after December 19, 2015 (the one year anniversary of enactment of this bill). Two new Universal Text Identifiers (UTIs), UTI WCP060 and WCP061 were created to comply with this change. 

Так это печать целый абзац, несмотря на очень незначительные изменения. Я полагаю, хорошо, что он показывает diff по полному абзацу, а не по предложению, но можем ли мы сделать вывод более зернистым каким-то образом? Как бы то ни было, кажется, что если я хочу выделить только текст, который изменился, Мне нужно будет провести дополнительное дельта-сравнение этих двух почти идентичных строк.

Answer 1

Вы можете использовать nltk.sent_tokenize() разделить суп строки в предложениях:

from nltk import sent_tokenize 

sentences = [sentence for string in soup.stripped_strings for sentence in sent_tokenize(string)] 
sentences2 = [sentence for string in soup2.stripped_strings for sentence in sent_tokenize(string)] 

diff = d.compare(sentences, sentences2) 
changes = [change for change in diff if change.startswith('-') or change.startswith('+')] 
for change in changes: 
    print(change) 

печатает только соответствующий приговор, где было обнаружено изменение:

- It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 65 to full retirement age (FRA). 
+ It contains a Title II provision that changes the age at which workers compensation/public disability offset ends for disability beneficiaries from age 68 to full retirement age (FRA).