From: Oleg Broytman <phd@phdru.name>
Date: Fri, 7 Mar 2008 20:56:06 +0000 (+0000)
Subject: Split the title into subparts, reassemble the subparts and recode.
X-Git-Tag: v4.5.3~168
X-Git-Url: https://git.phdru.name/?a=commitdiff_plain;h=fd325219011aaed865f074275b2270112b5e9732;p=bookmarks_db.git

Split the title into subparts, reassemble the subparts and recode.


git-svn-id: file:///home/phd/archive/SVN/bookmarks_db/trunk@208 fdd5c36f-1aea-0310-aeeb-c58d7e2b6c23
---

diff --git a/Robots/parse_html_beautifulsoup.py b/Robots/parse_html_beautifulsoup.py
index 8f82d24..db291f9 100644
--- a/Robots/parse_html_beautifulsoup.py
+++ b/Robots/parse_html_beautifulsoup.py
@@ -92,7 +92,12 @@ def parse_html(filename, charset=None, log=None):
    elif title.string:
       title = title.string.encode(_charset)
    else:
-      title = str(title)
+      parts = []
+      for part in title:
+         if not isinstance(part, basestring):
+            part = unicode(part)
+         parts.append(part.strip())
+      title = ''.join(parts).encode(_charset)
 
    meta = head.find(_find_contenttype, recursive=False)
    if meta: