So, a user will likely never see meta keywords displayed because they're an attribute of the <meta> tag. However, if you just want all of the text contained within a document, try this:<br><br><span style="font-family: courier new,monospace;">from lxml import html</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;">tree = html.fromstring(text_of_html)</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;">all_text = tree.text_content()</span><br>
<br>And then run your regexes against all_text. One caveat is that text_content() recursively gives you all text children of the nodes, so it will pull in JavaScript contained within <script> tags. If that's a problem, you can come up with some minor hackery to pull <script> nodes out of the tree.<br>
<br>Ted<br><br><br><div class="gmail_quote">On Fri, Jul 24, 2009 at 7:30 AM, Adam Nelson <span dir="ltr"><<a href="mailto:adam@varud.com">adam@varud.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Is there a shortcut method (or even a pasted script) that allows lxml to get all<br>
the 'user-visible' text?<br>
<br>
I'm writing a screen scraper that then takes that text and looks for<br>
banned words next to an<br>
advertiser's content - and therefore I need to run a regular<br>
expression on everything a user<br>
might see (including meta keywords, etc...) but I don't care<br>
about the actual tags<br>
themselves, or urls, etc...<br>
<br>
Right now, I'm just doing the regex on the entire HTML block.<br>
<br>
Thanks,<br>
Adam<br>
<br>
_______________________________________________<br>
lxml-dev mailing list<br>
<a href="mailto:lxml-dev@codespeak.net">lxml-dev@codespeak.net</a><br>
<a href="http://codespeak.net/mailman/listinfo/lxml-dev" target="_blank">http://codespeak.net/mailman/listinfo/lxml-dev</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>Ted Dziuba<br>Co-Founder and Engineer<br><br>Milo.com, Inc.<br>165 University Avenue<br>Palo Alto, CA, 94301<br><a href="http://milo.com">http://milo.com</a><br><br>Cell: (609)-665-2639<br>
<br>