TIDoS-Framework/lib/mechanize_mod/test/test_html.doctest at master · securityproject/TIDoS-Framework · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
>>> import mechanize
>>> from mechanize._response import test_html_response
>>> from mechanize._html import Factory


Link text parsing

>>> def get_first_link_text(html):
...     factory = Factory()
...     response = test_html_response(html)
...     factory.set_response(response)
...     return list(factory.links())[0].text

Whitespace gets compressed down to single spaces.  Tags are removed.

>>> html = ("""\
... <html><head><title>Title</title></head><body>
... <p><a href="http://example.com/">The  quick\tbrown fox jumps
...   over the <i><b>lazy</b></i> dog </a>
... </body></html>
... """)
>>> get_first_link_text(html)
u'The quick brown fox jumps over the lazy dog'

Empty <a> links have empty link text

>>> html = ("""\
... <html><head><title>Title</title></head><body>
... <p><a href="http://example.com/"></a>
... </body></html>
... """)
>>> get_first_link_text(html)
''

>>> html = ("""\
... <html><head><title>Title</title></head><body>
... <p><iframe src="http://example.com/"></iframe>
... </body></html>
... """)
>>> get_first_link_text(html)
''


Title parsing.  We follow Firefox's behaviour with regard to child
elements (haven't tested IE).

>>> def get_title(html):
...     factory = Factory()
...     response = test_html_response(html)
...     factory.set_response(response)
...     return factory.title

>>> html = ("""\
... <html><head>
... <title>T&nbsp;itle</title>
... </head><body><p>Blah.<p></body></html>
... """)
>>> get_title(html)
u'T\xa0itle'

>>> html = ("""\
... <html><head>
... <title>  Ti<script type="text/strange">alert("this is valid HTML -- yuck!")</script>
... tle &amp;&#38;
... </title>
... </head><body><p>Blah.<p></body></html>
... """)
>>> str(get_title(html))
'Ti<script type="text/strange">alert("this is valid HTML -- yuck!")</script> tle &&'


No more tags after <title> used to cause an exception

>>> html = ("""\
... <html><head>
... <title>""")
>>> get_title(html)
u''