The state of test coverage
When talking about testing software, and writing unit tests, one of the measures is code coverage — “describing the degree to which the source code of a program has been tested”. Our Nose test harness can tell us how well our tests touch the code base.
nosetests --with-doctest --doctest-extension=.doctest --with-coverage --cover-package apps
Results in a bunch of failing tests (currently being addressed). Ran 299 tests in 1229.766s FAILED (errors=2, failures=9)
But more interestingly, the coverage stats, with some arbitrary chosen colour highlighting (green is 90%+)!
Name Stmts Exec Cover Missing
——————————————————————————–
apps 1 1 100%
apps.a3c 59 57 96% 104-105
apps.a3c.admin 29 28 96% 39
apps.a3c.api 114 92 80% 29-30, 46, 57, 61, 65-70, 91, 103, 107, 111-116, 121, 138, 156, 166
apps.a3c.backends 60 54 90% 25-26, 45-48
apps.a3c.bll 123 119 96% 19, 22, 28-29
apps.a3c.context_processors 19 19 100%
apps.a3c.exceptions 14 14 100%
apps.a3c.forms 110 104 94% 32, 39, 126-127, 164-165
apps.a3c.management 34 33 97% 33
apps.a3c.management.commands 1 1 100%
apps.a3c.middleware 34 33 97% 57
apps.a3c.models 77 76 98% 37
apps.a3c.permissions 25 25 100%
apps.a3c.preferences 33 33 100%
apps.a3c.views 34 23 67% 14-15, 23, 28-29, 33-34, 40-41, 46-47
apps.about 1 1 100%
apps.about.urls 2 2 100%
apps.about.views 12 7 58% 5, 8, 11, 14, 17
apps.audit 1 1 100%
apps.audit.managers 8 8 100%
apps.audit.middleware 22 21 95% 36
apps.audit.models 153 111 72% 70-71, 77-89, 155-156, 165, 210-213, 226-234, 239-260, 264-276
apps.audit.signals 6 6 100%
apps.cdocs 1 1 100%
apps.cdocs.admin 3 3 100%
apps.cdocs.doc_blocks 40 40 100%
apps.cdocs.docs_audit 26 23 88% 7-11
apps.cdocs.doxygen_xmls 127 112 88% 11-15, 19, 69-73, 77, 88, 114-115, 188, 210
apps.cdocs.forms 25 25 100%
apps.cdocs.models 190 173 91% 50, 192, 262-263, 284-300
apps.cdocs.patches 35 29 82% 22-27
apps.cdocs.svn_backend 257 81 31% 19-20, 27-28, 33, 38, 55-71, 76-78, 83-91, 97-101, 106-112, 131, 138, 143-149, 154-173, 178-188, 195-196, 200-211, 216-239, 243-247, 254-260, 266-283, 289-293, 300-301, 305-325, 329, 336-342, 345, 348, 351, 354, 357, 361, 371-380, 385, 390-398, 405, 409-430, 433, 442-448, 451, 454-458, 461, 464, 467, 470, 480, 483, 486, 489, 492, 495, 502-503, 506
apps.cdocs.templatetags 1 1 100%
apps.cdocs.templatetags.docs 7 7 100%
apps.cdocs.urls 7 6 85% 11
apps.cdocs.utils 29 10 34% 15-20, 23-28, 35-44, 55-56, 59-67
apps.cdocs.views 90 87 96% 124-125, 163
apps.chatlog 1 1 100%
apps.chatlog.admin 4 4 100%
apps.chatlog.models 24 21 87% 27, 43-47
apps.chatlog.urls 2 2 100%
apps.chatlog.views 17 7 41% 11-24, 27-34
apps.dash 7 6 85% 11
apps.dash.models 2 2 100%
apps.dash.sections 32 31 96% 17
apps.dash.urls 2 2 100%
apps.dash.views 10 10 100%
apps.dashboard 1 1 100%
apps.dashboard.models 1 1 100%
apps.dashboard.urls 2 2 100%
apps.dashboard.views 110 104 94% 66, 81-82, 105-107
apps.django_evolution 11 9 81% 3, 6
apps.django_evolution.admin 4 4 100%
apps.django_evolution.db 4 4 100%
apps.django_evolution.db.common 120 20 16% 8-11, 14-20, 23-26, 29-30, 33-38, 41-82, 85-90, 94-98, 101-102, 105, 108-128, 131-136, 139-143, 146, 149-154, 157-166
apps.django_evolution.db.mysql 58 10 17% 8-42, 45-50, 53-63, 67-69, 72-81, 84-89
apps.django_evolution.db.mysql_old 1 1 100%
apps.django_evolution.db.postgresql 9 4 44% 7-13
apps.django_evolution.db.postgresql_psycopg2 1 1 100%
apps.django_evolution.db.sqlite3 149 19 12% 10-25, 28-35, 38-45, 48-60, 63-79, 83, 86-99, 102-134, 137-159, 162-175, 178, 181, 184, 187-200
apps.django_evolution.diff 117 45 38% 10-11, 15-17, 20, 23, 34-38, 71, 74, 84-85, 90-93, 99-104, 113-122, 131, 147, 151-167, 171-205
apps.django_evolution.evolve 45 22 48% 19, 36-60
apps.django_evolution.management 62 41 66% 3-4, 30, 39-40, 42-45, 49, 66-68, 73-74, 77-80, 95-98
apps.django_evolution.management.commands 1 1 100%
apps.django_evolution.management.commands.evolve 158 25 15% 6-7, 47-225
apps.django_evolution.models 20 16 80% 14-16, 27
apps.django_evolution.mutations 308 65 21% 23-34, 45-74, 77, 80-86, 89, 92, 95, 107-109, 112, 121-123, 127, 134, 143, 147-149, 152, 156-159, 163, 168-169, 172, 175-199, 202-218, 222-226, 229-237, 240-255, 258-261, 264-275, 278-295, 300-304, 307-314, 317-335, 338-370, 374-377, 380-392, 396-407, 411-432, 436, 439, 442-444, 447-461, 465, 468, 471-475
apps.django_evolution.signature 39 39 100%
apps.django_evolution.utils 13 3 23% 5-9, 16-22
apps.django_nose 1 1 100%
apps.django_nose.nose_runner 43 11 25% 33-42, 46-48, 52-54, 58-83
apps.errors 8 8 100%
apps.errors.exceptions 6 6 100%
apps.errors.log 22 13 59% 14, 28-35
apps.errors.middleware 15 14 93% 21
apps.events 1 1 100%
apps.events.admin 1 1 100%
apps.events.feed 25 15 60% 14-15, 18-22, 26, 29, 32-33, 36
apps.events.forms 15 15 100%
apps.events.models 1 1 100%
apps.events.urls 2 2 100%
apps.events.views 24 24 100%
apps.feeds 17 17 100%
apps.feeds.models 1 1 100%
apps.feeds.urls 2 2 100%
apps.feeds.views 33 31 93% 18-19
apps.fields 28 26 92% 24-25
apps.i18n 1 1 100%
apps.i18n.middleware 9 9 100%
apps.i18n.models 1 1 100%
apps.linkmedia 1 1 100%
apps.linkmedia.management 1 1 100%
apps.linkmedia.management.commands 1 1 100%
apps.linkmedia.management.commands.linkmedia 35 9 25% 31-62
apps.lists 36 27 75% 20-23, 25, 30-32, 49-50
apps.lists.exceptions 5 5 100%
apps.lists.models 1 1 100%
apps.lists.templatetags 1 1 100%
apps.lists.templatetags.lists 78 72 92% 54-58, 100-102
apps.mail 1 1 100%
apps.mail.admin 8 8 100%
apps.mail.api 45 40 88% 15, 32, 43, 50, 60
apps.mail.bll 52 42 80% 18, 24-27, 33-34, 47, 56, 64
apps.mail.exceptions 18 18 100%
apps.mail.feed 23 15 65% 14-15, 18, 21, 24, 27-28, 31
apps.mail.forms 42 40 95% 15-16
apps.mail.index 21 20 95% 38
apps.mail.links 12 11 91% 15
apps.mail.management 1 1 100%
apps.mail.management.commands 1 1 100%
apps.mail.management.commands.mail_receive 42 42 100%
apps.mail.models 54 53 98% 36
apps.mail.preferences 17 17 100%
apps.mail.urls 2 2 100%
apps.mail.utils 87 80 91% 80-81, 108-109, 172-177
apps.mail.views 38 27 71% 20, 43, 46, 53-60
apps.manage 1 1 100%
apps.manage.models 1 1 100%
apps.manage.urls 9 9 100%
apps.manage.views 48 37 77% 18, 50, 55, 70, 75, 84-85, 89-90, 94-95
apps.preferences 15 15 100%
apps.preferences.models 1 1 100%
apps.preferences.views 28 28 100%
apps.restapi 1 1 100%
apps.restapi.responder 41 28 68% 48, 74-85
apps.restapi.urls 2 2 100%
apps.search 17 13 76% 14-17
apps.search.document 24 22 91% 49, 58
apps.search.forms 66 56 84% 38, 67-75, 98, 100
apps.search.global_urls 2 2 100%
apps.search.index 16 10 62% 40-45, 50-51
apps.search.local_urls 2 2 100%
apps.search.management 5 5 100%
apps.search.management.commands 1 1 100%
apps.search.management.commands.fullindex 10 10 100%
apps.search.models 2 2 100%
apps.search.searching 33 27 81% 33-39, 53-57
apps.search.views 34 30 88% 44, 58-64
apps.tickets 1 1 100%
apps.tickets.admin 3 3 100%
apps.tickets.api 41 11 26% 11-16, 21-28, 35-40, 44-49, 54-59, 64-67
apps.tickets.bll 48 13 27% 9-17, 22-25, 30-31, 39-56, 61-67, 70-76
apps.tickets.dash_data 30 21 70% 12-15, 35-40
apps.tickets.feed 23 15 65% 14-15, 18, 21, 24, 27-28, 31
apps.tickets.forms 17 17 100%
apps.tickets.index 17 17 100%
apps.tickets.links 12 11 91% 15
apps.tickets.models 28 28 100%
apps.tickets.urls 2 2 100%
apps.tickets.views 23 23 100%
apps.tools 28 28 100%
apps.util 27 21 77% 23, 41-42, 69-71
apps.vcs 1 1 100%
apps.vcs.admin 5 5 100%
apps.vcs.diff_match_patch 947 144 15% 93, 143-216, 231-272, 282-286, 299-392, 405-437, 450-482, 537, 554-621, 630-669, 680-769, 778-846, 875-892, 901, 937-943, 946-952, 957, 970-989, 1000-1015, 1026-1030, 1041-1045, 1059-1070, 1086-1124, 1139-1152, 1167-1259, 1270-1275, 1287-1313, 1327-1395, 1408-1468, 1480-1525, 1534-1599, 1610-1613, 1628-1690, 1700-1704, 1714-1738
apps.vcs.forms 19 9 47% 13-17, 20-24
apps.vcs.links 26 25 96% 21
apps.vcs.management 1 1 100%
apps.vcs.management.commands 1 1 100%
apps.vcs.management.commands.post_commit 24 23 95% 35
apps.vcs.management.commands.update_authz 7 5 71% 10-11
apps.vcs.models 90 80 88% 46, 66-72, 84-85, 116-117, 137-139
apps.vcs.post_commit 2 2 100%
apps.vcs.svn_backend 229 119 51% 21-22, 48-64, 69-71, 76-84, 90-94, 99-105, 124, 131-135, 139-151, 157-178, 182-187, 205-228, 255, 258, 264, 278
——————————————————————————–
TOTAL 7051 4464 63%
———————————————————————-
[...] and the test suite has seen quite a few changes over the last couple of weeks, we can also revisit our state of test coverage. I’ve left out apps with 90+% coverage, and colour-coded the [...]
Basie’s test harness is in the clear! at Basie Blog
9 Nov 09 at 10:32 pm