Site Network: Home |

Actually, it works...

Though I've a few doubts. I need to discuss it with the devs if the solution I provided indeed is the best way to go... Titus, a mail coming your way soon :)

For the time being, lets talk about what has been done.

If you want to get the C coverage now with figleaf for your python repository, you have to make sure you execute figleaf from within the parent directory where you built python with make. Execute figleaf using your newly compiled python interpreter, passing -c/--c-coverage option on the command line with a list of comma separated directories to look for C code compiled for coverage with gcov:

$./python ../figleaf-github/figleaf/bin/figleaf -cModules Lib/test/regrtest.py test_zlib.py

In my case, I had only compiled zlib statically into python for C coverage, so I am running its test suite only yet. The module itself is located in Modules subdirectory of the Python source, so passing it along with -c option. And here is the output:

test_zlib
1 test OK.
File '/usr/include/sys/sysmacros.h'
Lines executed:0.00% of 6
/usr/include/sys/sysmacros.h:creating 'sysmacros.h.gcov'

File '/usr/include/sys/stat.h'
Lines executed:0.00% of 12
/usr/include/sys/stat.h:creating 'stat.h.gcov'

File './Modules/zlibmodule.c'
Lines executed:74.11% of 448
./Modules/zlibmodule.c:creating 'zlibmodule.c.gcov'
The gcov generates coverage report for the module in the current directory. Convert it to html using figleaf2html:

$../figleaf-github/figleaf/bin/figleaf2html

And here is the output for me:
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/getopt.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/contextlib.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/__future__.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/posixpath.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/os.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/random.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/test_support.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Modules/zlibmodule.c
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/warnings.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/fnmatch.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/__init__.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/sre_compile.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/genericpath.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/regrtest.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/sre_parse.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/test_zlib.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/unittest.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/socket.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/shutil.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/functools.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/re.py
reported on 21 file(s) total

figleaf: HTML output written to html
And here is a screen shot of the report:


And now all about how it works...

You start by compiling your Python with the modules you want to perform C code coverage for, built statically into it. This is something I've yet left to the user to do manually, after Seth suggested we move onto the report generation itself first as that was the priority. Later on may be I'll add auto static linkage of the C modules onto figleaf's functions.

Once you have compiled Python with the C modules statically linked in, and with the gcov options, there will be files generated for them by gcc with the filename structure of source.c.gcda. Passing the directory where these sources exist to figleaf with -c/--c-coverage option, makes figleaf look for the *.gcda files in that directory, and calling gcov on them for C report generation. Here is the function that performs this:

def get_c_coverage():
cov = {}
dir_file={}
dirs=c_code_dirs.split(",")
gcov_cmd=os.environ.get("COV", "gcov")
for d in dirs:
files=os.listdir(d)
for f in files:
if f.split(".")[-1]=="gcda":
os.system("%s %s -o %s" % (gcov_cmd, f.split(".")[0], d))
cov[d+"/"+".".join(f.split(".")[0:-1])+".c"]=d
try:
source=open(".".join(f.split(".")[0:-1])+".c.gcov", "r")
except IOError:
print "Can't open file: ", ".".join(f.split(".")[0:-1])+".c.gcov"
continue
exe_lines=[]
for line in source:
line=line.rstrip("\n")
if line.count(":") < 2:
continue
(count, lineno, code)=line.split(":", 2)
if count.strip()=="-" or count.strip()=="#####":
continue
else:
exe_lines.append(int(lineno))
cov[d+"/"+".".join(f.split(".")[0:-1])+".c"]=set(exe_lines)
return cov

This function has been added to figleaf's __init__ file, so it is located in the same file as the figleaf's main function. The above function is called from write_coverage():
def write_coverage(filename, append=True):
"""
Write the current coverage info out to the given filename. If
'append' is false, destroy any previously recorded coverage info.
"""
if _t is None:
return

data = internals.CoverageData(_t)

d = data.gather_files()

# sum existing coverage?
if append:
old = {}
fp = None
try:
fp = open(filename, 'rb')
except IOError:
pass

if fp:
old = load(fp)
fp.close()
d = combine_coverage(d, old)

# ok, save.
if c_code_dirs:
c_coverage=get_c_coverage()
if c_coverage:
d=combine_coverage(c_coverage, d)
outfp = open(filename, 'wb')
try:
dump(d, outfp)
finally:
outfp.close()

Here as you can see, figleaf checks if C coverage has been enabled, and calls get_c_coverage(). The result is appended to the Python coverage report, and consequently written to the output file.

Now to generate the html report, I've made a few modifications to a number of functions. Starting with...
def build_python_coverage_info(coverage, exclude_patterns, files_list):
keys = coverage.keys()

line_info = {}
lines=set([])
for pyfile in filter_files(keys, exclude_patterns, files_list):
try:
fp = open(pyfile, 'rU')
if pyfile.split(".")[-1]=="py":
lines = figleaf.get_lines(fp)
else:
lines = figleaf.get_c_lines(pyfile)
except KeyboardInterrupt:
raise
except IOError:
logger.error('CANNOT OPEN: %s' % (pyfile,))
continue
except Exception, e:
logger.error('ERROR: file %s, exception %s' % (pyfile, str(e)))
continue

# retrieve the coverage and merge into a realpath-based filename.
covered = coverage.get(pyfile, set())
realpath = os.path.realpath(pyfile)

# be careful not to overwrite existing coverage for different
# filenames that may have been canonicalized.
(old_l, old_c) = line_info.get(realpath, (set(), set()))
lines.update(old_l)
covered.update(old_c)

line_info[realpath] = (lines, covered)

return line_info

Here you can see a check for whether the file to generate the line information for is a Python source or a C source. The check isn't too generic, but works for the time being. In case of a C source, it calls a different function I wrote in __init__.py:

def get_c_lines(fp):
"""
Return the set of interesting lines in the C source code read
from this file.
"""
lines=[]
fp=os.path.basename(fp)
try:
fp=open("./"+fp+".gcov", 'r')
except IOError:
print "Can't open: ", "./"+fp+".gcov"

for line in fp:
line=line.rstrip("\n")
if line.count(":") < 2:
continue
(count, lineno, code)=line.split(":", 2)
if count.strip()=="-":
continue
else:
lines.append(int(lineno))
return set(lines)

It looks for all the lines in the source file that are marked as executable by gcov. Somewhat similar to what is already done in figleaf for Python code, but here is where the doubts arise.

I am not sure if this is the best way to check for the executable lines. It does generate accurate report compatible with what gcov generates, but I've seen gcov marking lots of lines as not executable that I would think should be marked otherwise. For example it skips on declaration statements. I was wondering if relying on gcov's interpretation of what lines are executable and what not is the right way to go here. Something to discuss with my supervisor... :|

0 Comments:

Post a Comment