Well GSoC came to an end. Thanks to my supportive mentor, Seth Lemons, for grading my work as satisfactorily meeting the intial requirements, and thus making my GSoC work a success.
Added 7 new test cases to test_os.py which tests the os module.
Following is a test class to test os.renames(). It starts by creating a temporary directory tree, and then renaming it. A walk is performed on the top most directory later on, and it is examined to verify that the renaming has taken place.
class RenamesDirTest(unittest.TestCase):
def setUp(self):
if os.path.exists(support.TESTFN):
os.remove(support.TESTFN)
os.mkdir(support.TESTFN)
def test_renames(self):
base=support.TESTFN
self.old=os.path.join(base, "dir1", "dir2", "dir3", "dir4")
os.makedirs(self.old)
self.new=os.path.join(base, "dir1", "dir2", "test3", "test4")
os.renames(self.old, self.new)
for (dirpath, dirnames, filenames) in os.walk(base, topdown=False):
self.assertEqual(self.new, dirpath)
break
def tearDown(self):
if os.path.exists(self.new):
os.removedirs(self.new)
elif os.path.exists(self.old):
os.removedirs(self.old)
A test for os.chdir(). Current directory for the current process is changed using os.chdir(), and later confirmed by calling os.getcwd(). A test case to see that os.chdir() fails when called with a directory argument that doesn't exist is also added.
#Tests for changing directory paths
class ChangePathTests(unittest.TestCase):
def setUp(self):
self.tempdir=support.TESTFN
os.mkdir(self.tempdir)
self.tempdir2=support.TESTFN+"2"
def test_chdir(self):
if os.path.exists(self.tempdir):
os.chdir(self.tempdir)
cwd=os.getcwd()
self.assertEqual(cwd, self.tempdir)
#Test to check chdir fails if nonexisting directory passed
def test_chdir_nonexistent(self):
if os.path.exists(self.tempdir2):
os.rmdir(self.tempdir2)
try:
os.chdir(self.tempdir2)
except OSError:
pass
else:
self.fail("Did not raise OSError")
def tearDown(self):
if os.path.exists(self.tempdir):
os.rmdir(self.tempdir)
A test class to test os.geteuid() and os.getgid(). Both are verified by comparing with values returned by os.stat().
class PosixGetUidGidTests(unittest.TestCase):
def setUp(self):
f=os.open(support.TESTFN, os.O_CREAT|os.O_RDWR)
os.close(f)
self.stats=os.stat(__file__)
def tearDown(self):
if os.path.exists(support.TESTFN):
os.remove(support.TESTFN)
if hasattr(os, "geteuid"):
def test_geteuid(self):
self.assertEqual(os.geteuid(), self.stats.st_uid)
if hasattr(os, "getgid"):
def test_getgid(self):
self.assertEqual(os.getgid(), self.stats.st_gid)
A test class for testing os.getenv() and os.putenv().
class GetPutEnvironTests(unittest.TestCase):
def test_putenv(self):
try:
os.putenv("KEY", "VALUE")
except:
self.fail("Not able to set environment variable")
def test_getenv(self):
keyvalue={"KEY":"VALUE"}
os.environ["KEY"]=keyvalue["KEY"]
value=os.getenv("KEY")
self.assertEqual(value, keyvalue["KEY"])
I am designing a few more tests to test the os.spawn* family of functions.
It's done and its working. And there isn't much new to tell. If you have read my report on the integration of figleaf for py2.6, the details are very similar, just that the syntax is of course py3kish. I used Titus' port of figleaf to py3k (http://github.com/ctb/figleaf/tree/py3k), with a few minor fixes.
The working is very similar to how the figleaf for py2.6 works. You have to use the -c/--c-coverage switch to give the directories holding C modules built with gcov support, and figleaf will incorporate their coverage report as well.
Now its time to work on writing new test cases for different modules. Many modules aren't 100% covered by its test suites, and there is a lot of room for improvements there. Of course I can't make all of them get completely covered, but I will write as much new tests as possible. I hope once I get started on the process and get used to it and get better familiarity with the code base, I can go on with the tests improvement well beyond GSoC.
Meanwhile, I am also going to give a try to make figleaf easy to use with gcov, and try to automate the static compilation of C modules for use with gcov.
Figleaf working with py3k now after a few minor modifications. Time to put its integration with gcov code in place.
This week's report; pace of work getting a bit slower again, need to pace it up.
Stuck at the following error for the night while porting figleaf to py3k... I hope it doesn't prove out to be a long night :/
test_decimal
Exception in thread Thread-53:
Traceback (most recent call last):
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 445, in getcontext
return _local.__decimal_context__
AttributeError: '_thread._local' object has no attribute '__decimal_context__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/threading.py", line 509, in _bootstrap_inner
self.run()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/threading.py", line 462, in run
self._target(*self._args, **self._kwargs)
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/test/test_decimal.py", line 1065, in thfunc1
test1 = d1/d3
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 1273, in __truediv__
context = getcontext()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 447, in getcontext
context = Context()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 3757, in __init__
for name, val in locals().items():
RuntimeError: dictionary changed size during iteration
Exception in thread Thread-54:
Traceback (most recent call last):
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 445, in getcontext
return _local.__decimal_context__
AttributeError: '_thread._local' object has no attribute '__decimal_context__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/threading.py", line 509, in _bootstrap_inner
self.run()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/threading.py", line 462, in run
self._target(*self._args, **self._kwargs)
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/test/test_decimal.py", line 1077, in thfunc2
test1 = d1/d3
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 1273, in __truediv__
context = getcontext()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 447, in getcontext
context = Context()
File "/home/shuaib/Tools/Projects/GSoC2009/py3k/Lib/decimal.py", line 3757, in __init__
for name, val in locals().items():
RuntimeError: dictionary changed size during iteration
Actually, it works...
Though I've a few doubts. I need to discuss it with the devs if the solution I provided indeed is the best way to go... Titus, a mail coming your way soon :)
For the time being, lets talk about what has been done.
If you want to get the C coverage now with figleaf for your python repository, you have to make sure you execute figleaf from within the parent directory where you built python with make. Execute figleaf using your newly compiled python interpreter, passing -c/--c-coverage option on the command line with a list of comma separated directories to look for C code compiled for coverage with gcov:
$./python ../figleaf-github/figleaf/bin/figleaf -cModules Lib/test/regrtest.py test_zlib.py
In my case, I had only compiled zlib statically into python for C coverage, so I am running its test suite only yet. The module itself is located in Modules subdirectory of the Python source, so passing it along with -c option. And here is the output:
test_zlibThe gcov generates coverage report for the module in the current directory. Convert it to html using figleaf2html:
1 test OK.
File '/usr/include/sys/sysmacros.h'
Lines executed:0.00% of 6
/usr/include/sys/sysmacros.h:creating 'sysmacros.h.gcov'
File '/usr/include/sys/stat.h'
Lines executed:0.00% of 12
/usr/include/sys/stat.h:creating 'stat.h.gcov'
File './Modules/zlibmodule.c'
Lines executed:74.11% of 448
./Modules/zlibmodule.c:creating 'zlibmodule.c.gcov'
$../figleaf-github/figleaf/bin/figleaf2html
And here is the output for me:
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/getopt.pyAnd here is a screen shot of the report:
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/contextlib.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/__future__.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/posixpath.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/os.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/random.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/test_support.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Modules/zlibmodule.c
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/warnings.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/fnmatch.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/__init__.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/sre_compile.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/genericpath.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/regrtest.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/sre_parse.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/test/test_zlib.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/unittest.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/socket.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/shutil.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/functools.py
reported on /home/shuaib/Tools/Projects/GSoC2009/release26-maint/Lib/re.py
reported on 21 file(s) total
figleaf: HTML output written to html
And now all about how it works...
You start by compiling your Python with the modules you want to perform C code coverage for, built statically into it. This is something I've yet left to the user to do manually, after Seth suggested we move onto the report generation itself first as that was the priority. Later on may be I'll add auto static linkage of the C modules onto figleaf's functions.
Once you have compiled Python with the C modules statically linked in, and with the gcov options, there will be files generated for them by gcc with the filename structure of source.c.gcda. Passing the directory where these sources exist to figleaf with -c/--c-coverage option, makes figleaf look for the *.gcda files in that directory, and calling gcov on them for C report generation. Here is the function that performs this:
def get_c_coverage():
cov = {}
dir_file={}
dirs=c_code_dirs.split(",")
gcov_cmd=os.environ.get("COV", "gcov")
for d in dirs:
files=os.listdir(d)
for f in files:
if f.split(".")[-1]=="gcda":
os.system("%s %s -o %s" % (gcov_cmd, f.split(".")[0], d))
cov[d+"/"+".".join(f.split(".")[0:-1])+".c"]=d
try:
source=open(".".join(f.split(".")[0:-1])+".c.gcov", "r")
except IOError:
print "Can't open file: ", ".".join(f.split(".")[0:-1])+".c.gcov"
continue
exe_lines=[]
for line in source:
line=line.rstrip("\n")
if line.count(":") < 2:
continue
(count, lineno, code)=line.split(":", 2)
if count.strip()=="-" or count.strip()=="#####":
continue
else:
exe_lines.append(int(lineno))
cov[d+"/"+".".join(f.split(".")[0:-1])+".c"]=set(exe_lines)
return cov
This function has been added to figleaf's __init__ file, so it is located in the same file as the figleaf's main function. The above function is called from write_coverage():
def write_coverage(filename, append=True):
"""
Write the current coverage info out to the given filename. If
'append' is false, destroy any previously recorded coverage info.
"""
if _t is None:
return
data = internals.CoverageData(_t)
d = data.gather_files()
# sum existing coverage?
if append:
old = {}
fp = None
try:
fp = open(filename, 'rb')
except IOError:
pass
if fp:
old = load(fp)
fp.close()
d = combine_coverage(d, old)
# ok, save.
if c_code_dirs:
c_coverage=get_c_coverage()
if c_coverage:
d=combine_coverage(c_coverage, d)
outfp = open(filename, 'wb')
try:
dump(d, outfp)
finally:
outfp.close()
Here as you can see, figleaf checks if C coverage has been enabled, and calls get_c_coverage(). The result is appended to the Python coverage report, and consequently written to the output file.
Now to generate the html report, I've made a few modifications to a number of functions. Starting with...
def build_python_coverage_info(coverage, exclude_patterns, files_list):
keys = coverage.keys()
line_info = {}
lines=set([])
for pyfile in filter_files(keys, exclude_patterns, files_list):
try:
fp = open(pyfile, 'rU')
if pyfile.split(".")[-1]=="py":
lines = figleaf.get_lines(fp)
else:
lines = figleaf.get_c_lines(pyfile)
except KeyboardInterrupt:
raise
except IOError:
logger.error('CANNOT OPEN: %s' % (pyfile,))
continue
except Exception, e:
logger.error('ERROR: file %s, exception %s' % (pyfile, str(e)))
continue
# retrieve the coverage and merge into a realpath-based filename.
covered = coverage.get(pyfile, set())
realpath = os.path.realpath(pyfile)
# be careful not to overwrite existing coverage for different
# filenames that may have been canonicalized.
(old_l, old_c) = line_info.get(realpath, (set(), set()))
lines.update(old_l)
covered.update(old_c)
line_info[realpath] = (lines, covered)
return line_info
Here you can see a check for whether the file to generate the line information for is a Python source or a C source. The check isn't too generic, but works for the time being. In case of a C source, it calls a different function I wrote in __init__.py:
def get_c_lines(fp):
"""
Return the set of interesting lines in the C source code read
from this file.
"""
lines=[]
fp=os.path.basename(fp)
try:
fp=open("./"+fp+".gcov", 'r')
except IOError:
print "Can't open: ", "./"+fp+".gcov"
for line in fp:
line=line.rstrip("\n")
if line.count(":") < 2:
continue
(count, lineno, code)=line.split(":", 2)
if count.strip()=="-":
continue
else:
lines.append(int(lineno))
return set(lines)
It looks for all the lines in the source file that are marked as executable by gcov. Somewhat similar to what is already done in figleaf for Python code, but here is where the doubts arise.
I am not sure if this is the best way to check for the executable lines. It does generate accurate report compatible with what gcov generates, but I've seen gcov marking lots of lines as not executable that I would think should be marked otherwise. For example it skips on declaration statements. I was wondering if relying on gcov's interpretation of what lines are executable and what not is the right way to go here. Something to discuss with my supervisor... :|
Hi again,
I thought I would update the blog now instead of waiting for my code to work properly so the progress reports keep coming in on time.
Well I've worked on making sure figleaf now takes into account the C code coverage too. I added an additional command line option to figleaf "-c/-c-coverage" which take a comma separated list of directories that would make figleaf look for C code in those directories linked in with gcov options for C code coverage. figleaf would make sure gcov is called for all the files it finds with a .gcda extension, thus generating a C code coverage report for it.
This does add the restriction on figleaf that now it is compulsory to call it from the directory where the make for the project was executed from, as gcov requires it being called from the compilation directory.
I've to make a few modifications to the code achieving the above task, as it is still not very generic. And now I am thinking of getting either my own fork of figleaf, or talking to Titus Brown about where to send the patches to.
Next comes in the integration of the report generated by gcov and figleaf. I made sure the gcov reported is filed in the .figleaf file generated by figleaf, but once you try to convert that file into an html report, the figleaf2html generates a number of indentation errors for the C code. That shows figleaf2html is very Python specific, but I haven't had a detail look at it yet, and that's what I am suppose to do next.
Once this is achieved, I am well on my way doing all of this again for Py3k.
Seth, my mentor suggested adding a command line option to figleaf so it would look for the C directories listed in a file, instead of giving them all on the command line with the "-c/-c-coverage" option. I'll make sure that's incorporated.
