<li><a class="reference" href="#can-i-create-my-own-functions-in-c" id="id7" name="id7">1 Can I create my own functions in C?</a></li>
<li><a class="reference" href="#id1" id="id8" name="id8">2 Can I create my own functions in C++?</a></li>
<li><a class="reference" href="#writing-c-is-hard-are-there-any-alternatives" id="id9" name="id9">3 Writing C is hard; are there any alternatives?</a></li>
<li><a class="reference" href="#how-can-i-execute-arbitrary-python-statements-from-c" id="id10" name="id10">4 How can I execute arbitrary Python statements from C?</a></li>
<li><a class="reference" href="#how-can-i-evaluate-an-arbitrary-python-expression-from-c" id="id11" name="id11">5 How can I evaluate an arbitrary Python expression from C?</a></li>
<li><a class="reference" href="#how-do-i-extract-c-values-from-a-python-object" id="id12" name="id12">6 How do I extract C values from a Python object?</a></li>
<li><a class="reference" href="#how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length" id="id13" name="id13">7 How do I use Py_BuildValue() to create a tuple of arbitrary length?</a></li>
<li><a class="reference" href="#how-do-i-call-an-object-s-method-from-c" id="id14" name="id14">8 How do I call an object's method from C?</a></li>
<li><a class="reference" href="#how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr" id="id15" name="id15">9 How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?</a></li>
<li><a class="reference" href="#how-do-i-access-a-module-written-in-python-from-c" id="id16" name="id16">10 How do I access a module written in Python from C?</a></li>
<li><a class="reference" href="#how-do-i-interface-to-c-objects-from-python" id="id17" name="id17">11 How do I interface to C++ objects from Python?</a></li>
<li><a class="reference" href="#i-added-a-module-using-the-setup-file-and-the-make-fails-why" id="id18" name="id18">12 I added a module using the Setup file and the make fails; why?</a></li>
<li><a class="reference" href="#how-do-i-debug-an-extension" id="id19" name="id19">13 How do I debug an extension?</a></li>
<li><a class="reference" href="#i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why" id="id20" name="id20">14 I want to compile a Python module on my Linux system, but some files are missing. Why?</a></li>
<li><a class="reference" href="#what-does-systemerror-pyimport-fixupextension-module-yourmodule-not-loaded-mean" id="id21" name="id21">15 What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean?</a></li>
<li><a class="reference" href="#how-do-i-tell-incomplete-input-from-invalid-input" id="id22" name="id22">16 How do I tell "incomplete input" from "invalid input"?</a></li>
<li><a class="reference" href="#how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual" id="id23" name="id23">17 How do I find undefined g++ symbols __builtin_new or __pure_virtual?</a></li>
<li><a class="reference" href="#can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance" id="id24" name="id24">18 Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?</a></li>
<li><a class="reference" href="#when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2" id="id25" name="id25">19 When importing module X, why do I get "undefined symbol: PyUnicodeUCS2*"?</a></li>
</ul>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id7" id="can-i-create-my-own-functions-in-c" name="can-i-create-my-own-functions-in-c">1 Can I create my own functions in C?</a></h1>
<p>Yes, you can create built-in modules containing functions,
variables, exceptions and even new types in C. This is explained in
the document "Extending and Embedding the Python Interpreter" (<a class="reference" href="http://docs.python.org/ext/ext.html">http://docs.python.org/ext/ext.html</a>).</p>
<p>Most intermediate or advanced Python books will also
cover this topic.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id8" id="id1" name="id1">2 Can I create my own functions in C++?</a></h1>
<p>Yes, using the C compatibility features found in C++.
Place <tt class="docutils literal"><span class="pre">extern</span> <span class="pre">"C"</span> <span class="pre">{</span> <span class="pre">...</span> <span class="pre">}</span></tt> around the Python include files and put
<tt class="docutils literal"><span class="pre">extern</span> <span class="pre">"C"</span></tt> before each function that is going to be called by the
Python interpreter. Global or static C++ objects with constructors
are probably not a good idea.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id9" id="writing-c-is-hard-are-there-any-alternatives" name="writing-c-is-hard-are-there-any-alternatives">3 Writing C is hard; are there any alternatives?</a></h1>
<p>There are a number of alternatives to writing your own C extensions,
depending on what you're trying to do.</p>
<p>If you need more speed, <a class="reference" href="http://psyco.sourceforge.net/">Psyco</a> generates x86 assembly code
from Python bytecode. You can use Psyco to compile the most
time-critical functions in your code, and gain a significant
improvement with very little effort, as long as you're running on a
machine with an x86-compatible processor.</p>
<p><a class="reference" href="http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/">Pyrex</a> is a compiler that accepts a slightly modified form of Python
and generates the corresponding C code. Pyrex makes it possible to write
an extension without having to learn Python's C API.</p>
<p>If you need to interface to some C library for which no Python
extension currently exists, you can try wrapping the library's data
types and functions with a tool such as <a class="reference" href="http://www.swig.org">SWIG</a>.
For C++ libraries, you can look at <a class="reference" href="http://www.riverbankcomputing.co.uk/sip/">SIP</a>, <a class="reference" href="http://cxx.sourceforge.net/">CXX</a>, <a class="reference" href="http://www.boost.org/libs/python/doc/index.html">Boost</a>, or <a class="reference" href="http://www.scipy.org/site_content/weave">Weave</a>.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id10" id="how-can-i-execute-arbitrary-python-statements-from-c" name="how-can-i-execute-arbitrary-python-statements-from-c">4 How can I execute arbitrary Python statements from C?</a></h1>
<p>The highest-level function to do this is <tt class="docutils literal"><span class="pre">PyRun_SimpleString()</span></tt> which takes
a single string argument to be executed in the context of the module
<tt class="docutils literal"><span class="pre">__main__</span></tt> and returns 0 for success and -1 when an exception occurred
(including <tt class="docutils literal"><span class="pre">SyntaxError</span></tt>). If you want more control, use <tt class="docutils literal"><span class="pre">PyRun_String()</span></tt>;
see the source for <tt class="docutils literal"><span class="pre">PyRun_SimpleString()</span></tt> in Python/pythonrun.c.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id11" id="how-can-i-evaluate-an-arbitrary-python-expression-from-c" name="how-can-i-evaluate-an-arbitrary-python-expression-from-c">5 How can I evaluate an arbitrary Python expression from C?</a></h1>
<p>Call the function <tt class="docutils literal"><span class="pre">PyRun_String()</span></tt> from the previous question with the
start symbol <tt class="docutils literal"><span class="pre">Py_eval_input</span></tt>; it
parses an expression, evaluates it and returns its value.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id12" id="how-do-i-extract-c-values-from-a-python-object" name="how-do-i-extract-c-values-from-a-python-object">6 How do I extract C values from a Python object?</a></h1>
<p>That depends on the object's type. If it's a tuple,
<tt class="docutils literal"><span class="pre">PyTupleSize(o)</span></tt> returns its length and <tt class="docutils literal"><span class="pre">PyTuple_GetItem(o,</span> <span class="pre">i)</span></tt>
returns its i'th item. Lists have similar functions, <tt class="docutils literal"><span class="pre">PyListSize(o)</span></tt>
and <tt class="docutils literal"><span class="pre">PyList_GetItem(o,</span> <span class="pre">i)</span></tt>.</p>
its length and <tt class="docutils literal"><span class="pre">PyString_AsString(o)</span></tt> a pointer to its value.
Note that Python strings may contain null bytes so C's <tt class="docutils literal"><span class="pre">strlen()</span></tt>
should not be used.</p>
<p>To test the type of an object, first make sure
it isn't NULL, and then use <tt class="docutils literal"><span class="pre">PyString_Check(o)</span></tt>, <tt class="docutils literal"><span class="pre">PyTuple_Check(o)</span></tt>,
<p>There is also a high-level API to Python objects which is
provided by the so-called 'abstract' interface -- read
<tt class="docutils literal"><span class="pre">Include/abstract.h</span></tt> for further details. It allows
interfacing with any kind of Python sequence
using calls like <tt class="docutils literal"><span class="pre">PySequence_Length()</span></tt>, <tt class="docutils literal"><span class="pre">PySequence_GetItem()</span></tt>, etc.)
as well as many other useful protocols.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id13" id="how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length" name="how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length">7 How do I use Py_BuildValue() to create a tuple of arbitrary length?</a></h1>
<p>You can't. Use <tt class="docutils literal"><span class="pre">t</span> <span class="pre">=</span> <span class="pre">PyTuple_New(n)</span></tt> instead, and fill it with
objects using <tt class="docutils literal"><span class="pre">PyTuple_SetItem(t,</span> <span class="pre">i,</span> <span class="pre">o)</span></tt> -- note that this "eats" a
reference count of <tt class="docutils literal"><span class="pre">o</span></tt>, so you have to <tt class="docutils literal"><span class="pre">Py_INCREF</span></tt> it.
Lists have similar functions <tt class="docutils literal"><span class="pre">PyList_New(n)</span></tt> and
<tt class="docutils literal"><span class="pre">PyList_SetItem(l,</span> <span class="pre">i,</span> <span class="pre">o)</span></tt>. Note that you <em>must</em> set all the tuple items to
some value before you pass the tuple to Python code --
<tt class="docutils literal"><span class="pre">PyTuple_New(n)</span></tt> initializes them to NULL, which isn't a valid Python
value.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id14" id="how-do-i-call-an-object-s-method-from-c" name="how-do-i-call-an-object-s-method-from-c">8 How do I call an object's method from C?</a></h1>
<p>The <tt class="docutils literal"><span class="pre">PyObject_CallMethod()</span></tt> function can be used to call an arbitrary
method of an object. The parameters are the object, the name of the
method to call, a format string like that used with <tt class="docutils literal"><span class="pre">Py_BuildValue()</span></tt>, and the argument values:</p>
<p>This works for any object that has methods -- whether built-in or
user-defined. You are responsible for eventually <tt class="docutils literal"><span class="pre">Py_DECREF</span></tt>'ing the
return value.</p>
<p>To call, e.g., a file object's "seek" method with arguments 10, 0
(assuming the file object pointer is "f"):</p>
<pre class="literal-block">
res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0);
if (res == NULL) {
... an exception occurred ...
}
else {
Py_DECREF(res);
}
</pre>
<p>Note that since <tt class="docutils literal"><span class="pre">PyObject_CallObject()</span></tt> <em>always</em> wants a tuple for the
argument list, to call a function without arguments, pass "()" for the
format, and to call a function with one argument, surround the argument
in parentheses, e.g. "(i)".</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id15" id="how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr" name="how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr">9 How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?</a></h1>
<p>In Python code, define an object that supports the <tt class="docutils literal"><span class="pre">write()</span></tt> method.
Assign this object to <tt class="docutils literal"><span class="pre">sys.stdout</span></tt> and <tt class="docutils literal"><span class="pre">sys.stderr</span></tt>.
Call print_error, or just allow the standard traceback mechanism to
work. Then, the output will go wherever your <tt class="docutils literal"><span class="pre">write()</span></tt> method sends it.</p>
<p>The easiest way to do this is to use the StringIO class in the standard
library.</p>
<p>Sample code and use for catching stdout:</p>
<pre class="literal-block">
>>> class StdoutCatcher:
... def __init__(self):
... self.data = ''
... def write(self, stuff):
... self.data = self.data + stuff
...
>>> import sys
>>> sys.stdout = StdoutCatcher()
>>> print 'foo'
>>> print 'hello world!'
>>> sys.stderr.write(sys.stdout.data)
foo
hello world!
</pre>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id16" id="how-do-i-access-a-module-written-in-python-from-c" name="how-do-i-access-a-module-written-in-python-from-c">10 How do I access a module written in Python from C?</a></h1>
<p>You can get a pointer to the module object as follows:</p>
<pre class="literal-block">
module = PyImport_ImportModule("<modulename>");
</pre>
<p>If the module hasn't been imported yet (i.e. it is not yet present in
<tt class="docutils literal"><span class="pre">sys.modules</span></tt>), this initializes the module; otherwise it simply returns
the value of <tt class="docutils literal"><span class="pre">sys.modules["<modulename>"]</span></tt>. Note that it doesn't enter
the module into any namespace -- it only ensures it has been
initialized and is stored in <tt class="docutils literal"><span class="pre">sys.modules</span></tt>.</p>
<p>You can then access the module's attributes (i.e. any name defined in
<p>Calling <tt class="docutils literal"><span class="pre">PyObject_SetAttrString()</span></tt> to assign to variables in the module also works.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id17" id="how-do-i-interface-to-c-objects-from-python" name="how-do-i-interface-to-c-objects-from-python">11 How do I interface to C++ objects from Python?</a></h1>
<p>Depending on your requirements, there are many approaches. To do
this manually, begin by reading <a class="reference" href="http://docs.python.org/ext/ext.html">the "Extending and Embedding" document</a>. Realize
that for the Python run-time system, there isn't a whole lot of
difference between C and C++ -- so the strategy of building a new Python
type around a C structure (pointer) type will also work for C++
objects.</p>
<p>For C++ libraries, you can look at <a class="reference" href="http://www.riverbankcomputing.co.uk/sip/">SIP</a>, <a class="reference" href="http://cxx.sourceforge.net/">CXX</a>, <a class="reference" href="http://www.boost.org/libs/python/doc/index.html">Boost</a>, or <a class="reference" href="http://www.scipy.org/site_content/weave">Weave</a>.
<a class="reference" href="http://www.swig.org">SWIG</a> is a similar automated tool that only supports C libraries.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id18" id="i-added-a-module-using-the-setup-file-and-the-make-fails-why" name="i-added-a-module-using-the-setup-file-and-the-make-fails-why">12 I added a module using the Setup file and the make fails; why?</a></h1>
<p>Setup must end in a newline, if there is no newline there, the build
process fails. (Fixing this requires some ugly shell script hackery,
and this bug is so minor that it doesn't seem worth the effort.)</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id19" id="how-do-i-debug-an-extension" name="how-do-i-debug-an-extension">13 How do I debug an extension?</a></h1>
<p>When using GDB with dynamically loaded extensions, you can't set a
breakpoint in your extension until your extension is loaded.</p>
<p>In your <tt class="docutils literal"><span class="pre">.gdbinit</span></tt> file (or interactively), add the command:</p>
<pre class="literal-block">
br _PyImport_LoadDynamicModule
</pre>
<p>Then, when you run GDB:</p>
<pre class="literal-block">
$ gdb /local/bin/python
gdb) run myscript.py
gdb) continue # repeat until your extension is loaded
gdb) finish # so that your extension is loaded
gdb) br myfunction.c:50
gdb) continue
</pre>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id20" id="i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why" name="i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why">14 I want to compile a Python module on my Linux system, but some files are missing. Why?</a></h1>
<p>Most packaged versions of Python don't include the
/usr/lib/python2.x/config/ directory, which contains various files required
for compiling Python extensions.</p>
<p>For Red Hat, install the python-devel RPM to get the necessary files.</p>
<h1><a class="toc-backref" href="#id21" id="what-does-systemerror-pyimport-fixupextension-module-yourmodule-not-loaded-mean" name="what-does-systemerror-pyimport-fixupextension-module-yourmodule-not-loaded-mean">15 What does "SystemError: _PyImport_FixupExtension: module yourmodule not loaded" mean?</a></h1>
<p>This means that you have created an extension module named "yourmodule", but your module init function does not initialize with that name.</p>
<p>Every module init function will have a line similar to:</p>
<p>If the string passed to this function is not the same name as your
extenion module, the <tt class="docutils literal"><span class="pre">SystemError</span></tt> exception will be raised.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id22" id="how-do-i-tell-incomplete-input-from-invalid-input" name="how-do-i-tell-incomplete-input-from-invalid-input">16 How do I tell "incomplete input" from "invalid input"?</a></h1>
<p>Sometimes you want to emulate the Python interactive interpreter's
behavior, where it gives you a continuation prompt when the input
is incomplete (e.g. you typed the start of an "if" statement
or you didn't close your parentheses or triple string quotes),
but it gives you a syntax error message immediately when the input
is invalid.</p>
<p>In Python you can use the <tt class="docutils literal"><span class="pre">codeop</span></tt> module, which approximates the
parser's behavior sufficiently. IDLE uses this, for example.</p>
<p>The easiest way to do it in C is to call <tt class="docutils literal"><span class="pre">PyRun_InteractiveLoop()</span></tt>
(perhaps in a separate thread) and let the Python interpreter handle
the input for you. You can also set the <tt class="docutils literal"><span class="pre">PyOS_ReadlineFunctionPointer</span></tt>
to point at your custom input function. See <tt class="docutils literal"><span class="pre">Modules/readline.c</span></tt> and
<tt class="docutils literal"><span class="pre">Parser/myreadline.c</span></tt> for more hints.</p>
<p>However sometimes you have to run the embedded Python interpreter in
the same thread as your rest application and you can't allow the
<tt class="docutils literal"><span class="pre">PyRun_InteractiveLoop()</span></tt> to stop while waiting for user input. The
one solution then is to call <tt class="docutils literal"><span class="pre">PyParser_ParseString()</span></tt> and test for
<tt class="docutils literal"><span class="pre">e.error</span></tt> equal to <tt class="docutils literal"><span class="pre">E_EOF</span></tt>, which means the input is incomplete).
Here's a sample code fragment, untested, inspired by code from Alex Farber:</p>
<pre class="literal-block">
#include <Python.h>
#include <node.h>
#include <errcode.h>
#include <grammar.h>
#include <parsetok.h>
#include <compile.h>
int testcomplete(char *code)
/* code should end in \n */
/* return -1 for error, 0 for incomplete, 1 for complete */
{
node *n;
perrdetail e;
n = PyParser_ParseString(code, &_PyParser_Grammar,
Py_file_input, &e);
if (n == NULL) {
if (e.error == E_EOF)
return 0;
return -1;
}
PyNode_Free(n);
return 1;
}
</pre>
<p>Another solution is trying to compile the received string with
<tt class="docutils literal"><span class="pre">Py_CompileString()</span></tt>. If it compiles without errors, try to execute the returned
code object by calling <tt class="docutils literal"><span class="pre">PyEval_EvalCode()</span></tt>. Otherwise save the input for
later. If the compilation fails, find out if it's an error or just
more input is required - by extracting the message string from the
exception tuple and comparing it to the string "unexpected EOF while parsing".
Here is a complete example using the GNU readline library (you may
want to ignore SIGINT while calling readline()):</p>
!strcmp (msg, "unexpected EOF while parsing")) /* E_EOF */
{
Py_XDECREF (exc);
Py_XDECREF (val);
Py_XDECREF (trb);
prompt = ps2;
}
else /* some other syntax error */
{
PyErr_Restore (exc, val, trb);
PyErr_Print ();
free (code);
code = NULL;
prompt = ps1;
}
}
else /* some non-syntax error */
{
PyErr_Print ();
free (code);
code = NULL;
prompt = ps1;
}
free (line);
}
}
Py_XDECREF(glb);
Py_XDECREF(loc);
Py_Finalize();
exit(0);
}
</pre>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id23" id="how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual" name="how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual">17 How do I find undefined g++ symbols __builtin_new or __pure_virtual?</a></h1>
<p>To dynamically load g++ extension modules, you must recompile Python, relink it using g++ (change LINKCC in the python Modules Makefile), and link your extension module using g++ (e.g., "g++ -shared -o mymodule.so mymodule.o").</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id24" id="can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance" name="can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance">18 Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?</a></h1>
<p>In Python 2.2, you can inherit from builtin classes such as int, list, dict, etc.</p>
provides a way of doing this from C++ (i.e. you can inherit from an
extension class written in C++ using the BPL).</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id25" id="when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2" name="when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2">19 When importing module X, why do I get "undefined symbol: PyUnicodeUCS2*"?</a></h1>
<p>You are using a version of Python that uses a 4-byte representation
for Unicode characters, but some C extension module you are importing
was compiled using a Python that uses a 2-byte representation for
Unicode characters (the default).</p>
<p>If instead the name of the undefined symbol starts with
<tt class="docutils literal"><span class="pre">PyUnicodeUCS4</span></tt>, the problem is the reverse: Python was built using
2-byte Unicode characters, and the extension module was compiled using
a Python with 4-byte Unicode characters.</p>
<p>This can easily occur when using pre-built extension packages. RedHat
Linux 7.x, in particular, provided a "python2" binary that is compiled
with 4-byte Unicode. This only causes the link failure if the extension
uses any of the <tt class="docutils literal"><span class="pre">PyUnicode_*()</span></tt> functions. It is also a problem if an
extension uses any of the Unicode-related format specifiers for
<tt class="docutils literal"><span class="pre">Py_BuildValue</span></tt> (or similar) or parameter specifications for