|
Interface C or C + + and Python with SWIG |
|
|
|
|
Tuesday, 13 May 2008 |
Introduction
The interface C is very rich and therefore costly to manage. Fortunately, automated tools can encapsulate libraries C or C + + with little code. SWIG specializes in this area but its presentation could be an entire book, only an introduction to its use will be made here.
SWIG is a utility to create from a configuration file. I modules encapsulation for several languages, including Python. The executable creates SWIG from a file module.i and with the option-python module.py a file and a file module_wrap.c (it is possible to encapsulate code C + + with the additional option-c + + which If the file has an extension. cpp). The file. C can then be compiled into a module called _module. (So, dll or pyd according to the platform and version).
distutils is capable of producing a module from the files. i and other files. c or. cpp necessary. It is important to add the argument swig_opt = [ '-c + +'] in creating an extension if a C + + code must be generated.
Unlike many indications on the Internet, SWIG is capable of handling C + + code and cases that do not bear have been comprehensively addressed. Indeed, these criticisms relate to older versions (1.1) while the 1.3 versions currently used are quite suited to the encapsulation of code C or C + +.
The guidelines begin with SWIG% and a code SWIG is not compiled by a compiler C or C + +. If a code SWIG must be included in a source file without being read by the compiler or if a code C / C + + should not be read by SWIG, it is possible to test the macro SWIG. If it is defined, SWIG is currently analysing the file, otherwise it is probably a compiler.
When generating code, any function returning a result complicated by value or taking complex arguments by value will be transformed into a function returning a pointer and taking into argument pointers. For complex, it is understood all types of structures or classes unreported at the time of execution of SWIG (indeed, it is more permissive than a compiler C or C + +).
I. Using simple
The overall architecture of a file. I is as follows:
%module mon_module
%{
/* Declarations supplementaires */
%}
/*List of functions, or constant variables to be included in the module */
void my_function (void);
Anything that is in the bloc% ()% will be fully copied to the source file generated, so any inclusion heading east to declare at this point. Similarly, it is important to add or file (s) header which are stored prototypes used functions. The declaration% module declares the name of the module to create, it is commonly file name without the extension (this is a convention). Finally, a list of functions, variables and constants which will be exhibited module must be written. This list of exposure may include a header where a sub-part is declared with the directive% include (similar to the directive # include preprocessor). For each plug compiled a Python module is created. This module Python encapsulates the plug, then the latter has another name, the prefix '_' to be added on behalf of the Python module at its compilation. This is the Python module to be used directly as stated in the tutorial on the API C. Some guidelines may specify specific behaviours:
% immutable and mutable%; prohibit or authorize the writing on the following attributes Directive
% rename (nouveau_nom) ancien_nom; change all references to ancien_nom and replaces them with nouveau_nom (which is handy in case ancien_nom is a keyword reserved Python)
% ignore name; can ignore all statements on behalf
When states able to function as a parameter another function and that these functions are described in the module, it is not possible to use them in Python. For example:
void fonction((int (*argument)(int, int)));
int add(int, int);
The function add accessible from Python is an encapsulated version of the C version that is no longer available to be given as an argument to function (). To solve this problem, use the% callback ( "% s_callback") and% nocallback to tell SWIG to create two objects: the first is the function add () classic, the second called add_callback () and may be given as an argument to function. It is however more callable from Python. In place of "% s_callback" it is possible to use "% (title) s" which will use the function name with a capital letter, "% (upper) s" to create a name in capital letters only or "% (lower) s "for only tiny. This channel is similar to the format string of function printf ().A great interest SWIG is to allow the creation of classes from a structure C. If a structure has been declared, the% extend helps create a bloc which can be defined methods working on the current instance of the structure (in the case, SWIG generates itself several wrappers to the attributes of structure): typedef struct structure{
} Structure;
%extend Structure{
Structure(/* Arguments */)
{
Structure* s;
s = (Structure*) malloc(sizeof(Structure));
/* Initializations */
return s;
}
~Structure()
{
free($self);
}
void methode()
{
/* Methode the structure */
}
}
The current instance in these methods is $ self. For the manufacturer, it does not exist and the user must allocate storage space and desallouer the destruction of the object. However, it is now possible to run a Python:
a = Structure()
a.methode()
The keyword% extend can be used at the time of the declaration of the structure (in the block struct). SWIG is also capable of detecting potential methods among the functions proposed free. For example, if new_Structure () exists, the statement by the manufacturer may be limited to Structure (/ * with the arguments */);, if delete_Structure () takes a single parameter, it may be used for destructive directly (no implementation of a destructive extend%) and if a function Structure_methode () whose first argument is a pointer to a structure exists, the method method () may be declared without implementations in% and extend this function will be called automatically.
If the free ends and more by _get or _set, SWIG implement an attribute in reading or writing and not a method. A structure need not be declared completely SWIG. Indeed, only the compiler thereafter will need a comprehensive definition, SWIG does the functions which will then work on the structure as a whole.
If a function needs to be declared in the bloc% ()% and in the list of exposure, the% inline may be added before %{%}. block In this case, SWIG and the compiler will use this code (very useful to declare utility which will be exposed as allowances or destruction, but it is not possible to add SWIG guidelines in the Code since it will be compiled C or C + +).
A macro SWIG is declared by a block define ma_macro%% enddef in the case of a constant or a function to define ma_macro% (ARG1, ARG2)% enddef.
Before moving to a more advanced, some rules are proposed to create a module encapsulating an existing code:
establish a precise list of functions, variables, structures to expose
include only the minimum worth of header files in blocks inclusion% ()%
create an interface file containing inclusions headers and definitions of the functions necessary
not hesitate to split into several pieces file interface and include the various sub-files in order to have a clear main interface
II. SWIG and C + +
SWIG supports almost any language C + +, a few surcharges such as new and delete or nested classes, but the rest is fully supported.
When it comes to classes, SWIG will generate a class called proxy in Python that will manage the interface with the underlying class C + +. The internal mechanisms are proposed for an instance of the proxy class can be assigned to an attribute of a class target. For example Classe1 has a pointer to Classe2, Python, it will be possible to assign a Python this pointer. In general, the internal mechanism is sufficient to properly manage memory. If this is not the case, it will manage to hand the cases using the methods disown () and acquire () of the proxy class. Attention also, if a Python object is created from a pointer C + + on Classe2, if the pointer is destroyed (because instance Classe1 was destroyed), the object will be in a state indefinitely because it destroys an object referencera .
SWIG default creates an encapsulation of the manufacturer and destructive, even if they are not explicitly generated (and the C + + generates a default version). To disable this setting, guidelines% nodefaultctor and nodefaultdtor%; are proposed and clearnodefaultctor% and clearnodefaultdtor%; to reactivate the generation. Regarding the manufacturer of copies, activation is by the% copyctor and deactivation by nocopyctor% (default constructor is not created copies).
If manufacturers or destructive are protected or private or if the class is encapsulated virtual pure, they will not be encapsulated.
Regarding the attributes and methods static SWIG automatically generates the appropriate code, or accessors is a simple call to the static function.
SWIG also manages a transparent manner inheritance, if a class inherits another in C + +, the encapsulated version of one also inherit the encapsulated version of the other.
Finally, SWIG is able to manage within the limits of the possible overloading of functions by testing the number of arguments and type to find out what function or what method call thereafter.
If a function taking a whole is overburdened by a function taking input another type of wide, SWIG will not be able to choose which to use in Python because there is only one type of whole (not counting whole length). One of the functions will be chosen and a warning message displayed. One solution to counter this problem is to ignore the function unnecessary or rename functions.
A major point of C + + is the concept of template. To successfully encapsulate a template (create a wrapper), we must instantiate. It is always possible to do so explicitly with a statement in the list exposure instantiated complete template for a type, but it is long and the type name should be changed (Python does not support vector <int > As the type of data). In this case, for a template presented in a block %{%}, Directive template (vector_int) vector <int>; allows instantiate the type of vector with the type int and call this type vector_int.
Regarding namespaces C + +, they are removed and all classes are exposed in the current module (to expose a subset namespace, a sub-module can be considered). It is imperative to check whether different classes in different namespaces have identical names.
The exceptions are supported through a native converters automatic (presented a little later) if exceptions are specified in the prototype of the function. In cases where a function or method raises an exception which is not in this list, a directive% catches () before a function declaration indicates that a certain number of exceptions (those proposed in parenthesis) are processed Python exceptions to the same and / or if ... is given in this bracket, any exception is caught and relaunched as a generic exception.
The smart pointers are managed by SWIG. If a class exposes the operator ->, all the attributes and methods of the class will be accessible by pointing a pointer intelligent, unless the attribute or function exists in the pointer.
III. Creation and use of converters
Several default converters are offered by SWIG and also by Numpy. Containers of the STL are set each in a file std_conteneur.i. By importing these files, SWIG knows how to convert the type C or C + + type Python.
Regarding the file exception.i which allows exceptions to redirect C + + exceptions Python, a directive can change the mode conversion : %exception /*a function or method to protect */{
try
{
$action
}
catch(const std::runtime_error& e)
{
SWIG_exception(SWIG_RuntimeError, const_cast<char*>(e.what());
}
} By default all functions or methods will be protected if any function or method is specified. In this case, SWIG generates the appropriate exception, the type of error may be proposed SWIG_MemoryError, SWIG_IOError, SWIG_RuntimeError, SWIG_IndexError, SWIG_TypeError, SWIG_DivisionZeroError, SWIG_OverflowError, SWIG_SyntaxError, SWIG_ValueError or SWIG_SystemError.
The main conversion tool is the directive and apply% clear directive to stop the effects of the first. The directive has this syntax:
%apply resultat{type_a_modifier};
/*code to convert */
%clear resultat ;
The result is the name of the rule of conversion is used and the type that will replace the type brackets.
This solution must be used to indicate a pointer to a std:: string should be transformed into a string const & or another solution based on references. For this, the% apply const string & (std:: string *); automates all conversions.
A major asset of conversions is to be able to specify an argument passed as a parameter is an argument to return, this argument will not be given to the call Python but returned by Python function (and if there are several arguments, a tuple be returned). Similarly arguments input and output are possible, they will always be returned and modified on the spot if their type permits.
%apply int* OUTPUT {int* resultat};
void ma_fonction(int a, int b, int* resultat);
This function will be named in Python:
resultat = ma_fonction(a, b); A conversion input / output is given by INPUT and a conversion only input is given by INPUT.
Constraints can also be applied:
POSITIVE to ensure that a number is strictly positive
NONNEGATIVE to ensure it is positive or zero
NEGATIVE number for a strictly negative
NONPOSITIVE for a zero or negative number
NONZERO for a different number of zero
NONNULL pointer for a non-zero.
If these constraints are not met, an exception Python is lifted.
Some of these contraines are available from the file constraints.i.
The real conversion mechanism allowing the user to define a method of conversion from one type to another is the typemaps. The syntax is as follows:
%typemap(methode[, modificateurs]) liste_types code; The method indicates that the typemap may make include:
in conducting a conversion type Python to C or C + +
carries out the reverse
typecheck can check the type of input (used to determine which overloaded function must be used) by returning 1 if the type is correct and 0 otherwise
argout can add to a result other values given in argument, the end result being a tuple or a list (that is what is used for variables passed argument which are actually results) check verifies the data input values (for example, allow only the positive values of a whole number) after conversion
arginit initiates a given before the conversion (usually this is not useful)
default can apply a default value to the argument of a function
freearg is drawn at the end of encapsulation to free up memory allocated for a typemap in
throw manages types of exceptions The list of types list_types is a separated list of reasons to be validated for conversion. It may be a single, a type with a name as an argument (in which case only the arguments with the same name and same type will be valid) or a list of types or with a simple name brackets (where an object Python be transformed into several objects in C or C + +). The types that are being reshaped with a typedef in different namespaces also validate the ground if the original definition validates this ground. Finally, with the% apply, it is possible to apply a new ground a ground defined by a typemap.
A typemap is valid for all the code that follows until it is redefined or deleted by% clear. However, if extend% is used to expand the possibilities of a class, the typemap must be defined before the definition of class.
The arguments given in typemap can be recovered in the statement. For example, $ n n is the argument given in the ground (this is the destination of conversion), $ n_name is his name, $ n_type its type and $ n_ltype simplified its type (without const keyword, Static tables are turned into pointers, ...). If $ * is used instead of $ type is the type returned and pointed $ & adds a pointer. $ n_basetype said about him the type underlying simple (int, float, ...) without any pointer or const. For tables passed as a parameter whose size is on file, $ n_dimi give the size depending on the size i.
The typemaps in, out and argout have additional variables. The first is $ symname, containing the name of the function being encapsulation. Then $ input is defined in and argout, and $ output and pour out argout.
If temporary values are needed to contain data for the call duration of the function, a typemap may specify in brackets after liste_types before the code. Even if several identical typemaps are needed to convert data, temporary variables do not overlap, SWIG add a distinguishing number at the end of the name of these variables. This temporary variable can be accessed in a typemap argout in postfixant the variable name by $ argnum which is the number of the argument amended (access to the temporary variable must be checked carefully to be sure that happens).
IV. SWIG with Numpy
Numpy offers typemaps for pre-existing SWIG in the file doc / numpy.i. This file can define:
tables input
tables changed on the spot
tables output but in past arguments
These typemaps can run on a blackboard in several dimensions (3 maximum), a pointer with dimensions data before or after the pointer. For example:
%apply (int IN_ARRAY2[ANY][ANY]) {(int matrix[ANY][ANY])};
void ma_fonction(int matrix[3][3]) ; The keyword ANY can tell that all sizes are allowed. Here, we apply the typemap indicating that we are working on a blackboard in 2 dimensions (IN_ARRAY2) and when my_function () will be encapsulated, a table Numpy may be passed as a parameter.
For tables changed on the spot, INPLACE_ARRAY * will be used for tables and output, it will ARGOUT_ARRAY *. Note that the dimension tables 2 and 3 can be specified in the form table [] [] or [][][] table and not in the form of pointers and dimensions involved. It is a limitation typemaps proposed and not SWIG.
To function properly, the plug of Numpy must be loaded in advance (depending import_array ()). For this, the following sequence is used: %{
#define SWIG_FILE_WITH_INIT
%}
%include "numpy.i"
%init %{
import_array();
%} The block init%%%) (glue its content directly into the module initialization proposed. A single call of this type must occur during the loading of the module, we should not copy in each sub-file interface.
The roster of numpy.i interface is a good example of comprehensive typemap but also a definition of macros and functions.
V. Example encapsulation of a structure with exposure of the interface Numpy
A complete Python class will be created, with the possibility of display and exhibition of the interface Numpy with __array_struct__.
Here is the structure in a file numpy_swig.h:
#ifndef NUMPY_SWIG
#define NUMPY_SWIG
typedef struct _SignedIntBuf
{
int* data;
int shape[2];
int strides[2];
} SignedIntBuf;
#endif The file interface numpy_swig.i is now on display in several parts:
%{
#define SWIG_FILE_WITH_INIT
%}
%include "numpy.i"
%init %{
import_array();
%}
%module (docstring="Ceci est un module Python encapsule par SWIG") numpy_swig
A module is created and associated documentation is proposed by the argument docstring. %{
#include "numpy_swig.h"
void delete_SignedIntBuf(SignedIntBuf* buffer)
{
free(buffer->data);
free(buffer);
}
void free_array_interface( void* ptr, void *arr )
{
PyArrayInterface* inter;
PyObject* arrpy;
inter = (PyArrayInterface*)ptr;
arrpy = (PyObject*)arr;
Py_DECREF(arrpy);
free(inter);
}
%} Two functions destruction which such data are in the code that will be generated. The first is the destroyer of a SignedIntBuf, structure, which was defined in the header precedent, and the second is the function of desallocation.
%inline %{PyObject* get__array_struct__(PyObject* self, int* shape, int* strides, int*data)
{
PyArrayInterface* inter;
PyObject* obj;
int nd;
nd = 2;
inter = (PyArrayInterface*)malloc(sizeof(PyArrayInterface));
if (inter==NULL)
return PyErr_NoMemory();
inter->two = 2;
inter->nd = nd;
inter->typekind = 'i';
inter->itemsize = sizeof(int);
inter->flags = NPY_NOTSWAPPED | NPY_ALIGNED | NPY_WRITEABLE;
inter->strides = strides;
inter->shape = shape;
inter->data = data;
Py_INCREF(self);
obj = PyCObject_FromVoidPtrAndDesc((void*)inter, (void*)self, free_array_interface);
return obj;
}
%} Here are the main function creating the interface Numpy from an object. In reality, this function takes a parameter being encapsulated SignedIntBuf but also directly the parameters necessary for the creation of the interface as the dimensions and data. Nothing is manually extracted here is SWIG which will do the job.
If the allocation of memory fails, an exception is raised, it is one of the exceptions provided in a previous section.
This code is carried in full in the code generated and SWIG knows the function, through the% inline.
Now comes the declaration of functions and variables exported with the extensions:
%include numpy_swig.h
%extend SignedIntBuf{
%feature("autodoc", "Une chaine de commentaire")SignedIntBuf;
SignedIntBuf(int width, int height)
{
SignedIntBuf* buffer = (SignedIntBuf*) malloc(sizeof(SignedIntBuf));
buffer->shape[0] = height;
buffer->shape[1] = width;
buffer->strides[0] = width * sizeof(int);
buffer->strides[1] = sizeof(int);
buffer->data = (int*) malloc(width*height*sizeof(int));
return buffer;
}
~SignedIntBuf();
char *__str__()
{
static char tmp[1024];
int i, j;
int used = 0;
used += sprintf(tmp, "Tableau :\n");
for(i=0; i < $self->shape[0]; i++)
{
for(j=0; j < $self->shape[1]; j++)
used += sprintf(tmp + used, "%d\t", $self->data[j + i*$self->shape[1]]);
used += sprintf(tmp + used, "\n");
}
return tmp;
}
%pythoncode
{
def __array_struct__get(self):
return get__array_struct__(self, self.shape, self.strides, self.data)
__array_struct__ = property(__array_struct__get, doc='Array protocol')
}
}; After included header C, an extension of the structure is proposed. The manufacturer is fully exposed here while the destroyer is simply prototype since it was established in a previous section.
% feature ( "autodoc" value); can create a chain of documentation automatically for value worth "0" or "1". Here, a chain is explicitly given for a particular function.
One solution is to go up a string describing the object. Here in C, a static array of characters is used and returned, but a string C + + would have been easier to handle.
Finally, a directive% pythoncode is given, it allows to fully block in the Python code generated. Here it is necessary because __array_struct__ must be a property Python. Moreover, if a method C is given directly to the accessor, it is impossible to recover the original Python pointer on the subject, which is necessary to indicate that the object is referenced in the interface (or on a risk memory corruption). That is why the accessor calls a function of free module with additional parameters to be automatically converted by SWIG in the underlying types.
Conclusion
This concludes the introduction to SWIG and its use for Numpy. The example given in the last part is unique on the Internet but is extremely useful to encapsulate a former scientific code and reuse easily with Numpy without making copies for each use.
This is the end of this first part devoted to the API C. To download the source code examples and other examples, see the page devoted to my book. |
|
|
