Adding a module to Micropython
Using Micropython on your own hardware, like I do on Raspberry Pi, is quite a challenge. First of all you have to port Micropython. But when it runs on your platform you will probably need some module for Python, to extent the language itself by some functionalities only accessible in C or Assembler. These could be I/O-functionalities like writing to device-registers or gathering the state of your platform, or you may just want to write something you could do in Python in a compiled language for performance and/or memory footprint reasons.
This article shows how to add simple modules to Micropython. Please be aware, that I cannot show everything here, and it is just an example scenario. You are off course free to investigate the stmhal-port of Micropython to find some examples of bigger modules. This article mostly got its information from [1].
Adding your own source-files
Adding C source files is easy enough. Have a look into the Makefile of the port you want to use. You will find a part which lists all of the source-files used to compile the specific port. This looks like this (from bare-rpi/Makefile):
SRC_C =
$(SRC)main.c
$(SRC)irq.c
$(SRC)module_rawptr.c
$(SRC)module_cdebug.c
$(SRC)module_cio.c
This example uses five C-files to build the port, three of which coincidentally add modules already. Create a source-file and add its name to the list of C-files. Be sure to use a relative path from the Makefile.
When using a platform with very limited (instruction) RAM be aware of the following [1]:
The second file you will need to add to is esp8266.ld, which is the map of memory used by the compiler. You have to add it to the list of files to be put in the .irom0.text section, so that your code goes into the instruction read-only memory (iROM). If you fail to do that, the compiler will try to put it in the instruction random-access memory (iRAM), which is a very scarce resource, and which can get overflown if you try to put too much there.
Create a first module
To create an empty Micropython module we need to use the type mp_obj_module_t, which is the type of the modules in Micropython. This type wants a property which defines its base and a dictionary of globally defined names. Let's add the following code to our file:
#include "py/mpconfig.h"
#include "py/nlr.h"
#include "py/misc.h"
#include "py/qstr.h"
#include "py/obj.h"
#include "py/runtime.h"
// create a table of global strings, here the only one is name
STATIC const mp_map_elem_t global_table[] =
{
{ MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_mymodule) },
};
// create a dictionary of strings from the table
STATIC MP_DEFINE_CONST_DICT(mymodule_globals, global_table);
// fill the mp_obj_module_t struct, which defines the module itself
const mp_obj_module_t mymodule =
{
.base = { &mp_type_module },
.globals = (mp_obj_dict_t*)&mymodule_globals,
};
This code create an "empty" module for Micropython. It fills the mp_obj_module_t-struct with the bare-minimum of information to be loadable by Micropython. The only thing in its global-dictionary is the string "mymodule". You have to add this name to the list of QStrings manually. Add the following line to the file qstrdefsport.h:
Q(mymodule)
Micropython can't just load the module by itself, it has to be told which modules exist and which modules you want to have built into the kernel/firmware. This can be done by adding it to the list of BUILTIN Modules in mpconfigport.h:
extern const struct _mp_obj_module_t mymodule;
#define MICROPY_PORT_BUILTIN_MODULES
{ MP_OBJ_NEW_QSTR(MP_QSTR_myfirstmodule), (mp_obj_t)&mymodule }
The first line shows the C-Compiler, that there is (or at least will be) a variable called mymodule when linking. This does not instantiate the variable itself, but rather tell the C-Compiler, that this variable will be defined and instantiated in another C-file. This line could also be put into a separate header-file.
The third and fourth lines are more interesting. The preprocessor-constant MICROPY_PORT_BUILTIN_MODULES tells the build-process which modules will be built in and will be callable from inside a Python-script. The first parameter will be the name of the module, which you need to load it from the script, the second one is the module-variable we filled earlier.
Loading this empty module from inside a Python-script can be done by:
import myfirstmodule
Add a function
As a first function in our module it would be nice to have one, which prints "Hello World", would it not? So let's define a C-function which does exactly that in the file of our module from earlier:
#include <stdio.h>
STATIC mp_obj_t mymodule_hello(void)
{
printf("Hello world!\n");
return mp_const_none;
}
This is not quite your normal "Hello World"-program, is it? At first it returns an mp_obj_t - a Python-object. The return value, which will actually be returned is mp_const_none.
A bit more technical
So let's look up those two things in the Micropython-code:
// This is the definition of the opaque MicroPython object type.
// All concrete objects have an encoding within this type and the
// particular encoding is specified by MICROPY_OBJ_REPR.
#if MICROPY_OBJ_REPR == MICROPY_OBJ_REPR_D
typedef uint64_t mp_obj_t;
typedef uint64_t mp_const_obj_t;
#else
typedef void *mp_obj_t;
typedef const void *mp_const_obj_t;
#endif
First off this is the definition of mp_obj_t. In the most simple way, this is a void-pointer, so a pointer without compile-time information about its type. So it is basically a blanko-check with no questions asked, when you or the Micropython-code want(s) to access it. Behind this pointer could be anything.
Now the mp_const_none:
// Constant objects, globally accessible
// The macros are for convenience only
#define mp_const_none (MP_OBJ_FROM_PTR(&mp_const_none_obj))
#define mp_const_false (MP_OBJ_FROM_PTR(&mp_const_false_obj))
#define mp_const_true (MP_OBJ_FROM_PTR(&mp_const_true_obj))
#define mp_const_empty_bytes (MP_OBJ_FROM_PTR(&mp_const_empty_bytes_obj))
#define mp_const_empty_tuple (MP_OBJ_FROM_PTR(&mp_const_empty_tuple_obj))
extern const struct _mp_obj_none_t mp_const_none_obj;
extern const struct _mp_obj_bool_t mp_const_false_obj;
extern const struct _mp_obj_bool_t mp_const_true_obj;
extern const struct _mp_obj_str_t mp_const_empty_bytes_obj;
extern const struct _mp_obj_tuple_t mp_const_empty_tuple_obj;
extern const struct _mp_obj_singleton_t mp_const_ellipsis_obj;
extern const struct _mp_obj_singleton_t mp_const_notimplemented_obj;
extern const struct _mp_obj_exception_t mp_const_MemoryError_obj;
extern const struct _mp_obj_exception_t mp_const_GeneratorExit_obj;
#include <stdlib.h>
#include "py/nlr.h"
#include "py/obj.h"
#include "py/runtime0.h"
typedef struct _mp_obj_none_t {
mp_obj_base_t base;
} mp_obj_none_t;
STATIC void none_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
(void)self_in;
if (MICROPY_PY_UJSON && kind == PRINT_JSON) {
mp_print_str(print, "null");
} else {
mp_print_str(print, "None");
}
}
const mp_obj_type_t mp_type_NoneType = {
{ &mp_type_type },
.name = MP_QSTR_NoneType,
.print = none_print,
.unary_op = mp_generic_unary_op,
};
const mp_obj_none_t mp_const_none_obj = {{&mp_type_NoneType}};
mp_const_none is a preprocessor-statement, which will be substituted to a pointer to mp_const_none_obj which in turn is an Python-object of mp_type_NoneType, which only has a print-function, that prints either "null" or "None". I don't want to go into more of a detail here.
Back to our function
For us to being able to use the function, we need to create a constant function-object. This is done by MP_DEFINE_CONST_FUN_OBJ_0. The last number stands for the number of arguments, this function will get. Our function does not want any arguments, so we will use 0:
STATIC MP_DEFINE_CONST_FUN_OBJ_0(mymodule_hello_obj, mymodule_hello);
Adding this function to our module is now quite easy, as we only need to add it to the global-names, so lets extent the global_table-variable from earlier:
STATIC const mp_map_elem_t global_table[] =
{
{ MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_mymodule) },
{ MP_OBJ_NEW_QSTR(MP_QSTR_hello), MP_OBJ_NEW_QSTR(MP_QSTR_hello_obj) }
};
This adds the object under the name "hello" to the dictionary of globally available functions of that module. We have to add the function to the list of QStrings again. Now it is possible to call our function by the following script:
import mymodule
mymodule.hello()
Function arguments
You probably already guessed it - the arguments to function will be of mp_obj_t type. So let's create a new function which greets someone with his/her actual name (when provided as argument):
STATIC mp_obj_t mymodule_greet(mp_obj_t who)
{
printf("Hello %s!\n", mp_obj_str_get_str(who));
return mp_const_none;
}
STATIC MP_DEFINE_CONST_FUN_OBJ_1(mymodule_greet_obj, mymodule_greet);
mp_obj_str_get_str receives a string-object (again: mp_obj_t itself is a void-pointer, the Micropython hopes to find a suitable struct at the end) and returns a C-string representation of the information stored inside. This function will btw. automatically raise a Python-exception if the type is not correct. Our function can be used to create a constant function-object with MP_DEFINE_CONST_FUN_OBJ_1, as it takes one argument.
Digging deeper
So what does MP_DEFINE_CONST_FUN_OBJ_1 do exactly? I wanted to know and began digging:
#define MP_DEFINE_CONST_FUN_OBJ_0(obj_name, fun_name)
const mp_obj_fun_builtin_fixed_t obj_name =
{{&mp_type_fun_builtin_0}, .fun._0 = fun_name}
#define MP_DEFINE_CONST_FUN_OBJ_1(obj_name, fun_name)
const mp_obj_fun_builtin_fixed_t obj_name =
{{&mp_type_fun_builtin_1}, .fun._1 = fun_name}
#define MP_DEFINE_CONST_FUN_OBJ_2(obj_name, fun_name)
const mp_obj_fun_builtin_fixed_t obj_name =
{{&mp_type_fun_builtin_2}, .fun._2 = fun_name}
The macro will be substituted by the creation of a variable of the type mp_obj_fun_builtin_fixed_t. The type-attribute will be set accordingly to the number of arguments of the function and the sub-union fun will be used to set either the _0, _1 or _2-attribute. The attributes share the same space in memory (see union). All of these attributes are of pointer-type to a function. The only distinguishing fact about these functions is the number of arguments.
typedef mp_obj_t (*mp_fun_0_t)(void);
typedef mp_obj_t (*mp_fun_1_t)(mp_obj_t);
typedef mp_obj_t (*mp_fun_2_t)(mp_obj_t, mp_obj_t);
typedef mp_obj_t (*mp_fun_3_t)(mp_obj_t, mp_obj_t, mp_obj_t);
typedef mp_obj_t (*mp_fun_var_t)(size_t n, const mp_obj_t *);
/// ...
typedef struct _mp_obj_fun_builtin_fixed_t {
mp_obj_base_t base;
union {
mp_fun_0_t _0;
mp_fun_1_t _1;
mp_fun_2_t _2;
mp_fun_3_t _3;
} fun;
} mp_obj_fun_builtin_fixed_t;
Adding a class
When adding a class we will have again to add the name of the class to the global name table of the module. But first we have to create the innards of the class itself. The class data itself is handled by a C-struct. This struct needs an mp_obj_base_t base-attribute. This will have some basic information like the type. Then you add every field you want to use within the class or objects, like this:
// this is the actual C-structure for our new object
typedef struct _mymodule_hello_obj_t
{
mp_obj_base_t base; // base represents some basic information, like type
uint8_t hello_number; // a member created by us
} mymodule_hello_obj_t;
This struct is not visible to Python in any way, but can be used within our C-functions and methods to access the data we want / need. See the methods below for more information on that.
Now let's create the type of the object:
// creating the table of local members
STATIC const mp_rom_map_elem_t mymodule_hello_locals_dict_table[] = { };
STATIC MP_DEFINE_CONST_DICT(mymodule_hello_locals_dict, mymodule_hello_locals_dict_table);
// create the class-object itself
const mp_obj_type_t mymodule_helloObj =
{
{ &mp_type_type }, // "inherit" the type "type"
.name = MP_QSTR_helloObj, // give it a name
.print = mymodule_hello_print, // give it a print-function
.make_new = mymodule_hello_make_new, // give it a constructor
.locals_dict = (mp_obj_dict_t*)&mymodule_hello_locals_dict, // and the local members
};
This code first defines a local dictionary for the local members of the type. Then define the class-structure itself. A class always needs a name, a new, and a print-method. This is required by Python, so we also need to define those, like this (above the code above):
mp_obj_t mymodule_hello_make_new(const mp_obj_type_t *type, size_t n_args, size_t n_kw, const mp_obj_t *args)
{
// this checks the number of arguments (min 1, max 1);
// on error -> raise python exception
mp_arg_check_num(n_args, n_kw, 1, 1, true);
// create a new object of our C-struct type
mymodule_hello_obj_t *self = m_new_obj(mymodule_hello_obj_t);
// give it a type
self->base.type = &mymodule_hello_type;
// set the member number with the first argument of the constructor
self->hello_number = mp_obj_get_int(args[0])
return MP_OBJ_FROM_PTR(self);
}
STATIC void mymodule_hello_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind)
{
// get a ptr to the C-struct of the object
mymodule_hello_obj_t *self = MP_OBJ_TO_PTR(self_in);
// print the number
printf ("Hello(%u)", self->hello_number);
}
The new-method will create a new object of the class-type. First it checks the number of arguments, then creates a new object of our struct-type, gives it a type, sets a number and returns the object itself. Be aware, that the method can receive loads of arguments (args), but the actual number of arguments is determined by n_args. The print-method is called, when a Python-script calls "print ( our_object )", i.e. is expected to print some information about the object to screen.
You will probably want to add "helloObj" to the list of QStrings, as well as to the list of global names of the module:
STATIC const mp_map_elem_t global_table[] =
{
{ MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_mymodule) },
{ MP_OBJ_NEW_QSTR(MP_QSTR_hello), MP_OBJ_NEW_QSTR(MP_QSTR_hello_obj) },
{ MP_OBJ_NEW_QSTR(MP_QSTR_helloObj), (mp_obj_t)&mymodule_helloObj_obj }
};
If you want to add a method or two, create a C-function then add it to the local dictionary of the class and add the name of the function to the list of QStrings:
STATIC mp_obj_t mymodule_hello_increment(mp_obj_t self_in)
{
mymodule_hello_obj_t *self = MP_OBJ_TO_PTR(self_in);
self->hello_number += 1;
return mp_const_none;
}
MP_DEFINE_CONST_FUN_OBJ_1(mymodule_hello_increment_obj, mymodule_hello_increment);
STATIC const mp_rom_map_elem_t mymodule_hello_locals_dict_table[] =
{
{ MP_ROM_QSTR(MP_QSTR_inc), MP_ROM_PTR(&mymodule_hello_increment_obj) },
}
Raising Exceptions
Raising Exceptions could be quite interesting from inside a C-module for Micropython. You might want to signal to Python, that some I/O-operation did not work or some other error-state. You can use the nlr_raise-Makro for that.
You can see mp_raise_msg for an example of the usage of that makro:
NORETURN void mp_raise_msg(const mp_obj_type_t *exc_type, const char *msg) {
if (msg == NULL) {
nlr_raise(mp_obj_new_exception(exc_type));
} else {
nlr_raise(mp_obj_new_exception_msg(exc_type, msg));
}
}
Conclusions
This article depicted briefly how you can go and create your own modules for the Python interpreter Micropython. You should now be able to use the interface, and I hope I could shed some light on the back-stage part.
References
[1] https://github.com/naums/micropython-dev-docs/blob/master/adding-module.rst
[2] https://github.com/micropython/micropython/blob/master/py/obj.h