I will clear out some time to give this a try. I can also take a stab at writing up some user documentation. How/where would you like that, and do you have any specific topics you think should be covered?

@dbc great if you could try it! Feedback would be the most useful thing at this point. If you want to start documenting it, it might be best to just make a simple page explaining how to get a demo working, and wait for more complete docs when this feature has become more stable. I'd say making a page on the github wiki is the best place at the moment, since it's mainly for developers.

Using source files as of today (17th Nov) I'm seeing

[adminpete@axolotl]: /mnt/qnap2/data/Projects/MicroPython/micropython/unix
$ make MICROPY_FORCE_32BIT=1 cross
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
make: *** No rule to make target `cross'. Stop.

@peterhinch It works for me. Did you checkout upy-cross-compiler branch from dpgeorge/micropython repo?

Ah, I'd missed that - it builds and runs fine now. A great facility - I'll give it some stick over the next few days :)

I have been playing with this functionality the last couple days, and was trying to use it in the unix build. I added MICROPY_PRESISTENT_CODE_LOAD.

I had mixed results with this python file:

def test1(num):
    print(num)

def test2(num):
    return (num+1)

./micropython_cross -v -X emit=bytecode test.py

MicroPython v1.4.4-672-gadc1251-dirty on 2015-11-20; linux version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> import test
>>> test.test1(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "test.py", line 2, in test1
TypeError: 'int' object is not callable
>>> test.test2(1)
2

I am really looking forward to this because I have a bit of IoT hardware that runs micropython and I upload user scripts to the flash on it and execute them. Right now I am parsing a minimized script which take up a massive amount of flash space. Please let me know if there is anywhere I can help, I am just getting caught up on this feature.

@btashton thanks for testing! The issue you are having is because the bytecode executed by standard unix interpreter uses opcode caching, whilst that produced by the cross compiler does not. There will eventually be a bit in the .mpy file which checks for this compatibility, but for now you just need to make sure that MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE is defined to the same value in the cross compiler and the interpreter. I've pushed a new commit to this branch that disables this caching in the unix port, so your example now works.

Awesome. Found another corner bug. Since I dont want the interpreter pulling in my .py file I called it test.py1 and ran:

./micropython_cross -v -X emit=bytecode test.py1

This ends up generating: test.pmpy
which contains the correct bytecode but a thrashed extension.

I just noted that load_obj is not implemented. Is this something that is intended to be added? I am happy to venture down this path.

micropython: ../py/emitglue.c:256: load_obj: Assertion `0' failed.

I just noted that load_obj is not implemented. Is this something that is intended to be added? I am happy to venture down this path.

@btashton Thanks for the offer to help but I needed this feature myself, so implemented it in 44e6e34.

This look fantastic. What typical space savings would one expect using precompiled micropython bytecode?

I have not been able to get this to compile recently.

LINK micropython_cross
build-cross/main.o: In function `main':
main.c:(.text.startup.main+0x391): undefined reference to `mp_verbose_flag'
build-cross/lib/utils/printf.o: In function `putchar':
printf.c:(.text.putchar+0x14): undefined reference to `mp_hal_stdout_tx_strn_cooked'
build-cross/lib/utils/printf.o: In function `puts':
printf.c:(.text.puts+0x1a): undefined reference to `mp_hal_stdout_tx_strn_cooked'
printf.c:(.text.puts+0x2e): undefined reference to `mp_hal_stdout_tx_strn_cooked'
collect2: error: ld returned 1 exit status
../py/mkrules.mk:83: recipe for target 'micropython_cross' failed

This patch should fix the build issues, including one with the minimal config:

diff --git a/unix/main.c b/unix/main.c
index 417c73a..3992d18 100644
--- a/unix/main.c
+++ b/unix/main.c
@@ -548,7 +548,9 @@ int main(int argc, char **argv) {
             if (strcmp(argv[a], "-X") == 0) {
                 a += 1;
             } else if (strcmp(argv[a], "-v") == 0) {
+                #if MICROPY_DEBUG_PRINTERS
                 mp_verbose_flag++;
+                #endif
             } else if (strncmp(argv[a], "-O", 2) == 0) {
                 if (unichar_isdigit(argv[a][2])) {
                     MP_STATE_VM(mp_optimise_value) = argv[a][2] & 0xf;
diff --git a/unix/unix_mphal.c b/unix/unix_mphal.c
index 1edf3bb..e7969a2 100644
--- a/unix/unix_mphal.c
+++ b/unix/unix_mphal.c
@@ -96,6 +96,14 @@ void mp_hal_stdio_mode_orig(void) {

 #endif

+mp_uint_t mp_hal_ticks_ms(void) {
+    struct timeval tv;
+    gettimeofday(&tv, NULL);
+    return tv.tv_sec * 1000 + tv.tv_usec / 1000;
+}
+
+#endif // MICROPY_ENABLE_RUNTIME
+
 int mp_hal_stdin_rx_chr(void) {
     unsigned char c;
     int ret = read(0, &c, 1);
@@ -120,11 +128,3 @@ void mp_hal_stdout_tx_strn_cooked(const char *str, mp_uint_t len) {
 void mp_hal_stdout_tx_str(const char *str) {
     mp_hal_stdout_tx_strn(str, strlen(str));
 }
-
-mp_uint_t mp_hal_ticks_ms(void) {
-    struct timeval tv;
-    gettimeofday(&tv, NULL);
-    return tv.tv_sec * 1000 + tv.tv_usec / 1000;
-}
-
-#endif // MICROPY_ENABLE_RUNTIME

This look fantastic. What typical space savings would one expect using precompiled micropython bytecode?

@ryannathans In terms of ROM savings, I'm not sure yet. Need to add an option to disable the compiler, but I guess it would save maybe 20k of ROM. But RAM savings are also significant because you no longer need room for the parse tree. It's a good option for very constrained systems.

@btashton thanks, now fixed.

I rebased this against the latest master. You can find my rebased branch here:
https://github.com/dhylands/micropython/tree/rebase-pr-1619

The changes needed were pretty minor. I did find that I had to add this line:

#define MICROPY_PY_BUILTINS_STR_UNICODE (1)

to unix/mpconfigport_cross.h before the mpy files could be loaded on the pyboard. I'll some testing with some real code soon.

I also had to enable MICROPY_EMIT_INLINE_THUMB in order to get @micropython.asm_thumb recognized.

We'll also have to decide how to deal with stm constants. When I tried to compile bytecode for this line:

    ldr(r3, [r0, stm.USART_CR1])

I got the error: SyntaxError: 'ldr' expects an integer

One solution would be have a different cross compiler for each target (I suggest it because its feasible).

I think that a better solution would be to have each target generate some source code which gets compiled into a shared library and have the cross compiler import a appropriate constants from an appropriately named shared library (probably derived from a command line option).

Even for stmhal, the constants in the stm module differ based on the MCU.

We'll also have to decide how to deal with stm constants.

Yes that is a good point. Having a binary per-MCU is the easiest solution.

Another idea would be to have a Python script with the constants defined (eg stm.py), and the cross compiler just loads and executes this script first, inserting the constants into its internal table. Like a C header file.

Another idea would be to have a Python script with the constants defined (eg stm.py), and the cross compiler just loads and executes this script first, inserting the constants into its internal table. Like a C header file.

Yeah - I like that even better. We can have the stmhal build (or whatever build is being used) generate a python file along with the current qstr/constant files that it generates now, and have a command line option on the cross-compiler to specify a file to be imported. We probably just need to decide on a particular module name (likes "constants") with a particular dictionaty name, that the cross compiler would use, and importing the command-line specified module would then add entries to that dictionary.

A cross compiler was merged in 56f76b8 which is dynamically configurable at runtime. So that makes this PR obsolete.

Only thing from the discussion here that would be good to remember is the ability to have constants specified by an external py script. But implementing such thing is for another day.