Skip to content

WIP: Add initial RIOT port#1249

Closed
kaspar030 wants to merge 20 commits intomicropython:masterfrom
kaspar030:add_riot_port
Closed

WIP: Add initial RIOT port#1249
kaspar030 wants to merge 20 commits intomicropython:masterfrom
kaspar030:add_riot_port

Conversation

@kaspar030
Copy link
Copy Markdown
Contributor

@kaspar030 kaspar030 commented May 11, 2015

This PR adds initial support for the RIOT operating system.

RIOT is a nice, portable, small but feature rich OS for small embedded devices, with small meaning from 4k RAM and up. The focus is on IoT applications, with a full-featured IPv6/6LoWPAN/RPL/... network stack nearing completion. It runs on 8bit, 16bit and 32bit platforms, sporting an O(1) preemptive scheduler, fast IPC and syncronization and nicely abstracted hardware support.

This port has been PRed for integration in RIOT as RIOT-OS/RIOT#2968. (Update: merged)

It cannot do much as no library has been ported.

TODO:

  • there's a bug that eats the parentheses when entering "help()" on upy's command line. But e.g., "print()" works fine. don't know why. (works fine on RIOT's native linux port) (seems to be pyterm related)
  • the garbage collector doesn't consider register contents, so it will probably not work correctly fixed
  • there needs to be either a mapping for riot C libs -> micropython libs or an adaption of micropythons "pyb." lib

This PR contains some cleanup commits:

  • one for accepting both 0xa and 0xd as newline for readline, needed to support our native port's console input
  • one that fixes obobject.c when not using MICROPY_CPYTHON_COMPAT (a .locals_dict had a broken definition when setting that flag)
  • one that changes a function pointer from void* to void(*)(void) to make it ISO C
  • one to make mp_binary_get_size unsigned as otherwise later comparisons trigger a sign-compare warning

@kaspar030
Copy link
Copy Markdown
Contributor Author

IMHO you guys should consider using RIOT as your HAL as to not have to reimplement hardware support for every new platform, but that's probably a veeeeery biased opinion. ;)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this shouldn't go into this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ups, changed back the wrong way.

@pfalcon
Copy link
Copy Markdown
Contributor

pfalcon commented May 11, 2015

IMHO you guys should consider using RIOT as your HAL as to not have to reimplement hardware support for every new platform, but that's probably a veeeeery biased opinion. ;)

I guess every RTOS could say the same, and that's a good reason to keep MicroPython core RTOS-free, while inviting as many as possible of them to participate on equal conditions. But RIOT is definitely something interesting which I for example had in my list for more detailed look for a long time. Guess this will be a nice chance to actually look a bit deeper into it ;-).

@kaspar030
Copy link
Copy Markdown
Contributor Author

I guess every RTOS could say the same, and that's a good reason to keep MicroPython core RTOS-free, while inviting as many as possible of them to participate on equal conditions.

You're probably right. ;)

Unfortunately RIOT is currently missing a file system, but I guess micropython will trigger development in that direction...

@danicampora
Copy link
Copy Markdown
Member

This PR adds initial support for the RIOT operating system.

Nice!

Unfortunately RIOT is currently missing a file system, but I guess micropython will trigger development in that direction...

We are already using FatFs in several ports, have you considered it?

@kaspar030
Copy link
Copy Markdown
Contributor Author

We are already using FatFS in several ports, have you considered it?

Actually, as I saw it in micropython, I got the same idea. So it works fine for you guys?

(We've had not-so-nice experiences with FAT in an early version of RIOT. Might have been the implementation.)

@danicampora
Copy link
Copy Markdown
Member

Actually, as I saw it in micropython, I got the same idea. So it works fine for you guys?

Works like a charm ;-)

(We've had not-so-nice experiences with FAT in an early version of RIOT. Might have been the implementation.)

What issues did you have in the past?

@dpgeorge
Copy link
Copy Markdown
Member

Certainly we can merge the small changes to the core.

But a whole new top-level directory which at the moment is almost the same as minimal/... that's hard to do.

@kaspar030 are you able to maintain this port?

@kaspar030
Copy link
Copy Markdown
Contributor Author

What issues did you have in the past?

It's been a while, but I remember that it was a nightmare to deal with all the variants and inconsistencies of FAT. But that was at a time when other things were more pressing, so this is no reason to not look into fatfs.

@kaspar030
Copy link
Copy Markdown
Contributor Author

@kaspar030 are you able to maintain this port?

I think so. Actually I'm pretty sure that my fellow RIOT friends will help fill in the pyb.* stuff and keep this updated.

I can keep this in my micropython fork for now until it has enough code to justify the top-level dir.

@kaspar030
Copy link
Copy Markdown
Contributor Author

BTW, great work so far with mircopython! Thanks a lot!

@danicampora
Copy link
Copy Markdown
Member

It's been a while, but I remember that it was a nightmare to deal with all the variants and inconsistencies of FAT.

So you guys were trying to write your own FAT implementation? Why reinvent the wheel? FatFs already takes care of all the dirty details, and it's AFAIK the most popular open source FAT implementation.

@kaspar030
Copy link
Copy Markdown
Contributor Author

@danicampora I don't remember... That was 2008, I was still a student and just WTFing over the commited code... I guess fatfs didn't fit somehow back than, or the reserach was very bad.

@danicampora
Copy link
Copy Markdown
Member

I don't remember... That was 2008

OK, I didn't know this project had that much history.

@kaspar030
Copy link
Copy Markdown
Contributor Author

OK, I didn't know this project had that much history.

Well, it had at least two different names until it came to new life as RIOT in 2013. The core was started in 2008.

I just realized that the ominous "help()" problem I'm experiencing seems to be with our terminal (pyterm), using picocom, it works fine.

@OlegHahm
Copy link
Copy Markdown

So you guys were trying to write your own FAT implementation? Why reinvent the wheel?

No, we were using the implementation from ChaN: http://elm-chan.org/fsw/ff/00index_e.html

@dpgeorge
Copy link
Copy Markdown
Member

We use the ChaN fatfs library. So far we have had very few issues with it (I think the only issue we had was recently fixed in the latest version).

@OlegHahm
Copy link
Copy Markdown

I cannot say much about the problems with FAT back then, because I never had a use case for it, but maybe it just improved during the last seven years. ;)

@dpgeorge
Copy link
Copy Markdown
Member

Do you have some info on how to build a complete binary image for a real board, such as the STM32F4 Discovery board?

@kaspar030
Copy link
Copy Markdown
Contributor Author

You'd checkout the branch, got to test/pkg_micropython, then "BOARD=stm32f4discovery make clean all flash", then "BOARD=stm32f4discovery make term". Maybe you also have to specify the serial port the board is connected to, e.g., "BOARD=stm32f4discovery PORT=/dev/ttyUSB1 make term".

Our default terminal does a lot of what mp-readline does, so expect the cli to be flakey.

edit with "checkout the branch" I mean check out the RIOT micropython branch mentioned in the respective PR (RIOT-OS/RIOT#2968).

@dpgeorge
Copy link
Copy Markdown
Member

I can confirm that it works on an STM32F4 Discovery board. Note that there is no USB driver in RIOT, rather the serial console is via UART2 on PA2=TX, PA3=RX.

@dpgeorge
Copy link
Copy Markdown
Member

I merged 3 of the commits making changes to the core (to obj.h, binary.[ch] and objobject.c), which should fix your warnings.

Regarding the change to readline, allowing '\n' to be equivalent to '\r': we could merge this, I think it doesn't hurt as there would be no other use for '\n' except to make a newline (is it ever the case that you can receive \n and \r to mean 1 newline?).

Edit: for reference here are the commits: cf5112b, b5cef5c, f5dd6f7.

@pfalcon
Copy link
Copy Markdown
Contributor

pfalcon commented May 17, 2015

Regarding the change to readline, allowing '\n' to be equivalent to '\r': we could merge this, I think it doesn't hurt as there would be no other use for '\n' except to make a newline (is it ever the case that you can receive \n and \r to mean 1 newline?).

Yes, I would like to know more details why this was needed too - I also think it won't hurt, but then sooner or later we may receive bug report of pressing Enter giving an extra empty line with someone using some weird terminal app, then we need to assess which fix is more important...

@dpgeorge
Copy link
Copy Markdown
Member

Such a change to the newline handling code would actually be useful also to the unix port when it uses mp-readline. When I made #1263 I was almost going to allow '\n' to be a newline, but instead did this: https://github.com/micropython/micropython/pull/1263/files#diff-15f11433aab2b0271dd5463445d67282R110

@kaspar030
Copy link
Copy Markdown
Contributor Author

Yes, I would like to know more details why this was needed too

Well, we have this "native port", which is basically a RIOT instance running as Linux or OSX user space process. That is tremendously useful for debugging. But as it's "serial console" is the host system's stdin, it receives a '\n' as newline character. So we're accepting both '\r' and '\n' as newline char.

We could also go for the way it's done in #1263, your call.

@kaspar030
Copy link
Copy Markdown
Contributor Author

Please excuse the lack of updates here. We're trying to get a release out, I will find time to update this afterwards (couple of weeks).

@jgriessen
Copy link
Copy Markdown

Is there a plan to let RIOT code be importable to work with micropython, or has this been dropped?

@dpgeorge
Copy link
Copy Markdown
Member

Is there a plan to let RIOT code be importable to work with micropython, or has this been dropped?

We would need @kaspar030 to work on it. Otherwise it'll be considered dropped (for now).

By default, tools/pyboard.py writes exec() code to serial very fast.
Most RIOT boards don't configure UART flow control, causing missing
bytes.

This commit adds a patch that makes pyboard.py write really slow,
allowing MicroPython's test suite to be executed.
@jedie
Copy link
Copy Markdown
Contributor

jedie commented Jan 17, 2020

The link in the initial Post is wrong here:

This port has been PRed for integration in RIOT as #2968 (in riot's github repo).

The right Pull Request @ RIOT is -> RIOT-OS/RIOT#2968

@kaspar030
Copy link
Copy Markdown
Contributor Author

The right Pull Request @ RIOT is -> RIOT-OS/RIOT#2968

Thanks, fixed!

We've merged this into RIOT. Working on getting at least basic peripherals in the machine module working.

@dpgeorge
Copy link
Copy Markdown
Member

@kaspar030 it's great you have integrated MicroPython into RIOT, and I encourage you to maintain that integration. But I think it makes much more sense for you to have that uPy-RIOT port in the RIOT code base, and built as part of your CI, etc. Rather than having it hosted here.

We are happy to make changes to core MicroPython to facilitate your out-of-tree port, so please to raise any PRs if you need changes like that made.

I'll close this PR, but thanks for all your efforts!

@dpgeorge dpgeorge closed this Jun 24, 2022
@OlegHahm
Copy link
Copy Markdown

May I ask why you think that RIOT support should not be integrated into upstream micropython? I see that already many different platforms and OS are supported, so basically RIOT would be just another candidate in the list. Testing its support for various supported hardware platforms in RIOT would still happen in the RIOT CI.

@jimmo
Copy link
Copy Markdown
Member

jimmo commented Dec 15, 2023

@OlegHahm I agree it would be nice to be able to take this on, but it's not that simple. Adding an upstream port requires:

  • An extensive and code review to initially merge it
  • Ongoing maintenance to keep it up to date as MicroPython evolves
  • Maintainer experience and knowledge with the underlying port/hardware/os (in this case, RIOT) to review future PRs
  • Possibly providing pre-built downloads / firmware images like we do for other ports

Right now we just don't have the capacity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ports Relates to multiple ports, or a new/proposed port

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants