Skip to content

Conversation

@samskiter
Copy link

just pulled in the latest master branch from the parent repo - i think this is correct - hopefully it helps you keep your PR up to date?

dpgeorge and others added 30 commits July 20, 2023 18:17
This adds named-pins support to the esp32 port, following other ports.
Since the name of esp32 CPU pins is just GPIOx, where x is an integer, the
Pin.cpu dict is not supported and CPU pins are just retrieved via their
existing integer "name" (the cost of adding Pin.cpu is about 800 bytes,
mostly due to the additional qstrs).

What this commit supports is the Pin.board dict and constructing a pin by
names given by a board.  These names are defined in a pins.csv file at the
board level.  If no such file exists then Pin.board exists but is empty.

As part of this commit, pin and pin IRQ objects are optimised to reduce
their size in flash (by removing their gpio_num_t entry).  The net change
in firmware size for this commit is about -132 bytes.

Signed-off-by: Damien George <[email protected]>
Includes fixing the SCK connection pin.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Re-enable some features required for the board to still build and the lora
driver to run.

This board only has 192KB of flash total, so default stm32 build is very
close to the limit.

Before:

LINK build-B_L072Z_LRWAN1/firmware.elf
   text    data     bss     dec     hex filename
 184352      68   14112  198532   30784 build-B_L072Z_LRWAN1/firmware.elf

(12256 bytes free)

After:

LINK build-B_L072Z_LRWAN1/firmware.elf
   text    data     bss     dec     hex filename
 155028      68   14052  169148   294bc build-B_L072Z_LRWAN1/firmware.elf

(41580 bytes free)

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Some targets like the ESP32-S3 use the IDF Component Manager to provide
additional dependencies to the build.  Make sure to include these extra
components when collecting properties used by MicroPython-specific build
steps, like qstr preprocessing.

Signed-off-by: Luca Burelli <[email protected]>
Implement a standard machine.bootloader() method for ESP32-series devices.
No default implementation, each board can enable it as required.

Signed-off-by: Luca Burelli <[email protected]>
Allow boards to define their own additional USB callbacks.

Signed-off-by: Luca Burelli <[email protected]>
On targets that provide a reference TinyUSB implementation, like ESP32,
the SDK already defines and implements standard callback functions such
as tud_cdc_line_state_cb(). This causes a symbol clash when enabling
shared implementations like the MicroPython 1200 touch functionality.

To avoid this symbol clash, add an optional macro to allow ports to
use a different function name in the shared implementation.

Signed-off-by: Luca Burelli <[email protected]>
The EXTRA_COMPONENT_DIRS variable is a list so adding a directory so should
be done via append, not set.  This enables boards to use other components
in the build. See:
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/build-system.html#optional-project-variables
This will be replaced with a new deflate module providing the same
functionality, with an optional frozen Python wrapper providing a
replacement zlib module.

binascii.crc32 is temporarily disabled.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
There are enough places that implement __exit__ by forwarding directly to
mp_stream_close that this saves code size.

For the cases where __exit__ is a no-op, additionally make their
MP_STREAM_CLOSE ioctl handled as a no-op.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
The compression algorithm implemented in this commit uses much less memory
compared to the standard way of implementing it using a hash table and
large look-back window.  In particular the algorithm here doesn't allocate
hash table to store indices into the history of the previously seen text.
Instead it simply does a brute-force-search of the history text to find a
match for the compressor.  This is slower (linear search vs hash table
lookup) but with a small enough history (eg 512 bytes) it's not that slow.
And a small history does not impact the compression too much.

To give some more concrete numbers comparing memory use between the
approaches:

- Standard approach: inplace compression, all text to compress must be in
  RAM (or at least memory addressable), and then an additional 16k bytes
  RAM of hash table pointers, pointing into the text

- The approach in this commit: streaming compression, only a limited amount
  of previous text must be in RAM (user selectable, defaults to 512 bytes).

To compress, say, 1k of data, the standard approach requires all that data
to be in RAM, plus an additional 16k of RAM for the hash table pointers.
With this commit, you only need the 1k of data in RAM.  Or if it's
streaming from a file (or elsewhere), you could get away with only 256
bytes of RAM for the sliding history and still get very decent compression.

In summary: because compression takes such a large amount of RAM (in the
standard algorithm) and it's not really suitable for microcontrollers, the
approach taken in this commit is to minimise RAM usage as much as possible,
and still have acceptable performance (speed and compression ratio).

Signed-off-by: Damien George <[email protected]>
Because we only use the streaming source, this is just extra code size.

Saves 64 bytes on PYBV11.

Signed-off-by: Jim Mussared <[email protected]>
This commit makes the following changes:
- Replace 256-byte reverse-bits-in-byte lookup table with computation.
- Replace length and distance code lookup tables with computation.
- Remove comp_disabled check (it's unused).
- Make the dest_write_cb take the data pointer directly, rather than the
  Outbuf.

Saves 500 bytes on PYBV11.

Signed-off-by: Jim Mussared <[email protected]>
This library used a mix of "tinf" and "uzlib" to refer to itself.  Remove
all use of "tinf" in the public API.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
This supports `wbits` values between +40 to +47.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Saves 68 bytes on PYBV11.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Collapsing the two adjacent calls to outbits saves 32 bytes.

Bringing defl_static.c into lz77.c allows better inlining, saves 24 bytes.

Merge the Outbuf/uzlib_lz77_state_t structs, a minor simplification that
doesn't change code size.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
For better abstraction for users of this API.

Signed-off-by: Jim Mussared <[email protected]>
This provides similar functionality to the former zlib.DecompIO and
especially CPython's gzip.GzipFile for both compression and decompression.

This class can be used directly, and also can be used from Python to
implement (via io.BytesIO) zlib.decompress and zlib.compress, as well as
gzip.GzipFile.

Enable/disable this on all ports/boards that zlib was previously configured
for.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Also update zlib & gzip docs to describe the micropython-lib modules.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
This replaces the previous zlib version.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
When MICROPY_SCHEDULER_STATIC_NODES is enabled, the logic is unchanged.

When MICROPY_SCHEDULER_STATIC_NODES is disable, sched_state is now always
initialised to MP_SCHED_IDLE when calling mp_init().  For example, the use
of mp_sched_vm_abort(), if it aborts a running scheduled function, can lead
to the scheduler starting off in a locked state when the runtime is
restarted, and then it stays locked.  This commit fixes that case by
resetting sched_state.

Signed-off-by: Damien George <[email protected]>
Since commit beeb74 we already check in modussl_mbedtls whether this
function is provided by the ESP-IDF before calling it, thus we no longer
need to define it here in order to compile.

Removing it so that if CONFIG_MBEDTLS_DEBUG is defined we do not cause any
'multiple definition' compile errors.

Signed-off-by: Daniël van de Giessen <[email protected]>
This changes the ESP32 WDT implementation to use a custom handle so that it
becomes possible to reset the WDT from a thread.

By default esp_task_wdt_add subscribes the task_id of the current task.
That means that if we're running in a different task we are unable to reset
the WDT, which prevents feeding the WDT from a thread directly, or even
from a timer (which may randomly run in a different task when there's
multiple threads).

As an added bonus, the name we set makes the error clearly specify that it
was the user-specified WDT that reset the chip.

Signed-off-by: Daniël van de Giessen <[email protected]>
Unless -o is given, output defaults to stdout unless a source file is
given (in which case the source file name is used to derive an output
file name).

Signed-off-by: Armin Brauns <[email protected]>
jimmo and others added 17 commits September 29, 2023 14:18
Defines the list of standard features and ensures that each board.json
only uses those ones. This list can be extended, but needs to be a
deliberate decision.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
The DHCP server broadcasts messages.  They are being sent via the default
netif which might be completely the wrong network.  We want to send
messages to the netif we got the original message from.

Original author: [Peter Harper](https://github.com/peterharperuk)
Source: raspberrypi/pico-examples#392

Signed-off-by: Samveen <[email protected]>
While cyw43 is deinitialized, an interrupt occurs.  That is handled with
these lines: ports/rp2/mpnetworkport.c#L59-L61 and as pendsv is disabled
while in network code, the poll function then just waits there.

When deinit has finished, the poll func is executed, but skipped:
src/cyw43_ctrl.c#L222-L225 this skips the `CYW43_POST_POLL_HOOK` which
would re-enable interrupts, but also reset `cyw43_has_pending`.

And in that state, the lightsleep code, will skip sleeping as it thinks
there is a network packet pending to be handled.

With this change applied, lightsleep works as expected when the wifi chip
is enabled, and when it's powered off.
At one point it was possible to internal_bench CPython vs MicroPython, but
seemingly not any more.

Signed-off-by: Angus Gratton <[email protected]>
Add "CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y" to
ports/esp32/boards/sdkconfig.base so that all micropython esp32 images
support OTA rollback in the bootloader.  These images can then be converted
to OTA-capable images as required by user tools.

Also remove CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y from board-specific
sdkconfig files as this is now the default.

Signed-off-by: Glenn Moloney <[email protected]>
This allows the MicroPython task stack size to be overridden by the
mpconfigboard.h settings.

Signed-off-by: dotnfc <[email protected]>
This was previously hard-coded to "MicroPy" / "pyboard Flash" / "1.00".

Now allow it to be overridden by a board.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
This was previously hard-coded to "Micropy" / "Mass Storage" / "1.0".

Now allow it to be overridden by a board.

Also change "Micropy" to "MicroPy" and "1.0" to "1.00" to match stm32.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Changes are:
- Disable internal flash storage and use the external QSPI for storage.
- Disable default bootloader entry mode.  The bootloader entry function
  exists in board_init.c.
- Remove OSC enable/disable macros (this board doesn't have an OSC).

Signed-off-by: iabdalkader <[email protected]>
Update expired certificate, increase time validity period to five years and
fix command arguments typos in commentaries.

Signed-off-by: Carlos Gil <[email protected]>
iobase_ioctl expects that an ioctl method must return an integer, and will
raise otherwise.

This was tripping up aioble on Unix, where the new hybrid modselect.c
implementation will attempt to extract a file descriptor from the pollable
via ioctl(MP_STREAM_GET_FILENO).

However, ThreadSafeFlag's ioctl only supported MP_STREAM_POLL, and returned
None otherwise.

This makes it return `-1` (to match tests/extmod/select_poll_custom.py). It
should probably be `-22` (corresponding to MP_EINVAL), but the value is
never checked, and MP_EINVAL can be a different value on different ports.

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
1. Remove the skip for detecting support for polling user-defined objects
   as this is always possible now on all ports.
2. Don't print when the scheduled task runs as the ordering of this
   relative to the other prints is dependent on other factors (e.g. if
   using the native emitter).

This work was funded through GitHub Sponsors.

Signed-off-by: Jim Mussared <[email protected]>
Prior to this commit, BTstack would only be notified of sent UART data when
the mp_bluetooth_hci_poll() function was called for some other reason, eg
because of incoming data over UART.  This is highly suboptimal.

With this commit, BTstack is now notified immediately after UART data has
been sent out.  This improves the multi_bluetooth/perf_gatt_char_write.py
performance test by about a factor of 10x for write-without-response, and
about 4x for write-with-response (tested on LEGO_HUB_NO6 as instance1).

Signed-off-by: Damien George <[email protected]>
# By Damien George (213) and others
# Via Damien George
* master: (710 commits)
  extmod/btstack/btstack_hci_uart: Trigger a poll after UART data is sent.
  tests/extmod/asyncio_threadsafeflag.py: Update for unix select.
  extmod/asyncio/event.py: Fix ThreadSafeFlag.ioctl return.
  examples/unix/machine_bios.py: Fix typo.
  tests/multi_net/ssl_cert_rsa.py: Update test certificate.
  stm32/boards/ARDUINO_GIGA: Update board config.
  rp2/msc_disk: Allow configuring the USB MSC inquiry response.
  stm32/usbd_msc_interface: Allow configuring the MSC inquiry response.
  esp32/main: Allow a board to override the MicroPython task stack size.
  esp32/boards: Add bootloader rollback support for all builds.
  tests/run-internalbench.py: Remove old CPython reference.
  tests/README: Document ./run-internalbench.py.
  rp2/modmachine: Fix lightsleep while wifi is powered off.
  shared/netutils/dhcpserver: Reply on correct netif.
  tools/mpremote: Add support for rfc2217, serial over TCP.
  tools/autobuild/build-downloads.py: Verify standard features.
  ports: Restrict board.json to standard features.
  {mimxrt,powerpc,samd}/mpconfigport: Don't override parse chunk alloc.
  esp32/boards: Fix VBAT voltage calculation for UM S3 boards.
  py/lexer: Add missing initialisation for fstring_args_idx.
  ...

# Conflicts:
#	ports/rp2/CMakeLists.txt
#	ports/rp2/modrp2.c
@samskiter
Copy link
Author

Worht saying that I'd also added some documentation support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.