At a first glance, it looks like sockets.

... with multi-stack support ...

@OlegHahm But the interface does work without dynamic memory.

@authmillenon I'm sorry, but this does not seem lightweight to me.

Can you remind me where I need dynamic memory for sockets?

@OlegHahm Sorry, I meant 'preallocated or dynamic memory'.

edit

As in, int fd = socket(blah) can at most return a pointer, and the system needs to either dynamically allocate the corresponding state structures, or have a couple of them preallocated.

+1 for the general approach, but I also don't see the light-weight here. I think we can do even more simple, like a stateless, one function call for sending UDP for example.

btw are we talking about a layer between application layer and lower layers or between applications and the network stack in general?

• I don't think we need multiple stacks within the interface.
• the API fixes address types to be used, making super-slim implementations impossible
• memcpy within an interface?
• while a generic interface for any connection is nice in any high-level system, RIOT-developers will know when to use tcp, udp, whatever, and can directly call the corresponding functions. No need to abstract here, better avoid the overhead and go for simplicity.

@authmillenon If you drop support for multiple interfaces, you basically end up with gnrc's interface, right?

@OlegHahm application <-> network stack

while a generic interface for any connection is nice in any high-level system, RIOT-developers will know when to use tcp, udp, whatever, and can directly call the corresponding functions. No need to abstract here, better avoid the overhead and go for simplicity.

and

@OlegHahm application <-> network stack

IMO that's contradicting. An application developer is not necessarily a RIOT-developer and usually doesn't care about which transport layer, routing, or whatever is used below. If we want to design an interface for applications to the network stack this has to be more generic and agnostic to lower layers. If we want to design an interface from application layer protocol implementations to lower layers (transport layer or similar), your assumption is valid (I guess).

@authmillenon Let me be more specific when I mean "this is not lightweight".

In order to bind to an udp port, all you need is "udp_bind(port)".
So now, if I would create a "null stack", with "null_udp_bind(port)" being empty, and write a "conn_*" implementation for it, integrate it like gnrc is integrated now,
then write a function that sets up the address struct needed for "conn_bind()", then calls "conn_bind(...)", thenn calls the "null stack" conn_null_bind() to get out the port number, and then calls "null_udp_bind()" (which does nothing). Like, a minimal usage of a stack implementing the conn API, with a stub function behind.

What do you estimate is the overhead in code, memory and stack for this bind, comparing to calling "null_udp_bind(port)"?

Whatever number you say, all of it is overhead brought by "conn", and should be avoided.

An application developer is not necessarily a RIOT-developer and usually doesn't care about which transport layer, routing, or whatever is used below.

Yes. Sure. He doesn't care. We all agree on that. We want our poor baby developers to be able to shout "DO!" at the RIOT node, and it does it. We want to make it even easier than Oracle does for its well-known enterprise language.

In reality, there are a lot of compromises to be made if we want a RIOT application developer to be able to do "connect me to X", not caring about whether it's TCP or not, abstracting that technicality away. I think we should summon all devs and have a big, long talk if we want RIOT to go that way. Or just provide sane, small, efficient APIs for people that do want to open a TCP connection.

In reality, there are a lot of compromises to be made if we want a RIOT application developer to be able to do "connect me to X", not caring about whether it's TCP or not, abstracting that technicality away. I think we should summon all devs and have a big, long talk if we want RIOT to go that way. Or just provide sane, small, efficient APIs for people that do want to open a TCP connection.

You're mixing things up. An application developer wants to transmit data over lwm2m, CoAP, MQTT, whatever... It's RIOT's duty to chose and provide the proper transport/network/link layer and routing protocol implementations for that and provide an API to the application developer that does not require internal knowledge about the stack. There's non inherent contradiction in making this sane, small, and efficient.

People who do want to open a TCP connection are typically no application developers.

You're mixing things up. An application developer wants to transmit data over lwm2m, CoAP, MQTT, whatever...

lwm2m, CoAP, MQTT are "application" when seen from the perspective of
this proposed API. Developers using libs implementing those protocols
will not use "conn__", but "lwm2m__", "coap__", "mqtt__".

Those libs will use whatever API they need to. If at that level we're
using "conn_*", we're producing bloat.

And if we re-use libs implementing those protocols, we will probably
writing wrappers from, e.g., "mqtt_open_tcp(uint32_t target_ipv4,
uint16_t port)" looking like

mqtt_open_tcp(uint32_t target_ipv4, uint16_t port)
{
conn_t conn;
conn_addr_ipv4_t addr;
conn_make_ipv4_addr(&addr, target_ipv4, port);
conn_open(&conn, get_default_network_stack(), AF_INET4, addr);
...
}

Complicated. Bloated.

lwm2m, CoAP, MQTT are "application" when seen from the perspective of this proposed API. Developers using libs implementing those protocols will not use "conn__", but "lwm2m__", "coap__", "mqtt__".

Go back to my initial question and your answer. You're talking about application layer <-> network stack.

@OlegHahm:

btw are we talking about a layer between application layer and lower layers or between applications and the network stack in general?

but never mind

@kaspar030 let me get your argument straight: you say it is not lightweight because it offers multi-task support / because of conn.

I tend to agree, that dynamic stack support isn't really necessary (and when thinking about it really hard to do with the concept I have for sockets in mind), so I guess I can make it a little more static, using macro magic instead of switch cases and inline functions. Multi-task support can also be achieved by writing a netapi adapter for a non-profit stack ;).

The reason this API looks so much like sockets is, because it IS a pointer based socket API. This is the main reason I did not re-purposed the socket_base header. The fact alone that I don't use file descriptors or other numerical id's makes the module itself very RAM-memory efficient (about ROM we can argue, but let me rewrite it first a little), because the module does not need to maintain a (possibly huge but largely unused) socket table, as socket_base did. The other reason I did not call it socket_whatever was to avoid user confusion *we already faced_ about which socket API to use. I say: If you are an inexperienced user: use POSIX sockets (or whatever library will wrap around them) because you are used to them, which will be the only API in RIOT that has "socket" in name, when I'm done. No ambiguity possible. More experienced users still can use conn without using sockets, master developers even can use "bare metal" stack APIs and if even that is to fat they can hacke the hardware directly, so that they can feel proud as the 1337 haxx0rs they are ;). In general, when it comes to user APIs I really want it to be friendly (as our slogan suggests), but optional.

It is a stack-agnostic light-weight "socket" API, the kind of API that
is used to write libs like mqtt or coap libraries or applications that
want to use e.g., UDP or TCP.

@authmillenon Did I get that right?

In general yes. It is the base for a very slim POSIX socket API, which allows then for very easy porting of external libs.

@kaspar030 https://github.com/kaspar030 let me get your argument
straight: you say it is not lightweight because it offers multi-task
support / because of |conn|.

It does not seem light-weight because of multi stack support and because
it forces use of a generalized address type.

More experienced users still can use |conn| without using sockets,
master developers even can use "bare metal" stack APIs and if even
that is to fat they can hacke the hardware directly, so that they
can feel proud as> the 1337 haxx0rs they are ;).

IMHO, we have a need for a "socket" kind of API which is not directly
using a stack's API, but wich is not adding unneccessary bloat.
Something that makes writing libraries and applications possible without
tying to a specific network stack.

IMHO, this proposed API is too inefficient.

In general, when it comes to user APIs I really want it to be
friendly (as our slogan suggests), but optional.

We need to be careful with that. At some point, user-friendlyness comes
at a price. This API is not at the highest application developer level,
but will be used for middle-layer libraries. Once used, slimming it down
will be hard.

In general, when it comes to user APIs I really want it to be friendly (as our slogan suggests), but optional.

An transport layer API is not a user API in my opinion.

An transport layer API is not a user API in my opinion.

What is it?

A networking API (like netapi).

Squashed and rebased to current master.

just these two optional remarks -> ACK once Travis...

Addressed comments and added test application for gnrc_conn_ip (for gnrc_conn_udp please continue to use the POSIX socket example). Please review the test, too.

addressed external comment.

re-ACK when comment for test application is addressed.

Addressed comments and squashed immediately.

Fixed doc error and squashed immediately.

Fixed avr-related bug in conn and squashed immediately.

Defined boards with insufficient memory for test application... Now it should work ………

(Hopefully) one last fix.