Skip to content

ieee802154: Set intra-PAN flag for packets where src/dst PANs match#5685

Closed
aeneby wants to merge 1 commit intoRIOT-OS:masterfrom
aeneby:intra_pan_fix
Closed

ieee802154: Set intra-PAN flag for packets where src/dst PANs match#5685
aeneby wants to merge 1 commit intoRIOT-OS:masterfrom
aeneby:intra_pan_fix

Conversation

@aeneby
Copy link
Copy Markdown
Member

@aeneby aeneby commented Jul 25, 2016

According to section 7.5.6.1 of 802.15.4-2003, packets destined for
the same PAN from which they were sent must have their intra-PAN
flag set, and the source PAN omitted from the transmitted frame.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Jul 25, 2016

Fixes #5684

@OlegHahm OlegHahm added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: network Area: Networking labels Jul 25, 2016
@PeterKietzmann
Copy link
Copy Markdown
Member

ACK for the change. Didn't test it until now. IMO this is not an urgent bugfix. Thus I'd like to wait until feature-un-freeze.

@PeterKietzmann PeterKietzmann added this to the Release 2016.10 milestone Jul 26, 2016
@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Jul 26, 2016

Fine by me. Thanks.

@PeterKietzmann PeterKietzmann added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Aug 12, 2016
@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Aug 17, 2016

What's Murdock complaining about?

@PeterKietzmann PeterKietzmann added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Aug 17, 2016
@PeterKietzmann
Copy link
Copy Markdown
Member

I think the source for lwip was down at that time. Just restarted Murdock

uint8_t type = (flags & IEEE802154_FCF_TYPE_MASK);
uint8_t bcast = (flags & IEEE802154_BCAST);

if (dst_pan.u16 == src_pan.u16) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be on the safe side: could you also clear the bit in case the pans are different?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, fixed.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Aug 19, 2016

@PeterKietzmann please wait to merge. I'd like to give some more thought to @thomaseichinger's comment first, when I get a chance.

According to section 7.5.6.1 of 802.15.4-2003, packets destined for
the same PAN from which they were sent must have their intra-PAN
flag set, and the source PAN omitted from the transmitted frame.
@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Aug 22, 2016

While I haven't managed to see the 802.15.4-2015 spec yet, there is this. Although it's only a submission draft, I have reason to believe (based on looking at what Wireshark has implemented, for example) that this - or something very close to it - is what ended up in the final specification. Perhaps someone in a position to do so can verify:

7.2.1.5 PAN ID Compression field

The PAN ID Compression field is used to indicate the presence of the PAN ID field. When the frame version field value is 0b00 or 0b01, the PAN ID compression field is interpreted as follows:

— If both destination and source addressing information is present, the MAC sublayer shall compare the destination and source PAN identifiers. If the PAN IDs are identical, the PAN ID Compression field shall be set to one, and the Source PAN ID field shall be omitted from the transmitted frame. If the PAN IDs are different, the PAN ID Compression field shall be set to zero, and both Destination PAN ID field and Source PAN ID fields shall be included in the transmitted frame.

— If only either the destination or the source addressing information is present, the PAN ID Compression field shall be set to zero, and the PAN ID field of the single address shall be included in the transmitted frame.

The PR has been updated to reflect this description; if there are no objections I'm happy for it to be merged as-is.

@PeterKietzmann
Copy link
Copy Markdown
Member

@aeneby, @miri64, @thomaseichinger I'm sorry I was off during latest discussions. After following your conversation (also in #5759) it is not completely clear to me which OSes/stacks will be affected in interoperability when adopting this PR to the latest specification which -again- seems to enforce the elision of the src PAN, in case it equals the dst PAN. Could someone please summarize that?

@miri64
Copy link
Copy Markdown
Member

miri64 commented Aug 30, 2016

@PeterKietzmann as long as a stack/OS implements IEEE 802.15.4 correctly interoperability isn't effected. The problem with lwIP's interop is a recognized bug

@PeterKietzmann
Copy link
Copy Markdown
Member

The question was, which stack/OS actually does it correctly.It is just lwip which does not implement it correctly?

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 1, 2016

I believe Linux (4.7) will completely ignore packets with two identical PAN IDs.

@thomaseichinger
Copy link
Copy Markdown
Member

@PeterKietzmann Management summary as far as I get it:
There are two sides concerning RIOT, (1) for the tx part we should set the intra pan bit and omit the source pan field. (2) Interoperability depends from many factors especially if hardware packet filtering is turned on this depends which revision of the 802.15.4 standard the HW implements.
Once we get our hands on the 802.15.4-2015 version we should implement this version for transceivers that are not filtering in hardware.

(Did I get you confused even more 😊)

@PeterKietzmann
Copy link
Copy Markdown
Member

@thomaseichinger thanks and yes :-). Is there any reason against proceeding as follows:

@thomaseichinger
Copy link
Copy Markdown
Member

thomaseichinger commented Sep 1, 2016

Seems fine to me if others agree too.
I am kind of reluctant agreeing on sending out corrupt frames to satisfy a buggy implementation. As it is an accepted lwip bug I'd say it is their move to take.

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 13, 2016

Because I think its unnecessary (because in the end only drivers use this function). It is (codewise and API-wise) simpler to just correct a potential developer's mistake and every byte we can save should be saved. We are working on constrained devices after all. Even if for some reason the device allows for the setting of that flag (should be 2 instructions, if I'm not mistaking) it's less code than checking the flag before the call and then again after (5 instructions, I think).

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 13, 2016

I disagree. Yes it's an extra comparison, but that's just a quirk of having a modular stack. Correcting the PAN_COMP flag automatically in ieee802154_set_frame_hdr() is what was causing the unit tests to fail, and IMO they were right to do so in this case.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 13, 2016

What is gnrc_netdev2->send() going to set the PAN_COMP flag to, if we remove this from the driver state?

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 13, 2016

It would leave it unset.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 16, 2016

It would leave it unset.

That seems arbitrary, and IMO reinforces the fact we should be setting the PAN compression flag before ieee802154_set_frame_hdr() is even called. Anyhow...

I apologise for taking so long with this PR, but I'm still trying to get my head around the unit tests. For example,test_ieee802154_set_frame_hdr_dst0_src8_pancomp() appears to be testing PAN compression for a beacon frame, but PAN compression is not valid for this frame type. What should this test expect of ieee802154_set_frame_hdr() in this case? That it "corrects" the frame header by setting the PAN compression bit to 0 for a beacon frame, as required by the 802.15.4 spec, or that it returns '0' due to being asked to create an invalid frame, as suggested by the function's documentation?

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 19, 2016

It would leave it unset.

That seems arbitrary, and IMO reinforces the fact we should be setting the PAN compression flag before ieee802154_set_frame_hdr() is even called. Anyhow...

Can we roll back for a minute? I think we are talking past each other here. If we remove the PAN_COMP flag from the driver state it does not know anything about PAN compression (because it does not exist for the driver), so naturally it would leave it unset. I don't see how this is arbitrary.

I apologise for taking so long with this PR, but I'm still trying to get my head around the unit tests. For example,test_ieee802154_set_frame_hdr_dst0_src8_pancomp() appears to be testing PAN compression for a beacon frame, but PAN compression is not valid for this frame type. What should this test expect of ieee802154_set_frame_hdr() in this case? That it "corrects" the frame header by setting the PAN compression bit to 0 for a beacon frame, as required by the 802.15.4 spec, or that it returns '0' due to being asked to create an invalid frame, as suggested by the function's documentation?

If you think they check an invalid state then correct them. I did not wrote them with regard to spec correctness, but API correctness. The API allows for this configuration so it needs to be tested.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 19, 2016

Can we roll back for a minute? I think we are talking past each other here. If we remove the PAN_COMP flag from the driver state it does not know anything about PAN compression (because it does not exist for the driver), so naturally it would leave it unset. I don't see how this is arbitrary.

If the flag is zero this doesn't mean "unset", it means "set to zero". This is an arbitrary decision to take if the code doesn't know/care what the value should actually be.

The API allows for this configuration so it needs to be tested.

Yes but I don't understand what the test result should be, given that we're sometimes correcting things and sometimes not.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 19, 2016

Actually, is it even currently possible for GNRC to send intra-PAN packets? I see that the gnrc_netdev2->send() function gets the PAN ID from the driver state, and uses it for both source and destination PANs when creating the packet header. Where would it get the destination PAN ID from for intra-PAN packets?

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 20, 2016

Actually, is it even currently possible for GNRC to send intra-PAN packets? I see that the gnrc_netdev2->send() function gets the PAN ID from the driver state, and uses it for both source and destination PANs when creating the packet header. Where would it get the destination PAN ID from for intra-PAN packets?

IIRC I implemented (at least on the NDP side) that if you set an address like this <PAN>:<short address> the PAN is included.

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Sep 26, 2016

IIRC I implemented (at least on the NDP side) that if you set an address like this : the PAN is included.

@miri64 could you please point me to an example of this?

Does this mean that the way to get a PAN ID in as part of the netif header would be to concatenate it with the src address?

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 27, 2016

@miri64 could you please point me to an example of this?

I couldn't find an actual example. Maybe I dropped it because non of the drivers did not supported it and it made things only more complicated (RFC 6775 doesn't even mention PAN IDs). The idea was early on that IEEE 802.15.4 devices would have 3 address types:

  • short (2 byte)
  • PAN+short (4 byte)
  • EUI-64 (8 byte)

Does this mean that the way to get a PAN ID in as part of the netif header would be to concatenate it with the src address?

Yes, at least for short addresses (long addresses would need a PAN coordinator still): as examplified above, they would be prepended.

@smlng
Copy link
Copy Markdown
Member

smlng commented Sep 29, 2016

This somehow breaks multicast messages with SRC_PAN == DST_PAN where it does not set the IEEE802154_FCF_PAN_COMP flag, what I found is that for multicast message dst_len = 0 when calling ieee802154_set_frame_hdr.

The result is that multicast messages such as RS have equal SRC_PAN and DST_PAN but the flag is not set, which makes Linux drop those RS packets. Unfortunately I run radvd with my RPi as 6LBR, but it does not send any RS.

@smlng
Copy link
Copy Markdown
Member

smlng commented Sep 29, 2016

This is really ugly in a way, currently it works with Master-branch and samr21-xpro. But only because the flag is set by the driver, which is IMO the wrong place to do that, because the actual header is written in layers above.

As both of you @miri64 and @aeneby proposed I think it would be best to not set this flag in the driver and it shouldn't be modified by the user. It must be checked and set where the hdr is build and the SRC_PAN and DST_PAN are written (or compressed).

@smlng
Copy link
Copy Markdown
Member

smlng commented Sep 29, 2016

btw. which makes it even more confusing and difficult to look up is that we have multiple #define for the same thing

@cgundogan
Copy link
Copy Markdown
Member

btw. which makes it even more confusing and difficult to look up is that we have multiple #define for the same thing
#define NETDEV2_IEEE802154_PAN_COMP (0x0040) in drivers/include/net/netdev2/ieee802154.h
#define IEEE802154_FCF_PAN_COMP (0x40) in sys/include/net/ieee802154.h

AFAIK, the current consensus is to remove the NETDEV2_IEEE802154_PAN_COMP flag. But I am still waiting with my PR in #5759 to reflect this change until the above discussion is resolved.

uint8_t bcast = (flags & IEEE802154_BCAST);

if (dst_pan.u16 == src_pan.u16 &&
(src_len != 0 && dst_len != 0)) {
Copy link
Copy Markdown
Member

@smlng smlng Sep 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem with this comparison of dst_len != 0 is that for multicast messages such as RS the length actually is 0 see 1. The result is the for RS messages we have SRC_PAN==DST_PAN but dst_len=0 and the flag will be unset which makes Linux drop those RS and never send an RA.

This could be fixed, for instance in 2, by not not passing netif_hdr->dst_l2addr_lendirectly but rather set this like with src_len. That is, if the packet is broadcast or multicast (see 3) then set dst_len = 2.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem with this comparison of dst_len != 0 is that for multicast messages such as RS the length actually is 0 see 1. The result is the for RS messages we have SRC_PAN==DST_PAN but dst_len=0 and the will not be unset which makes Linux drops those RS and never send an RA.

Just an explanation for this behavior: since netif_hdr_t is L2 agnostic and the format of a multicast address can be very L2 specific, multicast is just handled by setting a special multicast flag and not setting the destination address in the netif_hdr_t. The L2 then sets the multicast address to its liking according to e.g. the IPv6 destination address (or in IEEE 802.15.4 by just using the broadcast address).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miri

The L2 then sets the multicast address to its liking according to e.g. the IPv6 destination address

where does this happen in the standard case of 6lo over 802154?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

@miri64 miri64 Sep 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miri64 I don't understand why src address and length are set in _send here but the destination is not set the same way when it is multicast? I mean the code in gnrc_netdev2_ieee802154.c already knows about L2 so it should be able to set the address like here, or not?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what's the reasoning behind unset the broadcast flag here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miri64 I don't understand why src address and length are set in _send here but the destination is not set the same way when it is multicast? I mean the code in gnrc_netdev2_ieee802154.c already knows about L2 so it should be able to set the address like here, or not?

Because the driver is not "supposed" to know the length of the broadcast address. That's this function's job.

and what's the reasoning behind unset the broadcast flag here?

The flag is not part of the IEEE 802.15.4 specs so adding it to the MAC header seems like a bad idea ;-).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@miri64, this is no driver code (I thought)?! I was talking about these 2 files

  1. /sys/net/link_layer/ieee802154/ieee802154.c
  2. /sys/net/gnrc/link_layer/netdev2/gnrc_netdev2_ieee802154.c

both are not driver, but src is set in the GNRC (2.) part but dst if broad/multicast is set in the other file (2.). I argue for consistency, i.e., set broadcast address in GNRC too, and further get rid of this broadcast flag setting and later unsetting again.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to confess with "Because the driver is not 'supposed' to know […]" I was mixing terms myself. I meant "the glue code" gnrc_netdev2_ieee802154 ^^".

I argue for consistency, i.e., set broadcast address in GNRC too, and further get rid of this broadcast flag setting and later unsetting again.

Might indeed be a good idea!

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 30, 2016

btw. which makes it even more confusing and difficult to look up is that we have multiple #define for the same thing

#define NETDEV2_IEEE802154_PAN_COMP (0x0040) in drivers/include/net/netdev2/ieee802154.h
#define IEEE802154_FCF_PAN_COMP (0x40) in sys/include/net/ieee802154.h

They are not the same thing. NETDEV2_IEEE802154_PAN_COMP is a device configuration flag (which legitimacy we discuss here) and IEEE 802154_FCF_PAN_COMP a MAC header flag (which is required for proper IEEE 802.15.4 communication).

@smlng
Copy link
Copy Markdown
Member

smlng commented Sep 30, 2016

@miri64 thx for clarification! NETDEV2_IEEE802154_PAN_COMP is only used by at86rf2xx at the moment, but I think (too) that we shouldn't allow the user to manipulate that flag and/or that drivers could set it. This should be handled as proposed in this PR, though we have to fix the multicast issue, anyway.

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 30, 2016

@miri64 thx for clarification! NETDEV2_IEEE802154_PAN_COMP is only used by at86rf2xx at the moment, but I think (too) that we shouldn't allow the user to manipulate that flag and/or that drivers could set it. This should be handled as proposed in this PR, though we have to fix the multicast issue, anyway.

Yes! I'm working on an alternative PR now to finally clear (and clean) things up! ;-). It was used by at86rf2xx only because except cc2538 (which @aeneby ported) it was the only netdev2 IEEE 802.15.4 device up until now. All the others are currently still gnrc_netdev (kw2xrf being currently ported in #5469).

@smlng
Copy link
Copy Markdown
Member

smlng commented Sep 30, 2016

I --HAW Hamburg, that is-- now have several kw2x and cc2538 (remote-revb) for testing their netdev2 drivers, further I also use samr21-xpro with at86rf2xx and RPi with openlabs. Doing interop tests, this one popped up several times and it gets annoying. So this INTRA_PAN stuff has top priority to be solved!

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 30, 2016

Yes! I'm working on an alternative PR now to finally clear (and clean) things up! ;-).

See #5897

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 30, 2016

But the error says "stack smashing detected" […]

To clarify this: I ran into this error in #5897, too. It was happening with your PR due to the broadcast behavior @smlng reported here (PAN were equal, but broadcast flag was set => dst_len == 0 => PAN compression wasn't applied). Since my PAN compression handling looked a little bit different then, I just included the change into #5897 instead of basing that PR on this one.

@miri64
Copy link
Copy Markdown
Member

miri64 commented Sep 30, 2016

(PAN were equal, but broadcast flag was set => dst_len == 0 => PAN compression wasn't applied).

PAN compression wasn't applied => header was longer than expected => return address of test function was overridden => stack smashing detected ;-).

@aeneby
Copy link
Copy Markdown
Member Author

aeneby commented Oct 4, 2016

Closing in favour of #5897. Thanks, @miri64.

@aeneby aeneby closed this Oct 4, 2016
@aeneby aeneby deleted the intra_pan_fix branch October 4, 2016 08:52
@miri64 miri64 removed this from the Release 2016.10 milestone Oct 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: network Area: Networking CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants