UDP: set max buffer size to 4MB and fallback to 1MB by xunil-cloud · Pull Request #6989 · supercollider/supercollider

xunil-cloud · 2025-06-15T13:57:24Z

Purpose and Motivation

Fix #6969. Set UDP buffer size to 4 MB and fallback to 1MB if we fail to do that.

Types of changes

Bug fix

To-do list

Code is tested
All tests are passing
Updated documentation
This PR is ready for review

We fallback to 1MB if we fail to do that.

telephon

Thank you!

I think it would be good if sclang would inform us always about the current maximum OSC buffer size, and even keep a variable for this that can be accessed from within sclang.

There are places in the class library where we depend on the size limit, so these need to adjust themselves.

This could be a separate PR, of course.

elgiano · 2025-06-16T09:46:37Z

@telephon I found these four matches searching for 65535, do you think there are more?
In all these cases, SC is ensuring that a single message is not larger than the UDP limit for a single message, which is not dependent on the max buffer size (unless buffer size < max message size). That is, even if we have a 4MB udp buffer, we can't send a single message larger than 65535 bytes. (about why are we dividing 65535 by four, I think it's a conservative measure that could be the subject of a separate investigation :D)

supercollider/SCClassLibrary/Common/Audio/SynthDef.sc

Lines 598 to 605 in e8a06ee

    
           if (bytes.size < (65535 div: 4)) { 
        
           	server.sendMsg("/d_recv", bytes, completionMsg) 
        
           } { 
        
           	if (server.isLocal) { 
        
           		if(warnAboutLargeSynthDefs) { 
        
           			"SynthDef % too big for sending. Retrying via synthdef file".format(name).warn; 
        
           		}; 
        
           		this.writeDefFile(synthDefDir);

supercollider/SCClassLibrary/Common/Control/NetAddr.sc

Lines 101 to 102 in e8a06ee

    
           if(args.bundleSize > 65535) {// udp max size. 
        
           	args.clumpBundles.do { |item|

supercollider/SCClassLibrary/Common/Control/NetAddr.sc

Lines 121 to 122 in e8a06ee

    
           if(bundles.bundleSize > 20000/*65515*/) { // 65515 = 65535 - 16 - 4 (sync msg size) 
        
           	bundles.clumpBundles.do { |item|

supercollider/SCClassLibrary/JITLib/ProxySpace/ProxyInterfaces.sc

Lines 363 to 370 in e8a06ee

    
           if(server.options.protocol === \tcp or: { size < 16383}) { 
        
           	// full size: 65535, but does not work. 
        
           	bundle.addPrepare([5, bytes]); // "/d_recv" 
        
           } { 
        
           	// bridge exceeding bundle size by writing to disk 
        
           	if(server.isLocal.not) { 
        
           		Error("SynthDef too large (" ++ size 
        
           			++ " bytes) to be sent to remote server via udp").throw;

About being able to query the maxBufferSize (and perhaps even setting it?) I agree it would be nice to have a couple of primitives (something like _NetAddr_getMaxBufferSize, _NetAddr_setMaxBufferSize), especially because now different systems could have different buffer sizes and being able to query it would help debugging eventual future issues.

server/scsynth/SC_ComPort.cpp

elgiano · 2025-06-16T10:19:13Z

lang/LangPrimSource/SC_ComPort.h

-    static constexpr int receiveBufferSize = 8 * 1024 * 1024;
-    static constexpr int sendBufferSize = 8 * 1024 * 1024;
+    static constexpr int receiveBufferSize = 4 * 1024 * 1024;
+    static constexpr int sendBufferSize = 4 * 1024 * 1024;


I think we should add static constexpr int fallbackBufferSize = 1 * 1024 * 1024, and perhaps move all these constexpr to common/SC_OscUtils.hpp?

Actually, we could make a function setUDPBufferSizes in there, and call it from sclang/scsynth/supernova. But we could also do this in a subsequent refactoring PR

I think we should add static constexpr int fallbackBufferSize = 1 * 1024 * 1024

Most definitely.

Actually, we could make a function setUDPBufferSizes in there, and call it from sclang/scsynth/supernova.

I'm thinking about this, too. If that is what we want, I plan to move them all to SC_OscUtils.hpp.

I've seen that if we want to do this, there is a caveat to import SC_OscUtils.hpp in supernova

#define scprintf printf #include "SC_OscUtils.hpp" #undef scprintf

which is the same thing we do in lang/LangPrimSource/SC_ComPort.cpp.

But again, I don't think it's absolutely necessary to do it now, we could also do it after release in a subsequent PR, as you wish

Spacechild1 · 2025-06-16T11:14:15Z

I think it would be good if sclang would inform us always about the current maximum OSC buffer size, and even keep a variable for this that can be accessed from within sclang.

As @elgiano correctly pointed out, these size checks are only done to make sure we don't exceed the max. UDP packet size. The UDP socket buffers should always be large enough to hold at least a single UDP packet.

Spacechild1 · 2025-06-16T11:38:53Z

(about why are we dividing 65535 by four, I think it's a conservative measure that could be the subject of a separate investigation :D)

Yes, that's what I think as well. It seems pretty arbitrary to me, though.

Note that 65535 is only the max. value allowed in the size field of the UDP packet header (because it is a 16-bit unsigned integer.) For the actual max. data payload, you must subtract the UDP header size (8 bytes) and IP header size (20 bytes for IPv4). For IPv4 this is 65507 bytes. See https://en.wikipedia.org/wiki/User_Datagram_Protocol#UDP_datagram_structure.

But that does not mean that you can actually send and receive such large UDP packets, in particular when an actual network is involved (as opposed to loopback devices). The ethernet standard only guarantees an MTU of 1500 bytes. Most modern switches support jumbo frames (https://en.wikipedia.org/wiki/Jumbo_frame), but they also have a limit well below 65507. Once you send things over the internet, you might want to limit your UDP packets to something like 576 (!) bytes.

IMO, the actual solution to all of this is to use a reliable protocol, like TCP, but that's another story.

telephon · 2025-06-16T14:57:40Z

@telephon I found these four matches searching for 65535, do you think there are more?

No, methink that's it. Thank you for the explanation, sure!

telephon · 2025-06-16T15:00:24Z

about why are we dividing 65535 by four, I think it's a conservative measure that could be the subject of a separate investigation :D)

iirc, i did experiments and found that value :D. Might be wrong by now!

elgiano

Looks good to me, thanks! We can follow up later with a refactoring to reduce code duplication!

dyfer · 2025-06-18T22:07:25Z

@JordanHendersonMusic / @Spacechild1 is this good to be merged?

Spacechild1 · 2025-06-20T21:11:56Z

Good from my side!

JordanHendersonMusic · 2025-06-21T16:49:49Z

Does this fix #6969?

elgiano · 2025-06-21T16:56:04Z

Probably, but it would be nice if @ahihi could test it once we release rc2, since none of us could reproduce the error

-------- Original Message --------

On 6/21/25 18:50, Jordan Henderson wrote: JordanHendersonMusic left a comment [(supercollider/supercollider#6989)](#6989 (comment)) Does this fix [#6969](#6969)? — Reply to this email directly, [view it on GitHub](#6989 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AACNMWVTCQHEWDNEFGYFYB33EWEMHAVCNFSM6AAAAAB7LKDDI2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJTGY3DSOJRGI). You are receiving this because you modified the open/close state.Message ID: ***@***.***>

@github.com>

ahihi · 2025-06-21T17:16:59Z

Probably, but it would be nice if @ahihi could test it once we release rc2, since none of us could reproduce the error

i will! in fact i would be happy to try it already, but i dont currently have a SC build environment set up.

dyfer · 2025-06-21T17:31:27Z

@ahihi you can find builds from Github Actions under "Checks":

On macOS you need to right-click -> open (two times) to be able to run an unsigned build.

ahihi · 2025-06-23T07:44:24Z

@ahihi you can find builds from Github Actions under "Checks":

thanks! it is working fine here :)

Failing to set the previous 8MB buffer size could leave some systems with a too small default, causing failures when sending bigger synthdefs, with _NetAddr_SendMsg error "Message too long". 4MB is still more than the default in 3.13, while being acceptable for most systems. If it's still too much, fallback to 1MB if system's default is smaller.

Set UDP socket buffer size to 4MB

717c3c2

We fallback to 1MB if we fail to do that.

telephon reviewed Jun 15, 2025

View reviewed changes

elgiano requested changes Jun 16, 2025

View reviewed changes

xunil-cloud added 3 commits June 18, 2025 20:12

make sure we don't set buffer size smaller than default

7e59f01

print e.what()

5e54425

format code

48c5415

elgiano approved these changes Jun 18, 2025

View reviewed changes

dyfer added comp: sclang sclang C++ implementation (primitives, etc.). for changes to class lib use "comp: class library" comp: scsynth comp: supernova squash on merge labels Jun 18, 2025

dyfer linked an issue Jun 18, 2025 that may be closed by this pull request

_NetAddr_SendMsg "Message too long" errors in 3.14.0-rc1 on SuperDirt startup #6969

Closed

dyfer added this to 3.14.0 Jun 19, 2025

dyfer moved this to Committed in 3.14.0 Jun 19, 2025

Spacechild1 approved these changes Jun 20, 2025

View reviewed changes

elgiano changed the title ~~Fix _NetAddr_SendMsg "Message too long" errors~~ UDP: set max buffer size to 4MB and fallback to 1MB Jun 21, 2025

elgiano merged commit 4790b95 into supercollider:3.14 Jun 21, 2025
23 of 24 checks passed

github-project-automation bot moved this from Committed to Done in 3.14.0 Jun 21, 2025

ahihi mentioned this pull request Jun 23, 2025

_NetAddr_SendMsg "Message too long" errors in 3.14.0-rc1 on SuperDirt startup #6969

Closed

Comments

Conversation

xunil-cloud commented Jun 15, 2025

Purpose and Motivation

Types of changes

To-do list

Uh oh!

telephon left a comment

Choose a reason for hiding this comment

Uh oh!

elgiano commented Jun 16, 2025

Uh oh!

Uh oh!

Uh oh!

elgiano Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

elgiano Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

Spacechild1 Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

xunil-cloud Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

elgiano Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Spacechild1 commented Jun 16, 2025

Uh oh!

Spacechild1 commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

telephon commented Jun 16, 2025

Uh oh!

telephon commented Jun 16, 2025

Uh oh!

elgiano left a comment

Choose a reason for hiding this comment

Uh oh!

dyfer commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Spacechild1 commented Jun 20, 2025

Uh oh!

Uh oh!

JordanHendersonMusic commented Jun 21, 2025

Uh oh!

elgiano commented Jun 21, 2025 via email

Uh oh!

ahihi commented Jun 21, 2025

Uh oh!

dyfer commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahihi commented Jun 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

elgiano Jun 17, 2025 •

edited

Loading

Spacechild1 commented Jun 16, 2025 •

edited

Loading

dyfer commented Jun 18, 2025 •

edited

Loading

dyfer commented Jun 21, 2025 •

edited

Loading