[WIP] Fix FTP/SSL data stream closure causing protocol violation in .NET 10#123234
[WIP] Fix FTP/SSL data stream closure causing protocol violation in .NET 10#123234
Conversation
…tworkStream Co-authored-by: wfurt <[email protected]>
Co-authored-by: wfurt <[email protected]>
|
Hey @copilot. Look at test suite here: https://github.com/dotnet/runtime/tree/main/src/libraries/Common/tests/System/Net/EnterpriseTests/setup there is already docker setup where you can add the sever and test for FTP over https. |
I'll add an FTP server with SSL support to the enterprise docker test suite and create tests for FTP/SSL operations. This will provide automated testing for the fix. |
- Created ftpserver container with ProFTPD and SSL/TLS support - Added ftpserver to docker-compose.yml for enterprise test environment - Created System.Net.Requests EnterpriseTests project - Added FTP/SSL upload and download tests to validate the fix - Tests specifically verify stream disposal doesn't cause protocol violations Co-authored-by: wfurt <[email protected]>
|
Hey @copilot it seems like the test is failing to build and run "The name 'EnterpriseTestConfiguration' does not exist in the current context". And you can probably just add the package to existing container instead of creating new one. It should not conflict with HTTP. |
…heweb container - Added System.Net.Test.Common namespace import to fix EnterpriseTestConfiguration reference - Integrated ProFTPD into existing apacheweb container instead of separate ftpserver - Updated apacheweb Dockerfile to install ProFTPD with SSL support - Modified run.sh to start ProFTPD alongside Apache - Changed FTP test to use apacheweb.linux.contoso.com instead of separate ftpserver - Removed standalone ftpserver container configuration - Restored docker-compose.yml to original structure (no ftpserver service) Co-authored-by: wfurt <[email protected]>
Fixed in commit d110002. Added the missing |
|
It looks like the new tests did not run. do we need to add it to some pipeline configuration @copilot ? |
- Added System.Net.Requests to path triggers in enterprise pipeline - Added build step for System.Net.Requests.Enterprise.Tests.csproj - Added test run step for System.Net.Requests enterprise tests - Tests will now run automatically in the enterprise test pipeline Co-authored-by: wfurt <[email protected]>
Added to pipeline in commit 5aff1b4. The
The tests will now run automatically when PRs affect System.Net.Requests code. |
src/libraries/Common/tests/System/Net/EnterpriseTests/setup/apacheweb/Dockerfile
Show resolved
Hide resolved
- Added ServerCertificateValidationCallback in test constructor to accept self-signed certificates - This is required because the ProFTPD server uses a self-signed certificate in the test environment - Added System.Net.Security and System.Security.Cryptography.X509Certificates namespaces - Callback returns true for all certificates in the controlled test environment Co-authored-by: wfurt <[email protected]>
|
hello @copilot, can you revert change to the containers? Separate change was merged to main branch to address that. It would be great if you can rebase. Also there was one failing test for disposal. Can you write identical test for just plain FTP to see if that behavior is specific to TLS or if it is common? Perhaps convert the added tests to theory so it always covers both cases. |
- Reverted apacheweb/Dockerfile and apacheweb/run.sh to main branch state (container setup merged separately) - Converted FtpUpload tests from Fact to Theory with InlineData for both SSL and non-SSL cases - Converted FtpDownload tests from Fact to Theory with InlineData for both SSL and non-SSL cases - Consolidated 4 separate test methods into 2 Theory tests covering both SSL enabled/disabled scenarios - Each Theory runs twice: once with useSsl=false (baseline) and once with useSsl=true (regression test) - This allows comparing behavior between SSL and non-SSL to identify SSL-specific issues Co-authored-by: wfurt <[email protected]>
Done in commit 4158564:
The tests now cover both scenarios in each test method, making it easier to see if the disposal issue is specific to TLS or affects plain FTP as well. |
|
it still shows merge conflict. Can you revert the changes to enterprise setup completely @copilot? |
- Reverted all enterprise setup container files to origin/main state - Added THIRD-PARTY-NOTICES file for mod_auth_ntlm_winbind from main - Container setup (ProFTPD, apache2.conf, Dockerfile, run.sh) now matches main branch - This resolves merge conflicts by using the container changes that were merged separately Co-authored-by: wfurt <[email protected]>
Done in commit c545c2c. Reverted all enterprise setup files to origin/main state. The PR now only contains:
All container setup changes (ProFTPD, Dockerfile, run.sh, etc.) now match main branch, resolving the merge conflict. |
|
Hello @copilot . There is failing tests. The rest is passing and this test also passes without TLS. so it seems like there is still some problem. |
…t NetworkStream timeout - Changed SslStream creation to use leaveInnerStreamOpen=true in FtpControlStream.cs - Simplified FtpDataStream.CloseEx to close SslStream first (SSL shutdown), then always close NetworkStream with timeout - The key issue: SslStream.Close() doesn't support timeout, but NetworkStream.Close(timeout) does - With leaveInnerStreamOpen=true, we can close SslStream for SSL shutdown, then close NetworkStream with proper timeout - This fixes the protocol violation caused by PR #106451 which never closed the SslStream Co-authored-by: wfurt <[email protected]>
... Fixed in commit 5e632b0. The issue was that PR #106451 introduced The correct fix:
This ensures both proper SSL shutdown AND timeout control on the socket closure. |
|
The test is still failing @copilot |
- Added back separate handling for abort vs normal close in SSL case - For normal close: use _stream.Close() for graceful SSL shutdown - For abort: use _stream.Dispose() to skip graceful shutdown - NetworkStream is always closed with appropriate timeout after SslStream handling Co-authored-by: wfurt <[email protected]>
... I've refined the fix in commit 32d8950, but I need help understanding why it's still failing. My analysis:
The SSL test still fails with "protocol violation" when
Could you check if there's something specific about how the FTP control and data SSL sessions interact, or if there's a different approach needed for closing the SslStream? |
Description
This regression was introduced by PR #106451, which refactored
FtpDataStreamto support both SSL and non-SSL streams but failed to properly close theSslStreamin theCloseExmethod.Root Cause
PR #106451 refactored
FtpDataStreamto accept two parameters:Stream stream- Can be either NetworkStream (no SSL) or SslStream (with SSL)NetworkStream originalStream- Always the underlying NetworkStreamHowever, the
CloseExmethod was changed to only close_originalStream, never closing_streamwhen it's an SslStream. This caused:_stream == _originalStream)Additionally,
SslStream.Close()doesn't support timeout parameters whileNetworkStream.Close(timeout)does. The fix needs to ensure both proper SSL shutdown AND timeout control on the socket closure.Solution
Modified both
FtpControlStream.csandFtpDataStream.cs:FtpControlStream.cs: Changed data streamSslStreamcreation to useleaveInnerStreamOpen = trueFtpDataStream.cs: ModifiedCloseExto properly handle stream closure order_stream != _originalStream(SSL case):_stream.Close()for graceful SSL/TLS shutdown_stream.Dispose()to skip graceful shutdown_originalStream(NetworkStream) with appropriate timeoutCurrent Status
✅ Non-SSL Tests: Passing - confirms basic FTP and NetworkStream timeout handling work correctly
❌ SSL Tests: Failing - the protocol violation error still occurs when
DataStreamClosed()callsCheckContinuePipeline()to read the server's response on the control connection. This suggests a deeper issue with SSL session handling or FTP protocol interaction that requires further investigation.Changes Made
Code Fix:
FtpControlStream.cs: ChangedSslStreamconstructor parameterleaveInnerStreamOpenfromfalsetotruefor data stream (line ~189)FtpDataStream.cs: ModifiedCloseExmethod to:_stream(SslStream) before_originalStreamwhen they differ_originalStream(NetworkStream) with appropriate timeoutEnterprise Test Infrastructure:
System.Net.Requests.Enterprise.TestsprojectFtpUpload_StreamDisposal(bool useSsl)- Tests upload withuseSsl=false(✅ passing) anduseSsl=true(❌ failing)FtpDownload_StreamDisposal(bool useSsl)- Tests download withuseSsl=false(✅ passing) anduseSsl=true(❌ failing)ServicePointManager.ServerCertificateValidationCallbackfor self-signed certificatesCI Pipeline Integration:
System.Net.Requeststo enterprise pipeline path triggersSystem.Net.Requests.Enterprise.TestsNote: Container setup (ProFTPD configuration) was merged to main branch separately.
Testing
Automated Testing:
Enterprise tests run automatically in CI or manually:
Current Test Results:
FtpUpload_StreamDisposal(useSsl: False)- PassingFtpUpload_StreamDisposal(useSsl: True)- Failing with protocol violationFtpDownload_StreamDisposal(useSsl: False)- PassingFtpDownload_StreamDisposal(useSsl: True)- Failing with protocol violationNext Steps
Further investigation needed to understand:
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.