Skip to content

#1412 fix concurrency issue in CopyOnWriteMap#1419

Merged
asolntsev merged 3 commits intomainfrom
fix/1412-flaky-error
Nov 1, 2024
Merged

#1412 fix concurrency issue in CopyOnWriteMap#1419
asolntsev merged 3 commits intomainfrom
fix/1412-flaky-error

Conversation

@asolntsev
Copy link
Copy Markdown
Collaborator

@asolntsev asolntsev commented Oct 31, 2024

The default method Map.getOrDefault is not thread-safe: the condition if (v == null && containsKey(key)) leaves v = null if the key was added by another thread exactly at the same moment. Then both conditions are true: v==null and containsKey(key).

Now CopyOnWriteMap.getOrDefault does NOT check containsKey(key).
P.S. As a side-effect, CopyOnWriteMap doesn't allow putting null values to the map. But we don't need it anyway.

@asolntsev asolntsev self-assigned this Oct 31, 2024
@asolntsev asolntsev marked this pull request as draft October 31, 2024 17:03
@asolntsev asolntsev linked an issue Oct 31, 2024 that may be closed by this pull request
@what-the-diff
Copy link
Copy Markdown

what-the-diff bot commented Oct 31, 2024

PR Summary

  • Enhanced Import Section
    The code has been updated to include imports for 'ArrayList' and 'List'. These are data structures that help with the organization and manipulation of data within the program.

  • Addition of a Synchronized List Named 'Errors'
    A new feature has been introduced - a synchronized list called 'errors'. This element will collect any exceptions or errors that occur during testing of the application.

  • Improved Exception Handling in the fakeSomeData Method
    The system for managing unexpected situations or 'exceptions' in the fakeSomeData method has been revamped. Now, instead of just notifying about such exceptions, the revised method will also store them in the 'errors' list.

  • Extended the issue759Test Method
    The issue759Test method has been extended with an additional check. At the end of each test execution, it will now confirm that the 'errors' list remains empty. This change ensures that there have been no unexpected exceptions during the process.

@asolntsev asolntsev added the bug Something isn't working label Oct 31, 2024
@asolntsev asolntsev force-pushed the fix/1412-flaky-error branch from 938f3a2 to 4c4656a Compare October 31, 2024 17:27
@codecov
Copy link
Copy Markdown

codecov bot commented Oct 31, 2024

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.24%. Comparing base (0f50bbe) to head (0babf2f).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
.../net/datafaker/internal/helper/CopyOnWriteMap.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1419      +/-   ##
============================================
+ Coverage     92.22%   92.24%   +0.02%     
  Complexity     3165     3165              
============================================
  Files           320      320              
  Lines          6173     6177       +4     
  Branches        592      592              
============================================
+ Hits           5693     5698       +5     
  Misses          335      335              
+ Partials        145      144       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@asolntsev asolntsev force-pushed the fix/1412-flaky-error branch from 9242112 to 8ab0af1 Compare October 31, 2024 20:26
@asolntsev asolntsev requested a review from snuyanzin October 31, 2024 20:26
@asolntsev asolntsev marked this pull request as ready for review October 31, 2024 20:26
@asolntsev asolntsev changed the title #1412 avoid swallowing errors happened in child threads in Issue759Test #1412 fix concurrency issue in CopyOnWriteMap Oct 31, 2024
Copy link
Copy Markdown
Collaborator

@kingthorin kingthorin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable to me.

} catch (RuntimeException expected) {
assertThat(expected).hasMessageStartingWith("County is not configured for postcode " + zipCode);
} catch (RuntimeException | Error e) {
if (!e.getMessage().startsWith("County is not configured for postcode ")) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure there couldn't be exception/error with null message?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Yes, theoretically it can happen.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@snuyanzin Fixed.

When any unexpected error happened in Issue759Test#WorkerThread, it was not logged anywhere, just disappeared.
_countDownLatch was not decreased, and the test failed with "Test did not complete within 12 second" error.
the default method `Map.getOrDefault` is not thread-safe: the condition `if (v == null && containsKey(key))` leaves `v = null` if the `key` was added by another thread exactly at the same moment. Then both conditions are true: `v==null` and `containsKey(key)`.

Now `CopyOnWriteMap.getOrDefault` does NOT check `containsKey(key)`. As a consequence, `CopyOnWriteMap` doesn't allow putting `null` values to the map.
@asolntsev asolntsev force-pushed the fix/1412-flaky-error branch from 8ab0af1 to 0babf2f Compare November 1, 2024 06:19
@asolntsev asolntsev merged commit 87a3803 into main Nov 1, 2024
@asolntsev asolntsev deleted the fix/1412-flaky-error branch November 1, 2024 10:55
@asolntsev
Copy link
Copy Markdown
Collaborator Author

@kingthorin I think we should release DF 2.4.2
People sometimes complain about this concurrency issue.

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 1, 2024

@asolntsev I think we should perhaps first have the documentation for DF 2.4.1 before releasing 2.4.2?

@asolntsev
Copy link
Copy Markdown
Collaborator Author

@bodiam To be honest, I know nothing about documentation for DF 2.4.1 (and all other versions) :)

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 1, 2024

@asolntsev I know, I know. I'll organize a training session soon ;)

But maybe I should make a quick document with steps involved, I know it's not the most exciting thing to work on, but I think it's kinda nice when the documentation is a little up to date.

@asolntsev
Copy link
Copy Markdown
Collaborator Author

@kingthorin @bodiam Hi. What about releasing DataFaker 2.4.2?

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 18, 2024

I'm keen. @kingthorin ?

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 18, 2024

I just merged #1438 , happy to give that one a shot.

@kingthorin
Copy link
Copy Markdown
Collaborator

Ya if we can get that roll PR in first that'd be handy

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 18, 2024

Perhaps it would be nice to get this one in (#1414), or we keep this for 2.5?

@kingthorin
Copy link
Copy Markdown
Collaborator

Oops sorry I missed your comment about 1414 earlier somehow.

I just came in here to say I'd done the 2.4.2 release.

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 19, 2024

Thanks for that. It doesn't look the like version in the pom.xml got updated though?

@bodiam
Copy link
Copy Markdown
Contributor

bodiam commented Nov 19, 2024

Oh, it creates a PR. Oki, that works!

@asolntsev
Copy link
Copy Markdown
Collaborator Author

Oops sorry I missed your comment about 1414 earlier somehow.

I just came in here to say I'd done the 2.4.2 release.

Sorry, but README is still not updated. :)

image

@kingthorin
Copy link
Copy Markdown
Collaborator

Yup that hasn't been automated yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NPE at net.datafaker.service.FakeValuesService.fetchObject():268

4 participants