Skip to content

Add test demonstrating future cancellation data race #3749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
sfc-gh-anoyes wants to merge 2 commits into from
Closed

Add test demonstrating future cancellation data race #3749

sfc-gh-anoyes wants to merge 2 commits into from

Conversation

@sfc-gh-anoyes
Copy link
Collaborator

@sfc-gh-anoyes sfc-gh-anoyes commented Sep 5, 2020

I plan to merge a fix into 6.2, but just wanted to share my findings:

==================
WARNING: ThreadSanitizer: data race (pid=10793)
  Write of size 4 at 0x7b30000005c0 by thread T1:
    #0 ThreadSingleAssignmentVar<Void>::send(Void const&) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:387:16 (libfdb_c.so+0x46d38b)
    #1 (anonymous namespace)::DoOnMainThreadActorState<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23, (anonymous namespace)::DoOnMainThreadActor<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23> >::a_body1cont3(Void const&, int) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:554:11 (libfdb_c.so+0x4b70a7)
    #2 (anonymous namespace)::DoOnMainThreadActorState<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23, (anonymous namespace)::DoOnMainThreadActor<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23> >::a_body1cont2when1(Void const&, int) /home/anoyes/build/foundationdb/flow/ThreadHelper.actor.g.h:770:15 (libfdb_c.so+0x4b70a7)
    #3 (anonymous namespace)::DoOnMainThreadActorState<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23, (anonymous namespace)::DoOnMainThreadActor<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23> >::a_callback_fire(ActorCallback<(anonymous namespace)::DoOnMainThreadActor<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23>, 1, Void>*, Void const&) /home/anoyes/build/foundationdb/flow/ThreadHelper.actor.g.h:791:4 (libfdb_c.so+0x4b677f)
    #4 ActorCallback<(anonymous namespace)::DoOnMainThreadActor<Void, ThreadSafeTransaction::watch(StringRef const&)::$_23>, 1, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x4b677f)
    #5 SAV<Void>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x97dfe2)
    #6 RYWImpl::WatchActorState<RYWImpl::WatchActor>::a_body1cont1(int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:4704:36 (libfdb_c.so+0x97dfe2)
    #7 RYWImpl::WatchActorState<RYWImpl::WatchActor>::a_body1cont5(int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:4816:16 (libfdb_c.so+0x97dc01)
    #8 RYWImpl::WatchActorState<RYWImpl::WatchActor>::a_body1cont4(Void const&, int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:4740:15 (libfdb_c.so+0x97f068)
    #9 RYWImpl::WatchActorState<RYWImpl::WatchActor>::a_body1when1(Void const&, int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:4752:15 (libfdb_c.so+0x97f068)
    #10 RYWImpl::WatchActorState<RYWImpl::WatchActor>::a_callback_fire(ActorCallback<RYWImpl::WatchActor, 0, Void>*, Void const&) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:4773:4 (libfdb_c.so+0x97f068)
    #11 ActorCallback<RYWImpl::WatchActor, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x97ce04)
    #12 SAV<Void>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x53b1f2)
    #13 (anonymous namespace)::ChooseActorActorState<Void, (anonymous namespace)::ChooseActorActor<Void> >::a_body1when1(Void const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:13607:41 (libfdb_c.so+0x53b1f2)
    #14 (anonymous namespace)::ChooseActorActorState<Void, (anonymous namespace)::ChooseActorActor<Void> >::a_callback_fire(ActorCallback<(anonymous namespace)::ChooseActorActor<Void>, 0, Void>*, Void const&) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:13660:4 (libfdb_c.so+0x53b1f2)
    #15 ActorCallback<(anonymous namespace)::ChooseActorActor<Void>, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x53b1f2)
    #16 SAV<Void>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x53b632)
    #17 (anonymous namespace)::ChooseActorActorState<Void, (anonymous namespace)::ChooseActorActor<Void> >::a_body1when2(Void const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:13631:41 (libfdb_c.so+0x53b632)
    #18 (anonymous namespace)::ChooseActorActorState<Void, (anonymous namespace)::ChooseActorActor<Void> >::a_callback_fire(ActorCallback<(anonymous namespace)::ChooseActorActor<Void>, 1, Void>*, Void const&) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:13705:4 (libfdb_c.so+0x53b632)
    #19 ActorCallback<(anonymous namespace)::ChooseActorActor<Void>, 1, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x53b632)
    #20 SAV<Void>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x957712)
    #21 (anonymous namespace)::SuccessActorState<Optional<Standalone<StringRef> >, (anonymous namespace)::SuccessActor<Optional<Standalone<StringRef> > > >::a_body1cont1(Optional<Standalone<StringRef> > const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11669:37 (libfdb_c.so+0x957712)
    #22 (anonymous namespace)::SuccessActorState<Optional<Standalone<StringRef> >, (anonymous namespace)::SuccessActor<Optional<Standalone<StringRef> > > >::a_body1when1(Optional<Standalone<StringRef> > const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11690:15 (libfdb_c.so+0x957712)
    #23 (anonymous namespace)::SuccessActorState<Optional<Standalone<StringRef> >, (anonymous namespace)::SuccessActor<Optional<Standalone<StringRef> > > >::a_callback_fire(ActorCallback<(anonymous namespace)::SuccessActor<Optional<Standalone<StringRef> > >, 0, Optional<Standalone<StringRef> > >*, Optional<Standalone<StringRef> > const&) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:11711:4 (libfdb_c.so+0x957435)
    #24 ActorCallback<(anonymous namespace)::SuccessActor<Optional<Standalone<StringRef> > >, 0, Optional<Standalone<StringRef> > >::fire(Optional<Standalone<StringRef> > const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x957435)
    #25 SAV<Optional<Standalone<StringRef> > >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x98c572)
    #26 RYWImpl::ReadWithConflictRangeRYWActorState<RYWImpl::GetValueReq, RYWImpl::ReadWithConflictRangeRYWActor<RYWImpl::GetValueReq> >::a_body1when1(Optional<Standalone<StringRef> > const&, int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:2136:54 (libfdb_c.so+0x98c572)
    #27 RYWImpl::ReadWithConflictRangeRYWActorState<RYWImpl::GetValueReq, RYWImpl::ReadWithConflictRangeRYWActor<RYWImpl::GetValueReq> >::a_callback_fire(ActorCallback<RYWImpl::ReadWithConflictRangeRYWActor<RYWImpl::GetValueReq>, 0, Optional<Standalone<StringRef> > >*, Optional<Standalone<StringRef> > const&) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:2189:4 (libfdb_c.so+0x98e2c9)
    #28 ActorCallback<RYWImpl::ReadWithConflictRangeRYWActor<RYWImpl::GetValueReq>, 0, Optional<Standalone<StringRef> > >::fire(Optional<Standalone<StringRef> > const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x98c2b4)
    #29 SAV<Optional<Standalone<StringRef> > >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x98d913)
    #30 RYWImpl::ReadActorState<RYWIterator, RYWImpl::ReadActor<RYWIterator> >::a_body1cont7(Optional<Standalone<StringRef> > const&, int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:230:36 (libfdb_c.so+0x98d913)
    #31 RYWImpl::ReadActorState<RYWIterator, RYWImpl::ReadActor<RYWIterator> >::a_body1when1(Optional<Standalone<StringRef> > const&, int) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:344:15 (libfdb_c.so+0x98ddee)
    #32 RYWImpl::ReadActorState<RYWIterator, RYWImpl::ReadActor<RYWIterator> >::a_callback_fire(ActorCallback<RYWImpl::ReadActor<RYWIterator>, 0, Optional<Standalone<StringRef> > >*, Optional<Standalone<StringRef> > const&) /home/anoyes/build/foundationdb/fdbclient/ReadYourWrites.actor.g.cpp:365:4 (libfdb_c.so+0x98ddee)
    #33 ActorCallback<RYWImpl::ReadActor<RYWIterator>, 0, Optional<Standalone<StringRef> > >::fire(Optional<Standalone<StringRef> > const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x98d344)
    #34 SAV<Optional<Standalone<StringRef> > >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x546c32)
    #35 (anonymous namespace)::GetValueActorState<(anonymous namespace)::GetValueActor>::a_body1cont1loopBody1cont3(int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:8616:38 (libfdb_c.so+0x546c32)
    #36 (anonymous namespace)::GetValueActorState<(anonymous namespace)::GetValueActor>::a_body1cont1loopBody1cont8(int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:8781:16 (libfdb_c.so+0x546c32)
    #37 (anonymous namespace)::GetValueActorState<(anonymous namespace)::GetValueActor>::a_body1cont1loopBody1cont5(int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:8643:15 (libfdb_c.so+0x52bff0)
    #38 (anonymous namespace)::GetValueActorState<(anonymous namespace)::GetValueActor>::a_body1cont1loopBody1cont1when2(GetValueReply const&, int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:8668:15 (libfdb_c.so+0x52bff0)
    #39 (anonymous namespace)::GetValueActorState<(anonymous namespace)::GetValueActor>::a_callback_fire(ActorCallback<(anonymous namespace)::GetValueActor, 3, GetValueReply>*, GetValueReply const&) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:8738:4 (libfdb_c.so+0x52bff0)
    #40 ActorCallback<(anonymous namespace)::GetValueActor, 3, GetValueReply>::fire(GetValueReply const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x52bff0)
    #41 SAV<GetValueReply>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x546483)
    #42 (anonymous namespace)::RunAfterActorState<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface), (anonymous namespace)::RunAfterActor<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface)> >::a_body1cont1(GetValueReply const&, int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:151:38 (libfdb_c.so+0x546483)
    #43 (anonymous namespace)::RunAfterActorState<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface), (anonymous namespace)::RunAfterActor<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface)> >::a_body1when1(GetValueReply const&, int) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:170:15 (libfdb_c.so+0x546483)
    #44 (anonymous namespace)::RunAfterActorState<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface), (anonymous namespace)::RunAfterActor<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface)> >::a_callback_fire(ActorCallback<(anonymous namespace)::RunAfterActor<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface)>, 0, GetValueReply>*, GetValueReply const&) /home/anoyes/build/foundationdb/fdbclient/NativeAPI.actor.g.cpp:191:4 (libfdb_c.so+0x545f50)
    #45 ActorCallback<(anonymous namespace)::RunAfterActor<GetValueReply, Future<decltype(getReplyPromise(std::declval<GetValueRequest>()).getFuture().getValue())> (anonymous namespace)::loadBalance<StorageServerInterface, GetValueRequest>(DatabaseContext*, Reference<LocationInfo>, RequestStream<GetValueRequest> StorageServerInterface::*, GetValueRequest const&, TaskPriority, bool, QueueModel*)::'lambda'(StorageServerInterface)>, 0, GetValueReply>::fire(GetValueReply const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x545f50)
    #46 SAV<GetValueReply>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x542702)
    #47 (anonymous namespace)::LoadBalanceActorState<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface>, (anonymous namespace)::LoadBalanceActor<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface> > >::a_body1loopBody1loopBody2when1(ErrorOr<Optional<GetValueReply> > const&, int) /home/anoyes/build/foundationdb/fdbrpc/LoadBalance.actor.g.h:1770:42 (libfdb_c.so+0x542702)
    #48 (anonymous namespace)::LoadBalanceActorState<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface>, (anonymous namespace)::LoadBalanceActor<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface> > >::a_callback_fire(ActorCallback<(anonymous namespace)::LoadBalanceActor<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface> >, 6, ErrorOr<Optional<GetValueReply> > >*, ErrorOr<Optional<GetValueReply> > const&) /home/anoyes/build/foundationdb/fdbrpc/LoadBalance.actor.g.h:1873:4 (libfdb_c.so+0x53318b)
    #49 ActorCallback<(anonymous namespace)::LoadBalanceActor<StorageServerInterface, GetValueRequest, ReferencedInterface<StorageServerInterface> >, 6, ErrorOr<Optional<GetValueReply> > >::fire(ErrorOr<Optional<GetValueReply> > const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x53318b)
    #50 SAV<ErrorOr<Optional<GetValueReply> > >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x541602)
    #51 (anonymous namespace)::ErrorOrActorState<Optional<GetValueReply>, (anonymous namespace)::ErrorOrActor<Optional<GetValueReply> > >::a_body1cont2(Optional<GetValueReply> const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:904:37 (libfdb_c.so+0x541602)
    #52 (anonymous namespace)::ErrorOrActorState<Optional<GetValueReply>, (anonymous namespace)::ErrorOrActor<Optional<GetValueReply> > >::a_body1when1(Optional<GetValueReply> const&, int) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:923:15 (libfdb_c.so+0x541602)
    #53 (anonymous namespace)::ErrorOrActorState<Optional<GetValueReply>, (anonymous namespace)::ErrorOrActor<Optional<GetValueReply> > >::a_callback_fire(ActorCallback<(anonymous namespace)::ErrorOrActor<Optional<GetValueReply> >, 0, Optional<GetValueReply> >*, Optional<GetValueReply> const&) /home/anoyes/build/foundationdb/flow/genericactors.actor.g.h:944:4 (libfdb_c.so+0x541058)
    #54 ActorCallback<(anonymous namespace)::ErrorOrActor<Optional<GetValueReply> >, 0, Optional<GetValueReply> >::fire(Optional<GetValueReply> const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x541058)
    #55 SAV<Optional<GetValueReply> >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x53bff2)
    #56 (anonymous namespace)::MakeRequestActorState<GetValueRequest, (anonymous namespace)::MakeRequestActor<GetValueRequest> >::a_body1cont3(ErrorOr<GetValueReply> const&, int) /home/anoyes/build/foundationdb/fdbrpc/LoadBalance.actor.g.h:327:42 (libfdb_c.so+0x53bff2)
    #57 (anonymous namespace)::MakeRequestActorState<GetValueRequest, (anonymous namespace)::MakeRequestActor<GetValueRequest> >::a_body1cont1when1(ErrorOr<GetValueReply> const&, int) /home/anoyes/build/foundationdb/fdbrpc/LoadBalance.actor.g.h:372:15 (libfdb_c.so+0x53bff2)
    #58 (anonymous namespace)::MakeRequestActorState<GetValueRequest, (anonymous namespace)::MakeRequestActor<GetValueRequest> >::a_callback_fire(ActorCallback<(anonymous namespace)::MakeRequestActor<GetValueRequest>, 1, ErrorOr<GetValueReply> >*, ErrorOr<GetValueReply> const&) /home/anoyes/build/foundationdb/fdbrpc/LoadBalance.actor.g.h:393:4 (libfdb_c.so+0x53a6df)
    #59 ActorCallback<(anonymous namespace)::MakeRequestActor<GetValueRequest>, 1, ErrorOr<GetValueReply> >::fire(ErrorOr<GetValueReply> const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x53a6df)
    #60 SAV<ErrorOr<GetValueReply> >::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x53c9c3)
    #61 (anonymous namespace)::WaitValueOrSignalActorState<GetValueReply, (anonymous namespace)::WaitValueOrSignalActor<GetValueReply> >::a_body1loopBody1when1(GetValueReply const&, int) /home/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:3507:47 (libfdb_c.so+0x53c9c3)
    #62 (anonymous namespace)::WaitValueOrSignalActorState<GetValueReply, (anonymous namespace)::WaitValueOrSignalActor<GetValueReply> >::a_callback_fire(ActorCallback<(anonymous namespace)::WaitValueOrSignalActor<GetValueReply>, 0, GetValueReply>*, GetValueReply const&) /home/anoyes/build/foundationdb/fdbrpc/genericactors.actor.g.h:3560:4 (libfdb_c.so+0x53c9c3)
    #63 ActorCallback<(anonymous namespace)::WaitValueOrSignalActor<GetValueReply>, 0, GetValueReply>::fire(GetValueReply const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0x53c9c3)
    #64 SAV<GetValueReply>::finishSendAndDelPromiseRef() /home/anoyes/workspace/foundationdb/flow/flow.h:480:23 (libfdb_c.so+0x65d3e4)
    #65 void SAV<GetValueReply>::sendAndDelPromiseRef<GetValueReply&>(GetValueReply&) /home/anoyes/workspace/foundationdb/flow/flow.h:473:3 (libfdb_c.so+0x65d3e4)
    #66 NetSAV<GetValueReply>::receive(ArenaObjectReader&) /home/anoyes/workspace/foundationdb/fdbrpc/fdbrpc.h:101:12 (libfdb_c.so+0x65cf87)
    #67 non-virtual thunk to NetSAV<GetValueReply>::receive(ArenaObjectReader&) /home/anoyes/workspace/foundationdb/fdbrpc/fdbrpc.h (libfdb_c.so+0x65d024)
    #68 (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont1(int) /home/anoyes/workspace/foundationdb/fdbrpc/FlowTransport.actor.cpp:776:15 (libfdb_c.so+0xb1eea2)
    #69 (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1cont2(Void const&, int) /home/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:3546:15 (libfdb_c.so+0xb1eac8)
    #70 (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_body1when1(Void const&, int) /home/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:3558:15 (libfdb_c.so+0xb1eac8)
    #71 (anonymous namespace)::DeliverActorState<(anonymous namespace)::DeliverActor>::a_callback_fire(ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>*, Void const&) /home/anoyes/build/foundationdb/fdbrpc/FlowTransport.actor.g.cpp:3579:4 (libfdb_c.so+0xb1eac8)
    #72 ActorCallback<(anonymous namespace)::DeliverActor, 0, Void>::fire(Void const&) /home/anoyes/workspace/foundationdb/flow/flow.h:1007:86 (libfdb_c.so+0xb1eac8)
    #73 void SAV<Void>::send<Void>(Void&&) /home/anoyes/workspace/foundationdb/flow/flow.h:448:23 (libfdb_c.so+0x427c42)
    #74 void Promise<Void>::send<Void>(Void&&) const /home/anoyes/workspace/foundationdb/flow/flow.h:782:8 (libfdb_c.so+0xca9e0e)
    #75 N2::PromiseTask::operator()() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:921:11 (libfdb_c.so+0xca9e0e)
    #76 N2::Net2::run() /home/anoyes/workspace/foundationdb/flow/Net2.actor.cpp:1255:5 (libfdb_c.so+0xc97590)
    #77 runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/NativeAPI.actor.cpp:1547:13 (libfdb_c.so+0x4d9ff4)
    #78 ThreadSafeApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/ThreadSafeTransaction.cpp:375:3 (libfdb_c.so+0x4a4f4c)
    #79 MultiVersionApi::runNetwork() /home/anoyes/workspace/foundationdb/fdbclient/MultiVersionTransaction.actor.cpp:1203:20 (libfdb_c.so+0x43b2ad)
    #80 fdb_run_network /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:123:2 (libfdb_c.so+0x417469)
    #81 decltype(std::__1::forward<int (*)()>(fp)()) std::__1::__invoke<int (*)()>(int (*&&)()) /usr/local/bin/../include/c++/v1/type_traits:3539:1 (cancel_data_race_test+0x301006)
    #82 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()>&, std::__1::__tuple_indices<>) /usr/local/bin/../include/c++/v1/thread:273:5 (cancel_data_race_test+0x301006)
    #83 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, int (*)()> >(void*) /usr/local/bin/../include/c++/v1/thread:284:5 (cancel_data_race_test+0x301006)

  Previous read of size 4 at 0x7b30000005c0 by main thread:
    #0 ThreadSingleAssignmentVarBase::isReadyUnsafe() const /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:332:38 (libfdb_c.so+0x41d9cd)
    #1 ThreadSingleAssignmentVarBase::cancel() /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:315:6 (libfdb_c.so+0x41d9cd)
    #2 fdb_future_cancel /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:138:2 (libfdb_c.so+0x4176e6)
    #3 main /home/anoyes/workspace/foundationdb/bindings/c/test/cancel_data_race_test.cpp:30:3 (cancel_data_race_test+0x300745)

  Location is heap block of size 192 at 0x7b3000000540 allocated by main thread:
    #0 operator new(unsigned long, std::align_val_t) <null> (cancel_data_race_test+0x30040e)
    #1 ThreadFuture<decltype(std::declval<ThreadSafeTransaction::watch(StringRef const&)::$_23>()().getValue())> onMainThread<ThreadSafeTransaction::watch(StringRef const&)::$_23>(ThreadSafeTransaction::watch(StringRef const&)::$_23) /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:581:21 (libfdb_c.so+0x49f4a0)
    #2 ThreadSafeTransaction::watch(StringRef const&) /home/anoyes/workspace/foundationdb/fdbclient/ThreadSafeTransaction.cpp:259:9 (libfdb_c.so+0x49f4a0)
    #3 fdb_transaction_watch /home/anoyes/workspace/foundationdb/bindings/c/fdb_c.cpp:560:32 (libfdb_c.so+0x41a162)
    #4 main /home/anoyes/workspace/foundationdb/bindings/c/test/cancel_data_race_test.cpp:27:28 (cancel_data_race_test+0x300702)

  Thread T1 (tid=10795, running) created by main thread at:
    #0 pthread_create <null> (cancel_data_race_test+0x2877f2)
    #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:394:10 (cancel_data_race_test+0x300b26)
    #2 std::__1::thread::thread<int (*)(), void>(int (*&&)()) /usr/local/bin/../include/c++/v1/thread:300:16 (cancel_data_race_test+0x300b26)
    #3 main /home/anoyes/workspace/foundationdb/bindings/c/test/cancel_data_race_test.cpp:18:14 (cancel_data_race_test+0x300663)

SUMMARY: ThreadSanitizer: data race /home/anoyes/workspace/foundationdb/flow/ThreadHelper.actor.h:387:16 in ThreadSingleAssignmentVar<Void>::send(Void const&)
==================
ThreadSanitizer: reported 1 warnings

sfc-gh-anoyes added a commit to sfc-gh-anoyes/foundationdb that referenced this pull request Sep 5, 2020
@sfc-gh-anoyes
Fix data race described in apple#3749
0e49542
sfc-gh-mpilman
sfc-gh-mpilman approved these changes Sep 6, 2020
View reviewed changes
Copy link
Collaborator

@sfc-gh-mpilman sfc-gh-mpilman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apart from the missing new-line this looks good to me

@ajbeamon
Copy link
Contributor

ajbeamon commented Sep 8, 2020

I was just thinking about this particular race recently too and was curious about the fact that I hadn't seen it show up in thread safety checks before. Thanks for putting together a test to highlight it.

I wonder if trying to run the sanitizer on automatically generated tests such as the Flow binding tester may help to find other problems, too.

ajbeamon added a commit that referenced this pull request Sep 10, 2020
@ajbeamon
Merge pull request #3750 from sfc-gh-anoyes/anoyes/fix-cancel-data-race
0b8a4fc 
Fix data race described in #3749
@sfc-gh-anoyes
Copy link
Collaborator Author

sfc-gh-anoyes commented Sep 18, 2020

I'm going to close this and leave it as a reference. I plan to include this coverage in future work that would add automatic testing of the parts of the client that aren't exercised in simulation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sfc-gh-mpilman sfc-gh-mpilman sfc-gh-mpilman approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sfc-gh-anoyes @ajbeamon @sfc-gh-mpilman