Resolve iterators and async methods in stacktrace by benaadams · Pull Request #14655 · dotnet/coreclr

benaadams · 2017-10-23T09:50:36Z

Allows a developer to determine correct overload called for async and iterators, from a stacktrace.

Matches the complier generated containing Type to the source method using:
method.DeclaringType == StateMachineAttribute.StateMachineType attribute to allow the stack trace to resolve the mangled method name to the correct overload. Also appends the statemachine method name with + for iterators (addressing #14655 (comment)).

Changes:

async

System.Exception: Exception of type 'System.Exception' was thrown.
   at Program.RecursiveAsync(Int32 count)
   at Program.RecursiveAsync(Int32 count)
   at Program.RecursiveAsync(Int32 count)
   at Program.MethodAsync()
   at Program.Main(String[] args)

Instead of the current:

System.Exception: Exception of type 'System.Exception' was thrown.
   at Program.<RecursiveAsync>d__2.MoveNext()
   at Program.<RecursiveAsync>d__2.MoveNext()
   at Program.<RecursiveAsync>d__2.MoveNext()
   at Program.<MethodAsync>d__1.MoveNext()
   at Program.<Main>d__0.MoveNext()

iterators

System.Exception: Exception of type 'System.Exception' was thrown.
   at Program.Range(Int32 start, Int32 end)+MoveNext()
   at Program.Range(Int32 start, Int32 end)+MoveNext()
   at Program.Main(String[] args)

Instead of the current:

System.Exception: Exception of type 'System.Exception' was thrown.
   at Program.<Range>d__1.MoveNext()
   at Program.<Range>d__1.MoveNext()
   at Program.Main(String[] args)

Builds on the other PRs for improving async stacktrace readability:

Hide post exception stack frames Hide post exception stack frames #14652
Remove Edi boundary for async Remove Edi boundary for async #15781

Which removed the async noise:

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() 
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) 
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) 
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult() 
   at ...
--- End of stack trace from previous location where exception was thrown ---

Resolves: https://github.com/dotnet/corefx/issues/24627

PTAL @stephentoub @tmat @noahfalk @aelij

stephentoub · 2017-10-23T13:53:17Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

            return !(mb.IsDefined(typeof(StackTraceHiddenAttribute)) || (mb.DeclaringType?.IsDefined(typeof(StackTraceHiddenAttribute)) ?? false));
        }

+        private static Type ResolveStateMachineMethod(ref MethodBase mb, Type stateMachineType)


Why is mb passed by ref but stateMachineType passed as an arg and then returned?

Change to both be ref

stephentoub · 2017-10-23T13:55:46Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

+                            mb = method as MethodBase ?? mb;
+                            stateMachineType = mb.DeclaringType;
+                            break;
+                        }


if (asma != null && asma.StateMachineType == stateMachineType && method is MethodBase matched) { mb = matched; stateMachineType = matched.DeclaringType; break; }

?

stephentoub · 2017-10-23T13:59:35Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

                    sb.AppendFormat(CultureInfo.InvariantCulture, "   {0} ", word_At);

                    Type t = mb.DeclaringType;
+                    if (typeof(IAsyncStateMachine).IsAssignableFrom(t) || typeof(IEnumerator).IsAssignableFrom(t))


Check for [CompilerGenerated]?

stephentoub · 2017-10-23T14:01:46Z

Seems a bit strange that we're hiding the MoveNext method for compiler-generated enumerators, but we don't (and presumably wouldn't want to) for others (e.g. the one for list in your example stack trace). Why are the compiler-generated ones special?

benaadams · 2017-10-23T14:33:21Z

Why are the compiler-generated ones special?

User code mapping; like the line numbers refer to the original source its mapping the method name back to the original source.

The statemachine's MoveNext is a post-compile transform to the original source and can be impenetrable if there are overloads and no line numbers - which doesn't help with diagnosing the problem.

The List.MoveNext is the original source; whereas this isn't

at Program.<ValueTaskAsync>d__9.MoveNext()  in C:\Exceptions\Program.cs:line 126

The line number; while correct, certainly isn't in a method called <ValueTaskAsync>d__9.MoveNext() because that method does not exist in Program.cs

stephentoub · 2017-10-23T14:39:23Z

because that method does not exist in Program.cs

Neither does List'1.Enumerator.MoveNext.

There is a fundamental difference between an exception escaping from:

System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`3 predicate)

and an exception escaping from:

System.Linq.Enumerable.<SkipWhileIterator>d__179`1.MoveNext()

as they're two distinct methods. An exception escaping from the former means it's coming out of the synchronous call to SkipWhileIterator, whereas an exception coming out of the latter is coming from a call at some point later to MoveNext, which of course does exist on the IEnumerator<T> interface that's being used. So this transformation is both effectively lying (it's saying the exception is coming from one method when it's really coming from another) and it's suggesting somehow that calling IEnumerator<T>.MoveNext is calling SkipWhileIterator again, when it's not actually.

I get that it can help in some situations. But this is why it seems strange to me. If others are ok with the change, fine, but we should fully understand the ramifications.

benaadams · 2017-10-23T15:17:32Z

Neither does List`1.Enumerator.MoveNext.

Not sure what you mean?

Using the full exceptions (with line numbers):

at System.Collections.Generic.List`1.Enumerator.MoveNextRare() in 
 C:\GitHub\coreclr\src\mscorlib\shared\System\Collections\Generic\List.cs:line 1209
at System.Collections.Generic.List`1.Enumerator.MoveNext() in 
 C:\GitHub\coreclr\src\mscorlib\shared\System\Collections\Generic\List.cs:line 1202

Refers to source line 1209 in System.Collections.Generic.List<T>.Enumerator.MoveNextRare() which is a 1:1 mapping with the C# source?

Or do you mean the odd formatting of generics?

at System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`3 predicate)
 in C:\GitHub\corefx\src\System.Linq\src\System\Linq\Skip.cs:line 107

Is referring to source line 107 in SkipWhileIterator<TSource>(IEnumerable<TSource> source, Func<TSource, int, bool> predicate) where it evaluates the predicate, which throws the exception; again a 1:1 mapping with the C# source (other than the odd formatting of the generic types).

So this transformation is both effectively lying

Only is so much as System.Collections.Generic.List`1.Enumerator.MoveNextRare() is also lying because its not instead using the Jit's compiler output of

00007FF90001050E E8 2D ED B5 FF       call        00007FF8FFB6F240

VS also lies to you in the same way as its generally more productive.

I think I'm missing what you mean?

If you are not trying to work out a compiler bug (and you can still do this with the StackFrames); why would you prefer the C# compiler's output; to the C# compiler's input?

The later you just look at the source; the former you need to crack open ildasm or similar to even find the method. (Though if you have been programming .NET for a while you might be used to guessing the right method even without line numbers)

stephentoub · 2017-10-23T15:21:17Z

Not sure what you mean?

I simply meant it's not in the user's Program.cs file. In one case it's compiler-transformed, in the other it's using a 3rd party library from the user's code... in either case, it's not in the user's source.

I think I'm missing what you mean?

Given an iterator:

public IEnumerable<int> FooAsync()
{
    yield return 1;
    throw new Exception();
}

when I call:

IEnumerable<int> enumerable = FooAsync();

no exception will be thrown, because the Exception isn't coming from the synchronous call to FooAsync. It's only when I then do:

IEnumerator<int> enumerator = enumerable.GetEnumerator();
enumerator.MoveNext();
enumerator.MoveNext();

that the exception will be thrown. So I'm calling MoveNext and the exception is coming from MoveNext. I'm not again calling FooAsync(), yet that's what the call stack would show.

benaadams · 2017-10-23T15:31:13Z

In one case it's compiler-transformed, in the other it's using a 3rd party library from the user's code...

Maybe poor language choice I'm referring to the cs files as the "user's code" whether its their source, 3rd party or framework - e.g. its something you can directly look at; source step into the non-compiled files etc

when I then do: e.GetEnumerator().MoveNext();

Hmm... you may have me there...

Is async ok? No one (other than you 😉); is manually calling .MoveNext() on an AsyncStateMachine?

aelij · 2017-10-23T15:35:44Z

I agree with @stephentoub it is a bit confusing. FWIW, the VS debugger does display it like so (removes MoveNext).

The difference between async and an iterator is that an async method starts executing immediately when called, so it makes sense to hide the generated code there.

benaadams · 2017-10-23T15:53:40Z

FWIW, the VS debugger does display it like so (removes MoveNext).

It is the method where exception happens in the source file; so if you were trying to follow it through its the method you should look at - but yes I can see the disconnect with IEnumerable, if you call .MoveNext() manually.

Question though; does putting <FooAsync>d__1.MoveNext() in the stack trace add significant value over FooAsync()?

In your example where you immediately throw an exception

public IEnumerable<int> FooAsync()
{
    throw new Exception();
}

It cannot throw an exception (short of C# compiler bug) so FooAsync() will never appear in the stack trace; so it doesn't cause ambiguity doing the rename?

aelij · 2017-10-23T15:57:07Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

+            Type dt = declaringType.DeclaringType;
+            if (dt != null)
+            {
+                var methods = dt.GetMembers(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Instance);


I think dt.DeclaredMethods could be slightly cheaper here (don't need non-method members nor methods declared on base types).

Is an extension on TypeInfo rather than Type also IEnumerable rather than array. Will add BindingFlags.DeclaredOnly

And change to GetMethods

benaadams · 2017-10-23T16:04:20Z

e.g. for the Linq example there are two overloads

System.Linq.Enumerable.<SkipWhileIterator>d__177`1.MoveNext()
System.Linq.Enumerable.<SkipWhileIterator>d__179`1.MoveNext()

Whereas with this PR they would show as

System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`2 predicate)
System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`3 predicate)

stephentoub · 2017-10-23T16:07:32Z

This may not sit well with others, but what about showing both methods, e.g. rendering it as something like:
System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable'1 source, Func'2 predicate)+MoveNext()
?

benaadams · 2017-10-23T16:09:37Z

what about showing both methods

Seems a fair comprimise

benaadams · 2017-10-23T16:34:17Z

Output with that change

System.Exception: ...
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Collections.Generic.List`1.Enumerator.MoveNext()
   at Program.FirstException(Int32 value, Int32 index)
   at Program.<>c.<LinqIteration>b__4_0(Int32 x, Int32 i)
   at System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`3 predicate)+MoveNext()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Program.LinqIteration(Int32 count) 
   at Program.NonAsyncValueTask[TName](Int32 count, TName name)
   at Program.ValueTaskAsync(String name)+MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Program.TaskTAsync(Int32 count)+MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Program.MethodAsync(Int32 count)+MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Program.ConfiguredAsync(Int32 count)+MoveNext()

benaadams · 2017-10-23T17:15:37Z

@stephentoub so since .... redacted ...

stephentoub · 2017-10-23T17:18:52Z

@benaadams, I'd suggest that you limit PRs to a single purpose. There's a tendency here to push for additional changes that then slow the progress of the original PR.

benaadams · 2017-10-23T17:20:41Z

K, not included that change

stephentoub · 2017-10-23T17:22:27Z

K, not included that change

Thanks. Let's work on getting this change through, and if folks agree on it, then your subsequent one-line change can be debated on its own merits without tying it to debate on this one.

benaadams · 2017-10-24T10:32:01Z

Updated summary and squashed

BrunoMVPCosta · 2017-10-24T21:50:06Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

            return !(mb.IsDefined(typeof(StackTraceHiddenAttribute)) || (mb.DeclaringType?.IsDefined(typeof(StackTraceHiddenAttribute)) ?? false));
        }

+        private static bool TryResolveStateMachineMethod(ref MethodBase mb, ref Type declaringType)


I don't find anything particularly wrong with it, but may I suggest inverting the if statement to reduce the nesting? I think it will help readability, but I would like to hear your option.

private static bool TryResolveStateMachineMethod(ref MethodBase mb, ref Type declaringType) { Debug.Assert(mb != null); Debug.Assert(declaringType != null); Type dt = declaringType.DeclaringType; if (dt == null) return false; var methods = dt.GetMethods(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Instance | BindingFlags.DeclaredOnly); if(methods == null) return false; foreach (var method in methods) { var asma = method.GetCustomAttribute<StateMachineAttribute>(); if (asma != null && asma.StateMachineType == declaringType && method is MethodBase matched) { mb = matched; declaringType = matched.DeclaringType; return true; } } return false; }

benaadams · 2018-01-18T12:56:44Z

Rebased to fix merge clash with "Remove Edi boundary for async" #15781

For async

Pre

System.Exception: Exception of type 'System.Exception' was thrown.
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<RecursiveAsync>d__2.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.<MethodAsync>d__1.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 22
   at asyncexceptions.Program.<Main>d__0.MoveNext() in C:\Work\Things\asyncexceptions\Program.cs:line 12

Post

System.Exception: Exception of type 'System.Exception' was thrown.
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.RecursiveAsync(Int32 count) in C:\Work\Things\asyncexceptions\Program.cs:line 32
   at asyncexceptions.Program.MethodAsync() in C:\Work\Things\asyncexceptions\Program.cs:line 22
   at asyncexceptions.Program.Main(String[] args) in C:\Work\Things\asyncexceptions\Program.cs:line 12

benaadams · 2018-01-18T13:09:25Z

For iterators

Pre

System.Exception: Exception of type 'System.Exception' was thrown.
   at iteratorexceptions.Program.<Range>d__1.MoveNext() in C:\Work\Things\iteratorexceptions\Program.cs:line 35
   at iteratorexceptions.Program.<Range>d__1.MoveNext() in C:\Work\Things\iteratorexceptions\Program.cs:line 35
   at iteratorexceptions.Program.Main(String[] args) in C:\Work\Things\iteratorexceptions\Program.cs:line 12

Post

System.Exception: Exception of type 'System.Exception' was thrown.
   at iteratorexceptions.Program.Range(Int32 start, Int32 end)+MoveNext() in C:\Work\Things\iteratorexceptions\Program.cs:line 37
   at iteratorexceptions.Program.Range(Int32 start, Int32 end)+MoveNext() in C:\Work\Things\iteratorexceptions\Program.cs:line 35
   at iteratorexceptions.Program.Main(String[] args) in C:\Work\Things\iteratorexceptions\Program.cs:line 12

Code

Details

```csharp class Program { static void Main(string[] args) { try { foreach (var i in Range(0, 5)) { Console.WriteLine(i); } } catch (Exception e) { Console.WriteLine(e); } }

static IEnumerable<int> Range(int start, int end)
{
    for (var i = start; i < end; i++)
    {
        yield return i;
    }

    if (start == end)
    {
        throw new Exception();
    }

    foreach(var i in Range(end, end))
    {
        yield return i;
    }
}

}

</details>

danmoseley · 2018-01-18T16:53:26Z

@tmat @noahfalk any other feedback?

danmoseley · 2018-01-18T16:55:59Z

src/mscorlib/src/System/Diagnostics/Stacktrace.cs

+                }
+                catch
+                {
+                    // Linq.Expression compiled functions can throw when checking attributes


What does it throw? Is that a bug?

Don't need this; dynamic expressions have no declaring type so it wouldn't get here; and the other issues (benaadams/Ben.Demystifier#31, benaadams/Ben.Demystifier#20) are full framework bugs.

benaadams · 2018-01-23T09:29:35Z

@dotnet-bot test Windows_NT x64 Checked corefx_baseline
@dotnet-bot test Ubuntu x64 Checked corefx_baseline

benaadams · 2018-01-23T12:24:57Z

Tizen https://github.com/dotnet/coreclr/issues/15914

Failed "Ubuntu arm Cross Debug Innerloop Build" not a CI test anymore? #15959

noahfalk · 2018-01-23T23:54:43Z

no more feedback, I'm happy 👍

karelz · 2018-01-25T16:39:51Z

@tmat is it ready for merge from your point of view?
@noahfalk can you please mark it Approved? (easier to track ;))

tmat · 2018-01-25T17:43:49Z

👍

@karelz I don't see anything on the PR that would allow me to mark Approve.

karelz · 2018-01-25T17:46:59Z

Thanks @tmat -- on the top - "Files Changed" - right side top "Review changes" - "Approve" option.

karelz · 2018-01-25T17:49:29Z

@dotnet-bot test CentOS7.1 x64 Release Innerloop Build and Test
@dotnet-bot test Ubuntu arm Cross Debug Innerloop Build

benaadams · 2018-01-25T17:53:44Z

Ubuntu arm Cross Debug Innerloop Build doesn't exist anymore, so can't be retested

karelz · 2018-01-25T17:54:55Z

CentOS7.1 and Tizen armel legs are failing all over other PRs as well, so we are ready for merge I guess :)

tmat · 2018-01-25T18:05:58Z

@karelz Oh. OK. see it now. That's not very discoverable :(

noahfalk · 2018-01-25T20:23:11Z

@benaadams - thanks for all your work on this, I know it was a bit of slog but its a very nice improvement for .NET!

Allows a developer to determine *correct* overload called for *async* and *iterators*, from a stacktrace. Matches the compiler generated containing `Type` to the source method using: `method.DeclaringType == StateMachineAttribute.StateMachineType` attribute to allow the stack trace to resolve the mangled method name to the correct overload. Also appends the statemachine method name with `+` for iterators. Resolves: dotnet/corefx#24627

benaadams · 2018-01-28T11:44:57Z

netfx-port-consider?

dnfclas added the cla-already-signed label Oct 23, 2017

benaadams force-pushed the resolve-iterators-and-async branch 3 times, most recently from b09fd14 to 89a01b5 Compare October 23, 2017 13:14

stephentoub reviewed Oct 23, 2017

View reviewed changes

aelij reviewed Oct 23, 2017

View reviewed changes

benaadams force-pushed the resolve-iterators-and-async branch from e9b81d8 to b84a0cd Compare October 23, 2017 17:19

benaadams force-pushed the resolve-iterators-and-async branch from b84a0cd to 5c64d20 Compare October 24, 2017 07:35

benaadams changed the title ~~Resolve iterators and async methods~~ Resolve iterators and async methods in stacktrace Oct 24, 2017

BrunoMVPCosta reviewed Oct 24, 2017

View reviewed changes

benaadams mentioned this pull request Oct 25, 2017

Make stack traces readable dotnet/extensions#274

Closed

benaadams force-pushed the resolve-iterators-and-async branch from 3f45055 to e9a7ee9 Compare January 18, 2018 12:54

danmoseley requested a review from VSadov January 18, 2018 16:52

danmoseley reviewed Jan 18, 2018

View reviewed changes

Resolve iterators and async methods

1d83cdd

benaadams force-pushed the resolve-iterators-and-async branch from 20618a1 to 1d83cdd Compare January 20, 2018 17:00

karelz merged commit 8dfd31f into dotnet:master Jan 25, 2018

benaadams deleted the resolve-iterators-and-async branch January 25, 2018 18:45

benaadams mentioned this pull request Jan 25, 2018

Make stack traces readable mono/mono#6675

Merged

jkotas mentioned this pull request Feb 21, 2018

Show the expected stack trace from a rethrown exception. #16464

Merged

benaadams mentioned this pull request Jan 31, 2020

Does EDI message provide benefit? "End of stack trace from previous location..." dotnet/runtime#9517

Closed

ascott18 mentioned this pull request Apr 16, 2021

Captured stack traces do not benefit from improvements to async stack traces added in .NET Core 2.1 microsoft/ApplicationInsights-dotnet#2234

Closed

KalleOlaviNiemitalo mentioned this pull request Mar 17, 2024

Can we add a way to add attributes without requiring a dependency on their assembly? dotnet/runtime#99817

Open

cijothomas mentioned this pull request May 31, 2024

Adding Activity.AddException dotnet/runtime#102905

Merged

teo-tsirpanis mentioned this pull request Jul 7, 2024

F# program fails with an unprintable exception when FSharp.Core is not referenced. dotnet/runtime#104528

Closed

Conversation

benaadams commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephentoub Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephentoub commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentoub commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentoub commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Oct 23, 2017

Uh oh!

aelij commented Oct 23, 2017

Uh oh!

benaadams commented Oct 23, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benaadams commented Oct 23, 2017

Uh oh!

stephentoub commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Oct 23, 2017

Uh oh!

benaadams commented Oct 23, 2017

Uh oh!

benaadams commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentoub commented Oct 23, 2017

Uh oh!

benaadams commented Oct 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephentoub commented Oct 23, 2017

Uh oh!

benaadams commented Oct 24, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benaadams commented Jan 18, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

benaadams commented Jan 18, 2018

Uh oh!

danmoseley commented Jan 18, 2018

Uh oh!

Choose a reason for hiding this comment

benaadams commented Oct 23, 2017 •

edited

Loading

stephentoub Oct 23, 2017 •

edited

Loading

stephentoub commented Oct 23, 2017 •

edited

Loading

benaadams commented Oct 23, 2017 •

edited

Loading

stephentoub commented Oct 23, 2017 •

edited

Loading

benaadams commented Oct 23, 2017 •

edited

Loading

stephentoub commented Oct 23, 2017 •

edited

Loading

stephentoub commented Oct 23, 2017 •

edited

Loading

benaadams commented Oct 23, 2017 •

edited

Loading

benaadams commented Oct 23, 2017 •

edited

Loading

benaadams commented Jan 18, 2018 •

edited

Loading

benaadams Jan 18, 2018 •

edited

Loading

benaadams commented Jan 23, 2018 •

edited

Loading