Skip to content

Return object instead of IList from ReplicationLogic#13323

Merged
aparajit-pratap merged 11 commits intoDynamoDS:emitMSILfrom
aparajit-pratap:returnObj
Sep 27, 2022
Merged

Return object instead of IList from ReplicationLogic#13323
aparajit-pratap merged 11 commits intoDynamoDS:emitMSILfrom
aparajit-pratap:returnObj

Conversation

@aparajit-pratap
Copy link
Contributor

@aparajit-pratap aparajit-pratap commented Sep 20, 2022

Purpose

Return object instead of IList from ReplicationLogic

Hopefully this should address https://jira.autodesk.com/browse/DYN-5152

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.
  • This PR modifies some build requirements and the readme is updated

Release Notes

(FILL ME IN) Brief description of the fix / enhancement. Mandatory section

Reviewers

(FILL ME IN) Reviewer 1 (If possible, assign the Reviewer for the PR)

(FILL ME IN, optional) Any additional notes to reviewers or testers.

FYIs

(FILL ME IN, Optional) Names of anyone else you wish to be notified of

@pinzart90
Copy link
Contributor

pinzart90 commented Sep 21, 2022

@aparajit-pratap Looks like you simply replace the IList with object without any regressions (asides from mending a few tests)
I guess I am now thinking why we needed the IList in the first place ...

@aparajit-pratap
Copy link
Contributor Author

aparajit-pratap commented Sep 21, 2022

@aparajit-pratap Looks like you simply replace the IList with object without any regressions (asides from mending a few tests) I guess I am now thinking why we needed the IList in the first place ...

@pinzart90 I also made a few changes in the replication code. PTAL.

I guess choosing IList was the wrong choice. I know better now :)

if (o.IsEnumerable)
{
return o.Value as IList<CLRStackValue>;
return o.Value as IList<CLRStackValue> ?? new List<CLRStackValue>() { o };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to figure out why o.Value would not be IList<CLRStackValue>
The Marshaler should always convert enumerables to IList<CLRStackValue> I think...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, in this particular function I still haven't encountered the case where o.Value is something other than IList<CLRStackValue> so I guess I can revert this but definitely in the other places I needed to check if the Value property was something other than IList<CLRStackValue>.

Did your question pertain to just this function or all such changes in general?

Copy link
Contributor

@pinzart90 pinzart90 Sep 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well...all cases of CLRStackValue.Value
If a CLRStackValue.IsEnumerable== true then I think all nested values should be wrapped in a List < CLRStackValues > (this is done by the ArrayMarshaler)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not need to handle cases like if (o.IsEnumerable) { check if o is not IList<CLRStackValue>}

The root issue (as I interpret it) is that we currently do marshaling from CLR objects to CLRStackValue right at the beginning of ReplicationLogic. We do this marshaling without any consideration to the target types (i.e parameter types of functions)

The proper solution would be to stop doing this marshaling at the beginning of ReplicationLogic and only marshal function outputs then only unmarshal input arguments (considering the target parameter types) right before we invoke the functions.

There are 2 temporary solutions:

  1. Handle ArrayList in Marshaling code (so that it can be considered an array...not a class pointer)
  2. Stop unmarshaling to ArrayList altogether and go with something like List)

    I would go with option 2 since I remember we discussed this in the past, that the more performant ICollection implementation is List<> as opposed to ArrayList.

    The quickest way would be to just convert from ArrayList to List (I can remove all ArrayList usage later on)

     /// ProtoFFI.ArrayMarshaler
    /// internal override object UnMarshal(CLRStackValue dsObject, System.Type expectedCLRType, ProtoCore.MSILRuntimeCore runtimeCore)
              ...............
                if (collection is ArrayList arrList)
               {
                   var list = Activator.CreateInstance(typeof(List<object>));
                   typeof(List<object>).GetMethod(nameof(List<object>.AddRange)).Invoke(list, new object[] { arrList.ToArray(typeof(object)) });
                   return list;
               }
               return collection;
    

if (values.Count == 1) { return values[0]; }
return values;
var obj = n.GetCLRValue(p.Index, controller);
if (typeof(IEnumerable).IsAssignableFrom(obj.GetType()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible for o to be a string?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

o can be a string. but we check for that in o.IsEnumerable

}
if (collection is Array) return collection;

if(elementType == typeof(int))
Copy link
Contributor

@pinzart90 pinzart90 Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as an alternative we could use

  Array arr = null;
  unsafe
  {
      arr = Array.CreateInstance(elementType, collection.Count);
  }
  collection.CopyTo(arr, 0);
  return arr;

If we are ok with using unsafe...(the project would also need <AllowUnsafeBlocks>True</AllowUnsafeBlocks>)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any advantage to doing this? Is the advantage that it will support arbitrary types rather than just the primitive types that I have implemented?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty much just arbitrary types and less code.

Copy link
Member

@mjkkirschner mjkkirschner Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just curious - is there some way to convert to an array while avoiding iterating/copying the entire collection...
https://stackoverflow.com/questions/4972951/listt-to-t-without-copying

}
if (elementType == typeof(double))
{
return ConvertToTypedArray<double>(collection);
Copy link
Contributor

@pinzart90 pinzart90 Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another alternative is to use collection.Cast< T >().ToArray(); instead of the custom ConvertToTypedArray function you implemented.

@mjkkirschner
Copy link
Member

I think theres something to be gained by profiling and investigating ways to improve the performance of collection marshaling - but that seems like it's out of scope for this PR.

Copy link
Contributor

@pinzart90 pinzart90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Please use whatever conversion method you think best.

collection = (collection as ArrayList).ToArray(elementType);
collection = (collection as List<object>).ToArray();
}
return Activator.CreateInstance(expectedCLRType, new[] { collection });
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pinzart90 by your argument, wouldn't this existing code also be unsafe as this relies on there being a suitable constructor for whichever type is passed as expectedCLRType?

Copy link
Contributor

@pinzart90 pinzart90 Sep 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree that it looks a bit unsafe.
Not sure what we could do, except try to get the constructor first and check if it exists
Or we could whitelist the expectedType (Array, ArrayList, List etc) - knowing that the constructors exist for these types

@aparajit-pratap aparajit-pratap merged commit 39bd3f7 into DynamoDS:emitMSIL Sep 27, 2022
@aparajit-pratap aparajit-pratap deleted the returnObj branch September 27, 2022 19:00
sm6srw pushed a commit to sm6srw/Dynamo that referenced this pull request Jan 20, 2023
* return object instead of IList from ReplicationLogic

* cleanup

* fixes

* cleanup

* review comments

* remove comment

* replace ArrayList with List<object>

* cleanup

* revert unchanged files

* try supporting casting to arrays of arbitrary types

* review comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants