Showing posts with label WebAPI. Show all posts
Showing posts with label WebAPI. Show all posts

Thursday, 19 May 2016

Automapper, Dynamics CRM and excluding fields - Part 2

In my previous post, Automapper, Dynamics CRM and excluding fields, I introduced a concept of an "Excludable property". This is just 1 side of the call - POSTing/PUTtting records using a REST API. What about a GET? If you are using something like excludables you'll notice that the JSON returned does not look like the proposed JSON you POST or PUT. In fact, it looks something like this:

{
    Id:
    {
        Include: true,
        Value: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
    },
    Name:
    {
        Include: true,
        Value: "Hello"
    },
    Address1_Line1:
    {
        Include: true,
        Value: "Home!"
    }
}

Which is how our Excludables map to JSON. Here do we stop this?

Extend the IExcludable interface

To convert our excludables correctly we need to intercept the conversion and handle these properties manually. The first problem we hit is although we know it's an Excludable<> we don't know what the raw type is. The best way around this is to expand the IExcludable interface to allow exposing of a raw value, like this:

    public interface IExcludable
    {
        bool Include { get; set; }
        object RawValue { get; set; }
    }

The Excludable<> class just implements it on top of the existing value field, like this:

        public object RawValue
        {
            get
            {
                return value;
            }
            set { this.value = (T) value; }
        }


View Model Json Converter

Now that we can find the raw value without needing to know the underlying generic type we can intercept any excludable and convert it. This is the full converter class that results:

public class ViewModelJsonConverter : JsonConverter
    {
        public override bool CanRead => false;

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            JObject o = JObject.Parse(JsonConvert.SerializeObject(value, Formatting.Indented,
                new JsonSerializerSettings {ReferenceLoopHandling = ReferenceLoopHandling.Ignore}));
            foreach (var propertyInfo in value.GetType().GetProperties())
            {
                if (propertyInfo.CanRead)
                {
                    var currentValue = propertyInfo.GetValue(value);
                    IExcludable excludable = currentValue as IExcludable;
                    if (excludable != null)
                    {
                        if (excludable.Include)
                        {
                            if (excludable.RawValue == null || excludable.RawValue.GetType().IsValueType || excludable.RawValue.GetType().Name == "String")
                            {
                                o.Property(propertyInfo.Name).Value = new JValue(excludable.RawValue);
                            }
                            else
                            {
                                o.Property(propertyInfo.Name).Value = JObject.FromObject(excludable.RawValue);
                            }
                        }
                        else
                        {
                            o.Remove(propertyInfo.Name);
                        }
                    }
                }
            }
            o.WriteTo(writer);
        }

        public override bool CanConvert(Type objectType)
        {
            if (objectType.BaseType == typeof (ViewModel))
            {
                return true;
            }
            return false;
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            throw new System.NotImplementedException();
        }
    }

In the above code our view models code had inherited a ViewModel class. Also worth pointing out is how you handle the both value and object types (with strings needing an extra helping hand as they're a bit different!). For value types and strings you need to create a JValue where as for reference types you need to expose them as a JObject.

Now, just add this converter like so in your global.asax exactly how you added your excludable converter from the previous post:

GlobalConfiguration.Configuration.AddJsonConverter(new ViewModelJsonConverter());


This code effectively flattens the excludable class into the generic types and outside of the world of your REST api nobody is any the wiser.

Monday, 7 March 2016

Automapper, Dynamics CRM and excluding fields

In the past when I've built web based applications Automapper was always one of those libraries that I both loved and hated at the same time. In the Dynamics CRM world it can often be a bit of a dangerous tool to unleash on a website, especially when utilised by developers that don't know CRM very well. Let me explain why.

Attribute Collections

Most CRM Developers will already know what I'm talking about here, but for those of you not privy to this - the properties of an entity from a CRM database table are not actually surfaced quite like EntityFramework or nHibernate, they are surfaced using a dictionary. This is subsequently wrapped in an Attribute Collection. One of the main advantages of this is you can control partial updates/retrieves without worrying about the state of the entire entity. On the negative side this plays havoc when libraries like Automapper are used to populate the entities, especially when using early bound objects.

Here's a sample to give you and idea of what I'm getting at. Firstly, let's presume our domain models are Early Bound objects exported from dynamics. (self plug! Personally I use this open source tool : https://github.com/conorjgallagher/Dynamics.ExtendedSvcUtil).

Now, let's say we have an REST service and we want to utilise it to update an account. So we build a view model like this:

public class AccountViewModel
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Address1_Line1 { get; set; }
}

And we wire up automapper to map that across to our domain entity. In the latest version I believe that would look like this:

CreateMap<Account, AccountViewModel>().ReverseMap()

In this example we can create a new account via the this REST service with the following JSON:

{
    Id: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Name: "Hello",
    Address1_Line1: "Home!"
}

Within our controller we will receive an AccountViewModel populated with all the relevant data. We can then use Automapper to push this View Model into an early bound Account entity using the same method as above. As long as all the field names match we're good to go. Even though you are mapping to strongly typed fields what you end up with under the hood is an dictionary like this:

"accountid"="aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
"name"="Hello"
"address1_line1"="Home!"

Null vs Unset

A really nice feature of CRM is it differentiates between null and unset. If you exclude a field from the attribute collection it will also be excluded from the update statement that hits SQL. This is very useful for performance and limiting what plugins / workflows fire.

Back to Automapper and our view models above - a problem arises when you exclude fields from the JSON.  For example, if you subsequently send the following after the previous update:

{
    Id: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Address1_Line1: "Home line 1!"
}

This will hit our view model as this:

{
    Id = "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Name = null,
    Address1_Line1 = "Home line 1!"
}

Which I guess is expected, because how else can you represent an excluded value in the view model? If we don't deal with this we hit a more fundamental issue further down the chain in that our attribute collection will end up like this:

"accountid"="aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
"name"=null
"address1_line1"="Home line 1!"

And we'll blank our account name in CRM. Not good!

Excludable

This takes me on to a pattern I would like to propose for this type of issue: the concept of an excludable property. Looking at how nullable value types work surely we can come up with a similar concept! I won't take you through the entire evolution, but here is the interface and struct I now propose:

    public interface IExcludable
    {
        bool Include { get; set; }
    }

    public struct Excludable<T> : IExcludable
    {
        private bool hasValue;
        internal T value;
        private bool include;


        public Excludable(T value)
        {
            this.value = value;
            if (value != null)
            {
                this.hasValue = true;
            }
            else
            {
                this.hasValue = false;
            }
            this.include = true;
        }

        public bool HasValue
        {
         
            get
            {
                return hasValue;
            }
        }

        public bool Include
        {
            get { return include; }
            set { include = value; }
        }

        public T Value
        {
            get
            {
                return value;
            }
        }

   
        public T GetValueOrDefault()
        {
            return value;
        }

     
        public T GetValueOrDefault(T defaultValue)
        {
            return hasValue ? value : defaultValue;
        }

        public override bool Equals(object other)
        {
            if (!include || !hasValue) return other == null;
            if (other == null) return false;
            return value.Equals(other);
        }

        public override int GetHashCode()
        {
            return hasValue ? value.GetHashCode() : 0;
        }

        public override string ToString()
        {
            return hasValue ? value.ToString() : "";
        }

        public static implicit operator Excludable<T>(T value)
        {
            return new Excludable<T>(value);
        }

        public static explicit operator T(Excludable<T> value)
        {
            return value.Value;
        }

        public static bool operator ==(Excludable<T> x, T y)
        {
            return x.Equals(y);
        }

        public static bool operator !=(Excludable<T> x, T y)
        {
            return !x.Equals(y);
        }
    }

Overriding all the comparisons were required to get it to perform both value and null comparisons correctly. Finally, we can change our view model to use this instead:

    public class AccountViewModel
    {
        public Excludable<Guid> Id { get; set; }
        public Excludable<string> Name { get; set; }
        public Excludable<string> Address1_Line1 { get; set; }
    }

Json Converter gotcha

There's 1 final problem we need to solve that I'll highlight now. If we run our object without the address in it all seems to work just fine. Our property comes through as excluded as expected. But, if we instead try set it to null it is marked as included=false! The reason is down to how the JSON formatter deserializes the data into the given object. It doesn't actually call your constructor but does something dirty under the hood. How do we fix this? Create a custom json converter

    public sealed class ExcludableConverter : JsonConverter
    {
        public override bool CanConvert(Type objectType)
        {
            return objectType.IsGenericType && objectType.GetGenericTypeDefinition() == typeof(Excludable<>);
        }
        public override bool CanWrite { get { return false; } }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            return Activator.CreateInstance(objectType, reader.Value);
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
    }

And set this up on application start within your global.asax:

            var formatters = GlobalConfiguration.Configuration.Formatters;
            var jsonFormatter = formatters.JsonFormatter;
            var settings = jsonFormatter.SerializerSettings;
            jsonFormatter.SerializerSettings.Converters.Add(new ExcludableConverter());

Now, when a null comes in we explicitly tell the converter to create a new object for us.

Wiring up Automapper

The final step is getting automapper to utilise this new way of excluding items. In automapper you can set a condition when mapping so that you can do conditional ignores. I found the simplest way to add this rule was write an extension method:

public static void AddExcludableRule<S, D>(this IMappingExpression<S, D> m)
        {
            m.ForAllMembers(opt => opt.Condition(
                    s =>
                        !(s.SourceValue is IExcludable) || ((IExcludable)s.SourceValue).Include
                    ));
        }

Now, whenever I want to wire up an object that has excludables I just add that call to the end:

CreateMap<Account, AccountViewModel>().ReverseMap().AddExcludableRule();



Thursday, 25 February 2016

Dynamics CRM Web API - Pluralization bug

I started investigating the Web API while it was in preview mode so have noticed it move quite quickly over time. Mainly this is changes but sometimes it's bugs. For example, a change I spotted since my previous post (Web API Preview - Unrecognized 'Edm.String' literal...) is how it deals with lookup and regarding references. In the preview you performed the following:
/api/data/appointments?$select=subject&$filter=regardingobjectid eq a199a199-a199-a199-a199-a199a199a199

This has change to the following in the v8.0 release:
/api/data/v8.0/appointments?$select=subject&$filter=_regardingobjectid_value eq a199a199-a199-a199-a199-a199a199a199

The second thing I spotted is in fact a bug. This may have existed in the preview, but I never spotted it and it had me scratching my head for quite some time. We have the concept of an "event day" which we have name "inv_eventday". When creating this entity CRM creates a collection schema to allow you to reference it via the web api. For example the appointment entity has an equivalent collection schema called "appointments"; notice the pluralisation.

So, on to my event days. What you would expect to find as a collection schema is this:
/api/data/v8.0/inv_eventdays

Unfortunately, this is not the case. You get an error "Resource not found for the segment 'inv_eventdays'." I wondered if I had a case sensitivity issue or something, but every combination of captitals / camel case / pascal case just didn't seem to work.

After some time I decided I should check out the metadata to see if I could uncover the correct casing. For reference the following is how you get a list of all entity metadata:
/api/data/v8.0/EntityDefinitions

This uncovered a definition with the following schema name for the entity:
"SchemaName":"inv_eventday"

Which seems to match my expectations on what it should be. So what's the problem? Reading further down I find the Collection Schema name:
"CollectionSchemaName":"inv_eventdaies"

That's some funky plural we've got there! Looks like the algorythm decides that anything ending in "y" is pluralised as "ies". Obviously this isn't correct for all words. E.g. the word "sky" is indeed "skies", but "day" is in fact "days".

Now... where do you log CRM bugs...


Thursday, 3 December 2015

Web API Preview - Unrecognized 'Edm.String' literal...

Today I was playing around with the new Web API Preview to try and check out how the new REST API was going to work. If you're interested take a look here: https://msdn.microsoft.com/dynamics/crm/webapipreview. At a first glance this looks really good. It's much cleaner and does away with a long unnecessary url and the case sensitivity that always bugged me.

The first issue I hit was when trying to filter by a regarding object on an appointment. My first thought was is this not supported in the preview? I took a standard oData query from the 2011 endpoint and converted it so that this:
/xrmservices/2011/OrganizationData.svc/AppointmentSet?$select=Subject&$filter=RegardingObjectId/Id eq (guid'a199a199-a199-a199-a199-a199a199a199')

became this:
/api/data/appointments?$select=subject&$filter=regardingobjectid/id eq (guid'a199a199-a199-a199-a199-a199a199a199')


But I get presented with the error
Unrecognized 'Edm.String' literal 'guid'a199a199-a199-a199-a199-a199a199a199''


So what's going wrong? I tried many different variations, such as:

/api/data/appointments?$select=subject&$filter=regardingobjectid/id eq guid'a199a199-a199-a199-a199-a199a199a199'
/api/data/appointments?$select=subject&$filter=regardingobjectid eq (guid'a199a199-a199-a199-a199-a199a199a199')

etc... I always got 1 of 3 errors, the one previously mentioned, or
Can only bind segments that are Navigation, Structural, Complex, or Collections.

A binary operator with incompatible types was detected. Found operand types 'Edm.Guid' and 'Edm.String' for operator kind 'Equal'


The final error mentioned was by doing a direct comparison
/api/data/appointments?$select=subject&$filter=regardingobjectid eq 'a199a199-a199-a199-a199-a199a199a199'


And it got me thinking, maybe they've changed something. A quick look at oData 4.0 and I find that they've done away with both specifying the object type and the quotes for guids. The query is simply this:
/api/data/appointments?$select=subject&$filter=regardingobjectid eq a199a199-a199-a199-a199-a199a199a199


A sample on this query would have been nice (maybe there is one, I couldn't find it!)... But I'm liking these changes a lot!

Friday, 3 October 2014

Web API - Adding Xml Doc Comments to help files

One of the beautiful features of Web API is you get an out of the box set of help pages documenting your API and what calls are available. You will see this "HelpPage" section under "Areas" of your project. One thing I wanted to do to extend this is configure the help pages to read my Xml documentation comments within the code. It's not awfully difficult to achieve, simply follow these steps:

  1. In the file HelpPage/App_Start/HelpPageConfig.cs uncomment the following line:
     config.SetDocumentationProvider(new XmlDocumentationProvider(HttpContext.Current.Server.MapPath("~/App_Data/XmlDocument.xml")));  
    

  2. Open up the project properties for you Web API project and under the build tab enable the Xml documentation file (set the file path to App_Data\XmlDocument.XML)

This gives you the basic setup to get the XML comments outputting to your Help Pages. But how about making it look sexier? For example, let's say I had the following XML comment I wanted to display:
 /// <summary>  
 /// Get a list of products within the given product kit.  
 /// </summary>  
 /// <param name="id">Id of the parent product kit</param>  
 /// <returns>This api method returns a list of products in JSON format. To get a list of products in XML format add the following HTTP header:<br />  
 /// <br />  
 /// Accept: application/xml<br />  
 /// </returns>  

This will strip out the HTML tags and output on a single line like this:



To allow breaks in our comments we need to make a couple of changes. Firstly, we need to change how our XmlDocumentationProvider retrieves the tag value from the XML comment. Find the GetTagValue function in this class and change the line as follows:
     private static string GetTagValue(XPathNavigator parentNode, string tagName)  
     {  
       if (parentNode != null)  
       {  
         XPathNavigator node = parentNode.SelectSingleNode(tagName);  
         if (node != null)  
         {  
           return node.InnerXml;
         }  
       }  
       return null;  
     }  

Next, we need to modify all the display templates that display these tags to show the raw value instead. The Display Template you will need to modify are HelpPages/Views/Help/DisplayTemplates/ApiGroup.cshtml and HelpPageApiModel.cshtml located at. Find any of the following lines:
 <p>@controllerDocumentation</p>  

 <p>api.Documentation</p>  

 <p>@description.ResponseDescription.Documentation</p>  

And replace them with the equivalent as per this code:
 <p>@Html.Raw(controllerDocumentation)</p>  

 <p>@Html.Raw(api.Documentation)</p>  

 <p>@Html.Raw(description.ResponseDescription.Documentation)</p>  

This could cover you for all areas you want to add breaks or html formatting to. For example, our description in the given example should look much better now: