Showing posts with label C#. Show all posts
Showing posts with label C#. Show all posts

Friday, 7 October 2016

Performance: Metadata calls vs Exceptions

Today I was embarking on a mission to check the existence of a field on an entity. If it exists, do something, if not... well, do nothing! My first thoughts were "metadata... urgh!". As all of us in the CRM development world know, the metadata is notoriously slow. Reading stackoverflow I found that the recommended answer was always using the metadata. So I thought to myself there's got to be a better way to do this. Which gave me an idea... which performs faster, exceptions or the metadata? To hit the metadata you need to run something like this:
RetrieveEntityRequest request = new RetrieveEntityRequest
{
    EntityFilters = EntityFilters.Attributes,
    LogicalName = "account"
};
var response = (RetrieveEntityResponse)service.Execute(request);
AttributeMetadata first = null;
foreach (var element in response.EntityMetadata.Attributes)
{
    if (element.LogicalName == "xyz_fieldname")
    {
        first = element;
        break;
    }
}
var fieldExists = first != null;
I chose not to use linq/expressions when checking for the field just to try keep it as performant as possible. This code is fine in general, but boy is it slow. So I came with this as an alternative instead:
try
{
    var query = new QueryExpression("account");
    query.Criteria.AddCondition("accountid", ConditionOperator.Equal, "294450db-46c9-447e-a642-3babf913d800");
    query.NoLock = true;
    query.ColumnSet = new ColumnSet("xyz_fieldname");
    service.RetrieveMultiple(query);
}
catch
{
    // ignored
}
Using a query expression has 2 advantages, you're running the query against the primary key (accountid). You don't care about the id itself, the code will either throw an exception if the field doesn't exist, or return no records if it succeeds (1 record if you are the unluckiest guy on the planet to get a matching guid... but even then it would be faster). The second advantage of a query expression is you can run it using a nolock. You really don't care about the result set, the purpose isn't to find a record, it's to see if including the column forces an exception. So how do you test the execution of this? I wrote a console app that used a stop watch to wrap each call. The first time I ran the tests I ran each console app independently so not to skew the results by code optimization on what call ran first etc. And the results I got for a single call were the exception generally executed about 1.5 times faster. Sample code is this:
private static void RunExceptionTests(IOrganizationService service, int steps)
{
    Console.WriteLine("Testing exception with {0} steps", steps);
    var stopwatch = Stopwatch.StartNew();
    for (int i = 0; i < steps; i++)
    {
        try
        {
            var query = new QueryExpression("account");
            query.Criteria.AddCondition("accountid", ConditionOperator.Equal, "294450db-46c9-447e-a642-3babf913d800");
            query.NoLock = true;
            query.ColumnSet = new ColumnSet("xyz_fieldname");
            service.RetrieveMultiple(query);
        }
        catch
        {
            // ignored
        }
    }
    stopwatch.Stop();
    Console.WriteLine("Milliseconds taken: {0}", stopwatch.ElapsedMilliseconds);
}

private static void RunMetadataTest(IOrganizationService service, int steps)
{
    Console.WriteLine("Testing metadata with {0} steps", steps);
    var stopwatch = Stopwatch.StartNew();
    for (int i = 0; i < steps; i++)
    {
        RetrieveEntityRequest request = new RetrieveEntityRequest
        {
            EntityFilters = EntityFilters.Attributes,
            LogicalName = "account"
        };
        var response = (RetrieveEntityResponse)service.Execute(request);
        AttributeMetadata first = null;
        foreach (var element in response.EntityMetadata.Attributes)
        {
            if (element.LogicalName == "xyz_fieldname")
            {
                first = element;
                break;
            }
        }
        var fieldExists = first != null;
    }
    stopwatch.Stop();
    Console.WriteLine("Milliseconds taken: {0}", stopwatch.ElapsedMilliseconds);
}
I ran several tests on CRM online varying between multiple/single calls to the metadata vs multiple/single retrieve multiple calls throwing an exception. Exceptions always outperformed the metadata. If you're performing multiple calls within the same code it jumps to about 3 times faster (I'm guessing optimizations come into play). Because of how plugins are loaded into memory for faster execution I would wonder if it would regularly perform at 2 - 3 times faster than a metadata call. Either way, the bottle neck is the call to the Metadata service which cannot be optimized unless you introduce caching and more code complexity. Also, if the field exists you won't get an exception which means yet another performance bonus. The only scenario I haven't tested is running it against a massively heavily used entity... but if you're hitting your database so hard that a nolock retrieve cannot return in an acceptable time frame you probably have bigger problems to worry about! To conclude, can I just say the following. Micrososft, will you fix your metadata service already! It's been slow for donkeys years and is a real annoyance when you have to use it.

Thursday, 6 October 2016

Extended CrmSvcUtil - Exporting an attribute list

A little feature of the Extended CrmSvcUtil I neglected to mention in my previous post (it was a late feature!) is the ability to export a list of strongly typed attribute names. This helps remove "magic strings" from your source and introduce some strongly typed attribute checking. This is useful, even when using Early Bound, as you often need to check the existence of an attribute.

For example, let's say you are writing a plugin that fires on update of contact and you wish to include a pre image containing the parent account. What you will often see is code like this:

if (preImage.Contains("parentcustomerid") == false)
{
    //trace / throw an exception stating the parent count hasn't been provided...
}

Checking for null is not the same as checking for existence, because some contacts might not have a parent account set. So the check for existence is often quite important. Using an attribute list allows you to strongly type this instead as follows:

if (preImage.Contains(ContactAttributes.ParentCustomer) == false)
{
    //trace / throw an exception stating the parent count hasn't been provided...
}

Another area where this is incredibly useful is when building queries using Query Expressions or Fetch Expressions. If you want to include a set of columns, or set a condition on an attribute you will end up with this type of code:

QueryExpression qe = new QueryExpression();
qe.EntityName = "contact";
qe.ColumnSet = new ColumnSet();
qe.ColumnSet.Columns.Add("parentcustomerid");

Being able to specify the attribute strongly, like the following looks much better:

qe.ColumnSet.Columns.Add(ContactAttributes.ParentCustomer);

This helps work around many issues, like typo bugs or name change, like in cases where somebody accidentally creates a field called "new_ProjjectType and wishes to fix the name of the field. If 5 or 6 plugins already reference this field and perform some logic based on its value you might end up with multiple "magic strings" to fix across your code. Using an attribute list is a 1 fix solution to the problem.

The source for the Extended CrmSvcUtil can be downloaded from git hub with the latest release available to download from here

Wednesday, 28 September 2016

Extended CrmSvcUtil - A Neater Early Bound Generator

For quite some time I have persisted with late bound objects, because within plugins it makes life a bit simpler. The idea of Early Bound objects is good in theory, but the Microsoft provided CrmSvcUtil just doesn't cut it in terms of how it gives you the code. (1 big SDK file is not my idea of fun). You used to be able to split different aspects of this out, but they have removed functionality in recent versions. This has probably been the main reason I stuck with Late Bound until recently.

Issues with CrmSvcUtil

The biggest issue I have with the existing tool is the lack of naming ability. Say you have a custom entity called "new_project" with an OptionSet attribute on it called "new_projecttype". Ignoring all the other entities that will get exported regardless of whether you actually need them or not, and ignoring all the standard attributes that will be exported, the naming convention you'll end up with is this:

// Class...
public partial class new_project...

// Property...
public Microsoft.Xrm.Sdk.OptionSetValue new_projecttype

// Enum
public enum new_project_new_projecttype

I've chosen the type OptionSet in particular to convey an additional problem, but there are a host of issues with what is generated.
  1. We end up with 1 huge unmanageable file.
  2. Really bad naming convention by default that is not easy to change. Sure, you could manually edit them, but it will get overwritten with each generation of the metadata
  3. OptionSet properties created as the type OptionSetValue. Surely if we are using Early Bound then shouldn't our option set properties be the equivalent enum type? 

So the tool is quite lazy in what it does. It's a bit of a "bare minimum" to get you over the fence, and then you're left to your own devices. Quite frankly, you'd be quicker just writing the classes yourself and as long as you honor the correct attributes it would work perfectly fine.

I have investigated many tools, and all of them fell short. So this and all of the above issues caused the birth of my own pet project which has made my life a lot easier.

Extended Svc Util

It's named simply so, because all it does is extend and build on top of what the existing CrmSvcUtil does. Once the code has been generated it does not intercept the generation of the "monster file", but instead piggy backs the code generated for that to produce its own files. Let's take a look at what you can do.

To fix the problems in the above files you could set up a configuration like this:

<configuration>
 <configSections>
  <section name="schemaDefinition" type="CodeGenerator.Config.SchemaDefinition, CodeGenerator" allowLocation="true" allowDefinition="Everywhere"/>
 </configSections>
 <schemaDefinition groupOptionSetsByEntity="true" exportAttributeNames="true" entitiesFolder="..\MyProject.DomainModels" enumsFolder="..\MyProject.DomainModels">
  <entities>
   <entity name="new_project" friendlyName="Project">
    <attributes>
     <attribute name="new_name"  friendlyName="Name"/>
     <attribute name="new_projecttype" friendlyName="ProjectType" />
    </attributes>
   </entity>
  </entities>

  <optionSets>
   <!-- Global OptionSets-->
   <optionSet name="new_someglobaloptionset" friendlyName="SomeGlobalOptionSet" entity="Global" />

   <!-- Project OptionSets-->
   <optionSet name="new_project_new_projecttype" friendlyName="Project_ProjectType" entity="new_project" />
  </optionSets>
 </schemaDefinition>
</configuration>

So what does this do? Firstly, you can add friendly names to your entities, attributes and option sets. Secondly, the export will use the correct enum for your option sets rather than using the out of the box OptionSetValue. So what you'll end up with instead is this:

// Class...
public partial class Project

// Property...
public Proejct_ProjectType? ProjectType

// Enum
public enum Project_ProjectType

The next configuration item I'd like to point out is not only can you depict where the file is generated, but you can decide to group all of your option sets into 1 file per entity rather than separate classes. These are defined at the top of the configuration under Schema Definition. All of this causes 2 files to be generated named:

  • Project.cs
  • Project.Enums.cs


Finally, only entities you have specified within the list will be exported to their corresponding file, all others will be ignored. All of the code will still be exported to the output file you specify so if you wanted to double check that source against what this tool exports you can do so.

All of this makes life much easier and readable in the Early Bound world, and makes it much quicker to generate the classes exactly as you want. I have included a global option set option in there just as an example of how to deal with that. But in effect all of those option sets in this example will be exported to a file called Global.Enums.cs. You can rename out of the box fields, status fields and their accompanying enums too. So you're not just stuck to your custom entities and fields.

Source

I have uploaded the source to github (https://github.com/conorjgallagher/Dynamics.ExtendedSvcUtil). There are further instructions up there on how to utilise the DLL it builds with CrmSvcUtil. It's fully open source so feel free to download, edit, and use to your hearts delight. In the root folder of the project I have included the latest built version of the DLL, so if you just want that feel free to download it.

If you find bugs please feel free to submit a comment. I have not fully decided on how best to manage contributions, so if you are interested please contact me and we can discuss.

Enjoy!

Thursday, 19 May 2016

Automapper, Dynamics CRM and excluding fields - Part 2

In my previous post, Automapper, Dynamics CRM and excluding fields, I introduced a concept of an "Excludable property". This is just 1 side of the call - POSTing/PUTtting records using a REST API. What about a GET? If you are using something like excludables you'll notice that the JSON returned does not look like the proposed JSON you POST or PUT. In fact, it looks something like this:

{
    Id:
    {
        Include: true,
        Value: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
    },
    Name:
    {
        Include: true,
        Value: "Hello"
    },
    Address1_Line1:
    {
        Include: true,
        Value: "Home!"
    }
}

Which is how our Excludables map to JSON. Here do we stop this?

Extend the IExcludable interface

To convert our excludables correctly we need to intercept the conversion and handle these properties manually. The first problem we hit is although we know it's an Excludable<> we don't know what the raw type is. The best way around this is to expand the IExcludable interface to allow exposing of a raw value, like this:

    public interface IExcludable
    {
        bool Include { get; set; }
        object RawValue { get; set; }
    }

The Excludable<> class just implements it on top of the existing value field, like this:

        public object RawValue
        {
            get
            {
                return value;
            }
            set { this.value = (T) value; }
        }


View Model Json Converter

Now that we can find the raw value without needing to know the underlying generic type we can intercept any excludable and convert it. This is the full converter class that results:

public class ViewModelJsonConverter : JsonConverter
    {
        public override bool CanRead => false;

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            JObject o = JObject.Parse(JsonConvert.SerializeObject(value, Formatting.Indented,
                new JsonSerializerSettings {ReferenceLoopHandling = ReferenceLoopHandling.Ignore}));
            foreach (var propertyInfo in value.GetType().GetProperties())
            {
                if (propertyInfo.CanRead)
                {
                    var currentValue = propertyInfo.GetValue(value);
                    IExcludable excludable = currentValue as IExcludable;
                    if (excludable != null)
                    {
                        if (excludable.Include)
                        {
                            if (excludable.RawValue == null || excludable.RawValue.GetType().IsValueType || excludable.RawValue.GetType().Name == "String")
                            {
                                o.Property(propertyInfo.Name).Value = new JValue(excludable.RawValue);
                            }
                            else
                            {
                                o.Property(propertyInfo.Name).Value = JObject.FromObject(excludable.RawValue);
                            }
                        }
                        else
                        {
                            o.Remove(propertyInfo.Name);
                        }
                    }
                }
            }
            o.WriteTo(writer);
        }

        public override bool CanConvert(Type objectType)
        {
            if (objectType.BaseType == typeof (ViewModel))
            {
                return true;
            }
            return false;
        }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            throw new System.NotImplementedException();
        }
    }

In the above code our view models code had inherited a ViewModel class. Also worth pointing out is how you handle the both value and object types (with strings needing an extra helping hand as they're a bit different!). For value types and strings you need to create a JValue where as for reference types you need to expose them as a JObject.

Now, just add this converter like so in your global.asax exactly how you added your excludable converter from the previous post:

GlobalConfiguration.Configuration.AddJsonConverter(new ViewModelJsonConverter());


This code effectively flattens the excludable class into the generic types and outside of the world of your REST api nobody is any the wiser.

Monday, 7 March 2016

Automapper, Dynamics CRM and excluding fields

In the past when I've built web based applications Automapper was always one of those libraries that I both loved and hated at the same time. In the Dynamics CRM world it can often be a bit of a dangerous tool to unleash on a website, especially when utilised by developers that don't know CRM very well. Let me explain why.

Attribute Collections

Most CRM Developers will already know what I'm talking about here, but for those of you not privy to this - the properties of an entity from a CRM database table are not actually surfaced quite like EntityFramework or nHibernate, they are surfaced using a dictionary. This is subsequently wrapped in an Attribute Collection. One of the main advantages of this is you can control partial updates/retrieves without worrying about the state of the entire entity. On the negative side this plays havoc when libraries like Automapper are used to populate the entities, especially when using early bound objects.

Here's a sample to give you and idea of what I'm getting at. Firstly, let's presume our domain models are Early Bound objects exported from dynamics. (self plug! Personally I use this open source tool : https://github.com/conorjgallagher/Dynamics.ExtendedSvcUtil).

Now, let's say we have an REST service and we want to utilise it to update an account. So we build a view model like this:

public class AccountViewModel
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Address1_Line1 { get; set; }
}

And we wire up automapper to map that across to our domain entity. In the latest version I believe that would look like this:

CreateMap<Account, AccountViewModel>().ReverseMap()

In this example we can create a new account via the this REST service with the following JSON:

{
    Id: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Name: "Hello",
    Address1_Line1: "Home!"
}

Within our controller we will receive an AccountViewModel populated with all the relevant data. We can then use Automapper to push this View Model into an early bound Account entity using the same method as above. As long as all the field names match we're good to go. Even though you are mapping to strongly typed fields what you end up with under the hood is an dictionary like this:

"accountid"="aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
"name"="Hello"
"address1_line1"="Home!"

Null vs Unset

A really nice feature of CRM is it differentiates between null and unset. If you exclude a field from the attribute collection it will also be excluded from the update statement that hits SQL. This is very useful for performance and limiting what plugins / workflows fire.

Back to Automapper and our view models above - a problem arises when you exclude fields from the JSON.  For example, if you subsequently send the following after the previous update:

{
    Id: "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Address1_Line1: "Home line 1!"
}

This will hit our view model as this:

{
    Id = "aef7b4c1-98f6-4f53-9be3-2fa72d1e319d",
    Name = null,
    Address1_Line1 = "Home line 1!"
}

Which I guess is expected, because how else can you represent an excluded value in the view model? If we don't deal with this we hit a more fundamental issue further down the chain in that our attribute collection will end up like this:

"accountid"="aef7b4c1-98f6-4f53-9be3-2fa72d1e319d"
"name"=null
"address1_line1"="Home line 1!"

And we'll blank our account name in CRM. Not good!

Excludable

This takes me on to a pattern I would like to propose for this type of issue: the concept of an excludable property. Looking at how nullable value types work surely we can come up with a similar concept! I won't take you through the entire evolution, but here is the interface and struct I now propose:

    public interface IExcludable
    {
        bool Include { get; set; }
    }

    public struct Excludable<T> : IExcludable
    {
        private bool hasValue;
        internal T value;
        private bool include;


        public Excludable(T value)
        {
            this.value = value;
            if (value != null)
            {
                this.hasValue = true;
            }
            else
            {
                this.hasValue = false;
            }
            this.include = true;
        }

        public bool HasValue
        {
         
            get
            {
                return hasValue;
            }
        }

        public bool Include
        {
            get { return include; }
            set { include = value; }
        }

        public T Value
        {
            get
            {
                return value;
            }
        }

   
        public T GetValueOrDefault()
        {
            return value;
        }

     
        public T GetValueOrDefault(T defaultValue)
        {
            return hasValue ? value : defaultValue;
        }

        public override bool Equals(object other)
        {
            if (!include || !hasValue) return other == null;
            if (other == null) return false;
            return value.Equals(other);
        }

        public override int GetHashCode()
        {
            return hasValue ? value.GetHashCode() : 0;
        }

        public override string ToString()
        {
            return hasValue ? value.ToString() : "";
        }

        public static implicit operator Excludable<T>(T value)
        {
            return new Excludable<T>(value);
        }

        public static explicit operator T(Excludable<T> value)
        {
            return value.Value;
        }

        public static bool operator ==(Excludable<T> x, T y)
        {
            return x.Equals(y);
        }

        public static bool operator !=(Excludable<T> x, T y)
        {
            return !x.Equals(y);
        }
    }

Overriding all the comparisons were required to get it to perform both value and null comparisons correctly. Finally, we can change our view model to use this instead:

    public class AccountViewModel
    {
        public Excludable<Guid> Id { get; set; }
        public Excludable<string> Name { get; set; }
        public Excludable<string> Address1_Line1 { get; set; }
    }

Json Converter gotcha

There's 1 final problem we need to solve that I'll highlight now. If we run our object without the address in it all seems to work just fine. Our property comes through as excluded as expected. But, if we instead try set it to null it is marked as included=false! The reason is down to how the JSON formatter deserializes the data into the given object. It doesn't actually call your constructor but does something dirty under the hood. How do we fix this? Create a custom json converter

    public sealed class ExcludableConverter : JsonConverter
    {
        public override bool CanConvert(Type objectType)
        {
            return objectType.IsGenericType && objectType.GetGenericTypeDefinition() == typeof(Excludable<>);
        }
        public override bool CanWrite { get { return false; } }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            return Activator.CreateInstance(objectType, reader.Value);
        }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
    }

And set this up on application start within your global.asax:

            var formatters = GlobalConfiguration.Configuration.Formatters;
            var jsonFormatter = formatters.JsonFormatter;
            var settings = jsonFormatter.SerializerSettings;
            jsonFormatter.SerializerSettings.Converters.Add(new ExcludableConverter());

Now, when a null comes in we explicitly tell the converter to create a new object for us.

Wiring up Automapper

The final step is getting automapper to utilise this new way of excluding items. In automapper you can set a condition when mapping so that you can do conditional ignores. I found the simplest way to add this rule was write an extension method:

public static void AddExcludableRule<S, D>(this IMappingExpression<S, D> m)
        {
            m.ForAllMembers(opt => opt.Condition(
                    s =>
                        !(s.SourceValue is IExcludable) || ((IExcludable)s.SourceValue).Include
                    ));
        }

Now, whenever I want to wire up an object that has excludables I just add that call to the end:

CreateMap<Account, AccountViewModel>().ReverseMap().AddExcludableRule();



Tuesday, 14 July 2015

Dynamics CRM - Developing faster code

We're deep in the days of large corporate multinational CRM systems, integrating with other systems, being interrogated daily by websites and portals, and generally being poked and prodded by many different systems. Because of this we regularly hit performance issues in our systems that in a worst case scenario can grind CRM or the related access points to a halt. Outside of throwing servers at the problem what as a developer can you do? Here are my top 5 tips.

1. Always specify a column set

This is a no-brainer, or at least it should be, always specify a column set when performing a query otherwise you're pulling back every column in the table every time. The amount of times I see the following in code is not funny:
var cols = new ColumnSet(true);

I've even been a perpetrator myself! We really need to get better at specifying our columnsets. Breaking it down to basics, specifying a column set can mean the difference between this:
select * from contact

and this
select contactid, fullname, birthdate from contact

I have seen companies hit the max columns on either contact or account, so that might just give you an idea of what sort of impact specifying all columns will have.

2. Nolock = true

Unfortunately, this is only available on the QueryExpression or via FetchXml in a FetchExpression object. Be aware that this tip isn't necessarily as black and white as "always do this", as there are situations where it will not be feasible.

The upside of setting NoLock = true is that it won't request a record lock during the SQL read and therefore should neither block nor get blocked from making the call. This not only makes the call faster, but makes load balanced servers more efficient as they can read the same records without waiting on the data to be freed from a lock.

There is a downside to setting this which is the possibility of reading uncommitted data or even duplicate data. Depending on what you are reading you may or may not care about this, so you'll need to make a judgement accordingly.

For example, imagine I was reading a settings table to find out if I should display a field in red or blue. I happen to read some uncommitted data that changed the colour and was subsequently rolled back. This usually wouldn't be considered detrimental. In fact, this is probably a good example of when you should consider using NoLock.

On the other hand what if you were reading something more crucial? For example you read an uncommitted price of a stock on the market that in turn caused you to price an option incorrectly due to uncommitted data being rolled back. Probably an unrealistic scenario, but definitely something you would consider as a bigger issue!

The question you need to answer before applying this is - which is worse...
  • A deadlock
  • Inaccurate data
If inaccurate data isn't a huge problem then NoLock is going to offer a favourable solution.

3. OrganizationService.Retrieve locks the record!

This leads on from the previous point, if you perform an OrganizationService.Retrieve you'll end up locking the record on the database. If the retrieve doesn't need to lock then use a QueryExpression (or FetchExpression) to perform a NoLock and retrieve the record that way instead. This will return an entity collection which you'll just need the first record from, but if performance is king this shouldn't be a problem.

4. LINQ locks too!

Again, as with Retrieve, LINQ doesn't allow you to specify a NoLock. Which means you need to use a Query Expression (or FetchExpression) instead. This is particularly annoying as LINQ is such a beautiful way to access CRM, but such a pity you can't gain access to the QueryExpression it creates under the hood. (Well... you can... but not without performing some reflection to invoke a private method... let me know if you'd like to know how to do this)

5. Indexing

Finally, slightly beyond the realm of coding, but still something you could do - take a look at indexing in the database or lack thereof. If you have a DBA team these guys will be able to generate reports on missing indexes for you. On the other hand, if you know of fields that you are constantly searching / filtering then consider adding them to the quickfind views which will in turn create indexes in the background. Getting a DBA to add coverage is ultimately better, but quickfind is a quick (be it slightly dirty!) way to get a field indexed.

Take care though, indexes can deplete performance on other operations, such as inserts. So don't start adding indexes willy nilly all over the database! Something to be aware of.

Wednesday, 27 May 2015

The new LocalPluginContext (and new BasePlugin class)

As a .net developer who often embarks on CRM plugins there's 1 thing I have always took for granted and that is the code provided for plugins when you use the CRM Developer Toolkit for Visual Studio. I have often reviewed the code, performed some minor refactoring on it, and pretty much just overlooked the general logic of it. It's boiler plate stuff that you just learn to accept. But today things were different. Today I decided to read it properly. And let's all admit it, LocalPluginContext and Plugins from the CRM Developer Toolkit need a serious refactor.

Don't get me wrong, I have used and loved this class for years. It wraps up all the objects I need very nicely and provides them in properties for me. But you know what else it does? It makes your plugins un-unit-testable, if I am allowed to create such a word. This is of course if you don't change / rewrite the class yourself.

There are a few mistake that are made by the developer toolkit:

Firstly, it embeds the LocalPluginContext class as a protected class and makes all the important properties internal. Worse again, it privatizes all the property setters on those most used interfaces. Now I cannot mock them with most mocking libraries!

Secondly, it creates a list of registered events to check if the plugin should run for the given message (the infamous tuple that dirties your constructors and has driven me daft for years). This is like a double plugin registration and really isn't required. Think about it, you register your plugin to fire in the Plugin Registration Tool, but if you forget to register it a second time in the constructor it will refuse to execute!

Finally, it needs a serious code review with resharper installed. There are a few redundant statements and lines that could be simplified or even removed.

I think it's time for new Plugin and LocalPluginContext classes and here is my suggestion:

Rework the base plugin class

I don't like the idea of inheriting from a Plugin class that implements IPlugin. It feels wrong to me that we have a class we can register as a plugin, but it's not really a plugin! So I want to rework this as a proper Base Plugin class. And this is my version of it:


    public abstract class BasePlugin : IPlugin
    {
        private readonly string _className;

        protected BasePlugin()
        {
            _className = GetType().Name;
        }

        public void Execute(IServiceProvider serviceProvider)
        {
            // Construct the Local plug-in context.
            var localContext = new LocalPluginContext(serviceProvider);

            localContext.Trace("Entered {0}.Execute()", _className);

            try
            {
                Execute(localContext);
            }
            catch (FaultException<OrganizationServiceFault> e)
            {
                // Trace the exception before bubbling so that we ensure everything we need hits the log
                localContext.Trace(e);

                // Bubble the exception
                throw;
            }
            finally
            {
                localContext.Trace("Exiting {0}.Execute()", _className);
            }
        }

        public abstract void Execute(ILocalPluginContext localContext);
    } 

So we still have the Execute method as before, but instead of an over complicated execution process (using the dreaded tuple) I just expect you to implement an Execute method accepting an ILocalPluginContext when inheriting this class.. So it will work almost exactly as before. Also, although our BasePlugin implements IPlugin it is abstract, so it's not going to appear in the list of available plugins when you load the DLL.

You also may have noticed I have introduced an ILocalPluginContext. This brings me on to my next suggestion.

Rework LocalPluginContext

Another area that always concerned me was how this internal class was written. I'm going to simplify it a little. Rather than pulling everything out of the necessary areas in a constructor let's just create lazy loaded properties. At least then we're only pulling out an IOrganizationService if it's used, a tracer if it's used, etc. I generally expect that all plugins will use a tracer, but quite often you won't need an IOrganizationService, e.g. validation plugins.

I also want to add an interface to this to allow easier mocking while unit testing.

So here's my idea for the LocalPluginContext class:

    public interface ILocalPluginContext
    {
        IOrganizationService OrganizationService { get; }
        IPluginExecutionContext PluginExecutionContext { get; }
        ITracingService TracingService { get; }
        void Trace(string message, params object[] o);
        void Trace(FaultException<OrganizationServiceFault> exception);
    }

    public class LocalPluginContext : ILocalPluginContext
    {
        private readonly IServiceProvider _serviceProvider;
        private IPluginExecutionContext _pluginExecutionContext;
        private ITracingService _tracingService;
        private IOrganizationServiceFactory _organizationServiceFactory;
        private IOrganizationService _organizationService;

        public IOrganizationService OrganizationService
        {
            get
            {
                return _organizationService ?? (_organizationService = OrganizationServiceFactory.CreateOrganizationService(PluginExecutionContext.UserId));
            }
        }

        public IPluginExecutionContext PluginExecutionContext
        {
            get
            {
                return _pluginExecutionContext ??
                       (_pluginExecutionContext = (IPluginExecutionContext)_serviceProvider.GetService(typeof(IPluginExecutionContext)));
            }
        }

        public ITracingService TracingService
        {
            get
            {
                return _tracingService ?? (_tracingService = (ITracingService)_serviceProvider.GetService(typeof(ITracingService)));
            }
        }

        private IOrganizationServiceFactory OrganizationServiceFactory
        {
            get { return _organizationServiceFactory ?? (_organizationServiceFactory = (IOrganizationServiceFactory)_serviceProvider.GetService(typeof(IOrganizationServiceFactory))); }
        }

        public LocalPluginContext(IServiceProvider serviceProvider)
        {
            if (serviceProvider == null)
            {
                throw new ArgumentNullException("serviceProvider");
            }

            _serviceProvider = serviceProvider;
        }

        public void Trace(string message, params object []o)
        {
            if (PluginExecutionContext == null)
            {
                SafeTrace(message, o);
            }
            else
            {
                SafeTrace(
                    "{0}, Correlation Id: {1}, Initiating User: {2}",
                    string.Format(message, o),
                    PluginExecutionContext.CorrelationId,
                    PluginExecutionContext.InitiatingUserId);
            }
        }

        public void Trace(FaultException<OrganizationServiceFault> exception)
        {
            // Trace the first message using the embedded Trace to get the Correlation Id and User Id out.
            Trace("Exception: {0}", exception.Message);

            // From here on use the tracing service trace
            SafeTrace(exception.StackTrace);

            if (exception.Detail != null)
            {
                SafeTrace("Error Code: {0}", exception.Detail.ErrorCode);
                SafeTrace("Detail Message: {0}", exception.Detail.Message);
                if (!string.IsNullOrEmpty(exception.Detail.TraceText))
                {
                    SafeTrace("Trace: ");
                    SafeTrace(exception.Detail.TraceText);
                }

                foreach (var item in exception.Detail.ErrorDetails)
                {
                    SafeTrace("Error Details: ");
                    SafeTrace(item.Key);
                    SafeTrace(item.Value.ToString());
                }

                if (exception.Detail.InnerFault != null)
                {
                    Trace(new FaultException<OrganizationServiceFault>(exception.Detail.InnerFault));
                }
            }
        }

        private void SafeTrace(string message, params object[] o)
        {
            if (string.IsNullOrWhiteSpace(message) || TracingService == null)
            {
                return;
            }
            TracingService.Trace(message, o);
        }
    }

One extra feature I have added is how it traces. The new LocalPluginContext contains a function that performs better tracing of FaultExceptions (ToString does not cut it) and I've matched the standard trace to how the tracing service works and included a param of objects.

What the new plugins look like!

A basic plugin looks like this which in my opinion is a lot cleaner:

    public class MyPlugin : BasePlugin
    {
        override public void Execute(ILocalPluginContext localContext)
        {
            // do what needs to be done!
        }
    }

One big upside to doing it this way is we have a much more testable framework where we simply pass in a mocked up ILocalPluginContext.

Friday, 3 October 2014

Web API - Adding Xml Doc Comments to help files

One of the beautiful features of Web API is you get an out of the box set of help pages documenting your API and what calls are available. You will see this "HelpPage" section under "Areas" of your project. One thing I wanted to do to extend this is configure the help pages to read my Xml documentation comments within the code. It's not awfully difficult to achieve, simply follow these steps:

  1. In the file HelpPage/App_Start/HelpPageConfig.cs uncomment the following line:
     config.SetDocumentationProvider(new XmlDocumentationProvider(HttpContext.Current.Server.MapPath("~/App_Data/XmlDocument.xml")));  
    

  2. Open up the project properties for you Web API project and under the build tab enable the Xml documentation file (set the file path to App_Data\XmlDocument.XML)

This gives you the basic setup to get the XML comments outputting to your Help Pages. But how about making it look sexier? For example, let's say I had the following XML comment I wanted to display:
 /// <summary>  
 /// Get a list of products within the given product kit.  
 /// </summary>  
 /// <param name="id">Id of the parent product kit</param>  
 /// <returns>This api method returns a list of products in JSON format. To get a list of products in XML format add the following HTTP header:<br />  
 /// <br />  
 /// Accept: application/xml<br />  
 /// </returns>  

This will strip out the HTML tags and output on a single line like this:



To allow breaks in our comments we need to make a couple of changes. Firstly, we need to change how our XmlDocumentationProvider retrieves the tag value from the XML comment. Find the GetTagValue function in this class and change the line as follows:
     private static string GetTagValue(XPathNavigator parentNode, string tagName)  
     {  
       if (parentNode != null)  
       {  
         XPathNavigator node = parentNode.SelectSingleNode(tagName);  
         if (node != null)  
         {  
           return node.InnerXml;
         }  
       }  
       return null;  
     }  

Next, we need to modify all the display templates that display these tags to show the raw value instead. The Display Template you will need to modify are HelpPages/Views/Help/DisplayTemplates/ApiGroup.cshtml and HelpPageApiModel.cshtml located at. Find any of the following lines:
 <p>@controllerDocumentation</p>  

 <p>api.Documentation</p>  

 <p>@description.ResponseDescription.Documentation</p>  

And replace them with the equivalent as per this code:
 <p>@Html.Raw(controllerDocumentation)</p>  

 <p>@Html.Raw(api.Documentation)</p>  

 <p>@Html.Raw(description.ResponseDescription.Documentation)</p>  

This could cover you for all areas you want to add breaks or html formatting to. For example, our description in the given example should look much better now:

Thursday, 25 September 2014

Unit testing and Autofac

Today I'm going to write a little about unit testing while using a dependency injector. The particular DI used as part of this post is Autofac, but this applies to most DI/IOC solutions out there.


Autofac

One of the nice things about Autofac is how the modules are strucutred. If we look at a basic setup of Autofac within WebAPI it might look something like this:

 var builder = new ContainerBuilder();  
 builder.RegisterModule(new WebApiIocModule());  

 IocProxy.Container = _builder.Build();  
 var resolver = new AutofacWebApiDependencyResolver(IocProxy.Container);  
 GlobalConfiguration.Configuration.DependencyResolver = resolver;  
 _builder = null;  

The part I'd like to highlight is the "RegisterModule" function. This allows me to register a class responsible in the project for building up the IOC container with the correct types. It might look as simple as this for the sake of this example:

 public class WebApiIocModule : Module  
 {  
   protected override void Load(ContainerBuilder builder)  
   {  
     if (builder == null)   
       throw new ArgumentNullException("builder");  
 
     builder.RegisterApiControllers(Assembly.GetExecutingAssembly());  
     //Cascade  
     builder.RegisterModule(new DomainIocModule());  
   }  
 }  

And our DomainIocModule might be a little more complex, like this:

 public class DomainIocModule : Module  
 {  
   protected override void Load(ContainerBuilder builder)  
   {  
     if (builder == null)  
       throw new ArgumentNullException("builder");  

     builder.RegisterAssemblyTypes(typeof(IProductService).Assembly)  
       .Where(t => t.Name.EndsWith("Service") || t.Name.EndsWith("Mapper"))  
       .AsImplementedInterfaces()  
       .PropertiesAutowired()  
       .InstancePerLifetimeScope();  

     //Cascade  
     builder.RegisterModule(new DataAccessIocModule());
   }  
 }  

The DomainIocModule will exist in our domain layer which means we have an easy way to maintain the separation between our data layers and web layers without a DI/IOC breaking it.


Resolving Types by configuration using Autofac

The most common uses for resolving by configuration is when using plugins, or when you need to allow for "hot swapping" of a particular type. To give you an example, let's say your Web API will be deployed to different servers, each having a different back end system. The data sources for your API will vary from SQL Server to CRM Systems. For this reason you have a generic "DataAccess" layer, but you need to plugin a different implementation depending on where the API is running. What you can do in Autofac is use an Xml configuration to register your modules, like this:

Web.Config

 <configuration>  
  <configSections>  
   <section name="autofac" type="Autofac.Configuration.SectionHandler, Autofac.Configuration" />  
  </configSections>  

  <snip...>  

  <autofac>  
   <modules>  
    <module type="DataAccess.Sql.Ioc.SqlIocModule, DataAccess.Sql" />  
    <module type="DataAccess.Crm.Ioc.CrmIocModule, DataAccess.Crm" />  
   </modules>  
  </autofac>  
 </configuration>  

Data Access Module

 public class DataAccessIocModule : Module  
 {  
   protected override void Load(ContainerBuilder builder)  
   {  
     if (builder == null)  
       throw new ArgumentNullException("builder");  
     builder.RegisterModule(new ConfigurationSettingsReader("autofac"));  
   }  
 }  

All you require for the above to work is the Autofac Configuration dll/package.


Resolving all of this in a Unit Test project

You'll start running into problems pretty quickly when testing if you don't do some form of configuration. This could be as complex as writing a specific testing module to override/mock up your classes, or it could be as simple as just enabling Autofac so that it creates all objects as per your code base. There are a few different ways to solve this problem, but what I usually do is create a base class for my tests so that they can all benefit from the Autofac setup. It will look something like this:

 public class TestBase  
 {  
   public TestBase()  
   {  
     IocProxy.Container = TestsIocBuilder.Build();  
   }  
 }  

My TestIocBuilder looks pretty much identical to my code in the very first snippet, apart from it registers a class called "TestsIocModule" instead. My TestIocModule looks something like this:
 public class TestsIocModule : Module  
 {  
   protected override void Load(ContainerBuilder builder)  
   {  
     if (builder == null) throw new ArgumentNullException("builder");  
     //Cascade  
     builder.RegisterModule(new DomainIocModule()); 
   }  
 }  

That's pretty much it really. The DomainIocModule will cascade down for you so everything will be resolved as per your standard Autofac configuration.


What about my Xml Configuration!

The last step we need to think about is the Xml configuration for plugins. This doesn't get read by the testing project so we need to add an application configuration to our test project instead. It will contain exactly the same configuration as the Web.Config so you can pretty much copy it from above.


Some of my tests fail... but only if I run ALL my unit tests!!!

This can happen when you have dodgy Autofac config in your testing libraries. What happens is the first library that comes along and registers all of your Autofac modeles and this may cause subsequent tests to fail. For example, let's say I had an integrations test project and a unit tests project. If by accident I mistyped my module in the configuration file like this:

 <configuration>  
  <configSections>  
   <section name="autofac" type="Autofac.Configuration.SectionHandler, Autofac.Configuration" />  
  </configSections>  

  <snip...>  

  <autofac>  
   <modules>  
    <module type="DataAccess.Sql.Ioc.SqlIocModule, DataAccess.Slq" />  
    <module type="DataAccess.Crm.Ioc.CrmIocModule, DataAccess.Crm" />  
   </modules>  
  </autofac>  
 </configuration>  

But in the other config you have correctly named the module. This can cause you major confusion! Mainly because you might be happily writing and running unit tests that are passing first time, but as soon as you run all tests they start to fail! Be mindful of this when setting up IOC/DI across multiple test projects.


But I want to mock out my classes!!!

Enabling Autofac doesn't change how you mock. It just saves you setting up objects like Autofac would. In fact, it gives you a very easy way to mock out your data layer without having to write mocks up. You could write a full in memory Sql data layer and attach it using some Xml Config module instead. If you need to mock out all your other classes you can still do so as the constructors didn't suddenly disappear! Simply set up the class you want to test and pass in the mocked types as you normally would do.

Wednesday, 25 June 2014

EntityState must be set to null, Created (for Create message) or Changed (for Update message)

If you use Linq to retrieve entities from CRM quite a lot you may have come across this problem. In short what the above error means is you got the entity from an OrganizationServiceContext (i.e. Linq), but tried to update it using an OrganizationServiceProxy instead. There are a few ways you can make this mistake, for example take the following code snippet:

OrganizationServiceProxy _serviceProxy;
... <snipped connection code> ...
using (var svcContext = new OrganizationServiceContext(_serviceProxy))
{
    var query = from a in svcContext.AccountSet
                where a.Name.Contains("Contoso")
                select a;
    var account = query.First();
    account.SomeField = "SomeValue";
    _serviceProxy.Update(account);
}

As you can see, we use the service context when retrieving, but updating using the service proxy. CRM doesn't like this and will throw the above error. There are a few ways to get around this so let me explore each.

1. Update via the context.

using (var svcContext = new OrganizationServiceContext(_serviceProxy))
{
    var query = from a in svcContext.AccountSet
                where a.Name.Contains("Contoso")
                select a;
    var account = query.First();
    account.SomeField = "SomeValue";
    svcContext.UpdateObject(account);
    svcContext.SaveChanges();
}

Simple and straight forward. If you are reading via the context, just update via the context as well. This is the correct way to do this if you are staying within the scope of the service context.

2. Update the EntityState manually


There is a little nuance with how you must do this, but it might prove useful if you are outside the scope of the service context:
Account account;
using (var svcContext = new OrganizationServiceContext(_serviceProxy))
{
    var query = from a in svcContext.AccountSet
                where a.Name.Contains("Contoso")
                select a;
    account = query.First();
}
... <snipped other stuff that might happen> ... account.SomeField = "SomeValue"; account.EntityState = null; // Or you can set it to EntityState.Changed _serviceProxy.Update(account);

You'll notice that we have set the "EntityState" to null (or changed) outside of the scope of the ServiceContext using. If you try to do this within the using scope, like this:

Account account;
using (var svcContext = new OrganizationServiceContext(_serviceProxy))
{
    var query = from a in svcContext.AccountSet
                where a.Name.Contains("Contoso")
                select a;
    var account = query.First();
... <snipped other d stuff that might happen> ... account.SomeField = "SomeValue"; account.EntityState = null; // Or you can set it to EntityState.Changed _serviceProxy.Update(account);
}

you'll get the following error:

The entity is read-only and the 'EntityState' property cannot be modified. Use the context to update the entity instead.

The reason you get this problem is the ServiceContext adds the entity to a change tracking list which flags the entity as read only. Once you move outside the context of the using statement the Service Context is disposed and as a result is clears all tracked changes and the entity tracking list. This operation reverts the read only flag on the entity thus allowing you to set the EntityState flag. The issue it leaves behind is the EntityState flag is still set to "Unchanged" which makes it "un-update-able" - a side effect I consider a bug within CRM. (Note: I have not tested this in CRM 2013 yet to see if it was fixed, but it existed as a bug in 2011).

3. Create a new object


This is my preferred method when updating any entities. Why send the entire object back at CRM when you can send just the attributes you want to update? This method looks like the following:
Account account;
using (var svcContext = new OrganizationServiceContext(_serviceProxy))
{
    var query = from a in svcContext.AccountSet
                where a.Name.Contains("Contoso")
                select a;
    account = query.First();
}
... <snipped other stuff that might happen> ...
var accountForUpdate =  new Account {
        Id = account.Id,
        SomeField = "SomeValue"
    };
_serviceProxy.Update(accountForUpdate);

This is far more efficient and has the added benefit of bypassing plugins that might fire off certain field updates. This is because when you send the entire entity back any plugins that have an attribute filter on them will still fire. Just setting the required attributes will only post those to CRM and also make the update message smaller.

Happy CRM'ing!

Monday, 6 January 2014

Repository patterns - Fighting unclean code

Every company you work for is different. Different developers, different styles, different techniques. But one thing I find that is consistent across almost all places I have worked is a constant battle to maintain clean code and some places are worse than others. I have particularly come across this recently and no matter how many times I offer up tips the unclean code seems to creep in. Due to the fact I'm working with some really decent guys that just have a lot of bad habits I am struggling to find the right side of that fine line between sounding like a grumpy picky developer and just putting up with it!

I'm not just talking about different styles here, I'll give you an example. For our plugin development we use the repository pattern so the following type of code is used quite often:

public class AccountRepository
{
    IOrganizationService _service;

    public AccountRepository(IOrganizationService service)
    {
        _service = service;
    }

    /*...*/
}



This is somewhat fine, although the 1 to 1 mapping between entities and repository classes isn't ideal, but it works well for us so far. Unfortunately some of the developers don't fully understand what the repository pattern really is and what it should and shouldn't know of/know how to do. Recently I spotted that the following code was checked in some time back:

public class AccountRepository
{
    IOrganizationService _service;
    Entity Account { get; set; }

    public AccountRepository(IOrganizationService service)
    {
        _service = service;
        Account = null;
    }

    public AccountRepository(IOrganizationService service, Entity Account)
    {
        _service = service;
        this.Account = Account;
    }
    /*...*/
}

The problem this introduces is that my repository is now aware of an account record/entity. This immediately points the finger at the code breaking SRP (Single Responsibility Principle).

Let's refresh our memory on what a repository pattern is. Put simply, the repository should act as a data layer or mediator between our business logic, domain objects and our data source. It should know how to take a query and provide a resulting set of domain objects. It might also be able to add and remove objects, or even update objects. To perform these operations it should be aware of how to connect to the data source, how to map the result of a query to domain objects and it should make that list of objects available to the caller. In some instances you may move the "data mapping" logic out into its own classes, but in Dynamics CRM the Organization Service already provides you with Entity objects (or strongly typed objects if you prefer) so you may not need a separate data mapping abstraction.

Now, going back to the code above, the extra introduction of an existing record in the constructor causes us to add a capability to the repository. It is now record aware before you even run a query. This breaks SRP and most likely some other principles too. You'll find OCP, Open Closed Principle, hanging on a very fine thread as a result too! I am guessing that further down in the code (here where demons lie...) you will most likely see references to this object and decisions being made based on the object and its attributes. Oh you naughty naughty developers! What were you thinking!

So, what do I do? Refactor a load of methods I didn't develop and spend half a day fixing other peoples code? Send out a tip (yet another one!) saying why this is bad and look like Mr Grumpy (yet again!)? Or do I ignore it and hope it goes away in time...

Decisions decisions!

Monday, 30 September 2013

CRM - Lookup field name vs relationship field name

Today I thought I'd discuss a little gotcha in CRM 2011 that I'm amazed remained as a "feature" for so long! Have you ever seen "foreign key" fields in a CRM system and noticed that some are called "new_ContactId" where as others are called "new_Contact"? There's a fairly simple reason this occurs, it's basically a result of the developer/consultants preference when creating fields. Let's take a quick look at each method.

Old School - Creating relationships

If, like me, you have progressed from CRM 4 or earlier you generally create relationships between entities using the same method as always existed. You open up the entity customizations, navigate to 1:N or N:1 relationships and you get this screen:


You will notice I have highlighted what it will create the field as in the background: "new_ContactId". Most people comfortable with CRM throughout many versions will be familiar with this convention of creating fields.

New School - Lookups

Since CRM 2011 we now have a second way to create relationships. Rather than going via the relationship links on an entity simply pick the entity where you want the "foreign key" field to reside. In our example this is on the entity called "Custom Method". Navigate to the list of fields and simply add a new lookup field. Like this:


This time you'll notice I have highlighted the field name "new_Contact". We're missing the "Id" part!

Old School or New School?

I guess this is the question, which do you prefer? Personally, I prefer the new field naming convention for one simple reason. As CRM developers you don't really have your head in tables and SQL as much any more. Instead you're using plugins and Entities / Entity References. So generally in a plugin you'll use code like this:

var contactReference = (EntityReference)customMethod["new_ContactId"];

This "Id" always bugged me, because of this simple fact - it's not an Id in code, it's a reference. So if you just wanted the "Id" in code you use this:

var contactId = ((EntityReference)customMethod["new_ContactId"]).Id;

Which I don't like, because you're repeating the abbreviation "Id". In my opinion this kind of violates "DRY" so I will avoid it if at all possible. Using the new school way it looks cleaner without this repetition:

var contactId = ((EntityReference)customMethod["new_Contact"]).Id;

So whenever I create a field using the relationship convention I will manually remove the "Id" part.

Friday, 6 September 2013

Executing stand alone C# code within CRM

A question that I was asked recently, and have pondered for quite some time, is providing an ability to execute some independent/standalone C# code within a CRM instance. There are quite a few ways to do this, but I'll describe my preferred pattern here. The solution I put forward is an attempt to provide complex code solutions that do not require the use of another server or external application. And it should also work both Online and OnPremise.

Firstly, to support our "C#" calls we need a new entity that will trigger it. Let's call it "Custom Method". In it's most basic form it would be a very simple object with just 3 main attributes, the method name, parameters and result:



Next, what we would do is attach a C# plugin on the PreCreate of this that will allow us to run some C# code in the background. Using the developer tools a very basic plugin skeleton that has 1 custom method (contained within the plugin class for simplicity) will look something like this:

public class ExecuteCustomMethod: Plugin
{
    private readonly IDictionary<string, Func<string,string>> methods = 
     new Dictionary<string, Func<string,string>>();

    public ExecuteCustomMethod()
        : base(typeof(ExecuteCustomMethod))
    {
        methods.Add("Add2Numbers", Add2Numbers);
        RegisteredEvents.Add(new Tuple<int, string, string, 
            Action<LocalPluginContext>>
                (20, 
                 "Create", 
                 "xrm_custommethod", 
                 Execute));
    }

    protected void Execute(LocalPluginContext localContext)
    {
        if (localContext == null)
        {
            throw new ArgumentNullException("localContext");
        }
        
        var target = (Entity)localContext
                         .PluginExecutionContext
                         .InputParameters["Target"];

        try
        {
            var name = (string) target["xrm_name"];
            var parameters = (target.Contains("xrm_parameters") ? 
                              (string) target["xrm_parameters"] : 
                              string.Empty);
            target["xrm_result"] = methods[name].Invoke(parameters);
        }
        catch (Exception ex)
        {
            target["xrm_result"] = ex.ToString();
        }
    }

    private string Add2Numbers(string numbers)
    {
        var numberArray = numbers.Split('|');

        return (decimal.Parse(numberArray[0]) + 
                decimal.Parse(numberArray[1])).ToString();
    }
}

You could easily extend the parameters and/or result to use XML and serialize/de-serialize the results to provide a more robust/complete solution. For simplicity purposes I presume the parameter will be provided in a pipe delimited format. Now, you simply execute and retrieve the results as follows:



Effectively this may offer an easy solution to execute some complex stand alone C# for whatever reason required.



Tuesday, 12 March 2013

Silverlight - Passing javascript objects into a web page dialog arguments

I asked this question on Stack Overflow a while back and was a little surprised that it earned me the tumbleweed award. Has nobody seriously ever done this? Here's the original question: silverlight-passing-an-array-to-a-web-pages-dialog-arguments. It's a scenario quite a few Microsoft Dynamics CRM developers might come across if trying to invoke MSCRM web dialogs from silverlight. But for the purpose of this post I'll keep it somewhat generic.

Take this scenario, you have a web page dialog that requires some dialog arguments to work correctly. And it performs the following when loaded up:

var args = getDialogArguments();
if (args == null) return;
if (args.items == null) return;
var items = args.items;

var len = items.length;
for (var i = 0; i < len; i++)
{
  var item = items[i];
  cur.id = item.getAttribute("oid");
  cur.type = item.getAttribute("otype");
  cur.values = item.values;
  ... etc
}

We want to invoke this page via Silverlight, but the question is how do we pass in the arguments correctly? If we take a closer look at the dialog arguments (args) we can see that it has a member called "items" which is an array. Each of these items have attributes called "oid" and "otype". So let me explain what you need to do to set this up.

Before I continue, I want to add a "rule" before I explain how to achieve this. We need to do this without using "dynamic" because this causes you to have to reference the Microsoft.CSharp library which in turn causes your XAP file to bloat.

To start let's ask a slightly different question, what do these objects materialise themselves as within Silverlight? This I already knew the answer to, they are of type ScriptObject which is located in System.Windows.Browser. So why can't we just go and create one of these? Here is where I hit my first roadblock, it has an internal constructor. But, a quick search across the internet reveals that we can set this up using the following:

var dialogArgs = HtmlPage.Window.CreateInstance("Object");

And how about a property on this field?

dialogArgs.SetProperty("items", items);

Excellent, so now we're getting somewhere. Next up, how do you set up this array called "items"? Same way, but we can add indexers to it. Some code for setting up an array and an item will look something like this (I have just created a new GUID for the purpose of this example):

var item = HtmlPage.Window.CreateInstance("Object");
item.SetProperty("oid", Guid.NewGuid());
item.SetProperty("otype", "account");
var items = HtmlPage.Window.CreateInstance("Object");
items.SetProperty(0, item);

And finally, just pass that object straight into your dialog window like this:

var so = (ScriptObject)HtmlPage.Window.Invoke("showModalDialog", lookUpWindow, dialogArgs, "dialogWidth:600px;dialogHeight:600px;");

Job done.

Tuesday, 19 February 2013

CRM 2011 - How do you close an opportunity in Silverlight?

This is an interesting one I came across today. Due to using a different set of objects within the Silverlight framework you'll notice you're missing that valuable "WinOpportunityRequest" object! So, it begs the question, how do you close an opportunity in Silverlight?

Thankfully, we can go back to basics and raise an ordinary OrganizationRequest to achieve this. But before we do this let's take a look at what the code might look like in a plugin or standard CRM service request:


var opportunityClose = new Entity("opportunityclose");
opportunityClose.Attributes.Add("opportunityid",
    new EntityReference("opportunity", opportunityId));
opportunityClose.Attributes.Add("subject", "Opportunity expired");

var winOpportunity = new WinOpportunityRequest
    {
        OpportunityClose = opportunityClose,
        Status = new OptionSetValue(20001) // or whatever is valid for you
    };
service.Execute(winOpportunity);

You might be tempted to try achieve this using a SetState request, but you'll quickly find CRM complaining and telling you you're not allowed to do it. Instead, an easier way to do this is just replicate the above call as if you were using late binding. Like this:


var organizationRequest = new OrganizationRequest
   { RequestName = (won ? "WinOpportunity" : "LoseOpportunity") };
var opportunityClose = new Entity {LogicalName = "opportunityclose"};
SetAttribute(opportunityClose, "opportunityid", opportunityEntity.Id);
SetAttribute(opportunityClose, "subject", "Opportunity Expired");
organizationRequest["OpportunityClose"] = opportunityClose;
organizationRequest["Status"] = new OptionSetValue {Value = 20001}; // or whatever is valid for you



For reference, my SetAttribute function just looks like this:

private void SetAttribute(Entity entity, string attribute, object value)
{
  if (entity != null)
  {
    if (entity.Attributes == null) entity.Attributes = new AttributeCollection();
    if (entity.Attributes.ContainsKey(attribute))
    {
       entity.Attributes.SetItem(attribute, value);
    }
    else
    {
      entity.Attributes.Add(
          new Common.XrmSoapService.KeyValuePair<string, object> 
             { Key = attribute, Value = value });
    }
  }
}

Wednesday, 18 July 2012

Importing Marketing List Members... the fast way

A common bane in many developers lives is the inability to import marketing list members directly using an asynchronous import. Even in Microsoft Dynamics CRM 2011 this is still an issue. If you're reading this you've probably come across this too. The most common solution I've seen (and read) for this is importing one by one. A piece of code I came across recently did exactly this and (a stripped down version...) looked something like the following:


while (true)
{
    var record = MarketingListCSVFile.GetNextCSVRecord();

    if (record == null)
    {
        // eof - no more records to process
        break;
    }

    Entity contact = GetContact(record[ContactField]);
    
 AddListMembersListRequest request = new AddListMembersListRequest { ListId = MarketingList.Id, MemberIds = new Guid[] { contact.Id } };
    AddListMembersListResponse response = (AddListMembersListResponse)this.Service.Execute(request);
    // ...
}

AttachMarketingListToCampaign(MarketingList, MarketingListCSVFile.CampaignActivityCode);


... and so on. In practice this works, but there's a lot wrong with it. Aside from the fact that it's going to be slow due to hitting the server every time for each member, it's going to hammer that server until it get's all those members imported.

A better, and quite possibly the fastest way to get this imported is by using a custom "holding" entity, drive a Dynamic Marketing List off this entity and then convert this Dynamic Marketing List to a static marketing list (if required).


The Holding Entity 

Firstly, we need to create an entity that has all the required fields on it that will allow us to drive a query off. The only requirement here is that it has a relationship to the Cotact entity, because a query for a Dynamic Marketing list must return a list of contacts. In my case I needed to link these contacts back to a campaign activity, which involved importing a code to give me the ability to do this. All in all my new custom entity contains the following:

  • Contact (Lookup to contact) 
  • Campaign Activity Code (string) 


The query

Next we need to make sure we can drive the correct query off this. Pop open an advanced find and select the contact entity type from the list. My query looked like this:


This will bring back all the contacts required for my marketing list. All good so far.


The import

Next up, how do we kick off an asynchronous import via the code? If you haven't done this before it's worthwhile having a read of the following links first. Once you've digested all, or at least the applicable parts of this you're ready to write your import:

Sample: Import Data Using Complex Data Map
Data Import Entities


You'll notice another problem I ran into when researching this, and it's how everyone has gone dog crazy on early bound objects. I'm not saying they're bad, but I'm not as big a fan as most. Or maybe I'm just a freak for late bound objects. Mainly because it saves me the pain of:
  1. Making sure everyone has the latest and greatest definitions in their project
  2. Waiting for some other bloke to create his entity before I can write my "something or other" that  relies on just 1 field in that entity... 

Several ways around the above, but as you may have gathered by now, my favourite is use late binding ;)


So, we can break an import and what needs to happen down to about 7 basic steps (8 if you want to wait for the import to complete):

  1. Create an import map
  2. Create all your column mappings linked to your import map
  3. Create the import
  4. Create the import file linked to the import and import map
  5. Kick off the parse step
  6. Kick off the transform step
  7. Kick off the physical import.


An interesting point to note is that you don't have to wait for the parse to complete before kicking off the transform and import. When MS CRM receives these requests will just queue them up until the others have completed.

When you're done you'll end up with code that looks something like this (You'll notice that this doesn't look exactly like the sites I linked above due to late binding):


var importMap = new Entity("importmap");
importMap.Attributes["name"] = "Import Map Name";
importMap.Attributes["source"] = CsvFileName;
importMap.Attributes["description"] = "Import Description...";
importMap.Attributes["entitiesperfile"] = new OptionSetValue(1); // 1 = Single Entity Per File
Guid importMapId = service.Create(importMap);

// Create a column mapping for the contact lookup field.
var contactColumnMapping = new Entity("columnmapping");
contactColumnMapping.Attributes["sourceattributename"] = "Contact";
contactColumnMapping.Attributes["sourceentityname"] = "Contact_1";
contactColumnMapping.Attributes["targetattributename"] = "new_contact";
contactColumnMapping.Attributes["targetentityname"] = "new_marketinglistcontact";
contactColumnMapping.Attributes["importmapid"] = new EntityReference("importmap", importMapId);
contactColumnMapping.Attributes["processcode"] = new OptionSetValue(1); // 1 = Process
Guid contactColumnMappingId = service.Create(contactColumnMapping);

// If you have special codes you may need a lookup mapping for the contact
var contactLookupMapping = new Entity("lookupmapping");
contactLookupMapping.Attributes["columnmappingid"] = new EntityReference("columnmapping", urnColumnMappingId);
contactLookupMapping.Attributes["processcode"] = new OptionSetValue(1); // 1 = Process
contactLookupMapping.Attributes["lookupentityname"] = "contact";
contactLookupMapping.Attributes["lookupattributename"] = "new_code";
contactLookupMapping.Attributes["lookupsourcecode"] = new OptionSetValue(1); // 1 = Source
Guid contactLookupMappingId = service.Create(contactLookupMapping);

// Create a column mapping for the campaign activity code field.
var campaignActivityColumnMapping = new Entity("columnmapping");
campaignActivityColumnMapping.Attributes["sourceattributename"] = "Campaign Activity Code";
campaignActivityColumnMapping.Attributes["sourceentityname"] = "Contact_1";
campaignActivityColumnMapping.Attributes["targetattributename"] = "new_campaignactivityid";
campaignActivityColumnMapping.Attributes["targetentityname"] = "new_marketinglistcontact";
campaignActivityColumnMapping.Attributes["importmapid"] = new EntityReference("importmap", importMapId);
campaignActivityColumnMapping.Attributes["processcode"] = new OptionSetValue(1); // 1 = Process
Guid campaignActivityColumnMappingId = service.Create(campaignActivityColumnMapping);

// Create a column mapping for the name field.
var nameColumnMapping = new Entity("columnmapping");
nameColumnMapping.Attributes["sourceattributename"] = "Name";
nameColumnMapping.Attributes["sourceentityname"] = "Contact_1";
nameColumnMapping.Attributes["targetattributename"] = "new_name";
nameColumnMapping.Attributes["targetentityname"] = "new_marketinglistcontact";
nameColumnMapping.Attributes["importmapid"] = new EntityReference("importmap", importMapId);
nameColumnMapping.Attributes["processcode"] = new OptionSetValue(1); // 1 = Process
Guid nameColumnMappingId = service.Create(nameColumnMapping);

// Create Import
var import = new Entity("import");
import.Attributes["modecode"] = new OptionSetValue(0);
import.Attributes["name"] = "Importing data";
Guid importId = service.Create(import);

// Create the actual file...
var file = new Entity("importfile");
file.Attributes["content"] = File.ReadAllText(CsvFileName);
file.Attributes["name"] = CsvFileName;
file.Attributes["isfirstrowheader"] = true;
file.Attributes["source"] = CsvFileLocation;
file.Attributes["sourceentityname"] = "Contact_1";
file.Attributes["importmapid"] = new EntityReference("importmap", importMapId);
file.Attributes["importid"] = new EntityReference("import", importId);
file.Attributes["targetentityname"] = "new_marketinglistcontact";
file.Attributes["size"] = ((string)file.Attributes["content"]).Length.ToString();
file.Attributes["fielddelimitercode"] = new OptionSetValue(2); // 2 = Comma
file.Attributes["datadelimitercode"] = new OptionSetValue(1); // 1 = Double Quote
file.Attributes["processcode"] = new OptionSetValue(1); // 1 = Process
file.Attributes["usesystemmap"] = true;
Guid fileId = service.Create(file);

var parseRequest = new ParseImportRequest { ImportId = importId };
service.Execute(parseRequest);

var transRequest = new TransformImportRequest { ImportId = importId };
service.Execute(transRequest);

// Assign the request the id of the import we want to begin
var request = new ImportRecordsImportRequest { ImportId = importId };
var response = (ImportRecordsImportResponse)service.Execute(request);


You most likely won't end up with all that code in 1 place like this, or at least I hope not! But that's the general gist of what needs to happen.


Dynamic Marketing Lists

So what the above gives you is a very quick way to get the data imported into MS Dynamics CRM. But how do we use this? Let's grab that fetch xml from the earlier query and inject our campaign code into that:


string fetchXml = string.Format(
    "<fetch version='1.0' output-format='xml-platform' mapping='logical' distinct='true'>" +
    "  <entity name='contact'>" +
    "    <attribute name='fullname' />" +
    "    <attribute name='contactid' />" +
    "    <order attribute='fullname' descending='false' />" +
    "    <link-entity name='new_marketinglistcontact' from='new_contact' to='contactid' alias='aa'>" +
    "      <filter type='and'>" +
    "        <condition attribute='new_campaignactivitycode' operator='eq' value='{0}' />" +
    "      </filter>" +
    "    </link-entity>" +
    "  </entity>" +
    "</fetch>",
    campaignActivityCode);

Set up a new Marketing List, pop that into the "query" field of a Marketing list, and set the "type" to dynamic (1) and off we go.


Converting from a Dynamic to a Static Marketing List

Final step is to convert this to a Static Marketing List. This is easily achievable using the "CopyDynamicListToStaticRequest":

var copyDynamicListToStaticRequest = new CopyDynamicListToStaticRequest { ListId = ml.Id };
var response = (CopyDynamicListToStaticResponse)ml.Service.Execute(copyDynamicListToStaticRequest);
staticMarketingList = new MarketingList(ml.Service, response.StaticListId) 
    { Name = MarketingListCSVFile.MarketingListName, Locked = false };


Job Done.

I'd like to take this opportunity to thank that Hetfield dude for being frikken awesome and passing on the awesome. More specifically, his idea of using a dynamic marketing list.

Word.