So in a previous blog posting, I discussed creating joins using the SourceCode.SmartObjectClient API, but I also said that you could use the ADO.NET data provider as a simpler alternative. This weekend however, I found a pretty serious bug with the ADO.NET provider around joins which is large enough that I am now recommending that you avoid using the ADO.NET provider until it gets fixed.

Example of problem code:

SOConnection conn = OpenSmartClientConnection("Activity_Instance, 
    Process_Instance, Activity_Instance_Destination");  
 
SOCommand cmd = new SOCommand(); 
cmd.Connection = conn; 
cmd.CommandText = @" 
SELECT * FROM Process_Instance.List PI 
LEFT JOIN Activity_Instance_Destination.List AID ON 
    PI.ProcessInstanceID = AID.ProcessInstanceID 
LEFT JOIN Activity_Instance.List AI ON 
    AID.ProcessInstanceID = AI.ProcessInstanceID 
    AND AID.ActivityInstanceID = AI.ActivityInstanceID";  
 
using (SODataReader dr = cmd.ExecuteReader()) 
{ 
    DataSet ds = new DataSet("test"); 
    ds.Load(dr, LoadOption.OverwriteChanges, "Process_Instance"); 
}

The data in these tables are: 9 process instances, 3 activity_instance_destinations (each correspond to one of the 9 process instances), and 20 activity_instances (3 of which have destinations). So, I’d expect that the above query should return 9 rows, but instead it returns 13.

The problem is that the data provider ignores the second part of the ON (if I reverse the order it returns 11 rows). It doesn’t matter if I wrap it in () either.

If I craft the query using SmartObjectClientServer and joins then it works correctly, so it isn’t a problem with the SmartObjects themselves. Something like:

JoinDetails jd = new JoinDetails();  
 
jd.From = processInstances; 
Join join = new Join();  
 
join.AddCondition( 
    processInstances.Properties["ProcessInstanceID"], 
    activityDestinations.Properties["ProcessInstanceID"]);  
 
join.From = processInstances; 
join.To = activityDestinations; 
join.Scope = JoinScope.left; 
join.Type = JoinType.outer;  
 
Join join2 = new Join(); 
join2.From = activityDestinations; 
join2.To = activityInstances;  
 
join2.AddCondition( 
    activityDestinations.Properties["ActivityInstanceID"], 
    activityInstances.Properties["ActivityInstanceID"]);  
 
join2.AddCondition( 
    activityDestinations.Properties["ProcessInstanceID"], 
    activityInstances.Properties["ProcessInstanceID"]);  
 
join2.Scope = JoinScope.left; 
join2.Type = JoinType.outer;  
 
jd.Joins.Add(join); 
jd.Joins.Add(join2);  
 
SmartObjectList destinationsList = soServer.ExecuteList(jd,null);

works and properly returns 9 rows.

For those of you who love using the ADO.NET provider, there is a workaround (suggested by Jason Apergis), which is to move the conditions from the ON to the WHERE clause, like so:

SELECT * FROM Process_Instance.List PI 
LEFT JOIN Activity_Instance_Destination.List AID ON 
    PI.ProcessInstanceID = AID.ProcessInstanceID 
LEFT JOIN Activity_Instance.List AI ON 
    AID.ProcessInstanceID = AI.ProcessInstanceID 
    AND AID.ActivityInstanceID = AI.ActivityInstanceID 
WHERE ( 
    AID.ProcessInstanceID IS NULL 
    OR ( 
        AID.ProcessInstanceID = AI.ProcessInstanceID 
        AND AID.ActivityInstanceID = AI.ActivityInstanceID 
    ) 
)

But you shouldn’t have to write your queries to support a poor SQL language implementation on K2’s part though.

I have opened a trouble ticket with K2 and will update this blog entry with a resolution.

The documentation is a little light on how to do joins with SmartObjects when using the SourceCode.SmartObjects.Client API, especially when an association isn’t defined between them.

So here’s a quick post with an example. This example joins two standard reporting SmartObjects.

            //OpenSmartObjectConnection is a helper method I wrote
            SmartObjectClientServer soServer = OpenSmartObjectConnection();
 
            //Get first smart object which is going to be part of join
            SmartObject activityDestinations = 
                soServer.GetSmartObject("Activity_Instance_Destination");
            //Specify the SO method being called
            SmartListMethod getList = activityDestinations.ListMethods["List"];
            activityDestinations.MethodToExecute = getList.Name;
 
            //Get second smart object which is going to be part of join
            SmartObject processInstances = 
                soServer.GetSmartObject("Process_Instance");
            //You have to specify the method on this one too. You can also 
            //specify the method like this:
            processInstances.MethodToExecute = "List";
 
            //This would work to filter if we weren't doing a join, but 
            //seems to be ignored when a join is occuring. I just keep it
            //here so you can see how you'd filter if we weren't joining
            getList.Filter = destinationFilter;
 
            //We need to specify ResultNames for all smartobjects in our join.
            //This is so we can distinguish property names of the same name.
            //Think of this like a "SELECT * FROM Table1 t1 WHERE t.ID = ..."
            activityDestinations.ResultName = "AD";
            processInstances.ResultName = "PI";
 
            //Create an Equals filter which determines how our SmartObjects 
            //can be joined. If the filter is more complex then you'll have to
            //build a more complex filter using And and Or type filters.
            //Think of this as a SQL statement similar to:
            //SELECT * FROM Table1 t1
            //LEFT JOIN Table2 t2 ON t1.Id = t2.t1Id
            Equals joinFilter = new Equals(
                new PropertyExpression("AD.ProcessInstanceID", PropertyType.DateTime),
                new PropertyExpression("PI.ProcessInstanceID", PropertyType.DateTime)
                );
            //If you have other filters, such as only wanting a folio with a particular
            //name, add them using the other filter types
            And and = new And(
                destinationFilter, joinFilter);
 
            //Create our JoinDetails
            JoinDetails jd = new JoinDetails(); 
            jd.JoinFilter = and;
            jd.From = activityDestinations;
            Join join = new Join();
            join.From = activityDestinations;
            join.To = processInstances;
            join.Scope = JoinScope.left;
            join.Type = JoinType.outer;
            jd.Joins.Add(join);
 
            //When you want to run a join, you call ExecuteList passing in the
            //join description and a string called executionData. ExecutionData is
            //not used in the retrieval process, but is simply set on the resulting
            //SmartObjectList in the property ExecutionData. Not quite sure what
            //the purpose is.
            SmartObjectList destinationsList = soServer.ExecuteList(jd, 
                "I can put anything I want here");
 
            foreach (SmartObject activityDestination in destinationsList.SmartObjectsList)
            {
                //Just to demonstrate it is passed through
                string execData = destinationsList.ExecutionData;
                //Access a property on our left object
                string folio = activityDestination.Properties["Folio"].Value;
                //Access a property on our joined object
                string originator = activityDestination.JoinedProperties["Originator"].Value;
            }

This is a lot of code to write. Alternately, I could have used the SmartObject ADO.NET provider and written something more similar to a SQL statement which would have been a lot faster to write. I am not yet sure when/if one option is more appropriate than the other. If I have time, I will try to do a speed comparison and see if there is any advantage to one approach over the other.

We have a client who is currently in the process of moving from paper-based forms to an electronic process utilizing workflow and they were confused about what version of Office they needed and whether they needed an enterprise CAL or just a standard CAL for MOSS. They currently have Office 2003 installed throughout the enterprise.

Here is a quick overview of your primary options around upgrading Office/InfoPath, and Enterprise vs. Standard CALs as they relate to forms/workflow. It’s not an all encompassing list and doesn’t cover every permutation possible, but it does cover the most common scenarios: 

1. Stick with Office 2003 & InfoPath 2003, use standard MOSS CAL. Deploy InfoPath client to all form authors/consumers. Sacrifice some Office integration with MOSS 2007, such as being able to kick off workflows from other Office apps.
2. Upgrade to Office 2007 & InfoPath 2007, use standard MOSS CAL. Deploy InfoPath client to all form authors/consumers. Gain increased Office integration, some improved InfoPath capabilities.
3. Purchase enterprise MOSS CALs, deploy InfoPath 2007 to all form authors, allow form consumers to use web enabled forms via forms services (may not be sufficient for all their needs depending on form complexity). The rest of Office 2007 is optional depending on their desire to get benefits of increased Office integration.

 Workflows using SharePoint/Windows Workflow Foundation

If you are planning on just using SharePoint/Windows Workflow workflows, then workflow authors will likely also require Microsoft Office SharePoint Designer 2007 (and potentially Visual Studio). SharePoint Designer does not come as part of any Office suite, so the decision to deploy this to authors can be made independently of the decision to move to Office 2007 enterprise-wide.

 K2 [blackpearl] Workflows

This particular client also asked how the decision to go with K2 [blackpearl] instead of straight MOSS workflow would impact their decision. 

The choice to go with K2 has no real bearing on the Office/CAL decision as it will work with client based forms in 2003/2007, web enabled InfoPath forms (requires enterprise CALs for MOSS) and/or custom UI like ASP.NET, WebParts, etc. There is one caveat and that is that K2 [blackpearl] supports Visio 2007 as a workflow authoring tool. So if you wish to take advantage of Visio 2007 authoring, then Visio 2007 is obviously required for workflow authors.

Visio 2007 is sold as a stand-alone application, can run side-by-side with Office 2003, and doesn’t come as part of any of the Office suites anyway so this decision is really independent of whether Office 2007 is deployed enterprise-wide. Additionally, there are other ways to author workflow and, especially with the upcoming release of K2 Studio, Visio authoring is not really a must-have feature.

We received a question from a potential customer the other day around K2’s support for Business Process Modeling Language (BPML) and thought I’d write a quick post with the answer.

A Quick History

 Business Process Modeling Language was a proposed language standard by the BPMI for describing business processes. It went through several revisions and ultimately made it to a final draft form for v1.0, but never made it out of draft status. The BPMI was acquired by the Object Management Group (OMG) and BPML was subsequently abandoned for a different standard called Business Process Execution Language (WS-BPEL or BPEL4WS or just BPEL). Unlike BPML however, BPEL isn’t a complete standard and has some large functionality gaps which make it not capable of describing most workflow. Confused yet?

Business Process Modeling Notation (BPMN) is a graphical notation system which was originally designed for graphical representation of BPML. Even though BPML was abandoned, BPMN continues to live on as a partial means of expressing BPEL.

 K2 and Business Process Standards

K2’s official response to how it works with standards like BPEL can be found on this page. In short, their answer basically can be expressed as:

K2 supports BPMN (the notational standard) through support of Visio which has BPMN stencils and we have some underlying building blocks which might make it possible to export/import to BPEL with work.

The underlying building blocks are really Windows Workflow which has some support for importing/exporting BPEL and their XML file structure which conceivably makes it possible to use XSLT to transform K2 processes to BPEL.

 So to sum up:

• BPML is no longer in use and isn’t supported by (m)any workflow products anymore including K2.
• BPMN is supported through authoring workflow in Visio.
• BPEL is not officially supported, but with a significant amount of work you could probably get a BPEL process working within K2 or vice versa.

To illustrate that last point, there is a project on blackmarket which is supposed to be able to create a K2 workflow from a BPEL export from ARIS. I have no direct experience with the tool so can’t vouch for its effectiveness.

Warning: This post is targeted towards .NET developers. If you never dig in and write custom code when creating your workflows, this probably isn’t going to make sense to you.

In my last post, I discussed some of the issues around multi-developer teams and working in parallel on the same workflow with K2 blackpearl (summary: it’s really difficuly). I posted some best practice strategies on ways in which to mitigate these issues in that blog, so if you haven’t read it, you should.

In thinking about the issues around parallel development in preparation to write that blog entry, I came up with what I thought was an interesting solution and I spent the rest of this week fleshing/proving it out a bit.

The basic thought was, “What if there were a way to generate K2 project files directly from .NET assemblies?”

I saw several advantages to an approach like this:

• Allows you to leverage your existing .NET skills and quickly create workflow from business assemblies (which if you follow my proposed best practices is where all your code is anyway)
• Streamlines creation of workflows that make heavy use of custom code and enforces best practices by removing custom code from the workflow file
• Simplifies automated testing as you can test directly against your .NET assemblies (and not the workflow)
• Promotes code reuse
• Workflows become as granular as any other .NET project and assuming activities were broken out into their own files in the .NET assembly, developers can work on different pieces of a workflow code at the same time
• Makes it simple to consume external data sources such as Web Services, Databases/SmartObjects, etc. using the .NET practices you are already familiar with

Learning to Walk Before We Run

I mulled this over for a day and finally decided that this was something worth persuing and came up with a rough idea of how to make this work.

My plan for a proof-of-concept was to use Attributes on classes/properties/methods in the business assemblies to mark how items in the assembly corresponded to K2 workflow concepts. The code would be compiled into a .dll and then run through my tool. My tool would use reflection to pull out the attributes and would generate a K2 project file (*.kprx) with all the appropriate activities/events tied directly back to the business assembly by writing out the XML which makes up a *.kprx file.

I don’t want to bore you with the details of all the trouble I went through creating a *.kprx file, but trust me when I say it is complicated. I quickly abandoned the approach of accessing the XML directly as the file structure is just too complex. Knowing that K2 has three separate designers already (and more on the way), I figured they must have reusable routines to handle reading/authoring project files somewhere.

 So I broke out Reflector and began digging through the assemblies that sounded promising and quickly found SourceCode.Workflow.Authoring.dll and SourceCode.Workflow.Design.dll. These two .dll’s have just about everything you need (except for any kind of documentation) to read/create K2 project files.

After a lot of trial and error, I finally was able to create a workflow project which actually was readable from the K2 Designer for Visual Studio and which was buildable/deployable in K2.

Example Code & Resulting Workflow

Just to provide a simple example of what some of the code in your business assembly might look in order to generate a workflow with my tool:

    [Process("Colin's Process", "colinprocess.kprx")] 
    public class TestProcess 
    { 
        private string _userName = ""; 
        private int _testInt = 0; 
        private int _personId = 0;    
 
        public TestProcess() 
        { 
            TestActivity = new SimpleActivity(this); 
            TestActivity2 = new SimpleActivity(this); 
            TestActivity3 = new SimpleActivity(this); 
        }    
 
        [Activity("Test Activity")] 
        [Destination("UserName")] 
        [Escalation(60,0)] 
        public SimpleActivity TestActivity = null;    
 
        [Activity("Test Activity 2")] 
        [Destination("UserName")] 
        [Escalation(60, 0)] 
        public SimpleActivity TestActivity2 = null;    
 
        [Activity("Test Activity 3")] 
        [Destination("UserName")] 
        [Escalation(60, 0)] 
        public SimpleActivity TestActivity3 = null;    
 
        #region Line Rules    
 
        [LineRule("Start", "Test Activity")] 
        public bool DoesIntExceedThreshold() 
        { 
            if (TestInt > 1000) 
            { 
                return true; 
            }    
 
            return false; 
        }    
 
        [LineRule("Test Activity", "Test Activity 2")] 
        public bool IsPersonIdKnown() 
        { 
            if (PersonId == 15) 
            { 
                return true; 
            }    
 
            return false; 
        }    
 
        [LineRule("Test Activity", "Test Activity 3")] 
        public bool IsPersonIdNotKnown() 
        { 
            if (PersonId != 15) 
            { 
                return true; 
            }    
 
            return false; 
        }    
 
        #endregion Line Rules    
 
        #region Data Fields    
 
        public int PersonId 
        { 
            get 
            { 
                return _personId; 
            } 
            set 
            { 
                _personId = value; 
            } 
        }    
 
        public string UserName 
        { 
            get 
            { 
                return _userName; 
            } 
            set 
            { 
                _userName = value; 
            } 
        }    
 
        public int TestInt 
        { 
            get 
            { 
                return _testInt; 
            } 
            set 
            { 
                _testInt = value; 
            } 
        }    
 
        #endregion Data Fields 
    }    
 
    public class SimpleActivity 
    { 
        private TestProcess _testProcess = null;    
 
        public SimpleActivity(TestProcess process) 
        { 
            _testProcess = process; 
        }    
 
        [Event("Retrieve Data", 0)] 
        public void RetrieveData() 
        { 
            _testProcess.PersonId = 15; 
        } 
    }

The code above generates a workflow which looks like this:

Summing Up

Probably at this point you are saying, “What!?!?!? I wasted all my time reading this just to see you write a whole bunch of code which just creates a simple workflow I could have made in like five minutes in the K2 designer?”

As the heading of this section suggests, you have to walk before you can run, and this first post is just meant to provide an introduction to the idea. My second post in this series is going to drill down into understanding what the above code actually does and then my final post on the topic is going to (hopefully) provide a compelling example of the power of this approach using a complex workflow (along with the source for the generation tool).

Background 

To say that the user base K2 targetted for blackpearl is diverse almost doesn’t fully describe the situation. K2 is meant to be used by power users (via the SharePoint workflow editor), business analysts (via Visio), and developers (via Visual Studio) and that isn’t even the end of the story. K2 was (is?) working on an editor which could be used from Word 2007 and they are also working on a stand-alone editor called K2 Studio (similar to the K2 Studio from K2.NET 2003). So first off, I need to preface this blog by saying that K2 had very good reasons for developing blackpearl in the way that they did in order to support the large number of editors/user types that their business strategy has them focusing on.

This is a typical scenario which such a strategy facilitates:

• Developers create wizards and SmartObject Services/SmartObjects
• Business Analysts/Power user create a workflow in Visio/SharePoint which consumes the wizards, etc. the developers have created
• Business Analysys/Power user hand off the workflow  to developers to add any complex business logic, etc. if necessary
• Developer deploys workflow
• Business rejoices

Now, I am sure that there are businesses out there where this kind of scenario works great and certainly this is a form of team development and is one which K2 supports very well. It is a compelling story and makes you see the real power K2 can have in an organization if you empower your users with the right tools…

Problem 

That’s not the typical situation I find myself in as a consultant who does K2 work though (I’m sure it exists, I’m just saying I don’t get to see such utopias). I am usually brought in to a company that has no/little K2 experience and is wanting to create a workflow for the first time. Some business unit (with no concept of BPM/workflow solutions) is driving the requirements and an IT department is responsible for having to implement the workflow. There is usually some interaction between the two groups to determine the requirements and map out the flow, but the business unit does not actually do any development and the business analysists are busy just documenting the requirements so don’t participate in the actual development. And so it falls on the shoulders of developers to implement the workflow in K2.

 It then usually works like this, the IT group looks at  the problem which is being solved/requirements and says, “This is going to take one developer x number of weeks.” And then the business unit says, “x number of weeks! That is too long, we need it next week!” And then they negotate and eventually settle on y weeks, but to complete it in y weeks they need additional developers on the team.

So the “team development” I am talking about in this post is the multi-developer team where there is no cross-role development (developers trading workflow files with business analysts) going on, all collaboration is at a verbal level: requirements specs, flow diagrams, etc.

Another thing my workflows almost always have in them is some level of customized code. I am very rarely able to implement workflow solutions using just the drag-and-drop capabilities provided by the toolbox. Again, that’s not to say it isn’t possible to create workflows without writing any code, it is just to say that in my experience I am only brought in to assist with particularly gnarly workflows.

Some of the same things that make K2 easy to share between the different editors, etc., and which faciliate cross-role collaboration actually prove to be challenge when trying to share work among the same roles.

Some of the issues are:

• Multiple developers can not work on the same workflow (*.kprx) at the same time
  • Developers can’t just checkout an individual activity or event, they have to check out the entire workflow file
  • Visibility of changes is limited (such as in your source control file history) because they are only being tracked at the *.kprx file level.
• Detailed visibility is limited
  • K2 is very good at providing a high level view of a workflow (the workflow diagram) and this allows developers to quickly see all the activities/events within a particular workflow BUT a lot of important information is obscured. Examples:
    • No way to see when custom code has been added other than by drilling into each item
    • No way to see destinations or escallations other than drilling into each item
    • No way to see what SmartObject is called and what parameters are passed other than drilling into each item
    • No way to see what workflow is called in an IPC event other than drilling into each item
    • No way to see what logic is applied to line rules other than by drilling into each one
• Environment/Deployment/Sharing Issues
  •  As the solution becomes more complex (multiple wizards, SmartObject Services, Events, Roles, etc) it becomes very difficult to keep multiple environments in sync.
    • Because of this, most development teams only have a single K2 development environment which they must share. This leads to issues such as:
      • Inability to debug at the same time as another user
      • Inability to test at the same time as another user
    • The alternative to this is to give each developer their own development environment (usually on a VPC). This leads to issues such as:
      • Difficulty keeping VPC environment in sync
      • Difficulty in merging changes into the main development trunk/environment
      • Difficulty in accessing external resources
• Extreme difficulty in performing automated testing

Proposed Mitigation Strategies

• Specialize the team
  • A workflow commonly consists of multiple parts: GUI (InfoPath, SharePoint list, ASP.NET), Store/Retrieve data (SmartObjects, WebServices, etc.), Flow (K2.NET)
  • Where practical, you may see benefit by dividing the team so that each team member has non-overlapping responsibilities.
• Document the workflow
  • Give all activities/events useful, descriptive names
  • Add labels to all lines explaining the rule logic in english
  • Indicate escallating activities via a standard color (I use red)
  • Indicate customized code activities via a standard color (I use yellow)
•  Minimize custom code by offloading to
  • SmartObject methods (such as by calling a WebService)
  • External business assemblies
    • You still have custom code in this scenario, but the amount of customized code which exists in the workflow file itself is limited to just the call to your business assembly.
  • Custom Event Wizards
    • Updates to a wizard are not automatically reflected in a workflow. All workflows must be reopened and the event wizard design template must be updated (which overwrites any customizations you may have made to the code within that particular workflow). For this reason, I still recommend that custom event wizards should limit the amount of code which lives in the wizard and should keep most logic in external assemblies
• Split up large workflows
  • If workflows have logical groupings of activities, consider moving these activities to their own workflow and then calling them via an IPC Event Wizard.
    • Be sure to apply the KB000223 hotfix to your K2 blackpearl SP1 server
    • This has the benefit of allowing different developers to work on these different sub-workflows at the same time but increases
      • Deployment complexity: You now have to be sure to deploy the most recent versions of the workflow and all sub-workflows
      • Reporting complexity
      • Reduces workflow understandability: In order to understand the entire workflow you now have to view the workflow and the sub-workflows
      • May increase development complexity: As you pull out common activity blocks into their own workflows, you may need to generalize them in such a way that they can be consumed by multiple workflows which will lead to more complexity as you attempt to accomodate competing needs.
      • Increased testing requirements: If a sub-workflow is shared among many different workflows, a single change in that workflow impacts
  • Automated Testing
    • Complete test coverage is A TON of work.
      • My coworker, Jason Apergis, has a blog entry on how to perform automated testing on a workflow in K2.NET 2003 and the same strategies can be applied to blackpearl.
      • In the case where there is complex routing logic, the code and maintenance requirements quickly get out of hand.
    • Selective test coverage:
      • Offload as much complex logic as possible into custom business assemblies/web service calls
        • These options already have very mature testing frameworks.
        • Create full coverage unit tests for these external items independant of the workflow
        • This greatly reduces the surface on which bugs can occur and most issues become easy to trackdown within the workflow.
      • Perform complete workflow simulations only where the effort is justified (hint: if you offloaded properly, then it is rarely necessary)
    • K2 is working on a simulation capability for a future service pack.

I have two projects which I am working on to test some methods for improve some of the issues I have highlighted, but I’m not quite ready to announce them.

I expect to discuss the first one towards the end of this week. My first tool is a pretty radical departure from the standard K2 development paradigm so I am not sure anyone else will use it (hint: it gets rid of the K2 visual studio GUI entirely), but the potential it has for streamlining/enabling many of the suggested practices above is pretty awesome (IMHO). Look for the first preview later this week.

Before I came on board on my current project, there was a proof of concept done by a consultant from K2. This POC made use of a Dynamic Web Service SmartObject Service (similar to the Dynamic SQL SmartObject Service available on K2 blackmarket, but to call web services) and when I was brought in to implement the final solution I began using this service in the final solution.

This past weekend I brought home some of the workflows with the intent of working on them but then realized that I had forgotten the Dynamic Web Service broker. No problem I thought, I’ll just go grab it from it off of blackmarket (I had just assumed it would be on there) but it wasn’t. “Hmm”, I thought, “perhaps this is a good opportunity to learn how to write my own dynamic broker and make a few improvements over the one I had inherited.”

So in this post I’ll cover the steps I went through to create the service.

Structure of a Dynamic Service Object

A SmartObject Service consists of:

• Service Broker
  • Implements SourceCode.SmartObjects.Services.ServiceSDK.ServiceAssemblyBase
  • This is the mechanism with which K2 blackpearl interacts with your service object and is responsible
    • Telling K2 what configuration information is required (override GetConfigSection)
    • Telling K2 what methods are available to be called and what data is returned (override DescribeSchema)
    • Giving K2 a means to call the methods advertised by DescribeSchema and get results (override Execute)
    • Giving K2 a means to extend your service to support other service objects (override Extend) outside the scope of this discussion
• Backend communication class/methods
  • Called by the Service Broker
  • Responsible for implementing communication with your backend system and returning the data to your Service Broker

Sounds pretty simple doesn’t it? This can be a single class or (more typically) is divided into at least two classes (the service broker and the data access method).

Now one thing I didn’t know how to do was how to generate a web service proxy dynamically (I’ve always just used Visual Studio to do this for me), but that capability is pretty critical for our dynamic service broker.

Just a few house keeping details before we get started. I based a lot of my work off of the source code that Seb wrote for the Dynamic SmartObject Services sample on K2 blackmarket which creates a dynamic service object for SQL Server.

How to build a web service proxy dynamically

So a quick google search turned up some good resources around this:

• Dynamically Invoking a Web Service
• WebReference Class
• Adding a Web reference dynamically at Runtime

I’m not going to rehash the information covered in the posts above, but if you want to learn the details about how to create a web service proxy programmatically, those are some good resources.

Creating the web service access object

I used the code found in the first post in the list as a starting point for my proxy generation method and then modified it somewhat. The code looks like this:

        private static void GenerateProxy(string webServiceUrl) 
        { 
            WebClient client = new WebClient();        
 
            client.Credentials = GetCredentials();        
 
            //Does the url already have ?wsdl on the end? If not, let's add it. 
            if (!webServiceUrl.EndsWith("?wsdl")) 
            { 
                webServiceUrl += "?wsdl"; 
            }        
 
            Stream stream = client.OpenRead(webServiceUrl);        
 
            //Read in the WSDL 
            ServiceDescription description = ServiceDescription.Read(stream);        
 
            ServiceDescriptionImporter importer = new ServiceDescriptionImporter(); 
            importer.ProtocolName = "Soap12"; 
            importer.AddServiceDescription(description, null, null);        
 
            //We want to generate a client 
            importer.Style = ServiceDescriptionImportStyle.Client;        
 
            //Generate properties to represent primitive values. 
            importer.CodeGenerationOptions = 
              System.Xml.Serialization.CodeGenerationOptions.GenerateProperties;        
 
            // Initialize a Code-DOM tree 
            CodeNamespace codeNamespace = new CodeNamespace(); 
            CodeCompileUnit compileUnit = new CodeCompileUnit(); 
            compileUnit.Namespaces.Add(codeNamespace);        
 
            // Import the service into the Code-DOM tree. This creates proxy code 
            // that uses the service. 
            ServiceDescriptionImportWarnings warning = importer.Import(codeNamespace, 
               compileUnit);        
 
            if (warning == 0) 
            { 
                // Generate and print the proxy code in C#. 
                CodeDomProvider provider1 = CodeDomProvider.CreateProvider("CSharp");        
 
                // Compile the assembly with the appropriate references 
                string[] assemblyReferences = new string[2] { "System.Web.Services.dll", 
                   "System.Xml.dll" }; 
                CompilerParameters parms = new CompilerParameters(assemblyReferences); 
                CompilerResults results = provider1.CompileAssemblyFromDom(parms,
                    compileUnit);        
 
                _webServiceAssembly = results.CompiledAssembly; 
            } 
            else 
            { 
                throw new Exception("Warnings encountered creating service."); 
            } 
        }

The code above basically hits a webservice url to get the WSDL then creates a proxy object using the code-DOM. We use this object later for describing the schema and for executing code. This isn’t really central to creating a SmartObject service in general, so I am not going to go into this in detail

Describing the Schema

Now that the proxy object has been generated, we need to tell K2 about the schema. The bulk of this code looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

        public static void DescribeSchema() 
        { 
            //We always want to regenerate the schema when DescribeSchema is called 
            GenerateProxy(WebServiceUrl);        
 
            Type webServiceType = WebServiceAssembly.GetTypes()[0]; 
            TypeMappings map = CreateTypeMappings();        
 
            if (webServiceType != null) 
            { 
                ServiceObject serviceObject = null; 
                ServiceBroker.Service.ServiceConfiguration["TypeMapping"] = map;        
 
                try 
                { 
                    //Foreach method in our webservice... 
                    foreach (MethodInfo method in webServiceType.GetMethods()) 
                    { 
                        //which is actually a webmethod and 
                        //which has input parameters which are supported (an example of 
                        //parameters that aren't supported are arrays as input parameters 
                        //(arrays as return values are ok 
                        if (method.GetCustomAttributes( 
                              typeof(SoapDocumentMethodAttribute), false).Length > 0 
                            && AreAllParametersSupported(method) 
                            ) 
                        { 
                            //Create ServiceObject definition. 
                            serviceObject = CreateServiceObject(serviceObject, method);        
 
                            //Create SmartObject method from the methodinfo 
                            Method meth = CreateMethod(method);        
 
                            //Create input parameters to pass into the method 
                            CreateInputParameters(map, serviceObject, method, meth);        
 
                            //Create the properties to hold the return value from the method 
                            CreateOutputProperties(map, serviceObject, method, meth);        
 
                            //Add the method to the service object 
                            serviceObject.Methods.Add(meth); 
                            ServiceBroker.Service.ServiceObjects.Add(serviceObject); 
                        } 
                    } 
                } 
                catch (Exception ex) 
                { 
                    //Hmm, adding that method didn't work. Let's just swallow it and continue. 
                } 
            } 
        }

I’ve added line numbers to this code listing so I can describe what is going on in a bit more detail here.

In line 4 we generate our web service proxy object (which we will be using to describe the schema). In order to describe the schema, we’ll have to use Reflection to get information about the proxy type. Line 7 is something that I based off of Seb’s code. I’m not positive it is required, but didn’t really feel like fooling with it to see what would happen if I did my mapping another way. Line 17 is where we begin iterating through all the methods in our proxy object and generate the schema for them. I have a bunch of helper methods which do most of the actual work (I won’t go into each one in detail), but some of the ones to highlight are:

• IsSimpleReturnType: returns true/false depending on if the return type of the method is one which we definitely know how to deal with. Simple return types are all value types, strings, and arrays of value types.
• AreAllParametersSupported: returns true/false depending on if the method has parameters we know how to deal with. Supported parameter types are currently limited to value types and strings.
• CreateServiceObject: creates the service object
• CreateMethod: creates the method on the service object including specifying it’s type (possible types currently supported: Execute, Read, and List)
• CreateInputParameters: creates all the properties on the service object and the input property on the method
• CreateOutputParameters: creates all the properties on the service object and the return properties on the method

Creating the Execute Method

The actual Execute method which K2 calls is just a void Execute(), but in his source Seb uses an overloaded Execute which he calls from the Execute() method and I am following this practice. So my overloaded Execute method looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

        internal static void Execute(Property[] inputs, RequiredProperties required, 
                                              Property[] returns, MethodType methodType, 
                                              ServiceObject serviceObject) 
        { 
            Type webServiceType = WebServiceAssembly.GetTypes()[0]; 
            SoapHttpClientProtocol webService = 
               (SoapHttpClientProtocol)_webServiceAssembly.CreateInstance(webServiceType.Name);        
 
            //Use the credentials which are specified in the K2 service configuration 
            webService.Credentials = GetCredentials();        
 
            MethodInfo methodInfo = webServiceType.GetMethod(serviceObject.Name);        
 
            object[] parameters = new object[inputs.Length];        
 
            for(int i = 0; i < inputs.Length; i++) 
            { 
                parameters[i] = inputs[i].Value; 
            }        
 
            object returnValue = methodInfo.Invoke(webService, parameters);        
 
            if (methodType == MethodType.Read) 
            { 
                serviceObject.Properties.InitResultTable();        
 
                //simplest option is that the return type is just normal 
                if( IsSimpleReturnType( methodInfo.ReturnType ) ) 
                { 
                    serviceObject.Properties[0].Value = returnValue; 
                } 
                else 
                { 
                    for (int i = 0; i < returns.Length; i++) 
                    { 
                        serviceObject.Properties[i].Value = 
                            methodInfo.ReturnType.GetProperty( returns[i].Name ).GetValue(
                                returnValue, null); 
                    } 
                } 
                serviceObject.Properties.BindPropertiesToResultTable(); 
            } 
            else if (methodType == MethodType.List) 
            { 
                if (ServiceBroker.ServicePackage.ResultTable == null) 
                { 
                    ServiceBroker.ServicePackage.ResultTable = 
                       new DataTable(methodInfo.Name); 
                }        
 
                DataTable results = ServiceBroker.ServicePackage.ResultTable;        
 
                foreach (Property prop in returns) 
                { 
                    results.Columns.Add(prop.Name, Type.GetType(prop.Type)); 
                } 
                foreach (object obj in ((IEnumerable)returnValue)) 
                { 
                    //We know we are dealing with an array, but is it an array 
                    //of simple types (value objects, strings, etc) or is it an 
                    //array of custom objects? 
                    if (IsSimpleReturnType(methodInfo.ReturnType)) 
                    { 
                        //If it is a simple type then we can just assign the value 
                        //to our return values 
                        results.Rows.Add(obj); 
                    } 
                    else //it is a complex object, so let's use reflection to return 
                          //the properties 
                    { 
                        object[] values = new object[returns.Length];        
 
                        //iterate through our return values 
                        for (int i = 0; i < returns.Length; i++) 
                        { 
                            //if it is a complex type then we need to get the property 
                            //from the object 
                            values[i] = 
                                obj.GetType().GetProperty(returns[i].Name).GetValue(
                                   obj, null); 
                        }        
 
                        results.Rows.Add(values); 
                    } 
                } 
            } 
        }

Again, we use reflection here to access our proxy object and to execute the method on the object. Lines 4 - 11 are basically for the purpose of loading the method so that we can call it later. Lines 13 - 18 handle our input parameters for the method. Line 20 is where we actually invoke the web service call and get our return value. And then finally 22 - 78 is handling that return type in the event we are returning a single value (Read) or an array (List) and if we are returning a simple type or a complex/custom object.

Summing Up

I think that highlights the most interesting bits of the solution and I’ve created a K2 blackmarket project to hold the source code (I will be putting the source up there later today). If you need any further clarification or want additional details on a particular area, please comment and I will respond as I am able.