Building Message Specifications in Microsoft BizTalk Server 2004

This chapter is from the book

Microsoft BizTalk Server 2004 Unleashed

Learn More Buy

This chapter is from the book

This chapter is from the book 

Microsoft BizTalk Server 2004 Unleashed

Learn More Buy

In This Chapter

• Editor Details

• Flat-File Specifications

• Property Promotion and Property Schemas

• Using InfoPath Documents with BizTalk

Messaging is the foundation of BizTalk Server, and message schemas are the bedrock on which messaging is built. Schemas are used by the messaging runtime to understand the structure of data messages and find specific fields. The property fields specified by schemas can be used to help route a message to its destination. Messaging is the exchange of structured data, and schemas are where that structure is defined.

BizTalk Server makes use of XML Schemas to describe a wide range of message structures. Messages in BizTalk encompass the full range of features supported by XML and can also include positional and delimited flat-file formats. Because the same tool is used for both XML and flat-file formats, BizTalk abstracts messages into records and fields. This is a common notion that will be familiar to many programmers. Visually, BizTalk Editor treats both as nodes in a tree. We will begin by looking at how you can use Editor to visually describe a textual structure as nodes in this tree. After you've mastered that, we'll move on to see how various XML structures are described as the properties of these nodes. Then we'll move to the slightly more difficult task of doing the same thing for flat-file formats. Finally, we'll see how BizTalk's use of XML Schemas allows us to prepare the messaging system for use with another technology that uses forms in Microsoft Office as Web service clients.

Editor Details

Recall from Chapter 2, "Your First BizTalk Server Integration," that BizTalk Editor has three principal views: the Schema tree view, the XSD Source view, and the Property view. Because the XSD Source view is read-only, you'll be spending your time in the other two views. Because a lot of the properties you'll set and the options available to you depend on the type of the schema (XML or flat file), we'll concentrate on the basics of navigating the user interface and building a general document structure in this section.

Schema Properties

The first thing you see when you open a new schema is a <Schema> node with a single child node named Root. <Schema> is something of a metanode and as such is really a placeholder for the document as a whole. You'll set two kinds of properties for this node: properties that pertain to the schema as part of an overall BizTalk project and properties that pertain to the schema itself. The former are found and set in a properties dialog box that you open, whereas the latter are manipulated through the Properties window that is typically open throughout your Visual Studio session.

In the Solution Explorer node, right-click the schema file entry and click Properties. You'll see the Property Pages dialog box depicted in Figure 3.1.

Figure 3.1 Schema item properties for the project are set in the Property Pages dialog box.

There are four properties you can set, but they pertain to two features: Instance Document Interaction and Property Promotion. Editor lets you generate sample instance documents from the schema or validate sample instance documents. These are useful features that let you compare your schema to the sort of instance document you envision. Without setting instance document filenames, Editor can generate a temporary file for output, but you need an actual input document to use the validation feature. If you need a bit more permanence in output, you'll need to set the following properties:

• Output Instance Filename—The name and location of the sample instance document generated by Editor from the schema.

• Input Instance Filename—The name and location of the sample input document you provide for validation against the schema.

When generating instance documents, you may also need to set the Create Instance Output Type property. This defaults to XML, but when you are dealing with flat files, you'll need to set it to Native. You generate an instance document by right-clicking the schema document in Solution Explorer and clicking Generate Instance. When you want to validate the existing document pointed to by the Input Instance Filename property, you right-click as before and click Validate Instance.

The remaining property, Default Property Schema Name, refers to a schema that is used in conjunction with the property promotion feature we discuss in the "Property Promotion and Property Schemas" section later in this chapter. This property defaults to PropertySchema.xsd. Note that unlike the Filename properties we set for instance files, this is an unqualified filename.

Close the Property Page dialog box now and turn your attention to the Visual Studio Property window. With the schema file selected in the Solution Explorer, the Property window displays a variety of properties, mostly names and file locations. The Namespace property is a .NET-style namespace and differs from the Schema node's Target Namespace property. It turns up throughout the tools when qualifying .NET types. Similarly, the Type Name property is derived from the filename and is the unqualified typename.

If you select the <Schema> node in the tree view, however, you find a richer set of properties. In this view you can declare a namespace for the schema, indicate whether you are specifying a message or an envelope used to enclose messages, specify the standard for the schema, and even specify a root node to use when you are sending a message with multiple top-level nodes.

You can usually accept the defaults for Schema node properties, particularly when writing a schema for an XML message. There is one property, however, which may seem curious to you, especially if you have experience with previous versions of the Editor. If you peruse the Schema node properties for a newly created schema, you will not find a property that permits you to declare a flat file. Experienced BizTalk programmers will look at the Standard property without success. The answer to this dilemma lies in the fact that Editor is now composed of an XML subset plus extensions. To declare a flat-file schema, go to the Schema Editor Extensions property, click the ellipsis button, and check Flat File Extension in the Schema Editor Extensions dialog box. This dialog box also lets you load the Human Workflow extension. After the flat-file extension is loaded, new properties appear in the Property view, and new property values are added to the standard property list box. One of these is Flat File.

Adding Records and Fields

After you've created a schema and renamed the root node, you're ready to add some content. The commands to do this are found on the BizTalk menu off the main menu bar or directly through right-clicking and selecting from the context menu. Regardless of which menu you use, you point to Insert Schema Node and make a selection from the submenu.

The first distinction you have to worry about is whether to insert a record node or a field node. This distinction is easy when you are solely concerned with structure. Records can contain other records and fields or can have values directly. Fields take only values. This is directly analogous to database design. When you move into schemas for XML messages, you have another choice to make. Only elements can have child content, so they have to be implemented as elements. Both elements and attributes can have values, so BizTalk message fields can be implemented as either XML node type.

Now that you've decided what sort of node to add, you need to think about where the node is going. Your choices are all relative to the node currently selected in the tree view. Consequently, the Insert Schema Node menu has choices for Child Record, Child Field Attribute, and Child Field Element as well as Sibling Record, Sibling Field Attribute, and Sibling Field Element. After child content is created, you can change the order of the child nodes by dragging them within the tree view.

The XSD Schema format includes several kinds of group constructs. BizTalk takes advantage of this. By default, child content for a node is presumed to occur in order in the sequence provided. You can get the same effect explicitly by using the Sequence Group menu item, in which case a sequence element is created in the XSD Source view. Sometimes, though, you want to specify a series of records or fields from which to choose in creating a message instance. If you had a message schema representing this chapter, for example, you'd need to be able to select from a choice of paragraphs, figures, notes, and tables at any given point under the chapter record. To do this in BizTalk Editor, you insert a Choice group. The node in the tree view is labeled <Choice>, and the available choices are specified as the child nodes of this node. You control the number of times the message instance can choose from these options with the Min Occurs and Max Occurs properties of the <Choice> node.

XSD format schemas also allow you to specify an element that acts as a wildcard placeholder. At some point in the schema you know that you will have some element, and you know what the child content of that node will be, but you do not know in advance what the node itself will be. A family of applications might have various specialized views of an individual, but you are concerned only with some subset of the content. For example, a hospital information system might embrace Nurse, Doctor, and Other Staff types of employees, but all have a common substructure for the purposes of employee benefits. An HR application, then, might use a schema that ignores Nurse, Doctor, and OtherStaff elements while paying attention to the child content. This is the purpose of the All Group type of node. This option appears only when the selected node meets the constraints of the XSD Schema Recommendation for the use of the all element. To use this in BizTalk Editor, insert a child record, set the record's Content Type property to ComplexContent, and then point to Insert Schema Node and click All Group.

The final type of group you can create in Editor is an Attribute Group. You might have a series of field attributes that you need to apply to multiple records, and you want to avoid respecifying them every time you use them. You can do this by inserting an Attribute Group node and then inserting child field attribute nodes. When you want to reuse the group in another record, insert an Attribute Group node and then change the Group Name property value from what Editor generated to the value in the first attribute group. All the field attributes you defined there will automatically appear in the new group node.

Continuing our tour, you'll see that BizTalk Editor offers two singleton node types that, like the All Group node, allow you to inject some degree of ambiguity into a schema to accommodate uncertainty. The first is created with the Any Element command on the Insert Schema Node menu. When created, it inserts an <Any> node into the tree view and an XSD Schema any element into the schema source view. Like the All Group, the Any node accommodates some unknown well-formed element content. In contrast to the All Group node, it cannot have child content specified for it. The schema allows child content, but that content will be ignored for validation purposes. If you are specifying a schema for a very complex schema, you might, for example, want to skip specifying content for areas of the message that you know will never be used by the receiving application or partner. Similarly, if you are modeling an element that you know will have some unknown attributes (that you don't care about), you may use the Any Attribute command to insert an <AnyAttribute> node into the tree view. An XSD Schema anyAttribute element is added to the source view. Like the Any node, AnyAttribute is used to provide for the existence of content while ignoring its particulars.

Sample Schema: Structure

We've given you all the commands BizTalk Editor provides for creating structure, and that's a lot of material for one sitting. Let's ground this in a practical sample. You can examine this schema in the file BasicXMLSchema.xsd in the chapter download, or you can use the steps that follow to create it from scratch. The purpose of this message schema is to model a single person who serves as a point of contact, possibly as part of a customer management system. Each message instance must model the contact's name as well as a list of one or more home or office addresses. For simplicity, we'll assume United States style postal addresses and SSNs and ignore electronic means of contact.

1. Within a BizTalk project, right-click the project node in Solution Explorer, point to Add, then click Add New Item. From BizTalk Project Items, select the Schema template to create a new schema file.

2. Change the root node's Node Name property to Contact. Enter ContactType for the value of Data Structure Type.

3. Right-click Contact in the tree view, point to Insert Schema Node, and then click Sequence Group. Set the Max Occurs and Min Occurs properties of this new node to 1.

4. Right-click <Sequence>, point to Insert Schema Node, and then click Child Record. Change the Node Name property to Person and the Min Occurs and Max Occurs properties to 1.

5. Right-click Person and insert a Sibling Record. Name this Locations and set Min Occurs and Max Occurs to 1.

6. Return to Person and insert a Child Field Attribute. Name it SSN. Set the Use Requirement property to Required.

7. Return to Person and insert a Child Field Element. Name it FirstName and set Min Occurs and Max Occurs to 1.

8. Repeat step 7, inserting MiddleName (Min Occurs = 0, max Occurs = 1) and LastName (Min occurs = 1, Max Occurs = 1).

NOTE

Locations is intended to be a collection of one or more instances of home and or office addresses. These can occur in any order and any number, provided that at least one instance of an address record occurs. We'll model this with a Choice Group. Further, we note that home and office addresses have something in common: the street address. They differ in that a home address can have an apartment number, whereas an office can have a room number or mailstop. We can therefore create a reusable type to model addresses.

9. Right-click Locations, point to Insert Schema Node, and then click Choice group. Set the Min Occurs property to 1. This will ensure that at least one selection must be made from the permissible choices.

10. Right-click <Choice> and then insert a Child Record. Change the Node Name property to Home.

11. Right-click Home and insert a Child Field Attribute. Name this node apt.

12. Right-click apt and insert a Sibling Record. Name it Address and then type AddrType into the Data Structure Type property field. AddrType(ComplexType) appears in bold.

13. Insert a Sequence Group node as a child of Address. Set Min Occurs and Max Occurs to 1.

14. Select the <Sequence> node and insert the following five Child field Elements, setting their properties as shown in Table 3.1.

Table 3.1 Child Field Elements for the AddrType Global Type

Node Name	Min Occurs	Max Occurs
Street1	1	1
Street2	0	1
City	1	1
State	1	1
ZIP	1	1




15. Under Locations, select <choice>, and then insert a Child Record. Set Node Name to Office.

16. Right-click Office and insert a Child Field attribute. Name it room. Set Use Requirement to optional.

17. Repeat step 16 to create mailstop.

18. Right-click Office and insert a Child Record. Set Node Name to Address. Select AddrType (ComplexType) from the list in the Data Structure Type property. Note that the child content previously declared in step 14 appears.

19. Save your work as BasicXMLSchema.xsd, then right-click the Schema node in Solution Explorer and click Validate Schema. The Output window should indicate Validate Schema succeeded.

At this time, the tree view should look like Figure 3.2.

Figure 3.2 Final structure for your first sample schema.

Node Properties

We've covered a few of the properties of various nodes in the course of our previous discussion and introductory structural sample. The online documentation for BizTalk covers node properties in exhaustive detail. You need to know a bit more about node properties. Rather than repeat the BizTalk topics on node properties in their entirety, we'll cover the properties you are most likely to use in normal BizTalk development projects.

Schema Node Properties

Starting at the top, we should first consider the <Schema> metanode. Schema nodes are commonly configured with the properties in Table 3.2, although we'll be discussing several more properties when we get to flat-file schemas.

Table 3.2 Commonly Used Schema Node Properties

Some of the properties discussed previously require further discussion. Schema importation is an important mechanism for structuring your schemas and reusing common constructions. We deal with this issue in the section on building schemas from existing schemas. Property promotion is a very important topic for BizTalk messaging. We alluded to this in Chapter 2 when we promoted some properties from message instances into the message context. We discuss property promotion and property schemas later in this chapter. Envelopes are useful for message routing. Envelopes are processed by pipelines, so we shall go into greater depth on the topic of envelopes in Chapter 5, "Building Pipelines."

Record Node Properties

Record nodes, which include the document root element, are most important nodes after the Schema metanode because they are the foundation of message structure. The most commonly used record node properties in XML messages are listed in Table 3.3. We'll be adding additional properties when we get to flat-file formats.

Table 3.3 Commonly Used Record Node Properties

Field Node Properties

Field nodes have a greater number of commonly used properties. Because they represent data fields in your messages, field nodes need properties to specify and constrain the permitted values of that data. XSD Schemas provide rich mechanisms for type restriction, specification, and extension. This is reflected in BizTalk Editor's Property view for field nodes. Table 3.4 describes the properties you will most commonly use when working with message schemas.

Table 3.4 Commonly Used Field Node Properties

This list—which, remember, is only the list of commonly used properties—is somewhat lengthy, but most of the list pertains to type derivation. This is a complex and powerful topic. An exhaustive treatment is found in the W3C Recommendation XML Schema: Datatypes (http://www.w3.org/TR/xmlschema-2/). Type derivation is a useful technique, so we'll go over the basic ideas in the "Inheritance and New Types" section that follows.

Inheritance and New Types

You can go a very long way defining message schemas without once worrying about global types and type derivation. Business applications in the real world, however, frequently deal with objects that are variations on the same underlying idea. You will look at people, for example, in various ways, recording different attributes depending on whether they are employees, managers, customers, prospective customers, or vendor contacts. You can sell a physical good or perform a service, yet both are products. Each has slightly different attributes, but also have some things in common. Type inheritance in schemas is a useful tool for meeting the need of these variations while still capturing their commonality. Creating schemas from other schemas (multiple namespaces) is another tool, but that will have to wait for the "Creating Schemas from Existing Schemas" section.

Our basic XML sample defined a global type, AddrType, to capture common address fields. We used this as the data type for the Address child record in Home and Office. This was typing, though, not inheritance. Home and Office were variations on the idea of a place. We inserted what they had in common, Address, and then added the fields that made them unique. This met our admittedly simple needs but left no room for further refinement. Let's modify that schema to take advantage of inheritance.

Make a copy of your finished BasicXMLSchema.xsd file. In the download for this chapter, the file is called DerivedXMLSchema.xsd and it reflects the changes we are about to make. Now perform the following steps:

1. Select the Home record node. In the Properties view, select AddrType from the list provided for the Base Data Type property.

2. Click Yes to dismiss the dialog box warning you that you have performed an action that will delete structures defined in the schema.

3. Note that the Content Type property has the value ComplexContent and the Derived By property defaults to Extension.

4. Enter HomeType for the value of Data Structure Type.

5. Note that Home now has a sequence node that contains another sequence node. The latter node contains the field element nodes for the actual information defined in AddrType.

6. Select the sequence node that is the immediate child of Home. Right-click, point to Insert Schema Node, and then click Sibling Attribute.

7. Name the new node apt.

8. Repeat steps 1–3 for the Office record node.

9. Enter OfficeType for the value of the Data Structure Type property.

10. Repeat step 6 twice and name the new attribute field nodes room and mailstop.

At this point, you have the same message structure you had at the end of our basic XML schema, and you may be wondering what all the fuss is about. After saving your work, select any record node and insert a new record. Select either HomeType or OfficeType as the Data Structure Type property. When you do so, all the fields defined for the type, including the ones we added to differentiate Home and Office from AddrType, appear. We can now derive from these new types to create variations, perhaps to differentiate between a primary residence and a vacation home, or between primary and satellite offices.

Schema type derivation is about more than simply creating new types in the sense of class inheritance in an object-oriented programming language. The W3C Schema Recommendation devotes a considerable amount of space to deriving types through restriction. BizTalk Editor reflects a lot of this, mainly through the Pattern property and the Min and MaxFacetValue properties for numeric types.

Any application that includes a data entry form should include code for field-level data validation. BizTalk is integration middleware, though, so we have no idea whether the applications that are generating the messages flowing through BizTalk have checked their data thoroughly. Tightening up message schemas through the use of restriction is a very good way to ensure that bad data does not spread from one poorly written application to the rest of your system.

You make use of restriction by selecting Restriction for the value of the Derived By property of a node. When you do this, the value of Data Type is moved to the Base Data Type property. If you are working with a numeric type, you can then use MinFacetValue and MaxFacetValue to set lower and upper bounds, respectively. MinFacet Type and MaxFacet Type let you control whether the permissible range for your new type includes these boundary values.

Anyone who has used regular expressions in programming knows that regular expressions are a way to pack a lot of validation information into a compact expression. In general terms, the expression (or expressions) you provide for the value of the Pattern property is a string that specifies what can appear at any given point in the input data value. At runtime, the data in the message field must match the pattern expression for the message to be considered valid.

Our message schema includes a field for U.S. Social Security Numbers (SSNs). These numbers have a definite format, so the SSN field is an ideal candidate for demonstrating the use of restriction for string types. Perform the following steps using DerivedXMLSchema.xsd:

1. Select the SSN attribute field node.

2. Select Restriction from the drop-down list in the Derived By property.

3. Select the Pattern property and click the ellipsis button that appears.

4. Enter [0-9]{3}-[0-9]{2}-[0-9]{4} into the Edit field of the Pattern Editor dialog box.

5. Click OK to close the dialog box.

The regular expression that you entered says that an acceptable value for this field will be a string consisting of 3 digits selected from the range 0–9, followed by a literal hyphen character, then 2 digits from the same range, another hyphen, and then 4 digits, again from the same range. After you save the schema, you can experiment with instance validation and different values for this field to test your pattern.

Creating Schemas from Existing Schemas

The same concepts usually occur in many places throughout an organization, and this is reflected in the data structures in the organization's applications. Consequently, when you develop more than a few message schemas for a system, you'll probably find yourself repeating the same structures over and over. A better approach is to create individual schemas for these frequently recurring structures and then import the schema into more complicated message schemas.

BizTalk Editor offers three mechanisms for doing this, all adopted from XML Schemas. These mechanisms are import, include, and redefine. All three make the types in an external schema available to the schema on which you are working. Include requires that the two schemas either have the same target namespace or that the included schema must come from the empty namespace. Both include and import prevent you from deriving new types from the included or imported schema. Redefine works like import, except that you are free to derive from the external types. BizTalk Editor restricts you to including, importing, or refining schemas that are part of your current project or that are part of the project references.

Let's assume, for the sake of illustration, that you need to write a schema for messages describing projects. You quickly realize a need for describing a project contact and decide to borrow the Contact structure we created in DerivedXML.xsd. To do this, perform the following steps:

1. Add a new schema item to your current BizTalk project. Name it ProjectContactXML.xsd.

2. Rename the root node Project.

3. Add a child element field and name it Name. We'll leave other, non-Contact specifications as an exercise for the reader.

4. Select the <Schema> node.

5. Select the Imports property and click the ellipsis button.

6. Ensure that XSD Import is selected from the list for Import New Schema, as shown in the Imports Dialog window (see Figure 3.3).

7. Click Add.

8. Select DerivedXMLSchema from the Schemas branch in the BizTalk Type Picker dialog box that appears. This type will be qualified by the value of the Namespace property for the schema.

9. Click OK to close the type picker; then click OK again to close the Imports Dialog window.

10. Select the Project record node, insert a child record node, and name it Contact.

11. Select ns0:ContactType from the list of values for Data Structure Type property.

The Imports dialog box is shown in Figure 3.3. When you complete step 11, you'll see the nodes you defined in DerivedXML.xsd appear under the new node.

Figure 3.3 The BizTalk Type Picker enables you to select a schema to import, include, or redefine.

Creating a Schema from an Instance Document

BizTalk enables you to make use of any of several types of existing resources you may have for the specification of messages. Earlier versions of BizTalk used the XDR schema syntax proprietary to Microsoft. Before we had XSD Schemas as a standard, XML metadata was specified in document type definitions (DTDs). Without a prior definition, you can always compose a well-formed XML document or generate one from an existing application. You can use any of these to generate a new message schema for your BizTalk project.

To do this, you right-click the project node in the Solution Explorer, point to Add, and then click Add Generated Items. The Add Generated Items dialog box appears. Select the Generate Schemas folder under the Generated Schema Files folder in the categories pane, and then select the BtsImportSpecifications template in the templates pane and click Open. When that dialog box closes, the Generate Schemas dialog box appears. A typical view of this is shown in Figure 3.4.

By default, the software to import XDR schemas is loaded when you install BizTalk Server, but not the software for DTDs or well-formed XML documents. This is easily remedied. Execute the InstallDTD.vbs script (for DTDs) or InstallWFX.vbs script (for well-formed XML) as needed to install these components. These scripts are found in the SDK\Utilities\Schema Generator folder under your BizTalk installation.

When you have the proper components loaded, you select the type of resource from the Document type list, browse to the resource itself, and then click OK in the Generate Schemas dialog box. Visual Studio will then generate a message schema and add it to the project.

To get an idea of the process, we'll import the XML document shown in Listing 3.1 (found in the code download as sample_proj.xml).

Figure 3.4 You can select existing resources to generate a new schema using the Generate Schemas dialog box.

Listing 3.1 Sample XML for Schema Import

<?xml version="1.0" encoding="utf-8" ?> 
<Project xmlns="http://project.xsd">
  <Name>Project 1</Name>
  <Start priority="3">2003-08-01</Start>
</Project>

Referring to Listing 3.1, you should take note of three items: First is a default namespace, denoted by the xmlns attribute on the Project element; second is a numeric type for the attribute priority; and third is an ISO 8601 format date we've inserted as the textual content of the Start element. How does the importer handle these? Keep this in mind as we examine the schema over the next few paragraphs.

When the schema is generated (sample_proj.xsd in the code download), select the <Schema> node and examine the value of the Target Namespace property in the Properties view. The default namespace in the document was taken for this value in the schema. This will likely differ from the naming convention Visual Studio uses by default when you create a new schema, but if you have taken the trouble to declare namespaces in your documents, you are either pointing to a schema, in which case you don't need to import the instance document, or you have a meaningful scheme for creating namespace URIs.

Now if you look at the value assigned to the Start element, which had textual content in the sample, you'll see that the schema generator recognized the possibility that we meant to enter a date and therefore assigned Start the XSD date type as its data type. The priority attribute was recognized as a numeric type and was assigned the smallest type consistent with the value found. Unsigned byte might not have been what we had in mind.