- Framing the Problem
- LOB or Compose?
- Composition Techniques: Common Concepts
- Composition Techniques: Examples
- Shredding
- Implementation Concepts
- Conclusion
This chapter is from the book
This chapter is from the book
This chapter is from the book
Composition Techniques: Examples
In this section, we consider examples of composition techniques that illustrate the available variety. We divide composition techniques into three fundamental camps: default mappings, extended SQL, and annotated schemas. We illustrate each camp with a specific example—but it should be emphasized that these are just three current examples from a much wider selection, offered by commercial vendors, academic research and freeware.
Default Mapping
In the simplest kind of composition technique, a default mapping,1 the structure of the database is exposed directly, as a fixed form of XML document. This is an implicit composition technique, in which the user has little or no control over the generated XML. Listing 6.7 shows an example of a default mapping.
Listing 6.7 An Example of a Default Mapping from a Table to XML
table customer
name acctnum home
_____________________________________________________________________
Albert Ng 012ab3f 123 Main St., Anytown OH 99999
Francis Smith 032cf5d 42 1/2 Seneca, Whositsville, NH 88888
...
<customerSet>
<customer>
<name>Albert Ng</name>
<acctnum>012ab3f</acctnum>
<home>123 Main St., Anytown OH 99999</home>
</customer>
<customer>
<name>Francis Smith</name>
<acctnum>032cf5d</acctnum>
<home>42 1/2 Seneca, Whositsville, NH 88888</home>
</customer>
...
In a default mapping, the names of the tables and columns become (with escaping of reserved characters) the names of elements and/or attributes. These elements and attributes are arranged in a simple structure—usually some form of the three-level hierarchy you see here, or a simple variation that uses attributes instead of elements for column values. Datatypes follow some fixed conversion (e.g., SQL VARCHAR to XML String), and null values in the database either become missing elements or elements with missing values.
One advantage of default mapping is that it supports update very easily: Operations to update element values and add or delete elements have a very direct translation into update operations on the underlying database. However, default mapping does not by itself support many applications, since applications often need XML data in certain specific formats that don't happen to correspond to default mappings from the database. But if a default mapping is combined with a general-purpose query or transformation language such as XQuery, then XQuery may be used to “reformat” the implicitly mapped XML document into the desired XML form. Listing 6.8 illustrates a simple example of using XQuery to transform the document from Listing 6.7. This is a very simple example; a more realistic example would probably involve combining data from multiple tables, and so forth.
The combination of default mapping with XQuery is quite powerful in that it re-uses the same transformation capabilities that are supported in XQuery already. It is also generally efficient when the application needs to select from, query, or transform the resulting XML document, which is accomplished by composing the two operations (one query to generate the XML document, the other to select from or transform it). Since XQuery is defined to support query composition, a general XQuery implementation will be able to support this scenario.
Listing 6.8 Using XQuery to Transform a Default-Mapped XML Document
<ActiveCustomers xmlns="http://www.example.com/custlist"> {
for $i in doc("customer")/CustomerSet/Customer
let $firstname := substring-before($i/name, " ")
$lastname := substring-after($i/name, " ")
return
<customer acct="{$i/acctnum}">
<name type="full">
<first>{$firstname}</first>
<last>{$lastname}</last>
</name>
</customer>
}
</ActiveCustomers>
<ActiveCustomers xmlns="http://www.example.com/custlist">
<customer acct="012ab3f">
<name type="full">
<first>Albert</first>
<last>Ng</last>
</name>
</customer>
<customer acct="032cf5d">
<name type="full">
<first>Francis</first>
<last>Smith</last>
</name>
</customer>
...
</ActiveCustomers>
One thing to keep in mind is that the advantage of default mapping with respect to update applications is lost when combining default mapping and XQuery. While it is easy to update the directly mapped document itself, it is not usually possible to update an XML document that is the result of a query or transformation, since it suffers from the same “update through views” problem discussed in the previous section.
Extended SQL
Default mapping plus XQuery is one way to create a complex XML structure from relational data. Another is to add the capability to process and generate XML to SQL. An extended SQL language could be directly implemented by a database vendor, or it could be implemented in a layer on top of the database.
This is the approach taken with SQL/XML, which, as of this writing, is in draft form before ISO as an extension to the SQL query language [SQLXML]. SQL/XML could evolve to become a full solution to emitting, querying, transforming, and even updating XML in its own right. We cannot fully cover SQL/XML here, but we can summarize the major points with respect to mapping between XML and relational data.
How does SQL/XML generate XML? SQL/XML consists of a definition of a native XML datatype, as well as definitions of implicit and explicit ways of generating XML from relational data. The native XML datatype allows XML to be manipulated by the SQL query language, which means that XML documents can be treated as relational values (values of columns or procedure parameters, etc.). The language is intended to support both composed and LOB implementations of the native XML datatype.
SQL/XML also defines an implicit default mapping from relational data to XML. The standard defines how SQL identifiers are to be escaped to create legal XML identifiers and vice versa, as well as extensive conversion rules for standard SQL datatypes. The mapping can be applied to individual values, tables, and entire catalogs. Within SQL/XML, the default mapping is used to define the manner in which SQL datatypes are translated to XML when they appear in XML-generating operators.
Finally, SQL/XML defines a set of XML-generating operators that can be used in an SQL expression: XMLELEMENT, XMLFOREST, and XMLGEN. Each of these allows the creation of an XML element (or fixed sequence of elements) with substitution of values from relational data. For example, the XMLELEMENT operator creates an XML element with content mapped from its arguments:
SELECT e.id,
XMLELEMENT ( NAME "Emp",
XMLATTRIBUTES ( e.name AS "name" ),
e.hire,
e.dept ) AS "result"
FROM employees e
WHERE ... ;
This SQL statement generates a table that has two columns, one containing e.id, one containing an XML element Emp, with the indicated contents supplied by values from the employee table. The default mapping rules are used to determine how the data values are translated to XML and how the nested elements are named.
Interestingly, these generator functions lack features that directly support the generation of hierarchy through hierarchical joins. Although they allow you to create XML hierarchy, the hierarchy is all fixed and is populated directly from a relational set of values. Instead, the effect of generation of hierarchy through joins is piggybacked on existing object-relational features of SQL, in particular by applying the direct-mapping rules to nested-relational structures, as shown in Listing 6.9:
Listing 6.9 Generating Hierarchical Structure with Object-Relational Features of SQL/XML
SELECT XMLELEMENT( NAME "Department",
d.deptno,
d.dname
CAST(MULTISET(
select e.empno, e.ename
from emp e
where e.deptno = d.deptno)
AS emplist)))
AS deptxml
FROM dept d
WHERE d.deptno = 100;
<Department>
<deptno>100</deptno>
<dname>Sports</dname>
<emplist>
<element>
<empno>200</empno>
<ename>John</ename>
</element>
<element>
<ename>Jack</ename>
</element>
</emplist>
</Department>
SQL/XML does provide a function, XMLAGG, which directly supports generation of structure through grouping. It is used in collaboration with the SQL GROUP BY operation as in the example shown in Listing 6.10:
Listing 6.10 Generating Hierarchical Structure through Grouping in SQL/XML
SELECT XMLELEMENT
( NAME "department"
XMLELEMENT ( NAME "name" e.dept )
XMLELEMENT ( NAME "employees"
XMLAGG (
XMLELEMENT ( NAME "emp", e.lname )
)
)
) AS "dept_list"
FROM employees e
GROUP BY e.dept
The XMLAGG function acts like an aggregation operation, generating a list of XML content (in this case a single nested element): one item for each row in the group. The output of the query of Listing 6.10 might look like the example of Listing 6.11:
Listing 6.11 An Example XMLAGG Result
<department>
<name>Physics</name>
<employees>
<emp>Smith</emp>
<emp>Thompson</emp>
<emp>Johnson</emp>
</employees>
</department>
<department>
<name>Computer Science</name>
<employees>
<emp>Hinson</emp>
...
Extended SQL composition techniques are well-suited for supporting selection and transformation over XML documents. They usually cannot support update applications, unless the SQL expressions are very limited.
Annotated XML Template
Another way to express the construction of XML from relational data is to supply an XML template that is “decorated” with expressions that indicate how to compute corresponding XML. The template could be in the form of the XML document itself, or it could be an annotated DTD or schema. To illustrate this mapping technique, we use the Annotated XSD Schema language defined by Microsoft in Microsoft SQLXML 3.0 [SQLXML-MS]
To begin with, in MS SQLXML a completely unannotated schema has a default interpretation: Every complex element is presumed to correspond to a table (whose name is the same as the name of the element), and every simple-valued element or attribute is presumed to correspond to a column of that table (with the same name). Annotations are required to override this assumption, and to specify the relationships between nested complex elements.
Listing 6.12 An Annotated XML Schema
<xs:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
<xs:element name="Customer" type="CustomerType"
sql:relation="Customers"/>
<xs:complexType name="CustomerType">
<xs:sequence>
<xs:element name="Order" maxOccurs="unbounded"
sql:relation="Orders">
<xs:annotation>
<xs:appinfo>
<sql:relationship
parent="Customers" parent-key="CustomerID"
child="Orders" child-key="CustomerID"/>
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:attribute name="OrderID" type="xs:integer"
sql:field="ID"/>
<xs:attribute name="Amount" type="xs:integer"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="CustomerID" type="xs:string" />
<xs:attribute name="ContactName" type="xs:string"
sql:field="Name"/>
</xs:complexType>
</xs:schema>
Listing 6.12 shows an example, slightly modified from the Microsoft documentation. In this example, the outermost element, Customer, is populated from a table named Customers (indicated by the sql:relation annotation). Similarly, the nested element Order is populated from the table named Orders. The relationship between them is expressed by the sql:relationship annotation, which tells us that the Customers and Orders are joined through the indicated values.2 Multi-attribute joins are indicated by specifying the appropriate attribute names, in order, in the parent-key and child-key fields. The various simple fields (all of which are attributes in this example) either are mapped implicitly from table columns (e.g., CustomerID), or indicate, with the sql:field annotation, the name of the column from which they are derived (e.g., ContactName from Name).
Note that there are no SQL expressions in this technique. Instead, only names of existing tables and columns are allowed. A very simple variation on this mapping technique would be to allow the sql:relation annotation to contain an arbitrary SQL statement, or the sql:field annotation to contain an SQL expression. This would make it possible to generate more flexible XML documents from a given relational database.
But the restriction to table names and columns has a very powerful advantage: It enables the use of this mapping “in reverse,” to support both update operations and shredding of XML documents. If arbitrary expressions were allowed, it would not be possible to support either update or shredding with the same annotated schema. The same trade-off exists in other composition techniques, which either likewise do not allow arbitrary expressions, or in some cases, come in two varieties, where the user chooses which. For example, Microsoft SQLXML handles this by supporting two mapping techniques: annotated XSD to support update and shredding, and an extended SQL notation (the FOR XML clause) for arbitrarily computed XML.
We note below a few more interesting features of the annotated XSD before moving on to our next category:
-
The example in Listing 6.12 showed a generation of hierarchy through joins with a single join between a parent table, Customers, and a child table, Orders. You can also express a parent-child relationship that requires a chain of joins. For example if we wanted XML that contained information about customers and the products they had purchased (but not specifically about their individual orders), that might require a join from Customers to Orders to Products to generate the child elements. This is accomplished by inserting multiple sql:relationship annotations, in order, one for each required join.
-
While the XSD annotation does not permit general conditional expressions, it does allow one limited form of condition. Annotations sql:limit-field and sql:limit-value can be added to any element or attribute that specifies the sql:relation annotation, to indicate that only those rows of the relation that satisfy limit-field=limit-value are to be used. This is useful for selecting rows of a certain type; for instance, in the example shown in Listing 6.13, it selects billing addresses from a generic address table.
Listing 6.13 Generating XML Elements in Certain Conditions in MS SQLXML
<xs:element name="BillTo" type="xs:string" sql:relation="Addresses" sql:field="StreetAddress" sql:limit-field="AddressType" sql:limit-value="billing"> <xs:annotation> <xs:appinfo> <sql:relationship parent="Customers" parent-key="CustomerID" child="Addresses" child-key="CustomerID" /> </xs:appinfo> </xs:annotation> </xs:element> -
If the XML Schema contains xs:any, this can be annotated to indicate an “overflow” column where any extra data encountered during shredding will be stored (in string XML form). The system won't do anything with the information in this field, but it will at least be reproducible when the document is recomposed.
The chain of joins and limit-field features are examples of carefully pushing the limits of expressiveness while still maintaining the ability to update.
Additional Mapping Languages
The previous examples added extensions either to SQL or to existing XML standards to bridge the gap between XML and relational data. Another alternative is to express composition in a new, different, language, whose entire goal is to take relational data as input and generate XML data as output, or vice versa. This approach was initially quite popular, but it has waned because it adds an extra level of complexity—a new language to learn—over the other approaches, without adding much in the way of convenience or expressive power.
Additional languages are still in use by some vendors. They may also be used as an internal compiled form for one of the other mechanisms.