XML-Based Archiving

Purpose

XML-based data archiving has the same purpose as ADK-based archiving. Both are used to remove business complete data, that is data no longer needed in online business processes, from the database and to store the data in such a way that it is still accessible in the future. XML archiving is primarily used for XML archiving objects implemented by ABAP applications with XML interfaces. It is based on the following principles:

● A comprehensive use of standards

○ XML: Generally accepted markup language and exchange format for complex objects, especially business objects, which is supported by a wide range of tools; compared to ADK, this format is used to archive data objects as whole objects (not broken down into different records)

○ XML schema used to validate and describe the structures of XML documents

○ HTTP(s) used for secure communication between the application system and the XML data archiving service (XML DAS)

○ WebDAV used as a model for the hierarchical organization of archived data and as a general (not SAP-specific) protocol for connecting storage systems including archive systems

○ Java used as a platform-independent programming language, which is also widely used outside the SAP world

○ Java EE as a standard for the development of JAVA-based enterprise applications

● Data access is flexible and available even in the long term

○ Independent from the application system that generates the data (in case a system has a shorter life cycle than the data to be archived)

○ Independent from the application system technology (also relevant for JAVA applications)

○ Basis for cross-system data accesses and searches

● Central archiving service to minimize administration efforts for system landscapes with several components

Implementation Considerations

The XML DAS is a technology that was originally designed specifically for XML archiving objects. An example of an XML archiving object is CMX_XSTEPS used to archive Execution Steps (delivered with the business function EA-SCM of the SAP ECC Extensions).

In the future it will be possible to use XML DAS to create customer-specific archiving objects. However, this functionality has not yet been released.

Integration

XML-based archiving involves two logically separate systems, which are generally also physically separate: The application system whose database is to be cleaned up and which contains the data to be archived, and the archiving system in which XML DAS is running.

The application system contains another important component for XML archiving, called the XML DAS Connector. If you are working with an ABAP application system the XML DAS Connector for ABAP contains the XML Archive API, as well as the local archive administration, transaction SARA. The archiving programs of an XML archiving object use the XML Archive API instead of ADK. The XML Archive API communicates with XML DAS using HTTP(s).

XML DAS is part of the Application Server Java (AS Java). The meta data and the property indexes (used for value-based searches) are kept in the database of the AS Java, while the actual data that is archived is saved in a file system connected via the WebDAV protocol. Alternatively, you can also use a robust file system, which must be made available to all servers of the AS Java. The file system or rather, a part of it (folder or WebDAV collection) is used as an archive store for an archiving object via the XML DAS Administration. Because these and other settings are cross-application-system settings, the XML DAS Administration is part of the AS Java. Together with transaction SARA it is the most important tool for a data archiving administrator.

The following figure shows how all of these components fit together:

Figure 1: Relationship of the Different Components in XML-Based Archiving

¹ KM does not necessarily have to reside on the same AS Java system as XML DAS. It is also possible to have all AS Java functions and components reside on the same AS Java system.

² The diagram only shows the SLD connections that are relevant in a service-oriented archiving scenario.

³ Note that AS ABAP for mySAP Business Suite uses the XML DAS Java available in SAP NetWeaver 2004s.

Features

The XML DAS Connector for ABAP used by XML archiving objects contains the most important functions of XML DAS. It is very important that you understand the fundamental concepts and functions of XML DAS before you develop new archiving objects or start XML archiving sessions.

...

1. Components and use of the XML DAS hierarchy:

The data objects to be archived within an application either already exist as XML documents or they are read from the database via an XML interface. In addition you can archive the structure information of the XML documents in the form of one or more XML schemas together with the XML documents themselves. The same applies to XSL stylesheets. You can archive data objects of any format (to be transferred as type BIN).

○ Resources and collections

The general term for an archived data object in XML DAS is resource. Resources always reside inside collections. Collections contain other collections and subcollections, which together create the hierarchy. The concept that the XML DAS hierarchy comprises collections that contain resources is based on the WebDAV standard (RFC 2518). At the same time XML DAS uses the WebDAV protocol for the physical storage of archived data. This means that the collections and resources within the logical hierarchy are physically created in one or more WebDAV systems. The list of collections starting at the root of the XML DAS hierarchy and using the shortest path to a particular resource is the "address" of the resource: The list of collection names each followed by / is what we call the archive path. When you add the name of the resource to this path you have a URI. To be able to access an archived data object all you need is the URI.

A collection can also be addressed using a URI.

○ Properties and property indexes

You can also conduct value-based searches for resources. For this you need to assign properties to the resources. Whether or not the properties are derived from the XML contents or you use another type of attribution procedure, depends on the application. The concept of properties is also based on WebDAV. However, the following changes had to be made to adapt the WebDAV property concept to the fact that an XML archiving session usually processes many similar resources and to ensure more efficient searches.

- Properties are typized, meaning that name-value pairs contain values of a data type, such as VARCHAR (45), for character-like properties with property values that contain up to 45 characters.

- Properties are bundled in property indexes and have to be defined via the XML Archive API before they can be used.

- The application must specify the property index during archiving and value-based searches.

You can combine value-based searches with searching in specific collections (along an archive path).

○ Home collections and home paths

A part of the XML DAS hierarchy is reserved for each archiving object. This is done using home collections. Data objects from a specific application are archived only in or under the home collection of the archiving object they belong to. Also, the XML DAS Connector checks if read accesses are permitted: The archive path of a resource must begin with the home path of the archiving object, although it may have a path extension, meaning that it can reside further down in the hierarchy than the home collection.

2. Main functions of XML DAS

The XML archive API provides an application with the following XML DAS main functions:

○ MKCOL – Create collection

○ PUT – Write resource to archive

○ CHECK – Check resources for XML well-formedness or validate

○ HEAD – Existence check for resources or collections

○ GET – Read resource from archive

○ LIST – Output resource list (hierarchical search)

○ SELECT – Select resource (value-based search)

○ INFO – Information about XML DAS and about an archive path

○ FREEZE – Freeze collection

○ DELETE – Delete resources or collections

○ VERIFY_INTEGRITY – Asynchronously check the integrity of resources in one or several collections

○ INDEX_…

CREATE – Create property index

DESCRIBE – Describe properties of a property index

DROP – Delete property index

INSERT – Insert properties of a resource into an index

GET – Read properties of a resource from an index

In addition the XML Archive API offers methods that only affect the XML DAS Connector. For more details see the ABAP system documentation for class CL_XADK_API.

3. XML DAS Administration

○ Archive Store

To store the contents of resource and the collections on appropriate archive media, the XML DAS hierarchy is mapped to storage systems that are also hierarchically organized. The administrator can use storage system with WebDAV interfaces (called WebDAV systems) or file systems that must be reachable by all instances of the AS Java (for example several instances of an AS Java cluster).

More specifically, before you can archive, an archive store must be assigned to every home collection of an archiving object. Alternatively, you can also assign an archive store to those application collections that are direct or indirect subcollections of the home collection, as long as no archive store has already been assigned to any of their parent collections. This archive store is the link to an actual folder or an actual WebDAV collection. Before you can assign an archive store you must define it. To do so use the XML DAS Administration, which is available as part of the SAP NetWeaver Administrator (NWA) under:

Operations Management → Data and Database Management → Data Archiving

You can also use the XML DAS Administration to test the availability of archive stores, list all home paths known to the XML DAS and pack resources.

The XML DAS Administration contains more detailed documentation about the functions it offers. For information about the different Customizing steps in the application system and the configuration of the XML DAS see Customizing: XML Archiving and Administration of the XML Data Archiving Service.

○ Pack

When you pack the resources of a collection the resources are compressed and written to an openly documented file format, see Archiving Format for Packed Resources. Then the resources are physically deleted, although logically they remain unchanged in the XML DAS hierarchy. This means that accesses to packed and unpacked resources only differ in terms of processing time. Although packed resources may mean increased processing times during read operations (due to block accesses and temporary decompression), they considerably reduce the number of files or collections in your system (which means fewer management entries, such as I-Nodes). If necessary you can also permanently unpack packed resources using the XML DAS Administration.

Restrictions

● Archive Information System (SAP AS):

XML archiving does not make use of SAP AS. This means that it is not possible to create infostructures or use Archive Explorer to search for resources in the context of XML-based archiving.

● XML DAS Administration in NWA

For SAP NetWeaver 2007 SPS0 the XML DAS Administration can only be used with NWA in local mode.

Example

The example XML archiving object is BC_SBOOK_X.