Authorization Check in Archiving Programs

Authorization for archiving programs is controlled using authorization object S_ARCHIVE. ADK checks this authorization when one of the following function modules is called in a program:

• ARCHIVE_OPEN_FOR_WRITE
• ARCHIVE_OPEN_FOR_DELETE
• ARCHIVE_OPEN_FOR_READ
• ARCHIVE_OPEN_FOR_MOVE

The following authorizations can be assigned per archiving object and application (for example, FI or BC):

• 01: All authorizations

• Write, read, and reload archive files
• Execute delete programs
• Change mode in archive management

• 02: Change mode in archive management
• 03: Read archive files; display mode in archive management

Additional application-specific authorization checks may be made for database accesses.

Archive :Authorization Check

Definition

The authorization to carry out various archiving object programs is checked by the authorization object S_ARCHIVE. The Archive Development Kit (ADK) performs the check when an archive file is opened for one of the following actions:

● Write

● Delete

● Read

● Restore

Use

The following authorizations can be given per archiving object and solution (such as SAP ERP Financials or SAP ERP Human Capital Management):

● Everything is allowed

Write, read, and reload archives; execute delete programs; change mode in archive management (notes)

● Change mode in archive management

Maintain notes

● Read and analyze archives and display mode in archive management

There may also be additional access authorization checks for specific application components

XML Archiving Object

Definition

The XML archiving object has the same function in XML-based archiving as the archiving object in ADK-based archiving. It determines which data is archived and how, and facilitates the execution of the different programs.

Structure

As in ADK-based archiving, XML archiving objects are defined in transaction AOBJ. An XML archiving object can have the following programs: Preprocessing, Write, Delete, Read, Index and Postprocessing. The reloading of data is not supported in XML-based archiving, and the status Reloaded does not exist here.

You can recognize an XML archiving object by the indicator XML Archiving Object, which must be set in the detail screen of transaction AOBJ.

XML-Based Archiving

Purpose

XML-based data archiving has the same purpose as ADK-based archiving. Both are used to remove business complete data, that is data no longer needed in online business processes, from the database and to store the data in such a way that it is still accessible in the future. XML archiving is primarily used for XML archiving objects implemented by ABAP applications with XML interfaces. It is based on the following principles:

● A comprehensive use of standards

○ XML: Generally accepted markup language and exchange format for complex objects, especially business objects, which is supported by a wide range of tools; compared to ADK, this format is used to archive data objects as whole objects (not broken down into different records)

○ XML schema used to validate and describe the structures of XML documents

○ HTTP(s) used for secure communication between the application system and the XML data archiving service (XML DAS)

○ WebDAV used as a model for the hierarchical organization of archived data and as a general (not SAP-specific) protocol for connecting storage systems including archive systems

○ Java used as a platform-independent programming language, which is also widely used outside the SAP world

○ Java EE as a standard for the development of JAVA-based enterprise applications

● Data access is flexible and available even in the long term

○ Independent from the application system that generates the data (in case a system has a shorter life cycle than the data to be archived)

○ Independent from the application system technology (also relevant for JAVA applications)

○ Basis for cross-system data accesses and searches

● Central archiving service to minimize administration efforts for system landscapes with several components

Implementation Considerations

The XML DAS is a technology that was originally designed specifically for XML archiving objects. An example of an XML archiving object is CMX_XSTEPS used to archive Execution Steps (delivered with the business function EA-SCM of the SAP ECC Extensions).

In the future it will be possible to use XML DAS to create customer-specific archiving objects. However, this functionality has not yet been released.

Integration

XML-based archiving involves two logically separate systems, which are generally also physically separate: The application system whose database is to be cleaned up and which contains the data to be archived, and the archiving system in which XML DAS is running.

The application system contains another important component for XML archiving, called the XML DAS Connector. If you are working with an ABAP application system the XML DAS Connector for ABAP contains the XML Archive API, as well as the local archive administration, transaction SARA. The archiving programs of an XML archiving object use the XML Archive API instead of ADK. The XML Archive API communicates with XML DAS using HTTP(s).

XML DAS is part of the Application Server Java (AS Java). The meta data and the property indexes (used for value-based searches) are kept in the database of the AS Java, while the actual data that is archived is saved in a file system connected via the WebDAV protocol. Alternatively, you can also use a robust file system, which must be made available to all servers of the AS Java. The file system or rather, a part of it (folder or WebDAV collection) is used as an archive store for an archiving object via the XML DAS Administration. Because these and other settings are cross-application-system settings, the XML DAS Administration is part of the AS Java. Together with transaction SARA it is the most important tool for a data archiving administrator.

The following figure shows how all of these components fit together:

Figure 1: Relationship of the Different Components in XML-Based Archiving

¹ KM does not necessarily have to reside on the same AS Java system as XML DAS. It is also possible to have all AS Java functions and components reside on the same AS Java system.

² The diagram only shows the SLD connections that are relevant in a service-oriented archiving scenario.

³ Note that AS ABAP for mySAP Business Suite uses the XML DAS Java available in SAP NetWeaver 2004s.

Features

The XML DAS Connector for ABAP used by XML archiving objects contains the most important functions of XML DAS. It is very important that you understand the fundamental concepts and functions of XML DAS before you develop new archiving objects or start XML archiving sessions.

...

1. Components and use of the XML DAS hierarchy:

The data objects to be archived within an application either already exist as XML documents or they are read from the database via an XML interface. In addition you can archive the structure information of the XML documents in the form of one or more XML schemas together with the XML documents themselves. The same applies to XSL stylesheets. You can archive data objects of any format (to be transferred as type BIN).

○ Resources and collections

The general term for an archived data object in XML DAS is resource. Resources always reside inside collections. Collections contain other collections and subcollections, which together create the hierarchy. The concept that the XML DAS hierarchy comprises collections that contain resources is based on the WebDAV standard (RFC 2518). At the same time XML DAS uses the WebDAV protocol for the physical storage of archived data. This means that the collections and resources within the logical hierarchy are physically created in one or more WebDAV systems. The list of collections starting at the root of the XML DAS hierarchy and using the shortest path to a particular resource is the "address" of the resource: The list of collection names each followed by / is what we call the archive path. When you add the name of the resource to this path you have a URI. To be able to access an archived data object all you need is the URI.

A collection can also be addressed using a URI.

○ Properties and property indexes

You can also conduct value-based searches for resources. For this you need to assign properties to the resources. Whether or not the properties are derived from the XML contents or you use another type of attribution procedure, depends on the application. The concept of properties is also based on WebDAV. However, the following changes had to be made to adapt the WebDAV property concept to the fact that an XML archiving session usually processes many similar resources and to ensure more efficient searches.

- Properties are typized, meaning that name-value pairs contain values of a data type, such as VARCHAR (45), for character-like properties with property values that contain up to 45 characters.

- Properties are bundled in property indexes and have to be defined via the XML Archive API before they can be used.

- The application must specify the property index during archiving and value-based searches.

You can combine value-based searches with searching in specific collections (along an archive path).

○ Home collections and home paths

A part of the XML DAS hierarchy is reserved for each archiving object. This is done using home collections. Data objects from a specific application are archived only in or under the home collection of the archiving object they belong to. Also, the XML DAS Connector checks if read accesses are permitted: The archive path of a resource must begin with the home path of the archiving object, although it may have a path extension, meaning that it can reside further down in the hierarchy than the home collection.

2. Main functions of XML DAS

The XML archive API provides an application with the following XML DAS main functions:

○ MKCOL – Create collection

○ PUT – Write resource to archive

○ CHECK – Check resources for XML well-formedness or validate

○ HEAD – Existence check for resources or collections

○ GET – Read resource from archive

○ LIST – Output resource list (hierarchical search)

○ SELECT – Select resource (value-based search)

○ INFO – Information about XML DAS and about an archive path

○ FREEZE – Freeze collection

○ DELETE – Delete resources or collections

○ VERIFY_INTEGRITY – Asynchronously check the integrity of resources in one or several collections

○ INDEX_…

CREATE – Create property index

DESCRIBE – Describe properties of a property index

DROP – Delete property index

INSERT – Insert properties of a resource into an index

GET – Read properties of a resource from an index

In addition the XML Archive API offers methods that only affect the XML DAS Connector. For more details see the ABAP system documentation for class CL_XADK_API.

3. XML DAS Administration

○ Archive Store

To store the contents of resource and the collections on appropriate archive media, the XML DAS hierarchy is mapped to storage systems that are also hierarchically organized. The administrator can use storage system with WebDAV interfaces (called WebDAV systems) or file systems that must be reachable by all instances of the AS Java (for example several instances of an AS Java cluster).

More specifically, before you can archive, an archive store must be assigned to every home collection of an archiving object. Alternatively, you can also assign an archive store to those application collections that are direct or indirect subcollections of the home collection, as long as no archive store has already been assigned to any of their parent collections. This archive store is the link to an actual folder or an actual WebDAV collection. Before you can assign an archive store you must define it. To do so use the XML DAS Administration, which is available as part of the SAP NetWeaver Administrator (NWA) under:

Operations Management → Data and Database Management → Data Archiving

You can also use the XML DAS Administration to test the availability of archive stores, list all home paths known to the XML DAS and pack resources.

The XML DAS Administration contains more detailed documentation about the functions it offers. For information about the different Customizing steps in the application system and the configuration of the XML DAS see Customizing: XML Archiving and Administration of the XML Data Archiving Service.

○ Pack

When you pack the resources of a collection the resources are compressed and written to an openly documented file format, see Archiving Format for Packed Resources. Then the resources are physically deleted, although logically they remain unchanged in the XML DAS hierarchy. This means that accesses to packed and unpacked resources only differ in terms of processing time. Although packed resources may mean increased processing times during read operations (due to block accesses and temporary decompression), they considerably reduce the number of files or collections in your system (which means fewer management entries, such as I-Nodes). If necessary you can also permanently unpack packed resources using the XML DAS Administration.

Restrictions

● Archive Information System (SAP AS):

XML archiving does not make use of SAP AS. This means that it is not possible to create infostructures or use Archive Explorer to search for resources in the context of XML-based archiving.

● XML DAS Administration in NWA

For SAP NetWeaver 2007 SPS0 the XML DAS Administration can only be used with NWA in local mode.

Example

The example XML archiving object is BC_SBOOK_X.

Archiving Object

Definition

The archiving object is a central component of SAP Data Archiving. The archiving object specifies precisely which data is archived and how. It describes which database objects must be handled together as a single business object. and interprets the data irrespective of the technical specifications at the time of archiving (such as release and hardware).

Use

An archiving object has a name of up to ten characters in length. You need this name to carry out the archiving programs in archive administration (transaction SARA). However, the function for calling data archiving is often integrated in the application menu, where the archiving object name is set as the default so that you do not need to enter it yourself.

Structure

The following programs must (or can) be assigned to an archiving object. (Archiving objects are defined using transaction AOBJ) The SAP System contains programs (some of which are optional) for the following actions:

· Preprocessing (optional)

Some archiving objects require a preprocessing program that prepares the data for archiving. This preprocessing program marks data to be archived, but it does not delete any data from the database.

Preprocessing programs must always be scheduled manually and are run from Archive Administration.

· Write

This program creates archive files and writes data to them. At this point, however, no data is being deleted from the database.

You can specify in archiving object-specific Customizing whether the next phase (delete) is to take place automatically after the archive files have been created. Delete jobs can also be event-triggered. To do this, you set up the trigger event in archiving object-specific Customizing.

· Delete

This function can entail several activities. The activities are always dependent on the existing archive files.

Normally, the data is deleted from the data base. However, in some case, the archived data in the database may only have a delete indicator.

In archiving object-specific Customizing, you can specify that archive files, after successful processing, are to be transferred to an external storage system using the SAP Content Management Infrastructure (which contains the ArchiveLink/CSM interface).

· Postprocessing (optional)

This function is usually carried out after deletion has taken place. It is not available for all archiving objects. If the data has not yet been deleted from the database by the delete program, it is deleted by the postprocessing program. Depending on the archiving object used, postprocessing can also fulfill other tasks. For more information, refer to the documentation on the specific archiving object.

· Reload archive (optional)

You can reload archived data from the archive files into the database using this function. It is not available for all archiving objects. To access this function, choose Goto ® Reload.

· Index (optional)

This function builds (or deletes) an index that allows individual access. It is not included in every archiving object.

ADK-Based Data Archiving

Technical System Landscape

The technological foundation of data archiving is located in SAP Web Application Server and is available in two different variations: ADK-based and XML-based archiving. Most archiving objects are based on the Archive Development Kit (ADK). XML-based archiving is used for certain archiving objects. ADK and the XML archive API provide basic services and functions for developing and executing archiving programs. In both cases you use archive administration to set up and execute archiving.

Tasks

The archiving of application data has several phases, outlined below. Phase 3 "Accessing Archived Data" is not part of the actual archiving process, but is important later on when using the archived data and is therefore covered in this section.

The data archiving administrator role is available in SAP systems to help you execute data archiving. The technical name is SAP_BC_CCM_DATA_ARCHIVING. To call it, use role maintenance (transaction PFCG). The role provides the system or data archiving administrator with direct access to all information, tools, and systems that are needed for archiving data.

1. Analyzing

   You determine the largest or fastest-growing database tables and the archiving object for archiving the table contents.

   For more information, see Analyzing.

2. Executing and Monitoring

   1. Make system settings for data archiving

   2. Schedule, control, and monitor archiving jobs

   For more information, see Executing and Monitoring.

3. Accessing Archived Data

   You can access archived data and set up archive access.

   For more information, see Accessing Archived Data.

Additional Information

• Introduction to Data Archiving

  This documentation contains more information about the technology and processes used in data archiving.

• SAP Service Marketplace, Quick Link /data-archiving

  The SAP data archiving website provides extensive information about data archiving, for example guides, white papers, presentations, and so on.

• Data Management Guide

  This guide contains important and current information about handling large volumes of data in SAP systems. As well as data archiving, it also explains how to avoid, delete, and compress data.

  See SAP Service Marketplace at service.sap.com/data-archiving Media Library Literature & Brochures .

Data Archiving

Generally large amounts of database data is produced in the production system of a business application, which often times is reflected in a deterioration of system performance and an increased use of resources. Costs can also be negatively affected by large data volumes, since resources and efforts spent on database administration go up as a result.

Therefore, data from closed business processes, meaning data that is no longer needed in every day business processes, should be moved out of the database, in order to gain a reduction in costs, and improve performance and system availability. Simply deleting this data is often not a valid option, due to legal data retention requirements. This means that the data has to be removed from the database and stored in such a way that it is always accessible for reading if necessary. This is the task of SAP Data Archiving, which allows you to remove data from the database and store it in a consistent and secure manner. The archived data is stored in a file system and from there can be moved to other, more cost efficient storage media.

Note

Data archiving is an administrative task that must be performed regularly, particularly if you are facing fast growing data volumes. How often you archive largely depends on the growth rate of your database tables.

End of the note.

The technological foundation of data archiving is located in the SAP NetWeaver Application Server and is available in two different variations:

• ADK-based Archiving - ADK-based archiving is used only by ABAP applications that employ ABAP archiving objects in their data archiving projects.

• XML-based Archiving - XML-based archiving is employed for for archiving sets used by JAVA applications.

Introduction to Data Archiving

Data Archiving – a service provided by SAP NetWeaver – removes mass data that the system no longer needs online, but which must still be accessible at a later date if required, from the database. The following graphic illustrates the archiving process: Archiving objects are used to write documents to archive files, which can be stored on other media.

Data in the database can only be archived using archiving objects, which describe the data structure and context.

Financial Accounting documents are archived using the archiving object FI_DOCUMNT. It includes the document header, company code-dependent postings, change documents, SAPscript texts, and other elements.

In addition to the aforementioned technology SAP also offers XML-based archiving, as of SAP Web Application Server 6.40. With this new procedure XML archiving objects are used to write data in the form of resources either to a file system or directly to a WebDAV system, which takes on the role of the storage system shown in the figure above.

Integration

For the majority of the archiving objects the SAP Data Archiving concept is based on the Archive Development Kit (ADK). As of SAP Web AS 6.40 SAP offers an additional technology especially designed for XML-based archiving. This new technology comprises the XML Data Archiving Service (XML DAS) and the XML DAS Connector. Both ADK-based and XML-based archiving are carried out with the help of Archive Administration, transaction SARA.

For some applications it is possible to start archiving directly from the application menu. In these cases the application-specific parameters, such as the archiving object or the archiving program appear as default values.

Archiving objects for each application component are predefined in the system. Their structures are described in the application-specific sections.

Features

The archiving procedure is divided into three main steps:

· Creation of archive files

In the write phase the data to be archived is written sequentially into newly created archive files.

· Delete from the database

The delete program reads the data from the archive files and then deletes it from the database.

· Storage of archive files

The newly created archive files can then be moved to a storage system or copied to a tape. The removal to an external storage system can be triggered manually or automatically. It is also possible to store the data before the delete phase.

In XML-based archiving this phase does not exist as a separate step. The data can be written directly to a WebDAV system.

The archiving programs are generally scheduled in the background. However, they can also run in online mode.

The following documentation describes ADK-based archiving. It is important that you understand the basic archiving concepts described in this documentation before you begin with XML-based archiving. The differences between the two technologies and processes are described under XML-based Archiving.

Data Archiving

Data Archiving removes from the database application data from closed business transactions that are no longer relevant for the operational business. The archived data is stored in archive files that can be accessed by the system in read-only mode.

SAP Data Archiving

Introduction to Enterprise Data Archiving

Currently, a large number of enterprises use SAP R/3 as a platform for integration of business processes. The continuous usage of SAP results in huge amounts of enterprise data, which is stored in SAP R/3. With passage of time, the new and updated data is entered into the system while the old data still resides in the SAP enterprise system.

Since some of the old data is critical, it cannot be deleted. The difficulty is keeping the data you want, and deleting the data you do not want. Hence, a SAP database keeps on expanding rapidly and enterprise systems, which have limited data retention abilities for a few years, suffer from problems such as data overflow, longer transaction processing times, and performance degradation.

The solution of this problem has led to the concept of Data Archiving in SAP. Data Archiving removes out-of-date data from the SAP database that the R/3 system does not need online, but can be retrieved on a later date, if required. This data is known as archived data and is stored at an offline location. Data Archiving not only consistently removes data from the database but also ensures data availability for future business requirements.

One rule of thumb is that in a typical SAP enterprise system, the ratio of data required to be online and instantly accessible to old data, which could be archived, and stored offline is 1:6. For example, if an enterprise has 2100 GB of SAP database, the online data, which is frequently used by SAP users will be 300 MB and the rest (1800 MB) will be scarcely used and hence can be archived.

Data Archiving – Features

• It provides a protection layer to the SAP database and resolves underperformance problems caused by huge volumes of data. It is important that SAP users should keep only minimal data to efficiently work with database and servers. Data archiving ensures that the SAP database contains only relevant and up-to-date data that meet your requirements.

• Data archiving uses hardware components such as hard disks and memory. For efficient data archiving, minimum number of disks and disk space should be used.

• It also reduces the system maintenance costs associated with the SAP database. In the SAP database there are various procedures such as, data backup, data recovery, and data upgrade.

• SAP data archiving complies with statutory data retention rules that are common and well-proven techniques.

SAP data archiving can be implemented in two ways. In the next section both options will be discussed in detail.