Best Practices for Upgrading to ODS51 on Clients and Servers

Jon Champlin has been with IBM for 14 years and in the Messaging and Collaboration division (formerly Iris Associates) for 9 years, and he is currently a member of the Domino Database development team. Jon was assigned to the IBM Notes/Domino 8.x deployment, and helped to resolve discovered issues



Learn everything you need to know about ODS — what it is, why it’s important to run the most recent version, how to find out what version you’re currently running, and how to upgrade your databases to the latest ODS level on both servers and clients. Clear up some common misconceptions about ODS and get best practices for upgrading to ODS51 in releases 8.0 and higher.


You may have seen or heard references to a specific “ODS level” with regards to some new feature or function that has been introduced in IBM Lotus Notes and Domino. Many administrators and developers don’t know exactly what ODS represents, why they should care what version they’re running, or how to change it. That’s about to change. I’ll explain everything you need to know about ODS levels, focusing especially on upgrading to ODS51 in releases 8.0 and higher, and explaining how to get best results when enabling the Domino Attachment Object Service (DAOS) on the same server.
WhatODS Is
When people refer to the ODS level of a database, they are referring to the database’s “on disk structure” — the format used to store the internal structures the database maintains when those structures are written out to disk.Throughout this discussion, I’ll be focusing on the most recent ODS level, ODS51, which is the 51st revision of the original ODS format developed for Notes and Domino.
In Notes and Domino, all new releases are completely backward compatible with older ODS levels. For example, you can run a Notes 8.5 client and still be able to fully access a local database that was created by a Notes R5 client using the release 5.0 ODS level.
The table below lists the latest ODS level available for each Notes/Domino release from 2.0 through 8.5. You might see some of these levels in your own Notes and Domino deployments.
Figure 1 ODS levels by Notes/Domino release
Notes/Domino
Release Level
Latest
ODS Level Available
Release
2.0
ODS16
Release
3.0
ODS17
Release
4.0
ODS20
Release
5.0
ODS41
Release
6.0
ODS43
Release
8.0
ODS48
Release
8.5
ODS51
As the table indicates, ODS levels are upgraded only on major releases; they are not part of maintenance releases. The table also shows that not all revisions of the ODS make it into a public release of the product. For example, in Notes/Domino 8.0, the latest ODS version was ODS48, and in 8.5, the latest level was ODS51. ODS levels 49 and 50 were internal revisions during the 8.5 development process.
Prior to Notes/Domino 8.0, databases are automatically converted to the latest ODS level the first time a copy-style compaction is run on the database. Starting with Notes/Domino 8.0, that changes — the ODS is only upgraded by the compact task when a notes.ini parameter (CREATE_R8_DATABASES=1) is set. This change gives administrators the flexibility to decide when to upgrade databases to the latest ODS version. They can choose to continue running copy-style compaction on their databases while leaving the databases at an ODS level prior to the latest version. In Notes/Domino 8.5, the required notes.ini parameter is CREATE_R85_DATABASES=1.
A database at a certain ODS level can be read by the release in which that ODS level was introduced and by any higher levels of code. So, an ODS43 database can be accessed by a Domino server or Notes client running release 6.0 or higher. (The maximum supported ODS level is the same for a given release on both the client and the server.) In this scenario, any release of the client or the server running a release prior to 6.0 will not be able to read this database (thus, releases are backward compatible but not forward compatible regarding ODS levels).
If you try copy an ODS43 database to a release 5.0 client using the “copy” command from the operating system’s command prompt (an OS-level copy), the client will not be able to open it.
You should be aware that it’s possible to create a database with a file extension that locks that database into a certain ODS level for its entire lifetime. Normally when creating a Notes database, you specify the .nsf file extension (or the Notes/Domino software appends this extension if no file extension is supplied). In this scenario, Notes/Domino creates the database at the current ODS level for the client or server. However, you can specify the ODS for the database by specifying the file extension as .nsX, where X is the release number for the ODS level you want. (X has to be equal to or less than the release where the database is being created). For a database created with the .nsX extension, a copy-style compact does not upgrade the database. If you create a database named testdb.ns5, for example, the database is created at ODS41 even if the database is created on a Notes 7.03 client, and it will remain at that level even if you rename the database to testdb.nsf at the OS level and run compact on it. The only way to upgrade this type of database is to create a new replica of it and rename it with the .nsf file extension.
Myth Busting
Based on several questions regarding ODS levels that I’m frequently asked at conferences, I’ve become aware of several misconceptions about ODS that I’d like to clear up now.
Myth 1: Down-level Clients Cannot Access a Database Running a Higher ODS Level on the Server
The ODS level of a database only matters to the Notes/Domino software accessing it locally, and the access depends on the version of that software. For example, a Notes 7.0 client can access a database on a Domino 8.5 server that is at ODS51 with no problems. The remote procedure calls that are sent from the client to the server don’t care what the ODS level is on the server. When the server gets the request from the client, the server is at a proper level to read the ODS51 database, get the information requested, and send it back to the Notes 7.0 client. If an OS copy of the database was made on the 8.5 server and placed on a file share to which the 7.0 client had access, the client would then not be able to open the database. That’s because the client would not be going through the server code; it would be trying to open the database locally, which it would not be at the proper level to do.
Myth 2: Database Replicas at Different ODS Levels Cannot Replicate
As I mentioned earlier, when you create a replica of a database from one server to another or from a server to a client, the new replica is created at the highest ODS level for the release of code running on the target server or client. (For the notes.ini parameters to use for releases 8.0.x and 8.5.x, see “Upgrading the ODS Level of Databases on Domino 8.0.x and Higher Servers.”
When replication takes place between the replicas, the process is the same as for a client accessing a higher level ODS on the server. The remote procedure calls between the two servers or between the client and the server don’t care what ODS level is on the other side. You can replicate an ODS20 database replica with an ODS43 replica on another server or client with no problem. This process applies to all kinds of replication — streaming replication, cluster replication, and cluster streaming replication.
Myth 3: The Database ODS and Design Are Somehow Tied Together
I’ve heard people say they can’t upgrade the ODS level of their databases because they are still running an older design on their mail files. The design of a database and its ODS level are completely different things. The design of a database can replicate across multiple replicas, but, as I explained under “Myth 2,” the replicas of a database do not have to be at the same ODS level; they can all be at different ODS levels. So, you can have a mail file on a Domino 8.5 server that inherits from mail6.ntf (the release 6.0 mail template design) but that has ODS51, and it will work
fine.
Another misconception is that the design task will not continue to work unless the templates from which databases inherit are upgraded to the same ODS level as the database. The truth is that templates running at older ODS levels can still be read by the design task, and the design task can
apply design element changes to databases that are running newer ODS levels.
What ODS Level Are You Currently Running?
There are a few places where you can find out the current ODS level of a database.
For any database you open in your Notes client, you can check Database properties on the information tab, as shown below. Choose File  Database (or Application for Release 8.0 and higher)  Properties.
To discover the ODS level of multiple databases on a server, you can use the Domino Administrator client. On the Files tab, look at the File Format column, which lists the release and ODS format for each database. This column is sortable, so you can click on it to quickly determine which of your databases can be upgraded.
Click to Enlarge
Notice that Figure 3 shows some databases using the release 3 ODS level still being accessed on a Domino 8.5.1 server.
You can also use the Domino Administrator client to view all the local databases and their ODS levels on the client where Administrator is installed.
Why Upgrade the ODS Level of Your Databases?
Given that you can still access a database with an older ODS level using a newer client or server, why would you want to change the database’s ODS level?
One good reason is that upgrading the ODS level may improve performance or enable the database to benefit from features in the higher release.
Every time the ODS has changed, it has been the result of new features added to Notes and Domino that require changes to the way internal structures are stored within the database, or it has been for performance reasons to provide better response times to users.
For example, in release 5.0, ODS41 was required to take advantage of transaction logging for the first time. ODS43 (release 6.0) was needed for LZ1 compression and shared templates. ODS48 is required both for the design document compression developed in release 8.0 and the data document compression that followed in release 8.01.
Probably the most important reason for upgrading to ODS51 is to take advantage of the new Domino Attachment and Object Service (DAOS) feature introduced in release 8.5, which is a real space saver. DAOS allows attachments to be extracted from the database and stored securely in a
separate directory in the file system. Copies of the same attachment that are stored multiple times in the same database, or in different databases on the same server partition, are only stored once in the file system. One of the requirements for enabling DAOS is that databases are upgraded to ODS51, which makes it possible to store the “ticket” that replaces the actual attachment in the database and properly translate the ticket when any client or server tries to access it. For an in-depth understanding of DAOS, I recommend reading the DAOS articles in THE VIEW online knowledgebase. In addition, you may want to read some of the success stories from Lotus customers who have saved 60% or more of their total disk space after enabling DAOS on their servers. You can find quite a few with a Web search.
DAOS is currently implemented only on the Domino server. All local (client-based) replicas store attachments in the NSF files as they always have; they do not consolidate duplicate attachments within a single database or across multiple local databases. However, with Notes 8.5.1 there is a compelling reason to upgrade to ODS51 on your local databases as well. Notes 8.5.1 introduces a new feature, DAOS Exploitation, that saves end-user time by allowing the client to know, for the transfer of any attachment from the client to the server, whether or not the attachment already exists on the server, If the attachment is already on the server, the client does not re-send it across the network to the server. DAOS Exploitation requires ODS51 databases on the client and DAOS to be enabled on the server. Lotus recommends upgrading databases to ODS51 on the Notes client after the client is at release 8.5.1, since that is all that is needed to take advantage of DAOS (no end-user work required). If your users can’t or don’t want to upgrade all their local databases, Lotus recommends users at least upgrade their local mail file replica, mail.box, and the autosave database.
Another requirement of DAOS is that the server must be running with transaction logging enabled. For databases at ODS51, enabling transaction logging reduces CPU usage and I/O activity to the transaction log disk.
Internal testing at IBM has demonstrated up to a 50% reduction in CPU usage and a 45% I/O reduction when databases are enabled for transaction logging. This reduction is especially true when the advanced database property “Don’t overwrite free space” is enabled for databases, as shown below (Figure 4).
When this property is enabled, the disk space that was occupied by documents that were deleted from the database is not overwritten with a series of bytes. Thus, the I/O to disk and the transaction log is reduced. This property is appropriate for databases that do not contain confidential information or for databases that are on secure servers and which cannot be copied by users.
An additional CPU reduction can be realized after upgrading to ODS51. This reduction comes from the changed algorithm for document compression in this ODS level.
And finally, you can reduce I/O to the transaction log using the notes.ini parameter NSF_DONT_LOG_MAILBOX_BODY. When this parameter is set, the bodies of mail messages are not logged as they pass through the mail.box. This setting makes it possible for you to leave transaction logging enabled on mail.box to get the benefits of features like DAOS exploitation while avoiding a large I/O hit on the transaction log disk.
Now that you know the benefits that come with upgrading to the latest ODS level, it’s time to find out how best to upgrade the databases on your clients and servers.
Upgrading the ODS Level of Databases on Domino 8.0 and Higher Servers
As I mentioned earlier, in Domino 8.0 and higher, the ODS upgrade does not take place the first time you do a copy-style compaction unless a notes.ini parameter has been set. In release 8.5, the notes.ini parameter that needs to be set in order for the ODS level to be upgraded is CREATE_R85_DATABASES=1. In release 8.0, the parameter is CREATE_R8_DATABASES=1.
Below are the best practices to follow when upgrading the ODS of your databases. First, I lay out the steps for a straight upgrade to ODS51.
Then, I explain how to enable DAOS and upgrade to ODS51 simultaneously.
Note!
The steps below assume you are upgrading databases on Domino 8.5.x servers.
They apply equally to upgrading databases on 8.0.x servers, with the only difference being that you would use the notes.ini parameter for release 8 (CREATE_R8_DATABASES=1).
Upgrading Databases to ODS51
  1. After upgrading your server to Domino 8.5.x, and before upgrading to the new
    ODS, run the Fixup task on all databases. If you are running with transaction logging enabled, make sure it’s the server that runs Fixup by adding the switch to fixup.
    Be aware that the server will change the database instance ID (DBIID) of each database, which requires you to take a new full backup. Run the Fixup task with the server down if possible (e.g., from the Windows command prompt, enter nfixup.exe); that gives Fixup exclusive access to all the databases on the server. I realize that having the server down might not be possible on large production machines due to the amount of time it takes for Fixup to run on databases. However, if you can afford to have your server down for the time it takes, it’s worth doing. Running Fixup this way can clean up any lingering problems before you move on to the next steps.
  2. If you have the time, take a full backup of all the databases on the server (if you do this backup, ensure you do it after upgrading your server to Domino .x and before upgrading the ODS to 51). The full backup ensures you have clean copies of all databases at the old ODS level in case there are any problems and you need to roll back quickly.
  3. Next, run compact –c on the databases you wish to convert to the new ODS level. You have four options for compacting databases:
    • Individually, using compact –c <database name>
    • Per directory, using compact –c <directory>
    • On the entire server, using compact –c (and no additional parameters)
    • Using indirect files (the best practice recommended by Lotus)
Let me explain the last option in step 3. Indirect files are just text files with an .ind file extension — for example, dbs.ind.
They contain a list of files and/or directories that you want one of the Domino utilities to act upon. You can modify these files with any text editor. For example, running the command compact –c dbs.ind results in a copy-style compaction running on all the databases and/or directories listed in the indirect file. To reduce the time needed to do the conversion, you can create several indirect files, each with a subset of the databases to be converted, and run multiple compactions at the same time. A rule of thumb is to run one fewer concurrent compactions than the number of CPUs in the server. Following this rule avoids the thrashing of processes that are being swapped out. You also have the option to use the indirect files for the Fixup step.
  1. After the compaction process has completed, check the console log to see if any errors were reported that may have prevented the ODS upgrade from occurring (for example, insufficient space for performing a copy-style compaction of a particular database). After the server has been restarted, you will be able to check that all databases have been converted using the Domino Administrator client.
  2. If time permits before restarting the server, take another backup. The DBIIDs of your databases have changed again because of the copy-style compaction. Thus, a full backup will be required the next time your normal backup cycle runs if a full backup is not taken now.
  3. After you complete the upgrade of your databases to ODS51, restart the server. Your Notes clients should be able to access it normally, the same as they did before the upgrade.
Enabling DAOS and Upgrading to ODS51 at the Same Time
You may want to enable DAOS on the server at the same time you convert to ODS51. If you enable DAOS on a server and do not perform a copy-style compaction of the databases on that server, only new attachments will be added to the DAOS store. To get the existing attachments into DAOS and get back any additional space from the duplicated attachments, you need to run a copy-style compaction. You can run a single compaction that does the upgrade and pulls all the attachments out to the DAOS store. Here’s how:
  1. With the Domino 8.5 server running, configure DAOS on the DAOS tab of the server document and wait for the server to recognize that DAOS has been enabled. You can check to the status of DAOS on the server using the load daosmgr status console command. Figure 5 below shows some sample output from this command.
Click to Enlarge
  1. After the server recognizes the enablement of DAOS, shut down the server and make sure the notes.ini parameter CREATE_R85_DATABASES=1 is set.
  2. Optionally, run compact –c –ZU dbs.ind to enable LZ1 compression (if it is not already enabled) on the databases that you will be enabling for DAOS. You have this option only if the databases on the server are already at ODS43 or higher, which is the ODS level for 6.x (the release in which LZ1 compression was introduced). This command also upgrades to LZ1 any attachments that are not currently stored in LZ1 format and re-attaches them to the note.
    Note that it does not cause the document to replicate to other replicas after the conversion to LZ1. This compaction can be done as part of the next step, but I’ve found it’s faster to do it as its own step and then extract the attachments into DAOS as part of the ODS upgrade.
Note!
If all or most of your Notes clients are at release 6.0 or higher, Lotus recommends upgrading to LZ1 compression because it is the best compression algorithm available. A server using LZ1 compression needs to convert on the fly from LZ1 to Huffman compression (algorithm in releases prior to 6.x) before sending an attachment back to a release 5 Notes client. This fact doesn’t pose much of a concern as there are not many release 5 clients in use today.
You should also be aware that DAOS works best when all the attachments use the same compression algorithm. That’s because DAOS calculates a hash of the bytes in an attachment to determine if it already has a copy of that attachment or not. The same attachment using Huffman compression and LZ1 compression will hash to different values for DAOS so the attachment will be duplicated (in different forms) in the DAOS store.
  1. Perform the copy-style compaction as previously described, except with the – aos on flag added to the command line. This flag causes the server to pull attachments out of the databases as the databases are upgraded to ODS51 and to replace the attachments with “tickets.” The tickets point to the attachments now stored on the file system in the DAOS store.
  2. After the compaction process completes, restart the server and start planning
    the ODS upgrades on your clients.
Upgrading the ODS Level of Databases on Notes Clients
Upgrading the ODS level of databases on a Notes client is very similar to upgrading the ODS level of databases on the server — set the notes.ini parameter and copy-style compact the databases. The difference is you don’t have to think about DAOS, as it is not currently supported on the client. As mentioned earlier, the most urgent local databases to upgrade are mail file, mail.box, and the
autosave database (so users who work with local replicas can take advantage of DAOS).
One challenge in upgrading the ODS level of Notes client databases is there is no easy, centralized way to access them as there is on the server.
And, currently, there is no way to have a client automatically upgrade the ODS level when either the client is upgraded or some administrator-triggered event happens. Another challenge is that databases that are in use can’t be copy-style compacted. This challenge means administrators are hard pressed to find a time when they can run copy-style compaction on databases like names.nsf and log.nsf, which are always open by both the client and server; they need the client or server to be down to run copy-style compaction on these databases. My group at Lotus is looking into providing some options that will make it easier to upgrade client databases in future releases of Notes and Domino.
To upgrade the ODS level of databases on Notes clients, you can take any of several approaches depending on the level of sophistication of your end users.
  • Have your desk-side support or help desk team walk users through the process of shutting down the Notes client, updating the notes.ini file, and running the compact process — or just have someone visit end users and do it for them.
  • Send users an email message with a link to a batch file on a shared drive.When a user clicks the link, the batch file shuts down the client and performs the upgrade for the user.
  • Use a third-party tool that acts as a desktop manager. You can tell it perform the upgrade for end users while they are away from their desks.
Although the steps are the same, the process is usually more challenging since there are more machines to deal with and more end users to involve.
Returning to an Older ODS Level
I can’t think of a reason why you’d need to go back to an older ODS version, but there is a way to revert to an older version if the need ever arises. Once again, it involves using the compact task, except this time it’s with the –R switch (which stands for “revert one ODS level”).
Reverting to a former ODS version is a fairly straightforward process. For example, if you are running a Domino 6.0 server and you run compact –R on an ODS43 database on that server, the compact process will revert the database’s ODS back to ODS41.
The only time the reversion gets tricky is when you are running databases on a Notes/Domino 8.x client or server. In that case, the exact reversion depends on which notes.ini parameters you have set.
If you have an ODS51 database on a Notes 8.5 client, and you run compact –R on it, the resulting database ODS will be ODS48 if you have CREATE_R85_DATABASES=1 still set in the notes.ini. However, if you change that INI parameter to CREATE_R8_DATABASES=1 and run compact
–R
on another ODS51 database, the ODS for that database will be taken back to ODS43. The
reason for this behavior is that Notes/Domino calculates what the ODS level would be if you were creating a new database and then reverts the ODS one level from there. So, if you were to remove the CREATE_R*_DATABASES notes.ini parameter all together and run compact –R on an ODS51 database, it would be converted all the way back to ODS41 — because new databases created on Notes/Domino 8.5 without the notes.ini parameter set are ODS43 by default.
Again, I can’t see a reason you’d need this, but it’s good to know about in case you ever need to downgrade your servers for some reason or end up running into a problem with the new ODS level.
Anuncios
  1. No trackbacks yet.

Responder

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Cerrar sesión /  Cambiar )

Google+ photo

Estás comentando usando tu cuenta de Google+. Cerrar sesión /  Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Cerrar sesión /  Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Cerrar sesión /  Cambiar )

w

Conectando a %s

Anuncios
A %d blogueros les gusta esto: