Categoria: Performance Tuning

Deadlocks

A deadlock condition occurs when two users have to wait for each other in order to complete their respective transactions. For example, one user’s transaction needs records currently being used by a transaction started by another user. The second user’s transaction in turn requires records that the first user’s transaction is holding (while it waits for the second user’s transaction to be completed). Zim can detect deadlock situations and can abort the transaction of one of the users – the user’s updates are discarded, and locks released. The other user proceeds.

An application that normally responds very quickly, but is constantly deadlocking, is wasting its high performance. Deadlock situations are not errors, but rather the natural outcome of enabling more than one user to access the same data at the same time.

Completely eliminating deadlocks is not an acceptable solution. To do this, users are forced to wait in line to use the application. Operating system and application tuning can, however, reduce the likelihood of deadlocks, enabling the application, and its users, to maximize the amount of work done in a given period of time.

Page Locking

Zim uses automatic page locking to control concurrent access to the database. Locks force other users to wait until the user whose actions initiated the lock has finished using the data. This waiting can lead to deadlocks.

Two kinds of locks are used:

• a read lock on pages that are being read
• a write lock on pages that are being updated

A read lock enables other users to read the locked page; a write lock prevents other users both from reading and from updating the page.

Example

Two users, Bob and Macy, wish to access certain pages in a file at the same time:

• Bob wants to update page 5, and then read page 6.
• Macy wants to update page 6, and then read page 5.

Both users update their chosen pages successfully. However, Bob cannot read page 6 because Macy’s write lock precludes both updating and reading. Macy, on the other hand, cannot read page 5, because Bob has placed a write lock on it. A deadlock occurs because each user is waiting for the other. One user’s transaction (let’s assume Macy’s) is aborted, removing its page locks, and enabling the other user to continue. Macy’s work is lost.

File Locking

Zim can lock entire files (both read and write file locks) under certain conditions. A write file lock prevents other users from accessing any page in a file.

File locks can help reduce the incidence of deadlock. For example, Bob and Macy attempt the actions described under Page Locking, above, but, this time, file locking is employed.

When Bob updates page 5, a write file lock is placed on the file. Macy can neither read nor update pages 5 and 6. She must wait for Bob’s transaction to be completed before she can begin her own. Although waiting for Bob seems like a waste of time, this wait is better than the time wasted re-entering data after a deadlock.

Fine Tuning Locks

What the preceding example shows is that there is usually a trade-off between reducing the incidence of deadlock and maintaining a high degree of concurrent access to the application (several users accessing the database at the same time). In our example, file locking eliminated a deadlock, but it also reduced concurrent access (Macy had to wait for Bob). The administrator must  decide the level of trade-off that is appropriate for a particular application. A number of Zim Server configuration options can be used to fine tune the applications being serviced by Zim Server. The following must be considered before changing configuration options:

• All locks are maintained in memory by Zim Server. The operating system is not invoked to control any kind of locks managed by Zim Server. Therefore, the only limitation in configuring locks is the availability of memory;
• The locks configuration options are applicable to all databases being managed by Zim Server and not specifically to a single database or user.

All Zim Server configuration options documentation also discusses efficiency issues.

Of particular interest in the area of lock management are the configurations Maximum Locks, Maximum File Locks, Quick Locks, Secondary Lock Groups and Secondary Lock Group Size.

A well-designed database application is of no use if data and the application are not available 100% of the time. Data loss and corruption are not common in most systems, but occasional occurrences are unavoidable. Some of the events that can cause data loss include the following:

Because events such as these can happen anywhere, at any time, you should take the time now to plan how to handle a possible future data loss. Your plan should include a policy regarding the intervals at which you check the integrity of your database.

Another situation that may arise is when power failures or processing interruptions occur while data is being processed and/or committed. In these cases, ZimServer is perfectly capable of resolving the situation the next time it is restarted by applying previously unresolved commits to the database by means of the recorded modifications made to the database in a common transaction file. If transactions could not successfully end for any reason, these transaction records are ignored.

Among other practices, there are two common actions to be taken to guarantee data security:

Using ZimBackup and ZimServer to Make Online Backups

Zim:X provides the ability to perform online data commits in backup databases as soon as data is committed by ZimServer in the active databases. The backup databases can reside anywhere in the world, typically in different machines and/or different physical locations in relation to the active databases. Both ZimServer and ZimBackup need to be properly configured but this process takes only a few minutes to be performed.

a) Stop ZimServer

If ZimServer is currently running, stop it gracefully by means of the “-k” option. This guarantees that the databases under its control are in a valid state.

b) Stop ZimBackup

Make sure that Zimbackup is not running by stopping with the means of the “-k” option. This guarantees that the databases under its control are in a valid state.

c) Copy all databases involved in the backup operation

Make a copy of all databases under ZimServer control, the one that just has been stopped. This copy will not only save all databases but also create a replicate of the databases to be involved in the backup operation. In essence, the backup will start with the current valid state of the databases.

d) Change the server database configuration file “zimdb.zim”

Assuming that there are two Zim databases being serviced by ZimServer but only one needs to be backed up, you just need to add the keyword “backup” after the proper entry (in this case, only MyBase will be backed up):

10;MyBase;c:/mybase/;backup;
20;Example;c:/otherexample;

e) Change the server configuration file “zimconfig.srv”

The following lines in this configuration file should be changed:

audit path <a file directory>
backup path <another file directory>
backup port number <a port number, usually 6001>
backup server name <an ip address>

The audit path option tells where ZimServer should place the uncommitted data files. For safety reasons, it should point to some hardware different from the current serviced databases in order to preserve the commits in case of any failures. If not provided, it defaults to the current Zim installation.

The backup path option tells where ZimServer should place a compressed format of all committed data files ready to be sent to ZimBackup. For safety reasons, it should point to some hardware different from the current serviced databases and from the place pointed by the audit path so that the backed-up databases would preserve their integrity.

The backup port number and the backup server name are the TCP/IP identification where ZimBackup is operating to perform the backups.

f) Change the backup database configuration file “zimbk.zim”

In the environment where ZimBackup is going to run, change the “zimbk.zim” to provide the databases that are going to be backed up. Using the example above, only one reference to a database needed to be provided:

10;MyBase;c:\mybackupbase;

g) Change the backup configuration file “zimconfig.bkp”

In the environment where ZimBackup is going to run, change the following lines:

destination directory <temporary file directory>
backup port number <a port number, usually 6001>

The destination directory is a temporary location in the backup server environment to hold the backup files to be applied to the backup databases. After the corresponding application, these files are erased.

The backup port number is the same TCP/IP port used by ZimServer to “talk” to ZimBackup.

h) Copy the databases from the server to the backup place

Copy the databases that were saved in Step C to their proper places as described in “zimbk.zim”. In the above example, you should copy the contents saved from “c:/mybase” to “C:/MyBackupBase”.

i) Start ZimServer

All set, you can now start ZimServer.

j) Start ZimBackup

All set, you can now start ZimBackup. In fact, starting ZimServer before or after ZimBackup does not make a difference because when it’s time to back up data, one server waits for the other.

General guidelines to keep a database running can comprise:

• Using Zim 9 Backup Server, implement safe procedures to backup databases and to restore databases if they become unavailable;
• How to maintain the integrity of your database.

Because Zim 9 always operates in the client/server mode, memory operations performed by the client are not affected by what is being done in the server and vice-versa.

When running Zim on Windows in the client side, the local machine is dedicated to the client session; therefore, all resources available can be used solely for Zim. Consequently, memory is not a concern and the administrator can set configuration options to the maximum in order to improve performance.

The configuration options file for the client is the zimconfig.zim (the database configuration file) and the most relevant options are Maximum Forms, Maximum Form Fields, Runtime Buffers and Sort Buffers. These options can be set to their maximum values all the time. Other parameters like Directories, Document Line Length, Maximum Parameters and Parameter Size can be set to a size according to the needs of the application and can be left to their initial and default state, only changed if Zim states an error.

On the other hand, Zim sessions running on Unix will compete for memory and resources with all other Zim sessions and, most important of all, with Zim Server and its shared memory. Therefore, the administrator must balance the needs of Zim Server and the needs of Zim sessions. Thus, for Unix, the options Runtime Buffers and Sort Buffers must be privileged. The other options (as mentioned for Windows, above) can be configured to some values that still allow Zim sessions to run comfortably without expending too much memory. In general, these values are the default ones that can be changed depending the needs of the application.

Optimizing Memory Usage for Faster Response Times

If all the files an application needs at any given time could be stored in memory, response times would be significantly faster. Accessing main memory is much quicker than using disk input/output (I/O) because disk access times are much longer. However, in practice, it is impossible to keep everything in memory, especially if a database contains millions of records.

Efficient Data Transfer Between Memory and Disk

During an application session, much of the data must be continually transferred between memory and disk. This includes data stored in the database, database definitions, screen definitions, and application programs.

Controlling Data Locks

Maximizing disk access is crucial, but so is controlling locks over the data being processed by each user once it is available in memory. This topic is discussed in detail in the Locks and Deadlocks Conditions section.

Zim Server Efficiency

Zim Server’s efficiency relies heavily on the available memory allocated for its use. When Zim Server starts, it reads its configuration file, calculates the required memory, and allocates the corresponding shared memory for client connections.

Shared Memory Allocation

Shared memory is allocated using a mapping mechanism that associates an address space with physical memory. If the available real memory is sufficient to accommodate the allocated shared memory, Zim Server operations will be highly efficient. If not, the operating system will need to swap out and swap in portions of shared memory that do not fit in the real memory.

Impact of Swap Operations

Although swap operations are faster than regular file operations, they are still slower than real memory operations. Therefore, administrators must consider this factor when configuring Zim Server.

Configuration Options for Efficiency

All Zim Server configuration options address efficiency issues. Key options include Checkpoint Buffers, Checkpoint Transactions, Clustered Commits, Maximum Blocks per User, and Maximum Data Blocks.

When considering the performance of an application, individual users have different criteria in mind. To some, performance is the speed at which an application responds to the requests of an application user. To others, it is the ability of the application to run properly within the limits set by the hardware or operating system. Performance can also be seen as the overall ability of a multi-user application to process a great deal of work for a large number of users quickly and efficiently.

Performance tuning attempts to maximize the success of an application in achieving the criteria described above. Performance tuning can be done by altering the design of the database, using different coding techniques, modifying the design of multi-user transactions, changing the configuration of the operating system, and altering the software’s own configuration. The focus of this section is solely on altering the software’s configuration to fine-tune performance.

To increase the performance of any application, four areas should be considered:

The utility ZimAdmin can help with the dimensioning of the configuration options by dynamically  monitoring the status of ZimServer, just like this example: