web port number port
This configuration option establishes the port number used by Zim applications running on a Web browser. It’s reserved for future use.
Valid Settings
A number from 1024 to 32765.
| Windows | 6005 |
| UNIX | 6005 |
Category: ZIM Application Server
Java Options
java options <options>
The options are any valid parameters that are understood by Java when a Java instance is started. This occurs when ZimServer performs a connection to the JDBC Driver or JDBC SAM. The valid options are the ones described by the Java documentation.
Example
java options "-Xmx64M"
Backup Server Name
backup server name <name or IP address>
The name or IP address is the location in the network where ZimBackup Server is responding to perform an online application of the data just updated by ZimServer. These files correspond to committed transactions from all databases being served by Zim Server and can be placed anywhere in the network so that they are saved in case of a disk crash.
It should be used in conjunction with Backup Port Number configuration option.
Valid Settings
A valid Internet address.
| Windows | None is set |
| UNIX | None is set |
Backup Port Number
backup port number <port number>
The backup port number establishes the port number used by Backup Server to interact with Zim Server when applying the backup blocks to the backup databases.
Together with Backup Server Name, Zim Server establishes a connection to ZimBackup to keep sending backup blocks of data as soon as these data blocks are committed by ZimServer in its local databases.
Valid Settings
A valid number between 1024 and 65535 not conflicting with the Server Port Number configuration option.
| Windows | 6001 |
| UNIX | 6001 |
Maximum Log Size
maximum log size file-sizeThe number file-size, expressed in Kb, indicates the maximum size the log file ZIMSVLOG.ZIM can reach until ZimServer shuts itself down. This is to prevent the disk space be entirely used for logging.
Valid Settings
A number from 1,000 to 10,000,000 (or, between 1 Mb and 10 Gb) for both Windows and Linux. The default is 1,000,000, or 1 Gb.
The ZIMSVLOG.ZIM file
The log file zimsvlog.zim, located in the directory pointed to by the ZIM environment variable, records all events happening with Zim Server like, its start, its end, errors, tracing, and other information.
You can browse its contents using a text editor or using ZimAdmin.
After some time using ZimServer, this file can become very large. You should check the size of it from time to time, archive it (if you wish) and erase it. It can be erased at any time, even when Zim Server is running. Next time ZimServer needs to log some information, the file will be recreated.
File Management and Distribution
Zim uses the operating system’s file management system to store entity sets, relationships, programs (source and compiled), database definitions, ZIM directories, and work files. By default, all files are stored in the same area of your file system. You can however, change the locations of the various types of files by, for example, moving the files to different disks, to improve the performance of your application. Zim provides methods for controlling the files that it uses.
How to Maintain the Integrity of a Database
A database is always considered integral unless it presents problems. This may sound a little bit funny, but the general approach, in real life, is to run an application over a database until something wrong happens.
To avoid surprises, certain procedures can be preventively taken to guarantee that the database is always available. These procedures include:
- Check your database at regular intervals if your data is particularly important.
- Plan to check your database after the occurrence of a potentially corrupting event (e.g., power failure).
- Check your database when you get an error message that indicates corruption could have occurred;
In most circumstances, you can check the integrity of a database by two methods:
1) The ZIMFIX utility
If you suspect corruption of your database, or if you want to check the integrity of the database, run the Repair Facility (ZIMFIX) administrative utility against all database files. The Repair Facility checks the integrity of the database files associated with a specified entity set or relationship. In particular, it verifies that records are constructed properly and that indexes correspond properly to the data stored in the records.
Note: Before executing the Repair Facility (ZIMFIX) to detect corruption, run it with the EXTRACT FILE DEFINITIONS box checked (equivalent to ZIMDD).
For example, to check the integrity of the database files corresponding to EntitySets named Pollutants and WaterSamples, execute the following commands at the operating system level:
zimdd zimfix WaterSamples zimdd Pollutants
The Repair Facility can also perform structural integrity checks on application directory files, but it cannot check the integrity of the actual object definitions that those directories contain.
Note: The Repair Facility reports the records that fail the integrity checks, enabling recall and viewing of the damaged records.
2) Finding Data in your database
A fast way of detecting errors is to run a series of FIND commands over the entity set or relationship in question:
. A FIND command without using indexes to read all records;
. One FIND command per index to check the integrity of the indexes.
If there are no sparse indexes, the number of records found in each one of the FINDs must be exactly the same.
This method is faster than running ZIMFIX, but doesn’t check the contents of records. It indicates if something is wrong or it can directly stop a record that is corrupted. Later, you can run ZIMFIX.
How to Restore a Corrupted Database
There are two general ways to restore a corrupted database (in particular, a Zim file):
1. Use ZIMFIX with the repair option;
2. If you have found a record that is corrupt (trying to LIST that record, there is an error message), you can do:
- Use the FIND command to retrieve all records of the appropriate entity set or relationship;
- Use the NEXT command to locate the damaged record;
- Use the LIST command to display the record on the screen (it will display an error message indicating the corruption);
- Use the CHANGE command to correct the wrong fields or use the DELETE command to delete the entire record (you can later manually ADD the same record again).
3. If the corruption was in an index, you can ERASE and CREATE that particular index again (always erase and create indexes in the same order they were originally created to avoid the need to recompile all programs).
Zim Server Performance Tuning
To optimize Zim Server performance, it’s crucial to fine-tune the configuration settings in the zimconfig.srv file. Here are detailed technical guidelines for each of the three key factors:
1. Maximize Shared Memory Usage
- Buffers and Areas Allocation: Increase the number of buffers and areas allocated in shared memory. This reduces the need for frequent memory allocation and deallocation, which can be costly in terms of performance.
- Configuration Options:
SHARED_MEMORY_BUFFERS: Set this to a high value to ensure ample buffer space.SHARED_MEMORY_AREAS: Increase this to allow more areas to be stored in shared memory.- Example:
ini
SHARED_MEMORY_BUFFERS=1024
SHARED_MEMORY_AREAS=256
2. Balance Shared Memory Usage
- Avoid Over-Allocation: While maximizing shared memory usage is beneficial, it’s important not to exceed system-defined limits. Over-allocating can lead to overheads, such as increased context switching and memory paging, which degrade performance.
- Configuration Options:
MAX_SHARED_MEMORY: Set this to a value that balances usage without exceeding system limits.SHARED_MEMORY_THRESHOLD: Define a threshold to prevent over-allocation.- Example:
ini
MAX_SHARED_MEMORY=2048MB
SHARED_MEMORY_THRESHOLD=80%
3. Reduce Input and Output Operations
- Minimize I/O Operations: Excessive I/O operations can significantly slow down the server. Optimizing these operations can lead to substantial performance improvements.
- Configuration Options:
IO_OPERATION_LIMIT: Set a limit on the number of I/O operations.CACHE_SIZE: Increase the cache size to reduce the frequency of I/O operations.- Example:
ini
IO_OPERATION_LIMIT=1000
CACHE_SIZE=512MB
Additional Tips
- Monitoring and Tuning: Regularly monitor the performance metrics of your Zim Server and adjust the configuration settings as needed. Tools like
top,htop, or custom monitoring scripts can help you keep track of memory usage and I/O operations. - Documentation and Support: Refer to the official Zim Server documentation for detailed descriptions of each configuration option. If you encounter issues, consider reaching out to the Zim Server support community for advice and best practices.
By carefully adjusting these settings, you can achieve a well-balanced and high-performing Zim Server environment.
Checkpoint Buffers and Checkpoint Transactions
These options define the size of the checkpoint file, which is either the number of blocks or transactions that Zim Server will process before physically committing to disk. Both options refer to transactions that were effectively committed, meaning transactions that successfully finished.
Key Concepts
- Optimized Checkpoint Mechanism: Zim Server’s checkpoint mechanism ensures that only the last occurrence of a block is written to disk. This means that larger checkpoint files can lead to more efficient performance due to reduced input and output operations.
- Balancing Checkpoint Size: While larger checkpoint files can improve efficiency, they also mean that Zim Server will spend more time processing blocks and less time handling ongoing transactions. Additionally, larger checkpoint sizes can lead to longer wait times before the checkpoint file is produced and committed to disk.
Configuration Guidelines
To achieve optimal performance, it’s important to balance these options based on the size of your database and the number of users.
Small Databases
For small databases or a small number of users, use the following settings:
checkpoint buffers = 1000 # Small checkpoint file
checkpoint transactions = 100000 # Unlikely to reach this limit
- Explanation: These settings instruct Zim Server to update the database files after 1000 blocks are committed or after 100000 transactions are committed. Given the low activity, the 1000 blocks threshold will be reached frequently, ensuring efficient performance despite frequent checkpoints.
Large Databases
For larger databases or higher user activity, use these settings:
checkpoint buffers = 10000 # Medium size checkpoint file
checkpoint transactions = 100000 # Unlikely to reach this limit
- Explanation: These settings will result in less frequent checkpoints due to the higher block threshold, but the increased database activity will still ensure efficient performance.
Further Tuning
For even larger databases, consider increasing the checkpoint buffers value further:
checkpoint buffers = 50000 # Larger checkpoint file
checkpoint transactions = 100000 # Unlikely to reach this limit
- Explanation: Testing different values can help you find the optimal configuration for your specific use case. Larger values for
checkpoint bufferscan lead to improved performance by reducing the frequency of checkpoints, but it’s important to monitor and adjust based on actual performance metrics.
Conclusion
By carefully adjusting the checkpoint buffers and checkpoint transactions settings, you can optimize Zim Server performance for both small and large databases. Regular monitoring and testing different configurations will help you achieve the best balance for your specific environment.
Maximum Data Blocks
This option specifies the number of data blocks that Zim Server will keep in shared memory simultaneously. It significantly impacts Zim Server efficiency by optimizing the number of input and output operations. The larger this option, the more efficient Zim Server will be. However, it requires a substantial amount of shared memory. If insufficient real memory is available, the operating system will start swapping, which can drastically reduce performance.
Maximum Databases
This option determines how many Zim databases will be processed concurrently. The default is 3, but reducing this number to the minimum necessary will decrease the amount of shared memory required.
Maximum Tables
This option indicates the number of Zim tables (entity sets and relationships with fields) that will be processed across all databases. Keeping this number as low as possible will minimize shared memory usage.
Maximum Users
This option sets the maximum number of users allowed to connect to Zim Server, up to the number of Zim licenses purchased. Reducing this number to the minimum needed will conserve shared memory.
Object Sleep, Object Timeout, Pending Sleep, and Pending Timeout
These parameters define the wait and timeout durations when attempting to acquire various Zim Server internal objects. Lower values result in faster reactions to resolve issues (usually by deadlocking), allowing other users to proceed. Use the suggested values in the configuration files below.
Scatter Table Entries
This option specifies the maximum sizes of transactions in blocks that all users connected to Zim Server can perform simultaneously. If there are 10 users connected but only 5 performing transactions, only these will count. For example, if the average number of blocks processed is 500, the number of entries used will be 2,500. Once a transaction is finished, all entries used in that transaction are released. This number cannot be smaller than Scatter Table Links. A good approach is to calculate 3 to 5 times that number.
Scatter Table Links
This option sets the maximum size for a transaction in blocks, allowing transactions to process different blocks of data up to this limit. For instance, an ADD statement creating a very large Zim file (e.g., 500 MB) will require this parameter to be at least 500,000. However, such commands are not common daily, suggesting a specific configuration for large operations. The larger this number, the more efficient Zim Server will be. A good reference is to multiply Maximum Data Blocks by two or three. This parameter cannot be smaller than Maximum Data Blocks.
Configuration Files
Small Size Database Configuration File
blocks per read = 8
buffers per user = 1000
checkpoint buffers = 1000
checkpoint transactions = 100000
datablock timeout = 1
files per transaction = 20
first file to reclaim = 100
maximum blocks per user = 2000
maximum data blocks = 20000
maximum databases = <number>
maximum file locks = 200
maximum locks = 10000
maximum record groups = 2000
maximum tables = <number>
maximum users = <number>
object sleep = 1
object timeout = 1
pending sleep = 5
pending timeout = 5
processors = 1
quick locks = 10
reclaim object size = 1000
record group size = 256
scatter table entries = 70000
scatter table links = 50000
scatter table size = 100
secondary lock group size = 20
secondary lock groups = 7500
server port number = 6002
Medium Size Database Configuration File
blocks per read = 8
buffers per user = 1000
checkpoint buffers = 10000
checkpoint transactions = 100000
datablock timeout = 1
files per transaction = 20
first file to reclaim = 100
maximum blocks per user = 2000
maximum data blocks = 50000
maximum databases = <number>
maximum file locks = 200
maximum locks = 10000
maximum record groups = 2000
maximum tables = <number>
maximum users = <number>
object sleep = 1
object timeout = 1
pending sleep = 5
pending timeout = 5
processors = 1
quick locks = 10
reclaim object size = 1000
record group size = 256
scatter table entries = 150000
scatter table links = 100000
scatter table size = 100
secondary lock group size = 20
secondary lock groups = 7500
server port number = 6002
Large Size Database Configuration File
blocks per read = 8
buffers per user = 1000
checkpoint buffers = 50000
checkpoint transactions = 100000
datablock timeout = 1
files per transaction = 20
first file to reclaim = 100
maximum blocks per user = 2000
maximum data blocks = 100000
maximum databases = <number>
maximum file locks = 200
maximum locks = 20000
maximum record groups = 4000
maximum tables = <number>
maximum users = <number>
object sleep = 1
object timeout = 1
pending sleep = 5
pending timeout = 5
processors = 1
quick locks = 10
reclaim object size = 1000
record group size = 256
scatter table entries = 800000
scatter table links = 250000
scatter table size = 100
secondary lock group size = 20
secondary lock groups = 7500
server port number = 6002
By carefully adjusting these settings, you can optimize Zim Server performance for different database sizes. Regular monitoring and testing different configurations will help you achieve the best balance for your specific environment.
Database Backup
A well-designed database application is of no use if data and the application are not available 100% of the time. Data loss and corruption are not common in most systems, but occasional occurrences are unavoidable. Some of the events that can cause data loss include the following:
| Event | Description |
| User error | Individual records, or even entire files, can be erased by accident or with malicious intent |
| Hardware failure | Data can be lost if a disk suddenly becomes unreadable for some reason. |
Because events such as these can happen anywhere, at any time, you should take the time now to plan how to handle a possible future data loss. Your plan should include a policy regarding the intervals at which you check the integrity of your database.
Another situation that may arise is when power failures or processing interruptions occur while data is being processed and/or committed. In these cases, ZimServer is perfectly capable of resolving the situation the next time it is restarted by applying previously unresolved commits to the database by means of the recorded modifications made to the database in a common transaction file. If transactions could not successfully end for any reason, these transaction records are ignored.
Among other practices, there are two common actions to be taken to guarantee data security:
| Action | Description |
| Copy your databases at regular intervals | At specific times (usually overnight), your system is shut down (unavailable to users) and a physical copy of the databases is made. Data is guaranteed to be safe within a 24-hour span only. Data updates during this interval are probably going to be lost. The application system’s ability to come back online may vary from minutes to hours depending on the size of the database. |
| Online backup | Data is constantly backed up as soon as there is a commit to the data. Data loss is up to only a few seconds, if such, the application system may come back online as soon as users are routed to the backup computer. |
Using ZimBackup and ZimServer to Make Online Backups
Zim:X provides the ability to perform online data commits in backup databases as soon as data is committed by ZimServer in the active databases. The backup databases can reside anywhere in the world, typically in different machines and/or different physical locations in relation to the active databases. Both ZimServer and ZimBackup need to be properly configured but this process takes only a few minutes to be performed.
a) Stop ZimServer
If ZimServer is currently running, stop it gracefully by means of the “-k” option. This guarantees that the databases under its control are in a valid state.
b) Stop ZimBackup
Make sure that Zimbackup is not running by stopping with the means of the “-k” option. This guarantees that the databases under its control are in a valid state.
c) Copy all databases involved in the backup operation
Make a copy of all databases under ZimServer control, the one that just has been stopped. This copy will not only save all databases but also create a replicate of the databases to be involved in the backup operation. In essence, the backup will start with the current valid state of the databases.
| WARNING: If you have any “areas.zim” files, you will have to copy the corresponding files addressed by this configuration file to the proper database path. |
d) Change the server database configuration file “zimdb.zim”
Assuming that there are two Zim databases being serviced by ZimServer but only one needs to be backed up, you just need to add the keyword “backup” after the proper entry (in this case, only MyBase will be backed up):
10;MyBase;c:/mybase/;backup;
20;Example;c:/otherexample;e) Change the server configuration file “zimconfig.srv”
The following lines in this configuration file should be changed:
audit path <a file directory>
backup path <another file directory>
backup port number <a port number, usually 6001>
backup server name <an IP address>The audit path option tells where ZimServer should place the uncommitted data files. For safety reasons, it should point to some hardware different from the current serviced databases in order to preserve the commits in case of any failures. If not provided, it defaults to the current Zim installation.
The backup path option tells where ZimServer should place a compressed format of all committed data files ready to be sent to ZimBackup. For safety reasons, it should point to some hardware different from the current serviced databases and from the place pointed by the audit path so that the backed-up databases would preserve their integrity.
The backup port number and the backup server name are the TCP/IP identification where ZimBackup is operating to perform the backups.
f) Change the backup database configuration file “zimbk.zim”
In the environment where ZimBackup is going to run, change the “zimbk.zim” to provide the databases that are going to be backed up. Using the example above, only one reference to a database needed to be provided:
10;MyBase;c:\mybackupbase;| WARNING: The database number (10, in this case) and the database name (MyBase) MUST be the same as used in “zimdb.zim” for ZimServer. However, the destination directory does not need to be the same. |
g) Change the backup configuration file “zimconfig.bkp”
In the environment where ZimBackup is going to run, change the following lines:
destination directory <temporary file directory> backup port number <a port number, usually 6001>
The destination directory is a temporary location in the backup server environment to hold the backup files to be applied to the backup databases. After the corresponding application, these files are erased.
The backup port number is the same TCP/IP port used by ZimServer to “talk” to ZimBackup.
h) Copy the databases from the server to the backup place
Copy the databases that were saved in Step C to their proper places as described in “zimbk.zim”. In the above example, you should copy the contents saved from “c:/mybase” to “C:/MyBackupBase”.
| WARNING: Although Windows and Linux may be incompatible in many ways, it is perfectly possible to have ZimServer running on Windows and ZimBackup running on Linux (or vice-versa) because ZimBackup does not deal with the data within the database. However, before using the backed-up database, you will need to copy it to the original machine. |
i) Start ZimServer
All set, you can now start ZimServer.
j) Start ZimBackup
All set, you can now start ZimBackup. In fact, starting ZimServer before or after ZimBackup does not make a difference because when it’s time to back up data, one server waits for the other.
| WARNING: If you run any Zim statements changing data under the setting SET CHECKPOINT OFF all data changed in this way will NOT be backed up because this setting tells ZimServer to operate in single-user mode, that is, there are no transactions involved and there are no transactions to be committed. This setting is used to add data offline in bulk to avoid memory limitations which significantly increases the speed of the operation. However, in case of any failures, the database involved may be rendered corrupt and an off-line backup should be used to recover the database. |
General guidelines to keep a database running can comprise:
English 
Portuguese