Company

Products

Services

Partners

Media

 

 

 

Home

Empress Markets

News & Events


Presentations

Press Releases

Product Profile

Success Stories

Technical News


White Papers

Join Mailing List

For More Info

           

    Empress Technical News – December 2013

    Empress Expandable Mapped Files

    Efficient Shared Memory for Empress Application Developers

    Introduction

    The Empress Shared Memory feature provides users the ability to enhance the performance of the Empress RDBMS through the use of system shared memory. introduces additional flexibility in configuring Mapped File shared memory in the latest release of Empress 10.20 with Expandable Mapped Files.

    Normally, each database process has its own memory space allocated by the operating system and cannot access the memory space of another database process. Shared memory is an operating system feature that allows several processes to share a specified section of memory. By using shared memory, file I/O can be reduced, and database performance can be improved.

    Empress uses shared memory in three different ways to provide enhanced performance: lock managers, global buffers and mapped files Figure 1). All three concepts are great ways to place part of the Empress database located on disk into shared memory. In an extreme case, all Empress Database tables could be mapped into shared memory.

    Lock Managers

    Lock Managers allow placing of certain lock information in shared memory where processes can access it. Empress also allows the Lock Manager (shared memory vs. file) to be defined on a table-by-table basis. Some tables may be defined to use shared memory and some may continue to use files for managing locks.

                                                                                        Figure 1

     

    Figure 1: Empress Shared Memory

    Global Buffers

    Every time a process wishes to get data from a file, it must read that file. Buffers enable Empress to keep parts of a file or many files in shared memory. This speeds up access and therefore increases performance.

    When data is written to a file that has a Global Buffer, the data is simultaneously written to the Global Buffer and the file. Global Buffers are called write-through buffers. This also means that the data in the Global Buffer will represent the data in the file.

    Mapped Files

    Mapped Files are similar to Global Buffers in that they store data from files in shared memory. However, a Mapped File in shared memory has only one file associated with it. Also a Mapped File is not divided into blocks. Instead, it is one large block of memory directly associated with a given section of a file. This means that the Mapped File is essentially a window into a given file. The window can be the entire file, part of the file beginning at the start of the file, or a section of the file beginning and ending at user-defined points.

     

    Expandable Mapped Files

    If a file associated with a Mapped File in a shared memory is a read-only file than the size is well known in advance and a Mapped File can be precisely configured. However, if a file grows, there is no good estimation on how much it could grow. Having an unnecessarily large shared memory partition allocated in memory is wasteful of memory and has a performance hit as well.   

    To address these Mapped File constraints, introduced the Expandable Mapped File.

    When a file grows (e.g. new records are inserted), new Mapped File extensions are added to accommodate the new records (see Figure 2).

                                                                              Figure 2

    Figure 2: Empress Expandable Shared Memory Partition

     

    Defining Mapped File Shared Memory Partition

    In general, as shown in Figure 1, Shared Memory is divided into partitions. There are three types of partitions: Lock Manager, Global Buffer, and Mapped File. There can be any number of each type in shared memory but it is not necessary that all types be there. For example, shared memory could contain just two Mapped Files. The only limitation on the number and size of each partition is the amount of memory in the system available for shared memory.

    The shared memory partitions section has a specific layout which is set up by editing the Empress Database file to append the partition definition at the end of tabzero file:

    An example of the shared memory partitions section at the end of the Empress Database tabzero file that contains two Mapped Files is as follows:

        MSSHMKEY=123456

        MSSHMPROTECTION=

                   

                   

                               

                               

                               

                   

                   

                   

                               

                               

                               

                   

        MSSHMEND

    The contents of the file are discussed in the following sections.

    Shared Memory Key

    The first entry in the partition file is the Empress variable MSSHMKEY. The key is a user-defined number that the system uses to identify a given shared memory block. This number is arbitrary, but each site should have a procedure to assign keys. All databases that want to use the same shared memory block must use the same key to identify it. It looks like this:

        MSSHMKEY=123456

    Parameter for Data Dictionary Indicator  

    Next comes the Empress variable If this partition is used by sys_dictionary this should be set to " on" to get special protection on the shared memory partition. If it is not used by sys_dictionary then this variable will be unset. It looks like this:

        MSSHMPROTECTION=on

    Partition Name

    There can be any number of partitions in any order. All the partition definitions have the same first two lines. They are partition name and partition type, in that order. The partition name is any string of characters whose length is less than 32 (any more will be truncated). A partition name from the example looks like this:

        MSPARTNAME=Mapped1

    Partition Type

    There are three partition types that can be chosen: LOCK, GBUF, MAPF for Lock Manager, Global Buffer, and Mapped File respectively. Partition type from the example looks like:

        MSPARTTYPE=MAPF

    Each partition type has unique parameters that must be defined. For the purpose of this text we will focus only on Mapped File parameters.

    Parameters for Mapped Files

    A Mapped File partition has four possible parameters: starting offset, size, flush/sync behavior and number of extents. The starting offset is the offset from the start of the file where the mapping will begin (i.e., the start of the file window). The entries for both Mapped1 and Mapped2 in the example look like this:

    MSPARTMAPFSTART=0

    The size should be chosen to hold the most often used data, or if the file is small enough, the entire file. The entries for both Mapped1 and Mapped2 in the above example look like this:

    MSPARTMAPFSIZE=1048576

    The MSPARTMAPFNUMEXT value represents the maximum number of extra segments that Empress will create if the data cannot be mapped in the existing segment.

    An example of using the MSPARTMAPFNUMEXT value is:

    MSPARTMAPFNUMEXT=16

    If the above definition is used, the maximum shared memory allocation for the file to be mapped will be 1,048,576 * (16 + 1) = 17,825,792 bytes. The formula is:

    max_size = MSPARTMAPFSIZE *(MSPARTMAPFNUMEXT + 1)

    MSPARTMAPFNUMEXT cannot be greater than 16.

    The MSPARTMAPFSYNC value represents flush and sync Mapped File behavior.

    MSPARTMAPFSYNC=

    If MSPARTMAPFSYNC parameter is set to nothing or not set at all, on each database engine flush, local buffers will be flushed to the Mapped File and operating system buffers. This is a write-through concept.

    MSPARTMAPFSYNC=close

    With the close setting, on each database engine flush, local buffers will be flushed to the Mapped File. On close table operations (e.g. invoking an Empress C/C++ mr routine mrclose(), Mapped File will be flushed to the operating system buffers, if required.

    MSPARTMAPFSYNC=never

    With the never setting, on each database engine flush, local buffers will be flushed to the Mapped File. Mapped File will never be flushed to the operating system buffers or device.

    An example of using expandable Mapped file shared memory partition to demonstrate the example given in Figure 2 is as follows:

    MSSHMKEY=123456
    MSSHMPROTECTION=
                  MSPARTNAME=Mapped3
                  MSPARTTYPE=MAPF
                                  MSPARTMAPFSTART=0
                                  MSPARTMAPFSIZE=1048576
                                  MSPARTMAPFNUMEXT=2
                                  MSPARTMAPFSYNC=close
                  MSPARTEND
    MSSHMEND

     

    Instead of Summary

    If a file associated with a Mapped File in a shared memory is a read-only file than the size is well known in advance and a Mapped File can be precisely configured. However, if a file grows and there is no good estimation on how much it could grow, than a new way of configuring Mapped File shared memory partition is desirable.

    To address these situations Empress introduced Expandable Mapped Files, an excellent tool for Empress Application developers to control memory usage and achieve the best possible performance.

     

    Empress Software Inc.

     

                       
                             
                         

Company
Information
GSA Contracts
Careers
Privacy Policy 
Contact US

Products
Empress RDBMS
Empress Servers
Empress API
Free Trial 
Empress iPad Apps
 

Services
Consulting
Mobile Apps Development
Technical Support

Training Courses
Empress Extreme  

Partners
Distributors
Business Alliances
Partner Programs  
GSA Program
Distributors Program

Media 
Empress Markets
News and Events
Press releases
Technical News
Success Stories

        USA: 301-220-1919                 Canada & International:       905-513-8888         
Copyright © 2014           Empress Software Inc.           info@empress.com           Join Empress Mailing List