Understanding Oracle Solaris Time Sharing(TS) Dispatch Tables for Low- and High-end Servers

Asset ID:	1-71-1001653.1
Update Date:	2010-08-26
Keywords:

Solution Type Technical Instruction Sure

Solution 1001653.1 : Understanding Oracle Solaris Time Sharing(TS) Dispatch Tables for Low- and High-end Servers

Applies to:

Sun Fire E4900 Server
Sun Fire 6800 Server
Sun Fire E6900 Server
Sun SPARC Enterprise M4000 Server
Sun SPARC Enterprise M5000 Server
All Platforms

Goal

The Oracle Solaris[TM] process scheduler uses certain default values for scheduling in the time-sharing class.
The configuration is implemented as a simple lookup table and can be viewed using the command " dispadmin -c TS -g ".

The default values usually only depend on the Oracle Solaris release used and are independent of a specific platform or hardware. There are however exceptions for the high-end servers (currently Sun Enterprise[TM] 10000, Sun Fire[TM] 12K/15K/E20K/E25K). The dispatch tables for these high-end systems have been modified to better support typical server applications (especially databases).

Solution

The general dispatch table is a good trade-off between long-running processes and short-running processes such as interactive use. The dispatch table of the high-end servers has been modified to have longer default time quantums for each priority. If several processes are competing for CPU resources (load higher than the number of available CPUs), the new high-end dispatch table will cause less process switching while each process will stay longer on the CPU. As a result, interactive use (or response time of similar short time processes) might get delayed in such overload situations as they have to wait for one of the long term processes to exceed the given time quantum.

To illustrate the difference, here is an example of a system running Oracle Solaris 8:

Default dispatch table

     % dispadmin -c TS -g
      # Time Sharing Dispatcher Configuration
      RES=1000

      # ts_quantum  ts_tqexp  ts_slpret  ts_maxwait ts_lwait  PRIORITY LEVEL
         200         0        50           0        50        #     0
         200         0        50           0        50        #     1
         200         0        50           0        50        #     2
         200         0        50           0        50        #     3
         200         0        50           0        50        #     4
         200         0        50           0        50        #     5
         200         0        50           0        50        #     6
         200         0        50           0        50        #     7
         200         0        50           0        50        #     8
         200         0        50           0        50        #     9
         160         0        51           0        51        #    10
         160         1        51           0        51        #    11
         160         2        51           0        51        #    12
         160         3        51           0        51        #    13
         160         4        51           0        51        #    14
         160         5        51           0        51        #    15
         160         6        51           0        51        #    16
         160         7        51           0        51        #    17
         160         8        51           0        51        #    18
         160         9        51           0        51        #    19
         120        10        52           0        52        #    20
         120        11        52           0        52        #    21
         120        12        52           0        52        #    22
         120        13        52           0        52        #    23
         120        14        52           0        52        #    24
         120        15        52           0        52        #    25
         120        16        52           0        52        #    26
         120        17        52           0        52        #    27
         120        18        52           0        52        #    28
         120        19        52           0        52        #    29
          80        20        53           0        53        #    30
          80        21        53           0        53        #    31
          80        22        53           0        53        #    32
          80        23        53           0        53        #    33
          80        24        53           0        53        #    34
          80        25        54           0        54        #    35
          80        26        54           0        54        #    36
          80        27        54           0        54        #    37
          80        28        54           0        54        #    38
          80        29        54           0        54        #    39
          40        30        55           0        55        #    40
          40        31        55           0        55        #    41
          40        32        55           0        55        #    42
          40        33        55           0        55        #    43
          40        34        55           0        55        #    44
          40        35        56           0        56        #    45
          40        36        57           0        57        #    46
          40        37        58           0        58        #    47
          40        38        58           0        58        #    48
          40        39        58           0        59        #    49
          40        40        58           0        59        #    50
          40        41        58           0        59        #    51
          40        42        58           0        59        #    52
          40        43        58           0        59        #    53
          40        44        58           0        59        #    54
          40        45        58           0        59        #    55
          40        46        58           0        59        #    56
          40        47        58           0        59        #    57
          40        48        58           0        59        #    58
          20        49        59       32000        59        #    59

Dispatch table of high end servers

      % dispadmin -c TS -g
      # Time Sharing Dispatcher Configuration
      RES=1000

      # ts_quantum  ts_tqexp  ts_slpret  ts_maxwait ts_lwait  PRIORITY LEVEL
         400         0         1           2        40        #     0
         380         0         2           2        40        #     1
         380         1         3           2        40        #     2
         380         1         4           2        40        #     3
         380         2         5           2        40        #     4
         380         2         6           2        40        #     5
         380         3         7           2        40        #     6
         380         3         8           2        40        #     7
         380         4         9           2        40        #     8
         380         4        10           2        40        #     9
         380         5        11           2        40        #    10
         380         5        12           2        40        #    11
         380         6        13           2        40        #    12
         380         6        14           2        40        #    13
         380         7        15           2        40        #    14
         380         7        16           2        40        #    15
         380         8        17           2        40        #    16
         380         8        18           2        40        #    17
         380         9        19           2        40        #    18
         380         9        20           2        40        #    19
         360        10        21           2        40        #    20
         360        11        22           2        40        #    21
         360        12        23           2        40        #    22
         360        13        24           2        40        #    23
         360        14        25           2        40        #    24
         360        15        26           2        40        #    25
         360        16        27           2        40        #    26
         360        17        28           2        40        #    27
         360        18        29           2        40        #    28
         360        19        30           2        40        #    29
         360        20        31           2        40        #    30
         360        21        32           2        40        #    31
         360        22        33           2        40        #    32
         360        23        34           2        40        #    33
         360        24        35           2        40        #    34
         360        25        36           2        40        #    35
         360        26        37           2        40        #    36
         360        27        38           2        40        #    37
         360        28        39           2        40        #    38
         360        29        40           2        40        #    39
         360        30        41           2        41        #    40
         340        31        42           2        42        #    41
         340        32        43           2        43        #    42
         340        33        44           2        44        #    43
         340        34        45           2        45        #    44
         340        35        46           2        46        #    45
         340        36        47           2        47        #    46
         340        37        48           2        48        #    47
         340        38        49           2        49        #    48
         340        39        50           2        50        #    49
         340        40        51           2        51        #    50
         340        41        52           2        52        #    51
         340        42        53           2        53        #    52
         340        43        54           2        54        #    53
         340        44        55           2        55        #    54
         340        45        56           2        56        #    55
         340        46        57           2        57        #    56
         340        47        58           2        58        #    57
         340        48        59           2        59        #    58
         340        49        59           2        59        #    59

By default, processes will start at priority 59 and on normal machines will get 20ms time quantum on the CPU. If they exceed this CPU limit, they will be reduced in priority. If another process with higher priority is waiting for CPU resources, the current process will be taken off the CPU and the other waiting process will get the CPU.

In the case of a high-end server system, such a switch might be delayed up to 340ms instead of the default 20ms, and interactive users might get a slower response as they have to wait for the other server processes to exceed their time quantum. Please note that such a situation only occurs if all CPUs are busy dealing with long-running and CPU-intensive processes. Additionally, interactive or short term processes will get started.

If you are using one of the high-end servers in such a mixed mode (CPU-intensive, long-running processes together with interactive or short term processes in high load situations), then you might want to revert to the default dispatch table if faster response time for short term processes is needed. Keep in mind, though, that you already deal with a situation where all CPUs are overloaded. If you increase the number of process switches, this will delay the long-running processes.

The special dispatch table for high-end servers was introduced in Oracle Solaris 2.6 and went through several design changes in later Oracle Solaris releases. As of Oracle Solaris 8 and later, the alternative dispatch table is now included in the normal TS_DPTBL kernel module. The kernel module will check if Solaris runs on a high-end server and automatically activate the alternative dispatch table. Activation of this high end dispatch table can be controlled via the variable ts_dispatch_extended in the file /etc/system.

The default for ts_dispatch_extended is -1 which will select the dispatch table depending on the architecture. The new table is always selected for Sun Enterprise 10000 and Sun Fire 12K/15K/E20K/E25K. On other Sun Fire systems the new table should be activated manually via /etc/system.
Setting ts_dispatch_extended to 0 will always select the default dispatch table.
Setting ts_dispatch_extended to 1 will always select the high end server dispatch table.

The changes to the dispatch table are usually only helpful if your system is already overloaded and you want to have higher priority for the short term processes. You should consider adding more CPU resources or controlling your available resources using the Oracle Solaris Resource Manager (SRM) which has been integrated in Oracle Solaris 9 (and is available as a separate product for previous releases).

@Internal Comments
To find the extra code for Solaris 2.6:

http://aggregate.eng/on297_dhpg/source/usr/src/uts/common/disp/ts_dptbl.c

For Solaris 7 the changes are trivial (shell script) and for Solaris 8 (and later) you should check the regular ts_dptbl.c source plus the additions from usr/src/uts/sun4u/starcat/os/starcat.c and usr/src/uts/sun4u/starfire/os/starfire.c (which activate the new table).

Some architectures may select the new table if the system is using Ultra Sparc III+ or newer CPUs. This was true for Daktari and Serengeti when the fix for Bug ID 6239229 Click Here was introduced in Solaris Nevada build 17, but after half a year this was removed as part of a fix for Bug ID 6291734 Click Here. See SCCS logs for usr/src/uts/sun4u/daktari/os/daktari.c and usr/src/uts/sun4u/serengeti/os/serengeti.c for the history of bug fixes. Currently Serengeti and Daktari use default dispatch table out of the box. None of the released Solaris versions have used the new/temporary default for Serengeti or Daktari (as this was only added and removed in the internal development releases).

The new implementation in Solaris 8 was the result of PSARC 1999/434.

Discussion is ongoing if/which additional new architectures should also have the "server dispatch table" on by default. This question needs to be answered for each new architecture.

The dispatch table for high-end servers is also referenced in some of the tuning documents for databases. Users of the dispatch table with larger time quantums should know what they are doing and not be too surprised if the response time for interactive use increases if the machine is overloaded with database processes.

Regarding OPL systems: usr/src/uts/sun4u/opl/os/opl.c now also uses the server dispatch table(integration of Bug ID 6476273 Click Here which uses the now somewhat incorrect summary that OPL should use the same dispatch table as serengeit/daktari as the selection there has changed). The OPL information will be added in the next update.

In Oracle Solaris 2.6, special support for the Sun Enterprise 10000 was introduced. During installation on such a sun4u1 architecture, the package SUNWcar.u1 installs extra kernel modules in the directory /platform/sun4u1/kernel/sched/. Inside this directory you will find the modules TS and TS_DPTBL which will take precedence over the default modules in /kernel/sched/. If you want to disable the specific tables you can rename or delete the special modules, in which case the default modules will get used.

Starting with Oracle Solaris 7, the high-end dispatch table is not a separate kernel module. A new init script has been developed which gets installed as /etc/init.d/tsquantum. This script will check if Oracle Solaris[TM] 7 is running on a Sun Enterprise 10000 and load the dispatch table from the ASCII file /usr/lib/class/TS/TSbigquanta. You can deactivate the special dispatch table by removing or deleting the /etc/rc2.d/S99tsquantum file.

Attachments

This solution has no attachment