The mysql-test-run.pl Perl script is the main application used to run the MySQL test suite. It invokes mysqltest to run individual test cases. (Prior to MySQL 4.1, a similar shell script, mysql-test-run, can be used instead.)
Invoke mysql-test-run.pl in the
mysql-test
directory like this:
shell> mysql-test-run.pl [options
] [test_name
] ...
Each test_name
argument names a test
case. The test case file that corresponds to the test name is
t/
.
test_name
.test
For each test_name
argument,
mysql-test-run.pl runs the named test case.
With no test_name
arguments,
mysql-test-run.pl runs all
.test
files in the t
subdirectory.
If no suffix is given for the test name, a suffix of
.test
is assumed. Any leading path name is
ignored. These commands are equivalent:
shell>mysql-test-run.pl mytest
shell>mysql-test-run.pl mytest.test
shell>mysql-test-run.pl t/mytest.test
As of MySQL 5.1.23, a suite name can be given as part of the test name. That is, the syntax for naming a test is:
[suite_name
.]test_name
[.suffix
]
If a suite name is given, mysql-test-run.pl
looks in that suite for the test. With no suite name,
mysql-test-run.pl looks in the default list
of suites for a match and runs the test in any suites where it
finds the test. Suppose that the default suite list is
main
, binlog
,
rpl
, and that a test
mytest.test
exists in the
main
and rpl
suites. With
an argument of mytest
or
mytest.test
,
mysql-test-run.pl will run
mytest.test
from the
main
and rpl
suites.
To run a family of test cases for which the names share a common
prefix, use the
--do-test=
option. For example, prefix
--do-test=rpl
runs the
replication tests (test cases that have names beginning with
rpl
). --skip-test
has the
opposite effect of skipping test cases for which the names share
a common prefix.
As of MySQL 5.0.54/5.1.23/6.0.5, the argument for the
--do-test
and --skip-test
options allows more flexible specification of which tests to
perform or skip. If the argument contains a pattern
metacharacter other than a lone period, it is interpreted as a
Perl regular expression and applies to test names that match the
pattern. If the argument contains a lone period or does not
contain any pattern metacharacters, it is interpreted the same
way as previously and matches test names that begin with the
argument value. For example, --do-test=testa
matches tests that begin with testa
,
--do-test=main.testa
matches tests in the
main
test suite that begin with
testa
, and
--do-test=main.*testa
matches test names that
contain main
followed by
testa
with anything in between. In the latter
case, the pattern match is not anchored to the beginning of the
test name, so it also matches names such as
xmainytestz
.
To perform setup prior to running tests,
mysql-test-run.pl needs to invoke
mysqld with the --bootstrap
and --skip-grant-tables
options (see
Typical configure Options). If MySQL was configured
with the --disable-grant-options
option,
--bootstrap
,
--skip-grant-tables
, and
--init-file
will be disabled. To handle this,
set the MYSQLD_BOOTSTRAP
environment variable
to the full path name of a server that has all options enabled.
mysql-test-run.pl will use that server to
perform setup; it is not used to run the tests.
The init_file
test will fail if
--init-file
is disabled. This is an expected
failure that can be handled as follows:
shell>export MYSQLD_BOOTSTRAP
shell>MYSQLD_BOOTSTRAP=/full/path/to/mysqld
shell>make test force="--skip-test=init_file"
To run mysql-test-run.pl on Windows, you'll
need either Cygwin or ActiveState Perl to run it. You may also
need to install the modules required by the script. To run the
test script, change location into the
mysql-test
directory, set the
MTR_VS_CONFIG
environment variable to the
configuration you selected earlier (or use the
--vs-config
option), and invoke
mysql-test-run.pl. For example (using Cygwin
and the bash shell):
shell>cd mysql-test
shell>export MTR_VS_CONFIG=debug
shell>./mysqltest-run.pl --force --timer
shell>./mysqltest-run.pl --force --timer --ps-protocol
If you have a copy of mysqld running on the
machine where you want to run the test suite, you do not have to
stop it, as long as it is not using ports
9306
or 9307
. If either of
those ports is taken, you should set the
MTR_BUILD_THREAD
environment variable to an
appropriate value, and the test suite will use a different set
of ports for master, slave, NDB, and Instance Manager). For
example:
shell>export MTR_BUILD_THREAD=31
shell>./mysql-test-run.pl [
options
] [test_name
]
mysql-test-run.pl defines several environment variables. Some of them are listed in the following table.
Variable | Meaning |
MYSQL_TEST |
Path name to mysqltest binary |
MYSQLTEST_VARDIR |
Path name to the var directory that is used for
logs, temporary files, and so forth |
MYSQLD_BOOTSTRAP |
Full path name to mysqld that has all options enabled |
MASTER_MYPORT |
??? |
MASTER_MYSOCK |
??? |
Tests sometimes rely on certain environment variables being
defined. For example, certain tests assume that
MYSQL_TEST
is defined so that
mysqltest can invoke itself with
exec $MYSQL_TEST
.
mysql-test-run.pl supports the options in the
following list. An argument of --
tells
mysql-test-run.pl not to process any
following arguments as options. (A description of differences
between the options supported by
mysql-test-run.pl and
mysql-test-run appears following the list.)
Display a help message and exit.
Run the benchmark suite.
The directory where the benchmark suite is located. The
default path is ../../mysql-bench
.
Pass the --big-test
option to
mysqltest.
Check test cases for side effects.
The path to the directory where client binaries are located. This option was added in MySQL 5.0.66/5.1.27.
Start mysqltest in the ddd debugger.
Start mysqltest in the named debugger.
Start mysqltest in the gdb debugger.
The path to the directory where client libraries are located. This option was added in MySQL 5.0.66/5.1.27.
Extra options to pass to mysqld. The
value should consist of one or more comma-separated
mysqld options. This option is similar to
--mysqld
but should be given two or more
times. mysql-test-run.pl executes
multiple test runs, using the options for each instance of
--combination
in successive runs. If
--combination
is given only once, it has no
effect. For test runs specific to a given test suite, an
alternative to the use of --combination
is
to create a combinations
file in the
suite directory. The file should contain a section of
options for each test run. See
Section 4.9, “Passing Options from mysql-test-run.pl to
mysqld or mysqltest”.
This option was added in MySQL 5.1.23/6.0.4.
Write str
to the output.
Compress all information sent between the client and the server if both support compression.
Pass the --cursor-protocol
option to
mysqltest (implies
--ps-protocol
).
Start mysqld in the ddd debugger.
Dump trace output for all clients and servers.
Start mysqld using the named debugger.
Controls whether the Debug Sync facility for testing and debugging is enabled. The option value is a timeout in seconds. The default value is 300. A value of 0 disables Debug Sync. The value of this option also becomes the default timeout for individual synchronization points.
mysql-test-run.pl passes
--loose-debug-sync-timeout=
to mysqld. The N
--loose
prefix is used so that mysqld does not
fail if Debug Sync is not compiled in.
For information about using the Debug Sync facility for testing, see Section 4.14, “Thread Synchronization in Test Cases”.
This option was added in MySQL 5.1.41/5.5.0/6.0.6.
Run all test cases having a name that begins with the given
prefix
value. This option
provides a convenient way to run a family of similarly named
tests.
As of MySQL 5.0.54/5.1.23/6.0.5, the argument for the
--do-test
option allows more flexible
specification of which tests to perform. If the argument
contains a pattern metacharacter other than a lone period,
it is interpreted as a Perl regular expression and applies
to test names that match the pattern. If the argument
contains a lone period or does not contain any pattern
metacharacters, it is interpreted the same way as previously
and matches test names that begin with the argument value.
For example, --do-test=testa
matches tests
that begin with testa
,
--do-test=main.testa
matches tests in the
main
test suite that begin with
testa
, and
--do-test=main.*testa
matches test names
that contain main
followed by
testa
with anything in between. In the
latter case, the pattern match is not anchored to the
beginning of the test name, so it also matches names such as
xmainytestz
.
Use a version of mysqltest built with the embedded server.
Specify a file that contains a list of test cases that
should be displayed with the [ exp-fail ]
code rather than [ fail ]
if they fail.
This option was added in MySQL 5.1.33/6.0.11.
For an example of a file that might be specified via this
option, see
mysql-test/collections/default.experimental
.
Use an already running server.
Note: If a test case has an .opt
file
that requires the server to be restarted with specific
options, the file will not be used. The test case likely
will fail as a result.
Do not clean up from earlier test runs.
Normally, mysql-test-run.pl exits if a
test case fails. --force
causes execution
to continue regardless of test case failure.
Run tests with the gcov test coverage tool.
Start mysqld in the gdb debugger.
Run tests with the gprof profiling tool.
TCP/IP port number to use for the first mysqld, controlled by Instance Manager.
TCP/IP port number to use for the second mysqld, controlled by Instance Manager.
TCP/IP port number to use for mysqld, controlled by Instance Manager.
Pass the --log-warnings
option to
mysqld.
Use a server that has already been started by the user in a debugger.
Use a server that has already been started by the user in the gdb debugger.
Specify the path of the mysqld binary to use for master servers.
Specify the TCP/IP port number for the first master server to use. Observe that the option name has an underscore and not a dash.
Run the test suite in memory, using tmpfs or ramdisk. This
can decrease test times significantly.
mysql-test-run.pl attempts to find a
suitable location using a built-in list of standard
locations for tmpfs and puts the var
directory there. This option also affects placement of
temporary files, which are created in
var/tmp
.
The default list includes /dev/shm
. You
can also enable this option by setting the environment
variable
MTR_MEM[=
.
If dir_name
]dir_name
is given, it is added
to the beginning of the list of locations to search, so it
takes precedence over any built-in locations.
This option was added in MySQL 4.1.22, 5.0.30, and 5.1.13. It is not supported on Windows.
Extra options to pass to mysqld. The value should consist of one or more comma-separated mysqld options. See Section 4.9, “Passing Options from mysql-test-run.pl to mysqld or mysqltest”.
Extra options to pass to mysqltest. The value should consist of one or more comma-separated mysqltest options. See Section 4.9, “Passing Options from mysql-test-run.pl to mysqld or mysqltest”. This option was added in MySQL 6.0.6.
Pass
--ndb-connectstring=
to the master MySQL server. This option also prevents
mysql-test-run.pl from starting a
cluster. It is assumed that there is already a cluster
running to which the server can connect with the given
connectstring.
str
Pass
--ndb-connectstring=
to slave MySQL servers. This option also prevents
mysql-test-run.pl from starting a
cluster. It is assumed that there is already a cluster
running to which the server can connect with the given
connectstring.
str
Unknown.
--ndbcluster-port=
,
port_num
--ndbcluster_port=
port_num
Specify the TCP/IP port number that NDB Cluster should use.
--ndbcluster-port-slave=
port_num
Specify the TCP/IP port number that the slave NDB Cluster should use.
Run mysqld with options needed on NetWare.
Cause mysqltest not to generate a timing file.
Run tests using N
parallel
threads. By default, 1 thread is used. Use
--parallel=auto
for auto-setting of
N
. This option was added in MySQL
5.1.36.
Pass the --ps-protocol
option to
mysqltest.
Pass the --record
option to
mysqltest. This option requires a
specific test case to be named on the command line.
Reorder tests to minimize the number of server restarts needed.
Display the output of SHOW ENGINES
and
SHOW VARIABLES
. This can be used to
verify that binaries are built with all required features.
This option was added in MySQL 4.1.23, 5.0.30, and 5.1.14.
Enable debug output for mysql-test-run.pl itself.
Do not start Instance Manager; skip Instance Manager test cases.
Do not enable master server binary logging.
Do not start NDB Cluster; skip Cluster test cases.
--skip-ndbcluster-slave
,
--skip-ndb-slave
Do not start an NDB Cluster slave.
Skip replication test cases.
Do not enable master server binary logging.
Do not start mysqld with support for SSL connections.
Specify a regular expression to be applied to test case names. Cases with names that match the expression are skipped. tests to skip.
As of MySQL 5.0.54/5.1.23/6.0.5, the argument for the
--skip-test
option allows more flexible
specification of which tests to skip. If the argument
contains a pattern metacharacter other than a lone period,
it is interpreted as a Perl regular expression and applies
to test names that match the pattern. See the description of
the --do-test
option for details.
--skip-*
--skip-*
options not otherwise recognized
by mysql-test-run.pl are passed to the
master server.
Specify the path of the mysqld binary to use for slave servers.
Specify the TCP/IP port number for the first master server to use. Observe that the option name has an underscore and not a dash.
Pass --sleep=
to mysqltest.
N
Run the benchmarks with the --small-tests
and --small-tables
options.
For connections to localhost
, the Unix
socket file to use, or, on Windows, the name of the named
pipe to use.
Pass the --sp-protocol
option to
mysqltest.
If mysql-test-run.pl is started with the
--ssl
option, it sets up a secure conection
for all test cases. In this case, if
mysqld does not support SSL,
mysql-test-run.pl exits with an error
message: Couldn't find support for SSL
Initialize and start servers with the startup settings for the first specified test case. For example:
shell>cd mysql-test
shell>./mysql-test-run.pl --start alias &
This option was added in MySQL 5.1.32/6.0.11.
Initialize and start servers with the startup settings for the specified test case or cases, if any, and then exit. You can use this option to start a server to which you can connect later. For example, after building a source distribution you can start a server and connect to it with the mysql client like this:
shell>cd mysql-test
shell>./mysql-test-run.pl --start-and-exit
shell>../mysql -S ./var/tmp/master.sock -h localhost -u root
Start servers (without initialization) for the specified test case or cases, if any, and then exit. You can then manually run the test cases.
mysql-test-run.pl sorts the list of names
of the test cases to be run, and then begins with
test_name
.
Create strace output for mysqltest.
Run the stress test. The other
--stress-
options apply in this case.
xxx
file_name
is the location of the
file that contains the list of tests. The default file is
stress_init.txt
in the test suite
directory.
In sequential stress-test mode, the number of loops to execute before exiting.
This option indicates the test order in stress-test mode.
The mode
value is either
random
to select tests in random order or
seq
to run tests in each thread in the
order specified in the test list file. The default mode is
random
.
The name of the test suite to use for stress testing. The
default suite name is main
(the regular
test suite located in the mysql-test
directory).
For stress testing, the number of tests to execute before exiting.
For stress testing, the duration of stress testing in seconds.
The file that contains the list of tests to use in stress
testing. The tests should be named without the
.test
extension. The default file is
stress_tests.txt
in the test suite
directory.
The number of threads to use in stress testing. The default is 5.
Run the named test suite. The default name is
main
(the regular test suite located in
the mysql-test
directory).
Specify the maximum test suite runtime.
Specify the maximum test case runtime.
Cause mysqltest to generate a timing
file. The default file is named
./var/log/timer
.
The directory where temporary file are stored. The default
location is ./var/tmp
.
Use unified diff format when presenting differences between expected and actual test case results.
Do not install the test databases. (Use existing ones.)
Unused.
The MySQL user name to use when connecting to the server.
Run mysqltest and mysqld with valgrind.
Like --valgrind
, but passes the
--verbose
and
--show-reachable
options to
valgrind.
Run mysqltest with valgrind.
Like --valgrind-mysqltest
, but passes the
--verbose
and
--show-reachable
options to
valgrind.
Extra options to pass to valgrind.
Specify the path name to the valgrind executable.
Specify the path where files generated during the test run
are stored. The default location is
./var
.
Pass the --view-protocol
option to
mysqltest.
Specify the configuration used to build MySQL (for example,
--vs-config=debug
--vs-config=release
). This option is for
Windows only. It is available as of MySQL 4.1.23, 5.0.30,
and 5.1.14.
Unused?
Search the server log for errors or warning after each test
and report any suspicious ones; if any are found, the test
will be marked as failed. This is the default behavior, it
may be turned off with --nowarnings
.
Use NDB Cluster and enable test cases that require it.
Use NDB Cluster in all tests.
Run only test cases that have ndb
in
their name.
Unknown.
This option is a synonym for --ssl
.
mysql-test-run supports the following
options not supported by mysql-test-run.pl:
--local
, --local-master
,
--ndb-verbose
,
--ndb_mgm-extra-opts
,
--ndb_mgmd-extra-opts
,
--ndbd-extra-opts
,
--old-master
, --purify
,
--use-old-data
,
--valgrind-mysqltest-all
.
Conversely, mysql-test-run.pl supports the
following options not supported by
mysql-test-run:
--benchdir
,
--check-testcases
,
--client-ddd
,
--client-debugger
,
--cursor-protocol
,
--debugger
,
--im-mysqld1-port
,
--im-mysqld2-port
,
--im-port
, --manual-debug
,
--netware
, --notimer
,
--reorder
, --script-debug
,
--skip-im
, --skip-ssl
,
--sp-protocol
,
--start-dirty
, --suite
,
--suite-timeout
,
--testcase-timeout
, --udiff
,
--unified-diff
,,
--valgrind-path
, --vardir
,
--view-protocol
.