Integration with job schedulers is not discussed – examples will call the mpirun command directly. The remainder of this document will use OpenMPI for the examples. There is indeed, even a third option, hidden in the depths of the Lustre JIRA issue tracking system in ticket LU-56 that adds the ability to run against multiple mountpoints on a single client. LANL have added features not available in the LLNL version, most notable of which are some Lustre-awareness to allow striping across multiple MDTs and AWS S3 support. The preferred distribution of mdtest is available on GitHub at IOR as part of the IOR project.
INSTALL MPI ON NODE CODE
The mdtest application is distributed as source code and must be compiled for use on the target environment. MDTest provides a way to define a standard test that can be used to assess baseline performance of a file system, and provide a comparative measure against storage platforms. It is therefore important to be able to demonstrate the ability of Lustre to match and even exceed application requirements for file systems. Metadata performance is a critical measurement of file system capability and is increasingly relevant to parallel file system workloads in general. While mdtest can be run stand-alone to measure local file system performance, it is really intended to be run on parallel and shared file systems. In the case of Lustre, the machines are Lustre clients. The program works by creating, stat-ing and deleting a tree of directories and files in parallel across a population of machines (typically compute nodes in an HPC cluster).
MDTest measures the metadata performance of a given file system implementation and will run on any POSIX-compliant file system. The program can create directory trees of arbitrary depth and can be directed to create a mixture of work-loads, including file-only tests.
Multiple mdtest processes are run in parallel across several nodes using MPI in order to saturate file system I/O. The mdtest application runs on Lustre clients in a fully configured Lustre file system. For Lustre, this means the MGS, MDS and OSS services must be installed, configured and running, and that there is a population of Lustre clients running with the Lustre file system mounted. MDTest is not a Lustre-specific benchmark and can be run on any POSIX-compliant file system, but it does require a fully installed and configured file system implementation in order to run. MDtest is an MPI-based application for evaluating the metadata performance of a file system and has been designed to test parallel file systems.
0 kommentar(er)
