Difference between revisions of "Compiling M5"

From gem5
Jump to: navigation, search
(Required Software: Start new page for non-default Python install)
m (Added "Compiling with runtime checks")
 
(29 intermediate revisions by 6 users not shown)
Line 1: Line 1:
M5 runs on Linux, Mac OS X, OpenBSD, and Microsoft Windows (under Cygwin), but should be easily portable to other Unix-like OSes. Cross-endian support has been mostly added, however this has not been extensively tested. All the syscall emulation (_SE) targets support running regardless of host/target endianess, however at present ALPHA_FS does not fully support big endian machines.  
+
gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there.  Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.
  
 +
Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).
  
== Required Software ==
+
To build gem5, you need to ensure you have all the [[dependencies]] installed.
To build M5, you will need the following software:
 
 
 
* [http://gcc.gnu.org/ g++] version 3.4 or newer. Once upon a time, M5 built with g++ 2.95, but it probably doesn't anymore; < 3.4 seems to have some issues with templates.
 
* [http://www.python.org Python], version 2.4 or newer.  M5 links in the Python interpreter, so you need the Python header files and shared library (e.g., /usr/lib/libpython2.4.so) in addition to the interpreter executable.  These may or may not be installed by default.  For example, on Debian/Ubuntu, you need the "python-dev" package in addition to the "python" package.  If you need a newer Python version but can't or don't want to upgrade the default Python on your system, see our page on [[using a non-default Python installation]].
 
* [http://www.scons.org SCons], version 0.96.91 or newer.  SCons is a powerful replacement for make. See [http://sourceforge.net/project/showfiles.php?group_id=30337 here] to download SCons.  If you don't have administrator privileges on your machine, you can use the "scons-local" package to install scons in your m5 directory, or install SCons in your home directory using the '--prefix=' option.
 
* [http://www.swig.org SWIG], version 1.3.28 or newer.
 
* [http://www.zlib.net zlib], any recent version.  For Debian/Ubuntu, you will need the "zlib-dev" package to get the zlib.h header file as well as the library itself.
 
 
 
There a few utility scripts written in Perl, but Perl is not necessary to build or run the simulator.
 
 
 
Note that the required versions for both SCons and SWIG are fairly recent.  For example, they are newer than the packaged versions for the current Debian stable and Ubuntu 6.06 releases, so you may well need to install them from source (or go to Debian testing or unstable).
 
  
 
== Possible Targets ==
 
== Possible Targets ==
M5 can build many binaries each for a different guest architecture, simulation mode, and use. The currently available architectures are '''ALPHA''', '''SPARC''', and '''MIPS'''.
+
gem5 can build many binaries each for a different guest architecture. The currently available architectures are '''ALPHA''', '''ARM''', '''MIPS''', '''POWER''', '''SPARC''', and '''X86'''. In addition there is a '''NULL''' architecture.
 
 
Each of these can built in two simulation modes:
 
*'''FS''' - Full System mode. This mode simulates a complete system including a kernel, I/O devices, etc. This mode currently only works with the '''ALPHA''' architecture.
 
*'''SE''' - Syscall Emulation mode. This mode simulates statically compiled binaries by functionally emulating any syscall they make.
 
  
 
For each possible architecture and mode, several different executables can be built:
 
For each possible architecture and mode, several different executables can be built:
*'''m5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly. (-g3 -gdwarf-2 -O0).
+
*'''gem5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
*'''m5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target. (-g -O3)
+
*'''gem5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
*'''m5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof. (-O3 -g -pg).
+
*'''gem5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
*'''m5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). (-O3 -DNDEBUG)
+
*'''gem5.perf''' - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.  
 +
*'''gem5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization
  
 
== Compiling ==
 
== Compiling ==
  
Starting in the root of the source tree, you can build M5 using a command of the form:
+
Starting in the root of the source tree, you can build gem5 using a command of the form:
  
  % scons build/<arch>_<mode>/m5.<binary>
+
  % scons build/<arch>/gem5.<binary>
  
 
where the items between &lt;&gt; are the architectures, modes and binaries listed above.  For example:
 
where the items between &lt;&gt; are the architectures, modes and binaries listed above.  For example:
  
 
<pre>
 
<pre>
% cd m5
+
% cd gem5
% scons build/ALPHA_FS/m5.debug
+
% scons build/ALPHA/gem5.debug
  
 
scons: Reading SConscript files ...
 
scons: Reading SConscript files ...
 
Checking for C header file fenv.h... yes
 
Checking for C header file fenv.h... yes
Building in /tmp/m5/build/ALPHA_FS
+
Building in /tmp/gem5/build/ALPHA
Options file /tmp/m5/build/options/ALPHA_FS not found,
+
Options file /tmp/gem5/build/options/ALPHA not found,
   using defaults in build_opts/ALPHA_FS
+
   using defaults in build_opts/ALPHA
Compiling in ALPHA_FS with MySQL support.
+
Compiling in ALPHA with MySQL support.
 
scons: done reading SConscript files.
 
scons: done reading SConscript files.
 
scons: Building targets ...
 
scons: Building targets ...
g++ -o build/ALPHA_FS/base/circlebuf.do -c -pipe -fno-strict-aliasing
+
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA_FS
+
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
     build/ALPHA_FS/base/circlebuf.cc
+
     build/ALPHA/base/circlebuf.cc
 
...
 
...
  
 
</pre>
 
</pre>
  
If your output looked like the above, then congratulations, you've compiled M5!  The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA_FS/m5.debug.  For more build options and further details about the build system, see the [[SCons build system]] page.
+
If your output looked like the above, then congratulations, you've compiled gem5!  The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug.  For more build options and further details about the build system, see the [[SCons build system]] page.
  
 
== Installing full system files ==
 
== Installing full system files ==
  
Look for the path in configs/common/SysPaths.py (there are two to choose from) in which to install the contents of the tar file. Either use one of the existing paths, or edit the path in SysPaths.py. For example (where &lt;NNNN&gt; is a group containing all users of M5, and you will need to edit the tar file name and path to reflect the version you downloaded and where you put it):
+
If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the [[Download]] page.
 +
 
 +
The path to these files is determined in configs/common/SysPaths.py.  There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like <code>ImportError: Can't find a path to system files.</code> when you first attempt to run the simulator in full-system mode.
 +
 
 +
Note that the default path, <code>/dist/m5/system</code>, is designed for environments where you have root (sudo) access (to create <code>/dist</code>) and want the files in a place where they can be shared by multiple users.  If both of these are true, you can follow this example to put the system files at the default location:
  
 
<pre>
 
<pre>
 
% sudo mkdir -p /dist/m5/system
 
% sudo mkdir -p /dist/m5/system
 
% cd /dist/m5/system
 
% cd /dist/m5/system
% sudo tar vxfj &lt;path&gt;/m5_system_2.0b1.tar.bz2
+
% sudo tar vxfj &lt;path&gt;/m5_system_2.0b3.tar.bz2
% sudo chgrp -R <NNNN> /dist
+
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
 +
% sudo chgrp -R <grp> /dist # where <grp> is a group that contains all the m5 users
 
</pre>
 
</pre>
 +
 +
In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.
  
 
== Testing your build ==
 
== Testing your build ==
  
Once you've compiled M5, you can verify that the build worked by running regression tests.     Regression tests are also run via scons.  The command to run all tests for a particular is constructed as follows:
+
Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons.  The command to run all tests for a particular is constructed as follows:
  
 
  % scons build/<target>/tests/<binary>
 
  % scons build/<target>/tests/<binary>
  
For example, to run the regression tests on ALPHA_FS/m5.opt, type:
+
For example, to run the regression tests on ALPHA/gem5.opt, type:
  
  % scons build/ALPHA_FS/tests/opt
+
  % scons build/ALPHA/tests/opt
  
The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA_FS/m5.opt if necessary before running the tests.  Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run.  If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.
+
The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests.  Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run.  If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.
  
 
Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime.  You can run only the tests in a particular category by adding that category name to the target path, e.g.:
 
Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime.  You can run only the tests in a particular category by adding that category name to the target path, e.g.:
  
  % scons build/ALPHA_FS/tests/opt/quick
+
  % scons build/ALPHA/tests/opt/quick
  
(Note that currently the "medium" and "long" categories are empty; all of the tests are "quick".)
+
(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)
  
 
Specific tests can be run by appending the test name:
 
Specific tests can be run by appending the test name:
  
  % scons build/ALPHA_FS/tests/opt/quick/10.linux-boot
+
  % scons build/ALPHA/tests/opt/quick/fs/10.linux-boot
 +
 
 +
For more details, see [[Regression Tests]].
 +
 
 +
== Compiling with runtime checks ==
  
For more details, see [[Regression tests]].
+
GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, [https://github.com/google/sanitizers/wiki/AddressSanitizer AddressSanitizer] and [https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html UndefinedBehaviorSanitizer] are supported with the flags <code>--with-asan</code> and <code>--with-ubsan</code> respectively. When ASAN is enabled, tcmalloc must be disabled (<code>--without-tcmalloc</code>), because it confuses ASAN's shadow memory.

Latest revision as of 11:07, 24 September 2019

gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there. Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.

Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).

To build gem5, you need to ensure you have all the dependencies installed.

Possible Targets

gem5 can build many binaries each for a different guest architecture. The currently available architectures are ALPHA, ARM, MIPS, POWER, SPARC, and X86. In addition there is a NULL architecture.

For each possible architecture and mode, several different executables can be built:

  • gem5.debug - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
  • gem5.opt - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
  • gem5.prof - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
  • gem5.perf - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.
  • gem5.fast - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization

Compiling

Starting in the root of the source tree, you can build gem5 using a command of the form:

% scons build/<arch>/gem5.<binary>

where the items between <> are the architectures, modes and binaries listed above. For example:

% cd gem5
% scons build/ALPHA/gem5.debug

scons: Reading SConscript files ...
Checking for C header file fenv.h... yes
Building in /tmp/gem5/build/ALPHA
Options file /tmp/gem5/build/options/ALPHA not found,
  using defaults in build_opts/ALPHA
Compiling in ALPHA with MySQL support.
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
    -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
    -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
    -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
    build/ALPHA/base/circlebuf.cc
...

If your output looked like the above, then congratulations, you've compiled gem5! The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug. For more build options and further details about the build system, see the SCons build system page.

Installing full system files

If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the Download page.

The path to these files is determined in configs/common/SysPaths.py. There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like ImportError: Can't find a path to system files. when you first attempt to run the simulator in full-system mode.

Note that the default path, /dist/m5/system, is designed for environments where you have root (sudo) access (to create /dist) and want the files in a place where they can be shared by multiple users. If both of these are true, you can follow this example to put the system files at the default location:

% sudo mkdir -p /dist/m5/system
% cd /dist/m5/system
% sudo tar vxfj <path>/m5_system_2.0b3.tar.bz2
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
% sudo chgrp -R <grp> /dist  # where <grp> is a group that contains all the m5 users

In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.

Testing your build

Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons. The command to run all tests for a particular is constructed as follows:

% scons build/<target>/tests/<binary>

For example, to run the regression tests on ALPHA/gem5.opt, type:

% scons build/ALPHA/tests/opt

The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests. Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run. If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.

Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime. You can run only the tests in a particular category by adding that category name to the target path, e.g.:

% scons build/ALPHA/tests/opt/quick

(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)

Specific tests can be run by appending the test name:

% scons build/ALPHA/tests/opt/quick/fs/10.linux-boot

For more details, see Regression Tests.

Compiling with runtime checks

GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, AddressSanitizer and UndefinedBehaviorSanitizer are supported with the flags --with-asan and --with-ubsan respectively. When ASAN is enabled, tcmalloc must be disabled (--without-tcmalloc), because it confuses ASAN's shadow memory.