Difference between revisions of "Compiling M5"

From gem5
Jump to: navigation, search
(Source Tree)
m (Added "Compiling with runtime checks")
 
(45 intermediate revisions by 6 users not shown)
Line 1: Line 1:
M5 runs on Linux, Mac OS X and OpenBSD, but should be easily portable to other Unix-like OSes. Cross-endian support has been mostly added, however this has not been extensively tested. All the syscall emulation (_SE) targets support running regardless of host/target endianess, however at present ALPHA_FS does not fully support big endian machines.  
+
gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there.  Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.
  
 +
Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).
  
=== Required Software ===
+
To build gem5, you need to ensure you have all the [[dependencies]] installed.
To build M5, you will need the following software:
 
  
* [http://gcc.gnu.org/ g++] version 3.4 or newer. Once upon a time, M5 built with g++ 2.95, but it probably doesn't anymore; < 3.4 seems to have some issues with templates.
+
== Possible Targets ==
* [http://www.python.org Python], version 2.4 or newer.
+
gem5 can build many binaries each for a different guest architecture. The currently available architectures are '''ALPHA''', '''ARM''', '''MIPS''', '''POWER''', '''SPARC''', and '''X86'''. In addition there is a '''NULL''' architecture.
* [http://www.scons.org SCons], version 0.96.91 or newer.
 
* [http://www.swig.org SWIG], version 1.3.28 or newer.
 
  
SCons is a powerful replacement for make. See [http://sourceforge.net/project/showfiles.php?group_id=30337 here] to download SCons. If you do not have administrator privileges on the machine you are trying to compile on, you can use the "scons-local" package to install scons in your m5 directory, or install SCons in your home directory using the '--prefix=' option.
+
For each possible architecture and mode, several different executables can be built:
 +
*'''gem5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
 +
*'''gem5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
 +
*'''gem5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
 +
*'''gem5.perf''' - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.
 +
*'''gem5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization
  
There a few utility scripts written in Perl, but Perl is not necessary to build or run the simulator.
+
== Compiling ==
  
=== Possible Targets ===
+
Starting in the root of the source tree, you can build gem5 using a command of the form:
M5 can build many binaries each for a different guest architecture, simulation mode, and use. The currently available architectures are '''ALPHA''', '''SPARC''', and '''MIPS'''.
 
  
Each of these can built in two simulation modes:
+
% scons build/<arch>/gem5.<binary>
*'''FS''' - Full System mode. This mode simulates a complete system including a kernel, I/O devices, etc. This mode currently only works with the '''ALPHA''' architecture.
 
*'''SE''' - Syscall Emulation mode. This mode simulates statically compiled binaries by functionally emulating any syscall they make.
 
  
Possible Binaries
+
where the items between &lt;&gt; are the architectures, modes and binaries listed above.  For example:
*'''m5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly. (-g3 -gdwarf-2 -O0).
 
*'''m5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target. (-g -O3)
 
*'''m5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof. (-O3 -g -pg).
 
*'''m5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (including trace support).  (-O3 -DNDEBUG)
 
 
 
=== Source Tree ===
 
 
 
Once you have obtained and unpacked the M5 sources, the root of your of your M5 source tree should have three directories:
 
 
 
*'''build_opts''' - Default build options for each target (e.g., what set of build flags does the ALPHA_FS target imply)
 
*'''configs''' - Sample script files that can be used to run M5 and as a starting place for your own configurations
 
*'''ext''' - External packages used by M5 that are not commonly installed on systems
 
*'''src''' - Source code for the simulator itself
 
**'''arch''' -  Architecture specific files as well as the ISA parsing code
 
***'''alpha''' - Files specific to the Alpha architecture
 
****'''freebsd''' - Files specific to the Alpha architecture and to the FreeBSD operating system
 
****'''isa''' - ISA description files for the Alpha architecture
 
****'''linux''' - Files specific to the Alpha architecture and to the Linux operating system
 
****'''tru64''' - Files specific to the Alpha architecture and to the Tru64 operating system
 
***'''mips''' - Files specific to the MIPS architecture
 
****'''isa''' - ISA description files for the MIPS architecture
 
****'''linux''' - Files specific to the MIPS
 
***'''sparc''' - Files specific to the SPARC architecture
 
****'''isa''' - ISA description files for the SPARC architecture
 
****'''linux''' -  Files specific to the SPARC architecture and to the Linux operating system
 
****'''solaris''' -  Files specific to the SPARC architecture and to the Linux operating system
 
**'''base''' - General data structures and facilities that could be useful for another project
 
***'''loader''' - Code for loading binaries and reading symbol tables
 
***'''stats''' - Code for keeping statistics and writing the data to a file or a database
 
**'''cpu''' - CPU models for the simulator
 
**'''dev''' - I/O device models for the simulator
 
**'''docs''' - Doxygen templates & output
 
**'''encumbered''' - Code that has a different, more restrictive license than M5's
 
**'''kern''' - Operating system but architecture independent code (e.g. types of data structures).
 
***'''linux''' - Linux specific architecture independent code
 
***'''solaris''' -Solaris specific architecture independent code
 
***'''tru64''' - Tru64 specific architecture independent code
 
**'''mem''' - Models for the memory system
 
**'''python''' - Python code for configuration and higher level functions
 
**'''sim'''  - Simulator specific base functionality
 
*'''system''' - This directory contains platform dependent code such as BIOS, hypervisor, that someone using M5 may want to modify.
 
**'''alpha''' - Alpha console and palcode
 
**'''mips''' - none yet
 
**'''sparc''' - none yet
 
*'''tests''' - Regression test framework
 
**'''configs''' - Test configurations
 
**'''test-progs''' - Binaries and inputs for test programs
 
**'''quick''', '''medium''', '''long''' - Actual regression tests, including reference ouputs, grouped by runtime
 
*'''util''' - Random programs that are used in conjunction with M5 such as Python scripts to process data in an DB, submit jobs to a batch processing system, etc.
 
 
 
=== Compiling ===
 
Starting in the root of this tree, you can build M5 and test your build using a command of the form:
 
 
 
scons build/<arch>_<type>/m5.<target>
 
 
 
where the items between &lt;&gt; are the architectures, types and targets listed above.  For example:
 
  
 
<pre>
 
<pre>
% cd m5
+
% cd gem5
% scons build/ALPHA_FS/m5.debug
+
% scons build/ALPHA/gem5.debug
  
 
scons: Reading SConscript files ...
 
scons: Reading SConscript files ...
 
Checking for C header file fenv.h... yes
 
Checking for C header file fenv.h... yes
Building in /tmp/newmem/build/ALPHA_FS
+
Building in /tmp/gem5/build/ALPHA
Options file /tmp/newmem/build/options/ALPHA_FS not found,
+
Options file /tmp/gem5/build/options/ALPHA not found,
   using defaults in build_opts/ALPHA_FS
+
   using defaults in build_opts/ALPHA
Compiling in ALPHA_FS with MySQL support.
+
Compiling in ALPHA with MySQL support.
 
scons: done reading SConscript files.
 
scons: done reading SConscript files.
 
scons: Building targets ...
 
scons: Building targets ...
g++ -o build/ALPHA_FS/base/circlebuf.do -c -pipe -fno-strict-aliasing
+
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA_FS
+
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
     build/ALPHA_FS/base/circlebuf.cc
+
     build/ALPHA/base/circlebuf.cc
 
...
 
...
  
 
</pre>
 
</pre>
  
If your output looked like the above, then congratulations, you've compiled M5!  
+
If your output looked like the above, then congratulations, you've compiled gem5! The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug.  For more build options and further details about the build system, see the [[SCons build system]] page.
 +
 
 +
== Installing full system files ==
 +
 
 +
If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the [[Download]] page.
 +
 
 +
The path to these files is determined in configs/common/SysPaths.py.  There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like <code>ImportError: Can't find a path to system files.</code> when you first attempt to run the simulator in full-system mode.
 +
 
 +
Note that the default path, <code>/dist/m5/system</code>, is designed for environments where you have root (sudo) access (to create <code>/dist</code>) and want the files in a place where they can be shared by multiple users.  If both of these are true, you can follow this example to put the system files at the default location:
 +
 
 +
<pre>
 +
% sudo mkdir -p /dist/m5/system
 +
% cd /dist/m5/system
 +
% sudo tar vxfj &lt;path&gt;/m5_system_2.0b3.tar.bz2
 +
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
 +
% sudo chgrp -R <grp> /dist  # where <grp> is a group that contains all the m5 users
 +
</pre>
 +
 
 +
In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.
 +
 
 +
== Testing your build ==
 +
 
 +
Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons.  The command to run all tests for a particular is constructed as follows:
 +
 
 +
% scons build/<target>/tests/<binary>
 +
 
 +
For example, to run the regression tests on ALPHA/gem5.opt, type:
 +
 
 +
% scons build/ALPHA/tests/opt
 +
 
 +
The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests.  Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run.  If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.
 +
 
 +
Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime.  You can run only the tests in a particular category by adding that category name to the target path, e.g.:
 +
 
 +
% scons build/ALPHA/tests/opt/quick
 +
 
 +
(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)
 +
 
 +
Specific tests can be run by appending the test name:
 +
 
 +
% scons build/ALPHA/tests/opt/quick/fs/10.linux-boot
 +
 
 +
For more details, see [[Regression Tests]].
 +
 
 +
== Compiling with runtime checks ==
  
Each configuration is built in a separate subdirectory of m5/build. For each configuration, the various flavors of binaries are built in the same directory. The default binary that you built above is ALPHA_FS/m5.debug. You can build other binaries by specifying them on the scons command line, e.g. '<code>scons build/ALPHA_SE/m5.opt</code>' or '<code>scons build/MIPS_SE/m5.debug</code>'.
+
GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, [https://github.com/google/sanitizers/wiki/AddressSanitizer AddressSanitizer] and [https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html UndefinedBehaviorSanitizer] are supported with the flags <code>--with-asan</code> and <code>--with-ubsan</code> respectively. When ASAN is enabled, tcmalloc must be disabled (<code>--without-tcmalloc</code>), because it confuses ASAN's shadow memory.

Latest revision as of 11:07, 24 September 2019

gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there. Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.

Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).

To build gem5, you need to ensure you have all the dependencies installed.

Possible Targets

gem5 can build many binaries each for a different guest architecture. The currently available architectures are ALPHA, ARM, MIPS, POWER, SPARC, and X86. In addition there is a NULL architecture.

For each possible architecture and mode, several different executables can be built:

  • gem5.debug - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
  • gem5.opt - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
  • gem5.prof - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
  • gem5.perf - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.
  • gem5.fast - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization

Compiling

Starting in the root of the source tree, you can build gem5 using a command of the form:

% scons build/<arch>/gem5.<binary>

where the items between <> are the architectures, modes and binaries listed above. For example:

% cd gem5
% scons build/ALPHA/gem5.debug

scons: Reading SConscript files ...
Checking for C header file fenv.h... yes
Building in /tmp/gem5/build/ALPHA
Options file /tmp/gem5/build/options/ALPHA not found,
  using defaults in build_opts/ALPHA
Compiling in ALPHA with MySQL support.
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
    -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
    -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
    -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
    build/ALPHA/base/circlebuf.cc
...

If your output looked like the above, then congratulations, you've compiled gem5! The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug. For more build options and further details about the build system, see the SCons build system page.

Installing full system files

If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the Download page.

The path to these files is determined in configs/common/SysPaths.py. There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like ImportError: Can't find a path to system files. when you first attempt to run the simulator in full-system mode.

Note that the default path, /dist/m5/system, is designed for environments where you have root (sudo) access (to create /dist) and want the files in a place where they can be shared by multiple users. If both of these are true, you can follow this example to put the system files at the default location:

% sudo mkdir -p /dist/m5/system
% cd /dist/m5/system
% sudo tar vxfj <path>/m5_system_2.0b3.tar.bz2
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
% sudo chgrp -R <grp> /dist  # where <grp> is a group that contains all the m5 users

In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.

Testing your build

Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons. The command to run all tests for a particular is constructed as follows:

% scons build/<target>/tests/<binary>

For example, to run the regression tests on ALPHA/gem5.opt, type:

% scons build/ALPHA/tests/opt

The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests. Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run. If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.

Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime. You can run only the tests in a particular category by adding that category name to the target path, e.g.:

% scons build/ALPHA/tests/opt/quick

(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)

Specific tests can be run by appending the test name:

% scons build/ALPHA/tests/opt/quick/fs/10.linux-boot

For more details, see Regression Tests.

Compiling with runtime checks

GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, AddressSanitizer and UndefinedBehaviorSanitizer are supported with the flags --with-asan and --with-ubsan respectively. When ASAN is enabled, tcmalloc must be disabled (--without-tcmalloc), because it confuses ASAN's shadow memory.