Difference between revisions of "Documentation"

From gem5
Jump to: navigation, search
(Command line options)
(100 intermediate revisions by 13 users not shown)
Line 1: Line 1:
Outline for some new documentation. Maybe we should be doing this wit LaTeX and using pandoc (LaTeX->mediawiki converter)?
Welcome to the gem5 documentation. We've worked hard to add and organize everything in a more reasonable format, however we still haven't got to everything. All the documentation is a wiki, so we encourage you to add and edit sections as you find omissions and bugs.
==Getting Started==
For an overview, we have posted slides and most importantly '''video''' of a tutorial at HiPEAC Computer Systems Week. See [[Tutorials]].
===What is gem5?===
'''Old Documentation:''' We have some old documentation and have been trying to move everything to the new format, but we haven't quite succeeded yet. If you don't find the answer you're looking for here, you might want to check [[OldDocumentation | the old documentation]].
gem5 is a modular discrete event driven computer system simulator platform. That means that:
== Getting Started ==
# [[Introduction]] - A quick introduction to gem5.
# gem5's components can be rearranged, parameterized, extended or replaced easily to suite your needs.
# [[Source Code]] - Information about the source code itself.
# It simulates the passing of time as a series of discrete events.
# [[Dependencies]] - Things you'll need that aren't part of gem5 itself.
# It's intended use is to simulate one or more computer systems in various ways.
# [[Build System]] - How to run or modify gem5's build system.
# It's more than just a simulator, it's a simulator platform that lets you use as many of its premade components as you want to build up your own simulation system.
# [[Download | Full System and Benchmark Files]] - Disk images for operating system and benchmark applications, and how to install them.
# [[gem5 101]] - Six part course covering all the basics of gem5 (and some advanced material too).
gem5 is written primarily in C++ and python and most components are provided under a BSD style license. It can simulate a complete system with devices and an operating system in full system mode (FS mode), or user space only programs where system services are provided directly by the simulator in syscall emulation mode (SE mode). There are varying levels of support for executing Alpha, ARM, MIPS, Power, SPARC, and 64 bit x86 binaries on CPU models including two simple single CPI models, an out of order model, and an in order pipelined model. A memory system can be flexibly built out of caches and interconnects. Recently the Ruby simulator has been integrated with gem5 to provide even better, more flexible memory system modeling.
There are many components and features not mentioned here, but from just this partial list it should be obvious that M5 is a sophisticated and capable simulation platform. Even with all gem5 can do today, active development continues through the support of individuals and some companies, and new features are added and existing features improved on a regular basis. For the most up to date information you can check out the project's website at www.gem5.org.
===How am I expected to use gem5?===
gem5 is a framework to do computer architecture research. It has many features, but if you're trying to do something new and novel in the field it probably won't be able to evaluate your idea out-of-the-box. This is expected, because if it did have all the functionality required, that probably means that someone already evaluated a similar idea and has probably published about it.
To get the most out of gem5, you're going to have to get your hands dirty and add functionality. We hope we've made gem5 modular enough that you can do this with only the understanding of the portion of the simulator you're editing, but nothing is perfect.
Hopefully, you'll find gem easy to use, but if you run into gaps in this documentation please feel free to ask questions on the mailing lists after reading: http://www.m5sim.org/wiki/index.php/Reporting_Problems When you do solve your problem, please update the documentation as appropriate so that the next user can benefit from your experience. Additionally, please consider contributing your code back to the gem5 development repository. Only with contributions from the community can the simulator continue to grow.
===Pointer to gem5 Capabilities/Recipes===
===Getting a copy===
M5's source code is managed using the mercurial revision control system. There are several repositories you may be interested in:
# m5 – The main repository is where active development takes place.
# m5-stable – A repository which lags behind “m5” repository but has basically the same contents. It’s usually better to use “m5” instead of “m5-stable”
# encumbered – A repository for extensions to M5 that are under a different, more restrictive license. Currently this only includes support for SimpleScalar's EIO trace format.
# linux-patches – A repository for patches to the linux kernel that modify it so it can be simulated more efficiently. These patches are optional, but it's a good idea to use them if possible to cut down on simulation run time.
To check out a copy, first, make sure you have mercurial installed on your system and that you can run the '''hg''' command. Then use '''hg clone''' to create your own local copy using the URL '''http://repo.m5sim.org/XXX''' where XXX is replaced by the name of the repository your interested in. For example, to check out the main repository, you'd use the command:
hg clone http://repo.m5sim.org/m5
You can find out more about mercurial and its commands using its built in help by running:
hg help
or by visiting any of the following websites:
* [http://hginit.com/ hginit] provides a good introduction to the basic Mercurial concept
* [http://mercurial.selenic.com/guide/ Mercurial Guide] also provides a good basic introduction
* [http://hgbook.red-bean.com/read/ HGBook] is the definitive guide to Mercurial
M5 uses the scons build system which is based on python. To build the simulator binary, run scons from the top of the source directory with a target of the form '''build/<config>/<binary>''' where '''<config>''' is replaced with one of the predefined set of build parameters and '''<binary>''' is replaced with one of the possible m5 binary names. The predefined set of parameters determine build wide configuration settings that affect the behavior, composition, and capabilities of the binary being built. These include whether the simulator will run in FS or SE mode, if Ruby support is included, which ISA will be supported, which CPU models to build, and what coherence protocol Ruby should use. Examples are ARM_FS, X86_SE, and ALPHA_SE_MOESI_CMP_token. All of the available options can be found in the build_opts directory, and it should be fairly easy to see what each is for. We'll talk about the build system in more detail later. Valid binary names are m5.debug, m5.opt, m5.fast, and m5.prof. These binaries all have different properties suggested by their extension. m5.debug has optimization turned off to make debugging easier in tools like gdb, m5.opt has optimizations turned on but debug output and asserts left in, m5.fast removes those debugging tools, and m5.prof is built to use with gprof. Normally you'll want to use m5.opt. To build the simulator in syscall emulation mode with Alpha support, optimizations turned on, and debugging left in, you would run:
scons build/ALPHA_SE/m5.opt
In your source tree, you'd then find a new '''build/ALPHA_FS/''' directory with the requested '''m5.opt''' in it. For the rest of this chapter we'll assume this is the binary you're using.
Now that you've built M5, it's time to try running it. An M5 command line is composed of four parts, the binary itself, options for M5, a configuration script to run, and then finally options for the configuration script. Several example configuration scripts are provided in the “configs/example” directory and are generally pretty powerful. You are encouraged to make your own scripts, but these are a good starting point. The example script we'll use in SE mode is called se.py and sets up a basic SE mode simulation for us. We'll tell it to run the hello world binary provided in the M5 source tree.
build/ALPHA_SE/m5.opt configs/example/se.py -c tests/test-progs/hello/bin/alpha/linux/hello
This builds up a simulated system, tells it to run the binary found at the location specified, and kicks off the simulation. As the binary runs, it's output is sent to the console by default and looks like this:
M5 Simulator System
Copyright (c) 2001-2008
The Regents of The University of Michigan
All Rights Reserved
M5 compiled Feb 12 2011 20:43:55
M5 revision e00ef55a2c49 7933 default tip
M5 started Feb 12 2011 20:45:47
M5 executing on fajita
command line: build/ALPHA_SE/m5.opt configs/example/se.py -c tests/test-progs/hello/bin/alpha/linux/hello
Global frequency set at 1000000000000 ticks per second
0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000
info: Entering event queue @ 0. Starting simulation...
info: Increasing stack size by one page.
Hello world!
hack: be nice to actually delete the event here
Exiting @ tick 3240000 because target called exit()
You can see a lot of output from the simulator itself, but the line “Hello world!” came from the simulated program. In this example we didn't provide any options to M5 itself. If we had, they would have gone on the command line between m5.opt and se.py. If you'd like to see what command line options are supported, you can pass the --help option to either M5 or the configuration script. The two groups of options are different, so make sure you keep track of whether they go before or after the configuration script.
===Asking for help===
M5 has two main mailing lists where you can ask for help or advice. m5-dev is for developers who are working on the main version of M5. This is the version that's distributed from the website and most likely what you'll base your own work off of. m5-users is a larger mailing list and is for people working on their own projects which are not, at least initially, going to be distributed as part of the official version of M5. Most of the time m5-users is the right mailing list to use. Most of the people on m5-dev are also on m5-users including all the main developers, and in addition many other members of the M5 community will see your post. That helps you because they might be able to answer your question, and it also helps them because they'll be able to see the answers people send you. To find more information about the mailing lists, to sign up, or to look through archived posts visit:
===Cross-product status matrix===
gem5 combines several different ISAs, system modes (SE or FS), CPU models and memory models under a single infrastructure.  The cross-product of these combinations may or may not actually work.  The following page describes the current status of these combinations.
==Source Code==
===Tour of the tree===
These are the files and directories at the top of the tree:
AUTHORS  LICENSE  README  RELEASE_NOTES  SConstruct  build_opts  configs  ext  src  system  tests  util
AUTHORS, LICENSE, README are files with general information about the simulator. AUTHORS is a list of people who have historically contributed to M5. LICENSE has the license terms that applies to M5 as a whole, unless overridden by a more specific license. README has some very basic information introducing M5 and explaining how to get started.
The SConstruct file is part of the build system, as is the build_opts directory. build_opts holds files that define default settings for build different build configurations. These include X86_FS and MIPS_SE, for instance.
The configs directory is for simulation configuration scripts which are written in python. These are described in more detail later. The files in this directory help make writing configurations easier by providing some basic prepackaged functionality, and include a few examples which can be used directly or as a starting point for your own scripts.
The ext directory is for things M5 depends on but which aren’t actually part of M5. Specifically these are for dependencies that are harder to find, not likely to be available, or where a particular version is needed.
The src directory is where most of M5 is located. This is where all of the C++ and python source that contributes to the M5 binary is kept, excluding components in the ext directory.
The system directory is for the source for low level software like firmware or bootloaders for use in simulated systems. Currently this includes Alpha’s PAL and console code, and a simple bootloader for ARM.
The tests directory stores files related to M5’s regression tests. These include the scripts that build up the configurations used in the tests and reference outputs. Simple hello world binaries are also stored here, but other binaries need to be downloaded separately.
Finally, in the util directory are utility scripts, programs and useful files which are not part of the M5 binary but are generally useful when working on M5.
===Style rules===
===Generated files - where do they end up===
===.m5 config files===
===jobfile to run multiple jobs===
==Build System==
M5's build system is based on Scons, an open source build system implemented in python. You can find more information about scons at www.scons.org. The main scons file is called SConstruct and is found in the root of the source tree. additional scons files are called SConscript and are found throughout the tree, usually near the files they're associated with.
===Build targets===
In M5, scons build targets are of the form <build dir>/<configuration>/<target>. The <build dir> part of the target is a directory path that ends in "build". Typically this is simply "build" by itself, but you can specify a directory called "build" located somewhere else instead. The <configuration> part selects a set of preset build configuration variables values which correspond to common . The possible options are the file names in the build_opts directory, and are discussed more below. The build targets can be regression tests which are explained in more detail below, or they can be different versions of the M5 binary. The name of the binary is "m5" with an extension that specifies what version to build. Currently supported versions are m5.debug, m5.opt, m5.fast, and m5.prof.
m5.debug has optimizations turned off. This ensures that variables won't be optimized out, functions won't be unexpectedly inlined, and control flow will won't behave in surprising ways. That makes this version easier to work with in tools like gdb, but without optimizations this version is significantly slower than the others. You should choose it when using tools like gdb and valgrind and don't want any details obscured, but other wise more optimized versions are recommended.
m5.opt has optimizations turned on and debugging functionality like asserts and DPRINTFs left in. This gives a good balance between the speed of the simulation and insight into what's happening in case something goes wrong. This version is best in most circumstances.
m5.fast has optimizations turned on and debugging functionality compiled out. This pulls out all the stops performance wise, but does so at the expense of run time error checking and the ability to turn on debug output. This version is recommended if you're very confident everything is working correctly and want to get peak performance from the simulator.
m5.prof is similar to m5.fast but also includes instrumentation that allows it to be used with the gprof profiling tool. This version is not needed very often, but can be used to identify the areas of M5 that should be focused on to improve performance.
These versions are summarized in the following table.
{| border="1" cellpadding="10"
! Binary name !! Optimizations !! Run time debugging support !! Profiling support
| m5.debug || || X ||
| m5.opt || X || X ||
| m5.fast || X || ||
| m5.prof || X ||  || X
===Command line options===
Scons will recognize the following command line options specific to M5.
{| border="1" cellpadding="10" class="wikitable"
! Option !! Effect
| --color || Turn on colorized output
| --no-color || Turn off colorized output
===Environment variables===
The following environment variables are imported from the host environment for use in scons:
{| border="1" cellpadding="10"
! Variable !! Use
| AS || Assembler command
| AR || Archive tool command
| CC || C compiler command
| CXX || C++ compiler command
| HOME || User's home directory
| LD_LIBRARY_PATH || Path to search for library files
| PATH || Path to search for programs
| PYTHONPATH || Path to search for python files
| RANLIB || Ranlib command
| M5_CONFIG || Where to look for the special ".m5" directory
| M5_DEFAULT_BINARY || The default build target which overrides the default default build/ALPHA_SE/m5.debug
===Configuration variables===
These configuration variables are used to control the way M5 is built. Some are global, affecting all configurations built in the same build directory. Some are sticky, which means they retain their value once set. The available options are summarized in the following tables.
====Global sticky====
{| border="1" cellpadding="10"
! Variable !! Description !! Default
| CC || C Compiler || CC environment variable or value determined by scons
| CXX || C++ Compiler || CXX environment variable or value determined by scons
| BATCH || Use batch pool for build and tests || False
| BATCH_CMD || Batch pool submission command || qdo
| M5_BUILD_CACHE || Cache built objects in this directory || False
| EXTRAS || Add extra directories to the compilation ||
====Global non-sticky====
{| border="1" cellpadding="10"
! Variable !! Description !! Default
| VERBOSE || Print full tool command lines || False
| update_ref || Update test reference outputs || False
====Non-global sticky====
{| border="1" cellpadding="10"
! Variable !! Description !! Default !! Exported as config/*.hh
| TARGET_ISA || Target ISA || alpha || X
| FULL_SYSTEM || Full-system support || False || X
| CPU_MODELS || CPU Models || AtomicSimpleCPU,InOrderCPU,O3CPU,TimingSimpleCPU ||
| NO_FAST_ALLOC || Disable fast object allocator || False || X
| FAST_ALLOC_DEBUG || Enable fast object allocator debugging || False || X
| FAST_ALLOC_STATS || Enable fast object allocator statistics || False || X
| EFENCE || Link with Electric Fence malloc debugger || False ||
| SS_COMPATIBLE_FP || Make floating-point results compatible with SimpleScalar || False || X
| USE_SSE2 || Compile for SSE2 (-msse2) to get IEEE FP on x86 hosts || False ||
| USE_MYSQL || Use MySQL for stats output || whether mysql_config was found || X
| USE_POSIX_CLOCK || Use POSIX Clocks || whether posix clocks are available on this host || X
| USE_FENV || Use <fenv.h> IEEE mode control || whether fenv.h was found on this host || X
| USE_CHECKER || Use checker for detailed CPU models || False || X
| CP_ANNOTATE || Enable critical path annotation capability || False || X
| RUBY || Build with Ruby || False ||
| PROTOCOL || Coherence protocol for Ruby || MI_example || X
| NO_HTML || Do not create HTML files || False ||
| NO_VECTOR_BOUNDS_CHECKS || Don't do bounds checks || True || X
| GEMS_ROOT || Add debugging stuff to Ruby || the directory where GEMS is stored, currently src/mem || X
====Setting configuration variable values====
The first way you set configuration variable values is through the configuration name you choose as part of the build target. This file is loaded from build_opts and contains preset values for some of these variables which configures the build as the file name suggests. For instance, using the SPARC_FS configuration will turn on full system mode with the FULL_SYSTEM variable and set the TARGET_ISA to sparc.
It is important to note that the values in the file corresponding to the configuration you picked are -default- values and are only used if no directory already exists with its own values already in place. Those files are for defining reasonable starting points to configure M5 to behave the way you want it to, and are not intended to actively configure a particular build.
If you want to change a value after your build and configuration directory is already created, or if you want to override a value as it's created, you can specify the new values on the command line. The syntax is similar to setting environment variables at a shell prompt, but these go after the scons command. For example, to turn on RUBY support for an existing ALPHA_FS build, you could use the following command.
scons RUBY=True build/ALPHA_FS/m5.opt
It's often a good idea to add --help to the scons command line which will print out all of the configuration variables and what their values are. This way you can make sure everything is set up like you want, and that you don't have any typos in any variable names. If everything is as you expect, you can remove --help to actually start the build.
===Running regressions===
===Adding files to the build===
===Using EXTRAS===
===The python side===
====[[Python_Parameter_Types|Param types]] ====
====Special attribute names====
====Rules for importing - how to get what====
====Pro tips - avoiding cycles, always descend from root, etc.====
===The C++ side===
====create functions====
====Stages of initialization====
====Header files to include====
==Configuration Scripts==
===SimObject hierarchy===
===Explanation of infrastructure scripts in configs===
===How to use se.py and fs.py.===
===How to use other top level scripts (what are they?)===
===Where M5 looks for files===
==General memory system==
===Ports system===
====Ports in general====
====Various port types====
====Atomic/Timing/Functional accesses====
===Two memory systems: Classic and Ruby===
====Classic memory system====
Summarize functionality and highlight advantages and disadvantages.
Uses ports to connect all components.
====Ruby memory system====
Summarize functionality and highlight advantages and disadvantages.
Uses ports to connect cpus to the Ruby memory system.
Uses message buffers to connect components within the memory system.
==Classic memory system==
====Hooking them up====
====Anything else?====
=== High level components of Ruby ===
  Need to talk about the overview of Ruby and what are the major components.
''Rathijit will do it''
==== SLICC + Coherence protocols: ====
    Need to say what is SLICC and whats its purpose.
    Talk about high level strcture of a typical coherence protocol file, that SLICC uses to generate code.
    A simple example structure from protocol like MI_example can help here.
''Nilay will do it''
==== Protocol independent memory components ====
# Cache Memory
# Replacement Policies
# Memory Controller
''Arka will do it''
==== Topologies and Networks ====
There is a separation between the network topology and the implementation.
''Rathijit will do it''
=== Implementation of Ruby ===
  Low level details goes here. Need to explain code directory structure as well.
==== SLICC ====
    Explain functionality/ capability of SLICC
    Talk about
    AST, Symbols, Parser and code generation in some details but NO need to cover every file and/or functions.
    Few examples should suffice.
''Nilay will do it''
==== Protocols ====
    Need to talk about each protocol being shipped. Need to talk about protocol specific configuration parameters.
    NO need to explain every action or every state/events, but need to give overall idea and how it works
    and assumptions (if any).
===== MI example =====
MI protocols don't support LL/SC semantics.
''Rathijit will do it''
===== MOESI_hammer =====
''Somayeh will do it''
===== MOESI_CMP_token =====
''Shoaib will do it''
===== MOESI_CMP_directory =====
''Rathijit will do it''
===== MESI_CMP_directory =====
''Arka will do it''
==== Protocol Independent Memory components ====
===== System =====
''Arka will do it''
===== Sequencer =====
''Arka will do it''
===== CacheMemory =====
''Arka will do it''
===== DMASequencer =====
''Derek will do it''
===== Memory Controller =====
''Rathijit will do it''
==== Topologies and Networks ====
===== Topology specification =====
Python files specify connections.  Shortest path graph traversals program the routing tables.
===== Network implementation =====
# SimpleNetwork
# Garnet
==== Life of a memory request in Ruby ====
Cpu model generates a packet -> RubyPort converts it to a ruby request -> L1 cache controller converts it to a protocol specific message ...etc.
''Arka will do it''
===Event queue===
===EventManager objects (I don’t know a lot about these)===
===Event objects===
===Time sync===
===I/O device base classes===
===PCI devices===
===DMA devices===
===UARTs and serial terminals===
===Explanation of platforms and systems, how they’re related, and what they’re each for===
==Execution Basics==
===Microcode support===
==Architectural State==
====Register types - float, int, misc====
====Indexing - register spaces stuff====
==Address Translation==
===TLB management===
===Delayed translation===
===Page table walkers (defer to ISA discussion?)===
===Differences between SE and FS mode===
===Simple CPU===
====Flow of execution====
====Sharing of code between atomic and timing====
===O3 CPU===
====Anatomy of the pipeline====
====Load/store handling====
===InOrder (what to say here?)===
===Traditional handling with platform object===
===New, currently X86 only system===
====Interrupt sources/sinks/wires/pins====
====Interrupt messages====
===Interrupt managing objects in the CPU===
==ISA parser==
===decode tree===
===let blocks===
===microcode assembler===
====rom object====
===Lots more stuff===
==Multiple ISA support==
===Switching header files===
===Specialized classes and functions===
===Matrix of supported ISA/mode/CPU combinations===
==Alpha Implementation==
==ARM Implementation==
===Supported features and modes===
===Thumb support===
===Special PC management===
==MIPS Implementation==
==Power Implementation==
==SPARC Implemenation==
===Supported features and modes===
==X86 Implementation==
===[[X86_microop_ISA | Microcode ISA]]===
===Macroop specialization===
===Platform objects===
===Physical memory space allocation===
===Microcode registers and other hidden implementation details===
===Supported features and modes===
==Pseudo Instructions==
===List of pseudo instructions and what they do===
===How to get at and use them from a binary, script, etc.===
==SE mode==
===Loading binaries===
===System call instruction handling===
===Implementation of various system calls===
==Statistics (don’t know what to say here)==
== Summary: gem5 Capabilities/Recipes ==
=== Atomic mode ===
Only supported by the classic memory system.
=== Timing mode ===
==== Classic memory system ====
==== Ruby memory system ====
===== MOESI_hammer in ALPHA_SE mode running multiple SPEC benchmarks with Mesh topology =====
Highlight no DMA controllers
===== ??? FS mode =====
Highlight DMA controllers and how they are connected in the network.
=== Atomic mode then timing mode  ===
Only supported with the classic memory system.
==Utility Code==
===Bitfield functions===
===BitUnion classes===
===panic, warn, etc., when to use what===
===random number generation===
===reference counting pointers===
===Trace flags===
===State trace/Tracer objects===
===Remote GDB support===
===Unit tests===
==Development Tools/Contributing==
his page describes the methods available to extend M5 while preserving the ability to update to new versions. If you intend to make any changes to M5 we '''strongly''' advise you to follow one of the following methods. It will save you a great deal of time in the future and allow you to take advantage of new M5 versions without the error prone process of manually diffing and patching versions.
There are two recommend ways to add your own functionality to M5 and retain the ability to revision control your own code. The first suggested method relies on the queues feature of Mercurial (the revision control tool used for M5 development). The second relies on your own source control scheme (you could use mercurial or something else), and instead uses the [[Extras]] functionality in the build process to link extra objects into M5. In some situations a hybrid approach may be the best one, where the features you're attempting to evaluate are handled via the extras functionality, and minor changes to interfaces are done with Mercurial queues.
If you are extending M5 in a way that could be useful to the community, please consider pushing it back upstream. See [[#Contributing Your Changes|Contributing Your Changes]].
=== Using Mercurial Queues ===
The first method is using [http://hgbook.red-bean.com/hgbookch12.html Mercurial Queues] (MQ).  MQ provides management commands inis to provide management commands to create and apply patches to an upstream source tree. When the underlying source tree is updated (see above), you can remove your patches, get the new changes and reapply your patches very quickly. The patches themselves can be an complete mercurial repository that is revision controlled. It's essential to read the above chapter in the Mercurial manual to understand this process, but briefly you would begin by creating a mercurial queues repository. You can then add patches to the mercurial queues repository and automatically update them based on the changes you've made. With this method it is good to segment changes into logical blocks rather than have one large patch for all your changes. When it comes time to update to a new version of M5 you remove all your patches from the repository, update to the latest version and add re-apply all of your patches. Normally this requires minimal effort. For example:
hg qinit -c
hg qnew my_new_feature.diff
echo "// Not so much a new feature as an additional line in the source tree" >> src/sim/main.cc
hg qrefresh # my_new_feature.diff now contains the the extra line in the source tree
# Remove the patch by executing
hg qpop
# Reapply the patch by executing
hg qpush
# Commit the changes to the path
hg qcommit
#To update to the latest version of M5
hg qpop -a
hg fetch
hg qpush -a
# Again PLEASE read the manual
=== Using EXTRAS ===
The other method relies on the [[Extras]] functionality in the build process. This feature allows you to keep new M5 models in a directory of your choosing and have them compile and link code in with the M5 binary from arbitrary directories.  Because the location of this directory is completely independent of the M5 mercurial repository, you can use any revision control system you like, and you can include code that you don't want to release (or that was released under a different license) without worrying about contaminating the M5 repository.
The drawback of the Extras technique is that it only allows the addition of new source code to M5, not the replacement or modification of any existing source code.
The EXTRAS functionality is used to support EIO traces. The trace reader for EIO is licensed under the SimpleScalar license and the code to read these traces is not included in the M5 distribution.  
== Running gem5 ==
# [[Running gem5]] - Starting a simulation from the command line.
The next examples show how to compile the EIO code. By adding to or modifying the extras path, any other suitable extra could be compiled in. To compile in code using EXTRAS simply execute the following
# [[Output Files]] - Output obtained from simulation
scons EXTRAS=/path/to/m5_2.0_encumbered build/ALPHA_SE/m5.opt
# [[Checkpoints]], Fast Forwarding - How to create Checkpoints, restore the checkpointed state, fast forward simulation.
# [[Regression Tests]] - Running the regression tests.
# [[SimObjects]] - What SimObjects are and how they work.
# [[Configuration / Simulation Scripts]] - Explains how to configure the simulator and run simulations
# [[Visualization]] -- Tools to help visualize what is going on in a system
In the root of this directory you should have a SConscript that uses the <code>Source()</code> and <code>SimObject()</code> scons functions that are used in the rest of M5 to compile the appropriate sources and add any SimObjects of interest.
== Workloads ==
# [[Compiling workloads | Compiling]] - How to compile workloads to run within gem5.
# [[Multiprogrammed workloads | Multi-programmed]] - Multiprogrammed workloads in SE mode.
# [[Multithreaded workloads | Multi-threaded]] - Multithreaded workloads in SE mode.
# Benchmark suites
## [[SPEC benchmarks | SPEC]] - SPEC benchmarks in SE mode.
## [[Splash benchmarks | SPLASH]] - SPLASH benchmarks (SE or FS mode?)
## [[PARSEC benchmarks | PARSEC]] - PARSEC benchmarks in FS mode.
# [[Disk images]] - Setting up a disk image.
# [[Linux kernel]] - Preparing a Linux kernel for use in gem5.
# [[m5ops]] - How to talk to the simulator from within a simulated workload.
Note that EXTRAS is a "sticky" parameter, so after a value is provided to scons once, the value will be reused for future scons invocations targeting the same build directory (<code>build/ALPHA_SE</code> in this case) as long as it is not overridden.  Thus you only need to specify EXTRAS the first time you build a particular configuration or if you want to override a previously specified value.  For more information on sticky scons options, see the [[SCons build system]] page.
== Development ==
# [[Functional Testers]]
#* [[Directed Test]]
#* [[memtest]]
#* [[Garnet Synthetic Traffic]]
#* [[Ruby Random Tester]]
# Debugging - features designed specifically to help you figure out what's going on.
#* [[Trace Based Debugging | Trace-Based Debugging]]: figuring out what gem5 is doing using debug traces
#* [[Debugger Based Debugging | Debugger-Based Debugging]]: using gdb to debug gem5
#* [[Debugging Simulated Code]]: debugging the system/program running on top of gem5
# [[Coding Style]]
# [[Adding Functionality]]
#* [[Managing Local Changes with Mercurial Queues]]
#* [[Extras | Compiling Additional Code with EXTRAS]]
# [[Source Code Documentation]]
# [[Submitting Contributions]]
To run a regression with EXTRAS use a command line similar to the following:
== Infrastructure ==
./util/regress --scons-opts="EXTRAS=/tmp/m5_2.0b4_encumbered" -j 2 quick
# [[Events]]
# [[Statistics]]
# [[Utility Code]]
# [[Serialization]]
===Contributing Your Changes===
== Memory System ==
If you make a changes to gem5 that others would find useful, please consider sharing that code with the community. The best way to do this is create an account on our code review [http://reviews.m5sim.org website]. You can then either upload a diff directly to the website or use the hg post-review [http://mercurial.selenic.com/wiki/ReviewboardExtension extension] to send the patch directly to the website from your mercurial repository.
# [[General Memory System]] - Shared infrastructure between the Classic and Ruby models
# [[Classic Memory System]] - Description of our easily configurable, crossbar-based memory system
# [[Ruby]] - Description of our detailed and flexible memory system. Lots of information here.
#* [[SLICC|SLICC Overview]]
#* [[GEMS-gem5_SLICC_Transition_Guide|SLICC Transistion Guide]]
#* [[Interconnection_Network]]
===Mercurial repo browser===
== Full System ==
The source code of gem5 is available to browse online [http://repo.m5sim.org/ here].
# [[Devices]]
## [[IDE Devices]]
## [[NIC Devices]]
## [[Timers]]
## [[UARTs and serial terminals]]
# [[Disks and Disk Images]]
# [[Interrupts]]
==External Dependencies==
== Syscall Emulation ==
Like most software gem5 requires a number of packages and tools to be able to build itself. Since it's highly likely that any user will be required to make some changes to gem5 to use it unlike most software having an environment capable of building gem5 is require. The following tools are needed to build gem5:
# [[SE Mode]]
===Required versions===
To build M5, you will need the following software:
* [http://gcc.gnu.org/ g++] version 3.4.6 or newer. < 3.4 seems to have some issues with templates or internal compiler errors prevent compilation.
== Instruction Execution ==
* [http://www.python.org Python], version 2.4 or newer.  M5 links in the Python interpreter, so you need the Python header files and shared library (e.g., /usr/lib/libpython2.4.so) in addition to the interpreter executable.  These may or may not be installed by default.  For example, on Debian/Ubuntu, you need the "python-dev" package in addition to the "python" package.  If you need a newer or different Python installation but can't or don't want to upgrade the default Python on your system, see our page on [[using a non-default Python installation]].
# [[Execution Basics]]
* [http://www.scons.org SCons], version 0.98.1 or newer.  SCons is a powerful replacement for make. See [http://sourceforge.net/project/showfiles.php?group_id=30337 here] to download SCons.  If you don't have administrator privileges on your machine, you can use the "scons-local" package to install scons in your m5 directory, or install SCons in your home directory using the '--prefix=' option.
# [[Architectural State]]
* [http://www.swig.org SWIG], version 1.3.31 or newer.
# [[Address Translation]]
* [http://www.zlib.net zlib], any recent version.  For Debian/Ubuntu, you will need the "zlib-dev" or "zlib1g-dev" package to get the zlib.h header file as well as the library itself.
== CPU Models ==
* [http://www.gnu.org/software/m4/ m4], the macro processor.
# [[SimpleCPU | Simple CPU Model]]
# [[O3CPU | Out-of-Order CPU Model]]
# [[InOrder | In Order CPU Model]]
# [[TraceCPU | Trace CPU Model ]]
# [[Adding a New CPU Model]]
===Things in ext===
== ISA Implementations ==
There are a number of libraries that are in the ext directory under the source tree. This is done primarily so that users don't need to install rarely used packages. These include:
# [[Architecture Support]] -- What architectures are supported by gem5 and what features of them are implemented
* [http://libdnet.sourceforge.net/ dnet] -- dnet provides a simplified, portable interface to several low-level networking routines.
# [[Status Matrix]]
* [http://www.cs.unc.edu/Research/compgeom/gzstream/ gzstream] -- Gzstream is a small C++ library, basically just a wrapper, that provides the functionality of the zlib C-library in a C++ iostream.
# [[ISA Parser]]
* [http://www.mr511.de/software/english.html libelf] --  ELF object file access library.
# [[Alpha Implementation]]
* [http://www.dabeaz.com/ply/ PLY] -- PLY is an implementation of lex and yacc parsing tools for Python.
# [[ARM Implementation]]
* x11ksyms -- Keycodes from X11 for VNC support.
# [[MIPS Implementation]]
# [[Power Implementation]]
# [[SPARC Implementation]]
# [[X86 Implementation]]
# [[Defining ISAs]]

Latest revision as of 02:22, 28 November 2016

Welcome to the gem5 documentation. We've worked hard to add and organize everything in a more reasonable format, however we still haven't got to everything. All the documentation is a wiki, so we encourage you to add and edit sections as you find omissions and bugs.

For an overview, we have posted slides and most importantly video of a tutorial at HiPEAC Computer Systems Week. See Tutorials.

Old Documentation: We have some old documentation and have been trying to move everything to the new format, but we haven't quite succeeded yet. If you don't find the answer you're looking for here, you might want to check the old documentation.

Getting Started

  1. Introduction - A quick introduction to gem5.
  2. Source Code - Information about the source code itself.
  3. Dependencies - Things you'll need that aren't part of gem5 itself.
  4. Build System - How to run or modify gem5's build system.
  5. Full System and Benchmark Files - Disk images for operating system and benchmark applications, and how to install them.
  6. gem5 101 - Six part course covering all the basics of gem5 (and some advanced material too).

Running gem5

  1. Running gem5 - Starting a simulation from the command line.
  2. Output Files - Output obtained from simulation
  3. Checkpoints, Fast Forwarding - How to create Checkpoints, restore the checkpointed state, fast forward simulation.
  4. Regression Tests - Running the regression tests.
  5. SimObjects - What SimObjects are and how they work.
  6. Configuration / Simulation Scripts - Explains how to configure the simulator and run simulations
  7. Visualization -- Tools to help visualize what is going on in a system


  1. Compiling - How to compile workloads to run within gem5.
  2. Multi-programmed - Multiprogrammed workloads in SE mode.
  3. Multi-threaded - Multithreaded workloads in SE mode.
  4. Benchmark suites
    1. SPEC - SPEC benchmarks in SE mode.
    2. SPLASH - SPLASH benchmarks (SE or FS mode?)
    3. PARSEC - PARSEC benchmarks in FS mode.
  5. Disk images - Setting up a disk image.
  6. Linux kernel - Preparing a Linux kernel for use in gem5.
  7. m5ops - How to talk to the simulator from within a simulated workload.


  1. Functional Testers
  2. Debugging - features designed specifically to help you figure out what's going on.
  3. Coding Style
  4. Adding Functionality
  5. Source Code Documentation
  6. Submitting Contributions


  1. Events
  2. Statistics
  3. Utility Code
  4. Serialization

Memory System

  1. General Memory System - Shared infrastructure between the Classic and Ruby models
  2. Classic Memory System - Description of our easily configurable, crossbar-based memory system
  3. Ruby - Description of our detailed and flexible memory system. Lots of information here.

Full System

  1. Devices
    1. IDE Devices
    2. NIC Devices
    3. Timers
    4. UARTs and serial terminals
  2. Disks and Disk Images
  3. Interrupts

Syscall Emulation

  1. SE Mode

Instruction Execution

  1. Execution Basics
  2. Architectural State
  3. Address Translation

CPU Models

  1. Simple CPU Model
  2. Out-of-Order CPU Model
  3. In Order CPU Model
  4. Trace CPU Model
  5. Adding a New CPU Model

ISA Implementations

  1. Architecture Support -- What architectures are supported by gem5 and what features of them are implemented
  2. Status Matrix
  3. ISA Parser
  4. Alpha Implementation
  5. ARM Implementation
  6. MIPS Implementation
  7. Power Implementation
  8. SPARC Implementation
  9. X86 Implementation
  10. Defining ISAs