Chapter
4: Starting FinFarm
Introduction
The rapid fall in the prices
of computers lead to a new powerful concept for design verification, the
simulation farm. A simulation farm consists of a few tens or even hundreds
of inexpensive computers, each capable of simulating the entire design.
Fintronic's simulator has established itself as the primary simulator to
fit this paradigm because of its low memory utilization and its support
for 64-bit architectures which allow it to simulate even the largest circuits
on a single computer. To take advantage of the farm, the designer has only
to split the stimulus and launch a separate simulation for each of the
stimuli in the set. Previous attempts to partition the design rather than
the stimulus have failed because of the high performance penalty paid for
communicating between the different parts of the design. In addition, partitioning
the stimulus is much more natural and simple for the designer and can be
done in a straightforward manner, easy to understand and control by the
designer.
Once the stimulus has been
partitioned, there are three major problems faced by the harvester of the
farm: how to launch the large number of simulations, how to monitor the
status of the simulations and computers in the farm, and finally, how to
collect the results of the simulation, sort between the different types
of errors and take the appropriate action for each category. Fintronic
has introduced tools which make these tasks extremely simple and allow
for only one (or very few) harvesting engineers to manage the entire farm.
Farm management
Launching the simulations
on the farm
The farm is defined by a configuration
file. The entries in the configuration file are described in more detail
in the
Farm definition chapter.
There are two directories which control the number of simulations which
will be launched on the farm. The stimulus directory contains stimulus
files and the options directory contains command files for the simulator.
For each stimulus file and each configuration file, there will be one simulation
launched on any available computer on the farm. The configuration files
not only allow the user to have simulations with different options but
also allows for simulations of independent designs which are specified
in these files. To launch the simulations on the farm, the user only has
to click on the "Start Farm Simulations" button. Each simulation can be
stopped by selecting it from the list of running simulations and clicking
on the "Stop Simulation" button. When a simulation is stopped or finishes
with an error (see below on the different types of errors), the user can
re-launch that particular simulation by selecting it and clicking on the
"Start Simulation" button. The user has also the option of stopping all
simulations on the farm by clicking on the "Stop All Simulations" button.
A simulation which is deemed
to have failed because of a computer problem is queued for later processing
on a (possibly) different computer. Similarly, if there are not enough
computers or licenses available, the simulations are queued.
Monitoring the status
of the computers and simulations on the farm
The harvester is presented
with a real time report on the status of each individual simulation as
well as the activity/status of each computer on the farm. The status of
each simulation is shown in the main window. Each simulation is identified
by the name of the corresponding stimulus file and/or command file. Double
clicking on the name of a stimulus/command file name will open an editor
for that file. To select a simulation to start/stop individually, the user
has to click on the column where the current status of the simulation is
reported.
The status of each computer
is displayed in a separate window launched by clicking on the "Computers
status" button. For each computer on the farm, the name of the computer,
its status (Idle, Running or Eliminated) and some extra information depending
on the status are displayed. When a computer has the Running status, the
extra information reports the particular simulation that runs at the present
time on that computer. The simulation is again identified by the name of
the corresponding stimulus file and/or command file. When a computer has
the Eliminated status, the extra information reports the reason why the
computer was eliminated on the farm. By clicking on the message, a file
with extra information is displayed. A computer for which a failure has
been detected is removed from the available pool of computers on the farm
and can be easily reinserted once the problem has been fixed. This is done
by selecting the computer name and then clicking on the "Back in farm"
button.
Collecting the simulation
results and taking the appropriate actions
The results of each simulation
are analyzed and reported in a clear and straightforward way to the harvester.
The status of each simulation can be either queued, running, stopped, finished
successfully or finished with an error. For simulations that finished successfully
or with an error, the user can get more detailed information by clicking
on the "Check" button corresponding to the simulation. The harvester will
also be presented with the appropriate log files. If at any of the three
steps (analysis and code generation, building, actual simulation) an error
is detected, the farm tool reports the type of error, the stage at which
it occurred and a detailed message regarding the error. By clicking on
the message, the log file for each of the finished steps is displayed if
it is available.
Double clicking on an error
(or warning) in the log file will present the user with an editor window
with the cursor placed on the line that caused the error, if this is applicable.
Once the error is fixed, the
user can restart the simulation. The types of errors detected in each of
the three steps can be user errors, IO errors, memory errors, license errors,
fatal errors and internal errors. In addition to the above mentioned errors,
FinFarm has built-in support for reporting errors in self-checking simulations.
If the design is written in such a way that it prints a keyword when an
error is detected, this keyword can be specified in FinFarm's configuration
file, the log file will be checked for it and the result will be reported
accordingly. All the simulation results and log files are copied to the
central computer from which the farm is managed.
With the help of user defined
filters, the harvester can choose to view the results of a subset of the
simulations, such as the ones that failed or the ones that have a common
stimulus file but different options.
Filters can be defined from the Quick Filter option under the Filter
menu.
Fintronic also provides a
revolutionary way to report errors to our technical support over the internet.
All relevant source files, options and log files are collected and the
user has the options to send them to Fintronic.
Farm definition
The farm is defined by a configuration
file. The configuration file can be either edited using your favorite text
editor, or from the Config menu of FinFarm. When editing the file with
a text editor, outside of FinFarm, the entries are defined by keywords
as explained below. After a configuration file is edited outside of FinFarm,
it has to be re-opened in FinFarm for the changes to take effect. A configuration
file edited in FinFarm can be saved from the Config menu.
Selecting a line and clicking
on the "Add Computer" button will open an editor box where the user can
define a new computer for the farm. For simplicity the values for the selected
computer are used.
Selecting a line and clicking
the "Edit Computer" button will open an editor box where the information
for the selected computer can be modified.
The following are the entries
in a configuration file and how they would be specified with keywords using
a text editor.
A list of computer names available
to the farm, the local directories on each computer where the simulation
will take place and a source file which sets up the simulation environment
on the computer. A typical entry looks like this:
LOCAL_DIR /home/f4/finsim/farm
SOURCE_ENVIRONMENT /home/f4/finsim/4_6_06/setup.csh
The COMPUTER line may contain
one or more computers and if only one LOCAL_DIR and/or one SOURCE_ENVIRONMENT
is specified, it will apply to all computers.If each computer has a different
local directory name and source file, for each computer the user has to
specify 3 such entries.
Test directory
A test directory where the
compilations will take place; this directory should contain a file called
top.cf which specifies all the Verilog files, libraries and options common
to all simulations; this directory should be accessible by all computers
on the farm since the compilations of the design will take place in this
directory; if the user desires to have completely different simulations
with nothing in common, the top.cf in this directory should be empty but
should exist; each simulation will in this case be specified by the configuration
files in the OPTIONS_DIRECTORY and/or each Verilog file in the STIMULUS_DIRECTORY
(see below). A typical entry looks like this:
TEST_DIRECTORY /home/u4/demo_farm
Stimulus directory
A stimulus directory which
contains the Verilog files with stimulus for the simulation; accessible
by all computers. If no stimulus directory is specified, the simulations
are determined only by the configuration files in the OPTIONS_DIRECTORY
(see below) and the top.cf in the TEST_DIRECTORY. A typical entry looks
like this:
STIMULUS_DIRECTORY /home/u4/demo_farm/stim_dir
Options directory
An options directory containing
configuration files which specify different options for the simulation.
There will be as many simulations launched as the Carthesian product of
files in the STIMULUS_DIRECTORY and configuration files in the OPTIONS_DIRECTORY;
accessible by all computers. If no options directory is specified, the
simulations are determined only by the stimulus files in the STIMULUS_DIRECTORY
and the top.cf in the TEST_DIRECTORY. A typical entry looks like this:
OPTIONS_DIRECTORY /home/u4/demo_farm/opt_dir
Results directory
A results directory where
all the log files from all simulations will be collected; This directory
will also contain subdirectories with the results of each simulation. The
results of the simulation corresponding to the stimulus file stim3.v and
the configuration file top4.cf will be named stim3.v.top4.cf. A typical
entry looks like this:
RESULT_DIRECTORY /home/u4/demo_farm/results
Error string
An optional string which will
flag an error in the case of self checking Verilog test benches. A typical
entry looks like this:
ERROR_STRING ERROR
Starting FinFarm
FinFarm is started by calling
finfarm at the command line and currently takes no arguments. The user
is first prompted to select a configuration file. Once the configuration
file is opened, the user can access the entire farm management functionality
described in the
Farm management
chapter