FINFARM - A TOOL FOR MANAGING AND HARVESTING A SIMULATION FARM

TABLE OF CONTENTS
Chapter 1: Introduction
Chapter 2: Farm management
2.1 Launching the simulations on the farm
2.2 Monitoring the status of the computers and simulations on the farm
2.3 Collecting the simulation results and taking the appropriate actions
Chapter 3: Farm definition
3.1 Computers
3.2 Test directory
3.3 Stimulus directory
3.4 Options directory
3.5 Results directory
3.6 Error string
Chapter 4: Starting FinFarm

Introduction

The rapid fall in the prices of computers lead to a new powerful concept for design verification, the simulation farm. A simulation farm consists of a few tens or even hundreds of inexpensive computers, each capable of simulating the entire design. Fintronic's simulator has established itself as the primary simulator to fit this paradigm because of its low memory utilization and its support for 64-bit architectures which allow it to simulate even the largest circuits on a single computer. To take advantage of the farm, the designer has only to split the stimulus and launch a separate simulation for each of the stimuli in the set. Previous attempts to partition the design rather than the stimulus have failed because of the high performance penalty paid for communicating between the different parts of the design. In addition, partitioning the stimulus is much more natural and simple for the designer and can be done in a straightforward manner, easy to understand and control by the designer.
Once the stimulus has been partitioned, there are three major problems faced by the harvester of the farm: how to launch the large number of simulations, how to monitor the status of the simulations and computers in the farm, and finally, how to collect the results of the simulation, sort between the different types of errors and take the appropriate action for each category. Fintronic has introduced tools which make these tasks extremely simple and allow for only one (or very few) harvesting engineers to manage the entire farm.
 
 

Farm management

Launching the simulations on the farm
The farm is defined by a configuration file. The entries in the configuration file are described in more detail in the  Farm definition chapter. There are two directories which control the number of simulations which will be launched on the farm. The stimulus directory contains stimulus files and the options directory contains command files for the simulator. For each stimulus file and each configuration file, there will be one simulation launched on any available computer on the farm. The configuration files not only allow the user to have simulations with different options but also allows for simulations of independent designs which are specified in these files. To launch the simulations on the farm, the user only has to click on the "Start Farm Simulations" button. Each simulation can be stopped by selecting it from the list of running simulations and clicking on the "Stop Simulation" button. When a simulation is stopped or finishes with an error (see below on the different types of errors), the user can re-launch that particular simulation by selecting it and clicking on the "Start Simulation" button. The user has also the option of stopping all simulations on the farm by clicking on the "Stop All Simulations" button.



 
 

A simulation which is deemed to have failed because of a computer problem is queued for later processing on a (possibly) different computer. Similarly, if there are not enough computers or licenses available, the simulations are queued.
Monitoring the status of the computers and simulations on the farm
The harvester is presented with a real time report on the status of each individual simulation as well as the activity/status of each computer on the farm. The status of each simulation is shown in the main window. Each simulation is identified by the name of the corresponding stimulus file and/or command file. Double clicking on the name of a stimulus/command file name will open an editor for that file. To select a simulation to start/stop individually, the user has to click on the column where the current status of the simulation is reported.
 


 
 

The status of each computer is displayed in a separate window launched by clicking on the "Computers status" button. For each computer on the farm, the name of the computer, its status (Idle, Running or Eliminated) and some extra information depending on the status are displayed. When a computer has the Running status, the extra information reports the particular simulation that runs at the present time on that computer. The simulation is again identified by the name of the corresponding stimulus file and/or command file. When a computer has the Eliminated status, the extra information reports the reason why the computer was eliminated on the farm. By clicking on the message, a file with extra information is displayed. A computer for which a failure has been detected is removed from the available pool of computers on the farm and can be easily reinserted once the problem has been fixed. This is done by selecting the computer name and then clicking on the "Back in farm" button.
 
 

 
 
Collecting the simulation results and taking the appropriate actions
The results of each simulation are analyzed and reported in a clear and straightforward way to the harvester. The status of each simulation can be either queued, running, stopped, finished successfully or finished with an error. For simulations that finished successfully or with an error, the user can get more detailed information by clicking on the "Check" button corresponding to the simulation. The harvester will also be presented with the appropriate log files. If at any of the three steps (analysis and code generation, building, actual simulation) an error is detected, the farm tool reports the type of error, the stage at which it occurred and a detailed message regarding the error. By clicking on the message, the log file for each of the finished steps is displayed if it is available.
 
 

 
 
Double clicking on an error (or warning) in the log file will present the user with an editor window with the cursor placed on the line that caused the error, if this is applicable.
 


 

Once the error is fixed, the user can restart the simulation. The types of errors detected in each of the three steps can be user errors, IO errors, memory errors, license errors, fatal errors and internal errors. In addition to the above mentioned errors, FinFarm has built-in support for reporting errors in self-checking simulations. If the design is written in such a way that it prints a keyword when an error is detected, this keyword can be specified in FinFarm's configuration file, the log file will be checked for it and the result will be reported accordingly. All the simulation results and log files are copied to the central computer from which the farm is managed.
With the help of user defined filters, the harvester can choose to view the results of a subset of the simulations, such as the ones that failed or the ones that have a common stimulus file but different options.
 

Filters can be defined from the Quick Filter option under the Filter menu.

Fintronic also provides a revolutionary way to report errors to our technical support over the internet. All relevant source files, options and log files are collected and the user has the options to send them to Fintronic.

Farm definition

The farm is defined by a configuration file. The configuration file can be either edited using your favorite text editor, or from the Config menu of FinFarm. When editing the file with a text editor, outside of FinFarm, the entries are defined by keywords as explained below. After a configuration file is edited outside of FinFarm, it has to be re-opened in FinFarm for the changes to take effect. A configuration file edited in FinFarm can be saved from the Config menu.
 
 

 
 
Selecting a line and clicking on the "Add Computer" button will open an editor box where the user can define a new computer for the farm. For simplicity the values for the selected computer are used.
Selecting a line and clicking the "Edit Computer" button will open an editor box where the information for the selected computer can be modified.
 


 
 

The following are the entries in a configuration file and how they would be specified with keywords using a text editor.
 
Computers.
 
A list of computer names available to the farm, the local directories on each computer where the simulation will take place and a source file which sets up the simulation environment on the computer. A typical entry looks like this:
 
COMPUTER f4
LOCAL_DIR /home/f4/finsim/farm
SOURCE_ENVIRONMENT /home/f4/finsim/4_6_06/setup.csh
 
The COMPUTER line may contain one or more computers and if only one LOCAL_DIR and/or one SOURCE_ENVIRONMENT is specified, it will apply to all computers.If each computer has a different local directory name and source file, for each computer the user has to specify 3 such entries.
 
Test directory
 
A test directory where the compilations will take place; this directory should contain a file called top.cf which specifies all the Verilog files, libraries and options common to all simulations; this directory should be accessible by all computers on the farm since the compilations of the design will take place in this directory; if the user desires to have completely different simulations with nothing in common, the top.cf in this directory should be empty but should exist; each simulation will in this case be specified by the configuration files in the OPTIONS_DIRECTORY and/or each Verilog file in the STIMULUS_DIRECTORY (see below). A typical entry looks like this:
 
TEST_DIRECTORY /home/u4/demo_farm
 
Stimulus directory
 
A stimulus directory which contains the Verilog files with stimulus for the simulation; accessible by all computers. If no stimulus directory is specified, the simulations are determined only by the configuration files in the OPTIONS_DIRECTORY (see below) and the top.cf in the TEST_DIRECTORY. A typical entry looks like this:
 
STIMULUS_DIRECTORY /home/u4/demo_farm/stim_dir
 
Options directory
 
An options directory containing configuration files which specify different options for the simulation. There will be as many simulations launched as the Carthesian product of files in the STIMULUS_DIRECTORY and configuration files in the OPTIONS_DIRECTORY; accessible by all computers. If no options directory is specified, the simulations are determined only by the stimulus files in the STIMULUS_DIRECTORY and the top.cf in the TEST_DIRECTORY. A typical entry looks like this:
 
OPTIONS_DIRECTORY /home/u4/demo_farm/opt_dir
 
Results directory
 
A results directory where all the log files from all simulations will be collected; This directory will also contain subdirectories with the results of each simulation. The results of the simulation corresponding to the stimulus file stim3.v and the configuration file top4.cf will be named stim3.v.top4.cf. A typical entry looks like this:
 
RESULT_DIRECTORY /home/u4/demo_farm/results
 
Error string
 
An optional string which will flag an error in the case of self checking Verilog test benches. A typical entry looks like this:
ERROR_STRING ERROR

Starting FinFarm

FinFarm is started by calling finfarm at the command line and currently takes no arguments. The user is first prompted to select a configuration file. Once the configuration file is opened, the user can access the entire farm management functionality described in the Farm management chapter