# Documentation 1.0

2011

CeCiLL

## Sferes 2

### Introduction

Sferes2 is a framework for evolutionary computation (EC) experiments and especially for evolutionary robotics (ER). Its main goal is to help researchers in EC and ER to efficiently try new ideas. Sferes2 has been inspired by sferes http://sferes.lip6.fr, another (older and no more maintained) framework for ER.

Sferes2 allows to design efficient programs on modern (multicore) computers: our experiments typically require more than 24 hours of computation; in this situation it can be profitable to trade some complexity of implementation for some hours of computation. Nevertheless, we try to keep the framework as simple as possible. More specifically, the following choices were made in the initial design:

- use of modern c++ techniques (template-based programming) to employ object-oriented programming without the cost of virtual functions;

- use of Intel TBB to take full advantages of multicore and SMP systems;

- use of boost libraries when it's useful (shared_ptr, serialization, filesystem, test,...);

- use of MPI to distribute the computational cost on clusters;

- a full set of unit tests;

- no configuration file: a fully optimized executable is built for each particular experiment.

Appart from this documentation, information about Sferes 2 can be found in Sferes_v2: Evolvin' in the Multi-Core World. If you use Sferes 2 in your work, please cite the previous publication.

The source code of many experiments using sferes is available on the evorob_db website

This documentation assumes that the reader is reasonably familiar with evolutionary algorithms. For references you can check the following books about evolutionary computation:

- Deb, K. (2001). Multi-Objective Optimization Using Evolutionary Algorithms. Wiley.

- De Jong, K. A. (2002). Evolutionary Computation. The MIT Press.

### Main features

Sferes2 currently includes the following EA and toolboxes (this is a non-exhaustive list which contains only the most stable modules):

- EA

  - a rank-based single-objective EA

  - CMAES (Covariance Matrix Adaptation Evolution Strategie)

  - NSGA-II (multiobjective EA)

  - Epsilon-MOEA (multiobjective EA)

- Genotypes / Phenotypes:

  - Evolution of bit strings

  - Evolution of real parameters using different operators (SBX, gaussian, etc)

- Simulators

  - (optional) mobile robot (Khepera-like)

  - (optional) cartpole (single and double)

- Other

  - (optional) evolution of neural-networks

### Authors

- Jean-Baptiste Mouret mouret@isir.fr: main author and maintainer

- Stephane Doncieux

- Paul Tonellipaul@tonel.li : Documentation maintainer

- Many members of ISIR

### Where to go next?

Download Downloading sferes2

Compilation How to compile sferes2 on your system

Tutorial/Example A tutorial/example with explanation about the general structure of sferes2 experiment files, the example file is generated automatically in the next tutorials.

Neurobiology Tutorial I A tutorial to create an experiment using the neural network module of sferes2, this enables you to create a neural network and evolve its weights and parameters.

Neurobiology Tutorial II A tutorial to evolve the topology of a network in sferes2

Neurobiology Tutorial III A tutorial to evolve a network of maps using the EvoNeuro module from sferes II.

Reference A reference manual / guide to the source code

### Specific sections for ISIR members

Torque Introduction A reference to launching experiments on the ISIR cluster

## Download and unzip

### If you are not a member of ISIR

#### Download the last version

The last version of sferes2 is available here. The archive is updated daily and contains the main sferes code as well as core modules and a few sample experiments.

#### Unzip the archive

To unzip the archive type the following command in the folder in which you downloaded the archive:

    tar -xvjf sferes2.latest.tar.bz2

#### Go to sferes root

You can now change the directory to the root folder of sferes by typing the following command (replace ${VERSION} by your version number):

    cd sferes2-${VERSION}

### If you are a member of ISIR

#### Checkout the svn version

If you are a member of ISIR, you should checkout the last version of sferes2 from the svn repository:

    svn co https://hotline.isir.upmc.fr/svn/sferes2

#### Checkout the svn of experiments and modules

you should then checkout the svn containing the other isir experiments:

    svn co https://hotline.isir.upmc.fr/svn/er/src/sferes/exp

as well as the modules:

    svn co https://hotline.isir.upmc.fr/svn/er/src/sferes/modules

#### Go to sferes root and add additionnal experiments

You can now change the directory to the root folder of sferes by typing the following command :

    cd sferes2/trunk

and add the additionnal modules and experiments to the path:

    cd modules
    ln -sf ../../../modules/* .
    cd ../exp
    ln -sf ../../../exp/* .
    cd ..

this should create the correct shortcuts to link the experiments from the svn in Sferes 2.

## Launching experiments using Torque

This short tutorial is intended for isir members who are working on the ISIR cluster. It explains how to automatically launch a batch of experiments on the cluster nodes

### Script Files

The torque system used on the cluster can be launched using the qsub command. However, waf, used to compile sferes was scripted to accept experiment files.

The files used for experiments are in json format and should have the .json extension for easier maintenance.

each file has the following structure

    {
       "email" : "yourname@youraddress.com",
       "wall_time" : "00:59:59",
       "nb_runs": 4,
       "bin_dir": "/${path_to_binaries}/",
       "res_dir": "/${path_to_results_folder}",
       "exps" : ["${binary_name_2}","${binary_name_2}"]
    }

The email field is used by the system to report each job status after it ends.

The wall_time field defines the maximum allowed time for each job. If your experiment has not exited at the end of the time, it will be killed by the server. The wall time is used by torque to define the number of parrallel nodes allocated on the cluster :

Wall time and parrallel execution

Wall_time | cluster nodes used
--- | ---
<01:00:00 | 8
<12:00:00 | 4
<24:00:00 | 2
>23:59:59 | 1

The nb_runs defines the number of jobs which should be created.

The bin_dir sets the path to find the binaries to execute, you must put a complete path to your executables (avoid ~ as the user who will execute the program is not your regular user).

The res_dir sets the path to put the results

The exps sets the name of the binaries to execute. If you have more than one binary, the file will creat nb_runs jobs for each binary.

### Commands

After creating ${your_exp}.json file, you can launch it on the cluster using the following syntax:

    ./waf --qsub ${path_to_your_json}/${your_json_file}.json

This will create all the required jobs in the queue.

To check if the current queue, you can use the qstat command. Its output should look like this:

    Job id                    Name             User            Time Use S Queue
    ------------------------- ---------------- --------------- -------- - -----
    83356.cluster             ...e10_swap2_nov username2         73:50:34 R q1jour
    83357.cluster             ...e10_swap5_nov username2                0 Q q1jour
    83358.cluster             ...e10_swap5_nov username2                0 Q q1jour
    83359.cluster             ...odulation_nov username1         74:34:31 R q1jour
    83360.cluster             ...odulation_nov username1                0 Q q1jour
    83361.cluster             ...odulation_nov username1                0 Q q1jour

If you want to remove jobs from the queue (you can only remove [`your] jobs from the queue), you can use the qdel with the following syntax

    qdel ${job_id}

Usually, your experiment ids will be grouped, so you can delete multiple jobs using the following syntax:

    for i in `seq ${start_id} ${end_id}`;do qdel $i;done

### Result Files

The result from your experiments will be put in the res_dir folder. For each experiment, a folder with its name will be created containing one subdirectory for each job in the experiment named job_1, job_2... Each of these folder will contain:

- the usual generated files depending on your experiment

- a ${your_exp_name}.e??? containing the stderr output if relevant

- an stdout file containing the console output from the experiment

## Compilation

### Dependencies

We list here the Ubuntu package names:

- [required] libboost-dev >= 1.35 (including test, serialization, graph, program_options, filesystem); see http://www.boost.org;

- [required] libboost-test-dev >= 1.35

- [required] libboost-filesystem-dev >= 1.35

- [required] libboost-program-options-dev >= 1.35

- [required] libboost-graph-parallel-dev >= 1.35

- [required] python >= 2.4;

- [required] g++ >= 4.2 (this should work with icc too);

- [optional but highly recommended] libtbb-dev; see http://www.threadingbuildingblocks.org/

- [optional but recommended] libeigen2-dev >= 2.06; see http://eigen.tuxfamily.org/

- [optional but recommended] python-simplejson;

- [optional] libsdl1.2-dev , useful for visualisation of experiments

- [optional] libboost-mpi-dev , if you need the optional MPI support (Warning: the Ubuntu packages of boost are not compiled with MPI support; you will have to compile boost yourself to use MPI).

- [optional] OpenMPI or another MPI implementation (tested with IBM MPI and OpenMPI)

- [optional] libgoogle-perftools-dev.

if you are under debian, the complete command line to type is :

    sudo apt-get install libboost-dev libboost-test-dev libboost-filesystem-dev libboost-program-options-dev libboost-graph-parallel-dev python g++ libtbb-dev libeigen2-dev python-simplejson libgoogle-perftools-dev

### Compilation

#### Simple compilation

The next step is to configure sferes prior to compiling it. We will disable mpi as its use is complex.

    ./waf configure --no-mpi

If you have followed this tutorial, the output should look like this:

    Check for program g++ or c++             : /usr/bin/g++
    Check for program cpp                    : /usr/bin/cpp
    Check for program ar                     : /usr/bin/ar
    Check for program ranlib                 : /usr/bin/ranlib
    Checking for g++                         : ok
    boost headers                            : Version 1_46_1 (/usr/include)
    library boost_serialization              : ok
    library boost_filesystem                 : ok
    library boost_system                     : ok
    library boost_unit_test_framework        : ok
    library boost_program_options            : ok
    library boost_graph                      : not found
    library boost_mpi                        : not found
    library boost_python                     : not found
    library boost_thread                     : not found
    Checking for header tbb/parallel_for.h   : ok /usr/include
    Checking for library libtbb              : ok /usr/lib
    Checking for header tbb/parallel_for.h   : ok /usr/include
    Checking for library libtbb              : ok /usr/lib
    Checking for header mpi.h                : not found
    Checking for SDL (optional)              : ok
    Checking for header Eigen/Core           : ok /usr/include/eigen2
    Check for program ode-config             : not found
    Checking for ODE (optional)              : not found
    Checking for GSL (optional)              : not found

Assuming that everything has been properly installed you should be able to compile sferes2 by typing:

    ./waf build

which should produce the output

    'build' finished successfully (1m18.284s)

You should then run the unit tests:

    ./waf check

If everything is green, sferes2 is working! If not, please send an e-mail to the authors with as many details as possible about your problem.

#### Configuration options in waf

Sferes2 employs the waf build system (http://code.google.com/p/waf/) instead of autoconf/automake and make. Waf employs files named wscript to describe the compilation process in python.

The main wscript of sferes2 accepts the following configuration options:

- --boost-includes=BOOSTINCLUDES : path to the boost directory where the includes are e.g. /usr/local/include/boost-1_35

- --boost-libs=BOOSTLIBS : path to the directory where the boost libs are e.g. /usr/local/lib

- --no-mpi : disable mpi (MPI is automatically disabled if no MPI implementation is found)

- --mpi=MPI : path to MPI [e.g. /usr/lib/openmpi ]

- --apple : enable Apple (mac OSX) support

- --tbb=TBB : path to TBB if it's not installed in the default directory

- --64bits : compile for 64 bits architectures

- --ppc : compile for PowerPC (tested on IBM Power6)

- --rpath : set an optional rpath (useful in case of link failure)

To use these options, pass them to the waf script during the configuration step. For example:

    ./waf configure --mpi /usr/lib/openmpi/

The configure options used during the last call of the "configure" command are in the file build/configure.options.

#### compiling additionnal modules

The previous compilation provided you with a minimal sferes install. Several modules are available in the "modules" directory of sferes. Before using any of these modules in your experiment, you must compile these modules using sferes.

To do so, create a file named "modules.conf" at the root of sferes install (where waf is). You should then list the modules you want to use in this file, with one module per line.

for example, if you want to use the modules cartpole and nn2, your modules.conf file should look like:

    cartpole
    nn2

if you are using the command line, you can create and fill the file the following way:

    touch modules.conf
    echo nn2 >> modules.conf
    echo cartpole >> modules.conf

you should then recompile sferes with the updated file :

    ./waf build

This should generate test files in the build/default/modules/nn2 and build/default/modules/cartpole

You can then run the tests by using the waf check:

    ./waf check

WARNING Some tests, namely test_esn and test_hyper_nn may not work out of the box. But unless you use these specific modules, they should not impact the rest of the code.

## Tutorial

This short example shows how to set up a basic experiment to optimize parameters using a user-defined fitness.

### Running a sferes2 executable

Waf put all the compiled files in the "build" directory. Two variants are produced: "default" and "debug". You should always run the debug version first because the default version disables assert. Each variant mirrors the architecture of the main sferes2 directory.

Sferes2 contains some basic examples in the "examples" directory. For instance, let's check "ex_ea.cpp", a basic single-objective optimization of parameters. To launch the debug version, you should run:

    build/debug/examples/ex_ea

A verbose mode is also available:

    build/debug/examples/ex_ea -v all

And a more verbose one:

    build/debug/examples/ex_ea -v all -v trace

For the optimized version:

    build/default/examples/ex_ea

Running one of those commands should create a directory named using the schema hostame_pid-year-month-day_hours:min:seconds. In this directory there are:

- a file called "bestfit.dat", which contains, for each generation, the value of the best fitness;

- many files calles gen_xxx where xxx is a number.

These files are XML files which store the best candidates solutions for each generation (xxx is the generation number). For instance:

    <?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
    <!DOCTYPE boost_serialization>
    <boost_serialization signature="serialization::archive" version="5">
      <x class_id="0" tracking_level="0" version="0">
        <_best class_id="1" tracking_level="0" version="1">
          <px class_id="2" tracking_level="1" version="0" object_id="_0">
    	<_gen class_id="3" tracking_level="0" version="0">
    	  <_data>
    	    <count>10</count>
    	    <item_version>0</item_version>
    	    <item>0.50000423</item>
    	    <item>0.49999771</item>
    	    <item>0.49999756</item>
    	    <item>0.49999699</item>
    	    <item>0.50000197</item>
    	    <item>0.49999171</item>
    	    <item>0.5000003</item>
    	    <item>0.50000542</item>
    	    <item>0.50002229</item>
    	    <item>0.49999493</item>
    	  </_data>
    	</_gen>
    	<_fit class_id="5" tracking_level="0" version="0">
    	  <_value>-4.0586573e-14</_value>
    	  <_objs>
    	    <count>0</count>
    	    <item_version>0</item_version>
    	  </_objs>
    	</_fit>
          </px>
        </_best>
      </x>
      <x class_id="6" tracking_level="0" version="0">
        <_mean>-5376.9888</_mean>
      </x>
    </boost_serialization>

This XML file mirrors the statistics used in this experiment (more about this in the next sections). At this point of this overview, you can check:

- the _data subtree which lists the values of the "optimal" parameters (the result of the experiment);

- the _fit subtree which shows the fitness value of this "optimal" individual;

- the _mean value which corresponds to the mean fitness in the population (another statistics);

### Loading a result file

These XML files can be easily post-processed in your favorite language if needed. However, the same executable which generated the result files can also read them using the following syntax in the experience directory :

    ../build/default/examples/ex_ea --load gen_1000 -o output_file -n number

- gen_1000 is the generation file you want to load

- output_file is a text file which, most of the time (depending on the genotype and phenotype used), describes the best individual (or the Pareto-optimal set in multiobjective optimization); in this example, it contains the value of the parameters for the best individual:

    cat output_file
    8.46386e-05 -4.58956e-05 -4.88758e-05 -6.02007e-05 3.93391e-05 -0.000165701 5.96046e-06 0.00010848 0.000445843 -0.000101328

- number is the number of the individual described in the file you want to load.

### Building your own experiment

#### Setting everything up

##### Create directories and files

- At the root of the sferes repository (the main directory), use waf to create a new experiment. Let's call it "test":

    ./waf --create test

This should have created a new directory exp/test, a waf file exp/test/wscript and a basic file exp/test/test.cpp. You can now edit/customize them.

##### Compiling

- In the main sferes2 directory (not in the experiment's directory):

      ./waf --exp my_exp

  If the experiment is called "test", the command line is:

      ./waf --exp test

##### Running

    cd exp/test
    ../../build/debug/exp/test/test

and for the optimized version:

    ../../build/default/exp/test/test

#### Customizing / writing the experiment

Let's start with exp/test/test.cpp, from the end of the file.

##### Main function

At the end of the file, you should see a main() function. It defines some types using typedef then run the evolutionary algorithm. The types describe our experiment (what kind of genotype? what kind of fitness? what kind of algorithm?). We will go back to the Params class in the following section. Ignore it for now.

    int main(int argc, char **argv)
    {
      // Our fitness is the class FitTest (see above), that we call
      // fit_t. Params is the set of parameters (struct Params) defined in
      // this file.
      typedef FitTest<Params> fit_t;
      // We define the genotype. Here we choose EvoFloat (real
      // numbers). We evolve 10 real numbers, with the params defined in
      // Params (cf the beginning of this file)
      typedef gen::EvoFloat<10, Params> gen_t;
      // This genotype should be simply transformed into a vector of
      // parameters (phen::Parameters). The genotype could also have been
      // transformed into a shape, a neural network... The phenotype need
      // to know which fitness to use; we pass fit_t to it.
      typedef phen::Parameters<gen_t, fit_t, Params> phen_t;
      // The evaluator is in charge of distributing the evaluation of the
      // population. It can be simple eval::Eval (nothing special),
      // parallel (for multicore machines, eval::Parallel) or distributed
      // (for clusters, eval::Mpi).
      typedef eval::Eval<Params> eval_t;
      // Statistics gather data about the evolutionary process (mean
      // fitness, Pareto front, ...). Since they can also store the best
      // individuals, they are the container of our results. We can add as
      // many statistics as required thanks to the boost::fusion::vector.
      typedef boost::fusion::vector<stat::BestFit<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
      // Modifiers are functors which are run once all individuals have
      // been evalutated. Their typical use is to add some evolutionary
      // pressures towards diversity (e.g. fitness sharing). Here we don't
      // use this feature. As a consequence we use a "dummy" modifier that
      // does nothing.
      typedef modif::Dummy<> modifier_t;
      // We can finally put everything together. RankSimple is the
      // evolutionary algorithm. It is parametrized by the phenotype, the
      // evaluator, the statistics list, the modifier and the general params.
      typedef ea::RankSimple<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;
      // We now have a special class for our experiment: ea_t. The next
      // line instantiates an object of this class
      ea_t ea;
      // We can now process the command line options and run the
      // evolutionary algorithm (if a --load argument is passed, the file
      // is loaded; otherwise, the algorithm is launched).
      run_ea(argc, argv, ea);
      //
      return 0;
    }

##### Include part

Let's now go back to the top of the file. The file starts with the usual include files, which obviously depends on which classes (genotype, phenotype, ea, ...) you selected in the main function:

    #include <iostream>
    #include <sferes/phen/parameters.hpp>
    #include <sferes/gen/evo_float.hpp>
    #include <sferes/ea/rank_simple.hpp>
    #include <sferes/eval/eval.hpp>
    #include <sferes/stat/best_fit.hpp>
    #include <sferes/stat/mean_fit.hpp>
    #include <sferes/modif/dummy.hpp>
    #include <sferes/run.hpp>

##### Params

Then, the Params structure defines the parameters of the algorithm. This particular way of setting them allows the compiler to propagate constants to the whole programm (i.e. it replaces the parameters in the code by their values), allowing some optimizations. This parameters will depend on the algorithms you chose to use in your main function.

    struct Params
    {
      struct evo_float
      {
        // we choose the polynomial mutation type
        static const mutation_t mutation_type = polynomial;
        // we choose the polynomial cross-over type
        static const cross_over_t cross_over_type = sbx;
        // the mutation rate of the real-valued vector
        static const float mutation_rate = 0.1f;
        // a parameter of the polynomial mutation
        static const float eta_m = 15.0f;
        // a parameter of the polynomial cross-over
        static const float eta_c = 10.0f;
      };
      struct pop
      {
        // size of the population
        static const unsigned size = 200;
        // number of generations
        static const unsigned nb_gen = 2000;
        // how often should the result file be written (here, each 5
        // generation)
        static const int dump_period = 5;
        // how many individuals should be created during the random
        // generation process?
        static const int initial_aleat = 1;
        // used by RankSimple to select the pressure
        static const float coeff = 1.1f;
        // the number of individuals which are kept from one generation to
        // another (elitism)
        static const float keep_rate = 0.6f;
      };
      struct parameters
      {
        // maximum value of the phenotype parameters
        static const float min = -10.0f;
        // minimum value
        static const float max = 10.0f;
      };
    };

##### Fitness function

Last, it's time to write the fitness function. It's a special class with an "eval()" function which derives from fit::Fitness. It has to fill this->_value in single-objective optimization and this->_objs in multiobjective optimization. In this example, we want to maximize -\sum_i p_i^4, where p is the individual's phenotype.

    SFERES_FITNESS(FitTest, sferes::fit::Fitness)
    {
     public:
      // indiv will have the type defined in the main (phen_t)
      template<typename Indiv>
        void eval(const Indiv& ind)
      {
        float v = 0;
        for (unsigned i = 0; i < ind.size(); ++i)
          {
    	float p = ind.data(i);
    	v += p * p * p * p;
          }
        this->_value = -v;
      }
    };

## Neurobiology Tutorial I

This page provides a step-by-step approach for neuroscientists wanting to optimize their models using sferes. In the course of this work you will learn how to optimise the number values of a fixed model. A second tutorial is available for people who want to optimize the topology of models using evoneuro. For all the tutorials, a complete code is available at the bottom of the webpage.

### Download and install Sferes2

The first step is to install and compile Sferes2. To do so, please follow the download and compile pages before continuing. If you have successfully followed these two pages, you should be in the "trunk" directory of sferes2, else, type the following command after replacing SFERES2_PARENT_DIRECTORY by the path where you put sferes2:

    cd SFERES2_PARENT_DIRECTORY/sferes2/trunk

### Create a new experiment

To create a new experiment in sferes, you must first create its directory and go in it. You should replace "test" by the name of your new experiment in all the following command lines.

    ./waf --create test

This should have created a new directory exp/test, a waf file exp/test/wscript which is used to compile the experiment, and a basic file exp/test/test.cpp. You can now edit/customize them.

In the next section, we will transform the sample experiment provided to create a neural network using the NN2 module. This experiment will optimise the weights and bias parameters of a hand made feed-forward network with a set number of hidden neurons. The objective of this network will be to approximate a simple 3D gaussian function (two inputs, one output).

nn1

### Add the nn2 module to our experiment

nn2 is a module available in the sferes archive you have downloaded. you should have added it already to the list in your "modules.conf" file. If not, do it now.

we can now add a neural network to our experiment. To do so, we need to do three things:

### Update the wscript configuration file

We must add the library EIGEN2, required by nn2 module in the list of the required uselib section of the file exp/test/wscript.

    #! /usr/bin/env python
    def build(bld):
        obj = bld.new_task_gen('cxx', 'program')
        obj.source = 'test.cpp'
        obj.includes = '. ../../'
        obj.uselib = 'EIGEN2'
        obj.target = 'test'
        obj.uselib_local = 'sferes2'

### Modify the parameters

We must add the nn2 parameters in the params section of the experiment file test.cpp.

In sferes, all the parameters which will not change during an experiment should be set in the params section. For our experiment, we will just provide the number of hidden neurons in our neural network in a fit section of params.

    struct fit
    {
      static const size_t nb_hidden_neurons = 3 ;
      SFERES_ARRAY(float,x_pos, 0.69, 0.52, 0.80, 0.95, 0.97, 0.82, 0.11, 0.13, 0.27, 0.59);
      SFERES_ARRAY(float,y_pos, 0.24, 0.30, 0.63, 0.30, 0.81, 0.50, 0.47, 0.53, 0.51, 0.66);
    };

The two other lines use the macro SFERES_ARRAY which can be used to define arrays in the params. The first line defines two functions:

-     x_pos(size_t i)

  which returns the float value at index i

-     x_pos_size()

  which returns the size of the array

The arrays x_pos and y_pos specify all the positions at which we will assess the output of our neural network against the reference function.

### Add the relevant include files

In order to create a neural network, we will now add the relevant include file (as well as math.h for our function definition):

    #include <modules/nn2/nn.hpp>
    #include <math.h>

### Defining the function to approximate

In this test, the function is really simple and computes a gaussian function in two dimensions:

    float _gaussian(float x, float y)
    {
     return exp(-(x*x+y*y)/2);
    }

We need to put this function in the FitTest class, which is defined by the macro SFERES_FITNESS which derives it from the default sferes fitness class. This class is the core of all experiments and must define the "eval" function which will be called to evaluate each individual in our experiment.

### Neural network type

The definition of our network type require several templates to be set. These template parameters must be set in the main function of the experiment which always contains the following elements:

    * The templates parameters of the genotype and phenotype

    * the declaration of the algorithm instanciation : [teletype]ea_t ea;

    * the call to the main function of sferes which runs the algorithm :

    run_ea(argc, argv, ea);

. This function parses the user arguments at execution.

The first template parameter for a neural network, is the type of values passed around the network called "IO". In our situation, we will use float numbers.

    typedef float io_t;

At each activation a neuron will be passed a array of IO (usually a vector). and will output a single IO value. The neuron uses two functions to do so: a potential function "Pot" and an activation function "Act". in the following way:

    output = Act(Pot(inputs_array))

For our example, we will use a sum function call PfWSum as the potential function and a sigmoid like function AfTanh as the activation function. The AfTanh requires an additionnal parameter to set a bias in the neuron which will be set during the network creation. These two function are defined as well as a few other common functions in the files af.hpp and pf.hpp in the nn2 module folder. We now can define the type of our neurons in the network:

    typedef Neuron<PfWSum<>, AfTanh<io_t> > neuron_t;

The next thing we need to define is the connection type. Again, we will use simple connections which have a single parameter: the weight type. Here, the weight type will be a float.

    typedef Connection<float> connection_t;

We can now define the complete network type:

    typedef NN< neuron_t, connection_t > nn_t;

### Setting the number of evolved parameters

We will instantiate a network at each evaluation. The evolved parameters will be used for two things:

- setting each neuron bias

- setting each connection weight

We will therefore need a number of evolved parameters 4 times bigger than the number of hidden neurons (each hidden neuron has one internal parameters, one outbound and two inbound connections). We can now modify the gen_t line of the test experiment:

    typedef gen::EvoFloat<4 * Params::fit::nb_hidden_neurons, Params> gen_t;

These parameters will have a value set by the parameters in Params.

### Instantiating the neural network

by using the sekeleton provided, we can create a new network at the beginning of the eval function:

    template<typename Indiv>
       void _generate_net(const Indiv& ind)
       {
         _hidden_neuron_handles = std::vector<neuron_handle_t>(0);
         // iterator on the parameters
         size_t k = 0;
         // set the neural network number of inputs and outputs
         _nn1.set_nb_inputs(2);
         _nn1.set_nb_outputs(1);
         // create the hidden neurons and set their internal bias
         for (size_t i = 0; i < Params::fit::nb_hidden_neurons; ++i)
         {
           std::ostringstream tmp;
           tmp << i;
           _hidden_neuron_handles.push_back(_nn1.add_neuron(tmp.str()));
           //set the neuron bias using the evolved parameters
           _nn1.get_neuron_by_vertex(_hidden_neuron_handles[i]).set_afparams(ind.data(k++));
         }
         // create the first layer of connections and set their weight
         for (size_t i = 0; i < _nn1.get_nb_inputs();++i)
         {
           for (size_t j = 0; j < _hidden_neuron_handles.size();++j)
           {
             _nn1.add_connection(_nn1.get_input(i),_hidden_neuron_handles[j],ind.data(k++));
           }
         }
          //the output is a real neuron, its bias must be initialised (this is not the case of the inputs)
           _nn1.get_neuron_by_vertex(_nn1.get_output(0)).set_afparams(0.0f);
         // create the second layer of connections and set their weight
         for (size_t j = 0; j < _hidden_neuron_handles.size();++j)
         {
           _nn1.add_connection(_hidden_neuron_handles[j], _nn1.get_output(0), ind.data(k++));
         }
       }

In this function, a few points are important.

- Input and output neurons are automatically created by the functions set_nb_inputs and set_nb_outputs. These functions must be called before creating any hidden neurons.

- add_neuron takes a identifier string as parameter and outputs a handle to the vertex, not the neuron! You need to use get_neuron_by_vertex to get the real neuron.

- While the inputs of the network are not real neurons, the output contains Pot and Act functions. Their parameters must set.

- add_connection, which takes as parameters the pre-synaptic neuron, the post-synaptic neurons and the weight can take a single value as parameter if the number of parameter is equal to one, or an array if there are more parameters.

The variables used : "_hidden_neuron_handles" and "_nn1" must also be added in the protected section of the class.

    protected:
      std::vector<neuron_handle_t> _hidden_neuron_handles;
      nn_t _nn1;

### Execute the neural network in a loop

We can now run the instantiated neural network in a loop and test the points defined in Params:

    template<typename Indiv>
      void eval(const Indiv& ind)
    {
      //create the network
      _generate_net(ind);
      //initialize network (reset accumulators and set the weight buffers if pf)
      _nn1.init();
      float v = 0;
      float tmp_val = 0;
      for (size_t i = 0; i < Params::fit::x_pos_size(); ++i)
      {
        for (size_t j = 0; j < Params::fit::y_pos_size(); ++j)
        {
          //set the network inputs
          std::vector<io_t> inputs(0);
          inputs.push_back(Params::fit::x_pos(i));
          inputs.push_back(Params::fit::y_pos(j));
         //iterate the network
         for (size_t k = 0;k < 3; ++k)
         {
           _nn1.step(inputs);
         }
         tmp_val=(_gaussian(inputs[0],inputs[1])-_nn1.get_outf()[0]);
         v+=tmp_val*tmp_val;
        }
      }
      this->_value = -v;
    }

In this function, you should note that initializing the network before using it is mandatory. The step function, used three times here cycles through the neurons without taking the network topology into account. Therefore, you should execute a number of steps equivalent to the depth of the network (counting the input neurons) to be sure the output values have been updated.

The input neurons do not apply the activation function or the potential function on the input vector, on the contrary, the output neurons do.

Finally, reading the network outputs is done by the function get_outf() which provides an array of io_t of size the numuber of output neurons.

### Compiling and debugging the experiment

Once your code is done you can create the binary files for your experiment by using the following command:

    ./waf --exp test

If your code is correct, you should get binary files of your experiment in to folders: * in

    build/debug/exp/test

you will find a version compiled with debug information * in

    build/default/exp/test

you will find the final version, with code optimized for speed, but no debug information

If your code contains errors, two tools may help you debug using the debug version : * valgrind can be used to easily find memory leaks or analyse which parts of your code are too slow in combination with kcachegrind. * gdb is really useful to backtrace errors.

### A posteriori analysis of solutions

Sferes2 provides several methods to analyse the results of your experiments. an experiment in sferes2 will create a new folder in the launch directory which should look like that:

    machineneame_2001-01-01_13_52_00_2170

and can be decomposed in the following elements separated by an underscore: # the hostname of the machine running the experiment (especially usefull for running experiments on multiple machines) # the date and hour the run was launched # a random generated string to identify the experiment

This experiment will contain several files named

    gen_X

with X being the generation. These files contain the pareto front of the experiment in xml format (using the serialization library from boost). The interval at which these individuals are dumped is defined in the source code by the parameter

    Params::pop::dump_period

.

If you want to test one of these individuals, you can load it using the executable of the experiment from the

    build/debug/exp/test/

directory:

    ./test --load=machineneame_2001-01-01_13_52_00_2170/gen_50 --out gna --number 0

this will load the generation 50 of the run

    machineneame_2001-01-01_13_52_00_2170

, load the individual number 0 in the file and write the debug information in a file named

    gna

.

### Adding code executed when loading files

If you want additionnal debug information, you can define code in the fitness function which will only be run when loading files. This is done by using the

    this->mode()

function. This code snippet writes the topology of the loaded network in a file named

    graph.dot

.

    if (this->mode() == fit::mode::view)
    {
      std::ofstream ofs(std::string("graph.dot").c_str());
      _nn1.write(ofs);
    }

the graph.dot file created can then be transformed into a graphical view of your network using the dot software (not included in sferes 2 code):

    dot graph.dot -T eps -o graph.eps

### Complete file

Below is the complete example file:

    #include <iostream>
    #include <sferes/phen/parameters.hpp>
    #include <sferes/gen/evo_float.hpp>
    #include <sferes/ea/rank_simple.hpp>
    #include <sferes/eval/eval.hpp>
    #include <sferes/stat/best_fit.hpp>
    #include <sferes/stat/mean_fit.hpp>
    #include <sferes/modif/dummy.hpp>
    #include <sferes/run.hpp>
    #include <modules/nn2/nn.hpp>
    #include <math.h>

    using namespace sferes;
    using namespace sferes::gen::evo_float;
    using namespace nn;

    struct Params
    {
      struct evo_float
      {
        // we choose the polynomial mutation type
        static const mutation_t mutation_type = polynomial;
        // we choose the polynomial cross-over type
        static const cross_over_t cross_over_type = sbx;
        // the mutation rate of the real-valued vector
        static const float mutation_rate = 0.1f;
        // the cross rate of the real-valued vector
        static const float cross_rate = 0.5f;
        // a parameter of the polynomial mutation
        static const float eta_m = 15.0f;
        // a parameter of the polynomial cross-over
        static const float eta_c = 10.0f;
      };
      struct pop
      {
        // size of the population
        static const unsigned size = 200;
        // number of generations
        static const unsigned nb_gen = 2000;
        // how often should the result file be written (here, each 5
        // generation)
        static const int dump_period = 5;
        // how many individuals should be created during the random
        // generation process?
        static const int initial_aleat = 1;
        // used by RankSimple to select the pressure
        static const float coeff = 1.1f;
        // the number of individuals that are kept from on generation to
        // another (elitism)
        static const float keep_rate = 0.6f;
      };
      struct fit
      {
        static const size_t nb_hidden_neurons = 3 ;
        SFERES_ARRAY(float,x_pos, 0.69, 0.52, 0.80, 0.95, 0.97, 0.82, 0.11, 0.13, 0.27, 0.59);
        SFERES_ARRAY(float,y_pos, 0.24, 0.30, 0.63, 0.30, 0.81, 0.50, 0.47, 0.53, 0.51, 0.66);
      };
      struct parameters
      {
        // maximum value of parameters
        static const float min = -10.0f;
        // minimum value
        static const float max = 10.0f;
      };
    };

    SFERES_FITNESS(FitTest, sferes::fit::Fitness)
    {
      typedef float io_t;
      typedef Neuron<PfWSum<>, AfTanh<io_t> > neuron_t;
      typedef Connection<float> connection_t;
      typedef NN< neuron_t, connection_t > nn_t;
      typedef nn_t::vertex_desc_t neuron_handle_t;
     public:
      // indiv will have the type defined in the main (phen_t)
      template<typename Indiv>
        void eval(const Indiv& ind)
      {
        //create the network
        _generate_net(ind);
        //initialize network (reset accumulators and set the weight buffers if pf)
        _nn1.init();
        float v = 0;
        float tmp_val = 0;
        for (size_t i = 0; i < Params::fit::x_pos_size(); ++i)
        {
          for (size_t j = 0; j < Params::fit::y_pos_size(); ++j)
          {
            //set the network inputs
            std::vector<io_t> inputs(0);
            inputs.push_back(Params::fit::x_pos(i));
            inputs.push_back(Params::fit::y_pos(j));
           //iterate the network
           for (size_t k = 0;k < 3; ++k)
           {
             _nn1.step(inputs);
           }
           tmp_val=(_gaussian(inputs[0],inputs[1])-_nn1.get_outf()[0]);
           v+=tmp_val*tmp_val;
          }
        }
        if (this->mode() == fit::mode::view)
        {
          std::ofstream ofs(std::string("graph.dot").c_str());
          _nn1.write(ofs);
        }
        this->_value = -v;
      }
     protected:
      std::vector<neuron_handle_t> _hidden_neuron_handles;
      nn_t _nn1;
      float _gaussian(float x, float y)
      {
        return exp(-(x*x+y*y)/2);
      }
      template<typename Indiv>
        void _generate_net(const Indiv& ind)
        {
          _hidden_neuron_handles = std::vector<neuron_handle_t>(0);
          // iterator on the parameters
          size_t k = 0;
          // set the neural network number of inputs and outputs
          _nn1.set_nb_inputs(2);
          _nn1.set_nb_outputs(1);
          // create the hidden neurons and set their internal bias
          for (size_t i = 0; i < Params::fit::nb_hidden_neurons; ++i)
          {
            std::ostringstream tmp;
            tmp << i;
            _hidden_neuron_handles.push_back(_nn1.add_neuron(tmp.str()));
            //set the neuron bias using the evolved parameters
            _nn1.get_neuron_by_vertex(_hidden_neuron_handles[i]).set_afparams(ind.data(k++));
          }
          //the output is a real neuron, its bias must be initialised
          _nn1.get_neuron_by_vertex(_nn1.get_output(0)).set_afparams(0.0f);
          // create the first layer of connections and set their weight
          for (size_t i = 0; i < _nn1.get_nb_inputs();++i)
          {
            for (size_t j = 0; j < _hidden_neuron_handles.size();++j)
            {
              _nn1.add_connection(_nn1.get_input(i),_hidden_neuron_handles[j],ind.data(k++));
            }
          }
          // create the second layer of connections and set their weight
          for (size_t j = 0; j < _hidden_neuron_handles.size();++j)
          {
            _nn1.add_connection(_hidden_neuron_handles[j], _nn1.get_output(0), ind.data(k++));
          }
        }
    };

    int main(int argc, char **argv)
    {
      // Our fitness is the class FitTest (see above), that we will call
      // fit_t. Params is the set of parameters (struct Params) defined in
      // this file.
      typedef FitTest<Params> fit_t;
      // We define the genotype. Here we choose EvoFloat (real
      // numbers). We evolve 10 real numbers, with the params defined in
      // Params (cf the beginning of this file)
      typedef gen::EvoFloat<4 * Params::fit::nb_hidden_neurons, Params> gen_t;
      // This genotype should be simply transformed into a vector of
      // parameters (phen::Parameters). The genotype could also have been
      // transformed into a shape, a neural network... The phenotype need
      // to know which fitness to use; we pass fit_t.
      typedef phen::Parameters<gen_t, fit_t, Params> phen_t;
      // The evaluator is in charge of distributing the evaluation of the
      // population. It can be simple eval::Eval (nothing special),
      // parallel (for multicore machines, eval::Parallel) or distributed
      // (for clusters, eval::Mpi).
      typedef eval::Eval<Params> eval_t;
      // Statistics gather data about the evolutionary process (mean
      // fitness, Pareto front, ...). Since they can also stores the best
      // individuals, they are the container of our results. We can add as
      // many statistics as required thanks to the boost::fusion::vector.
      typedef boost::fusion::vector<stat::BestFit<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
      // Modifiers are functors that are run once all individuals have
      // been evalutated. Their typical use is to add some evolutionary
      // pressures towards diversity (e.g. fitness sharing). Here we don't
      // use this feature. As a consequence we use a "dummy" modifier that
      // does nothing.
      typedef modif::Dummy<> modifier_t;
      // We can finally put everything together. RankSimple is the
      // evolutianary algorithm. It is parametrized by the phenotype, the
      // evaluator, the statistics list, the modifier and the general params.
      typedef ea::RankSimple<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;
      // We now have a special class for our experiment: ea_t. The next
      // line instantiate an object of this class
      ea_t ea;
      // we can now process the comannd line options an run the
      // evolutionary algorithm (if a --load argument is passed, the file
      // is loaded; otherwise, the algorithm is launched).
      run_ea(argc, argv, ea);
      //
      return 0;
    }

## Neurobiology Tutorial II

This second tutorial will guide you through the process of evolving not only the weights, but also the topology of a neural network. Using the nn2 module from sferes2, we will test both the evolution of a a recurrent network on the same task as previously (function approximation).

### Download and install Sferes2 and crete a new experiment

The first step is to install and compile Sferes2. To do so, please follow the download and compilation pages before continuing. If you have successfully followed these two pages, you should be in the "trunk" directory of sferes2, else, type the following command after replacing $SFERES2_PARENT_DIRECTORY by the path where you put sferes2:

    cd $SFERES2_PARENT_DIRECTORY/sferes2/trunk

you should now create a new experiment:

    ./waf --create test2

We can now enter the experiment source code and edit the wscript to add the necessary modules:

    cd exp/test2
    edit wscript

The required modules are the same as the previous tutorial:

    #! /usr/bin/env python
    def build(bld):
        obj = bld.new_task_gen('cxx', 'program')
        obj.source = 'test2.cpp'
        obj.includes = '. ../../'
        obj.uselib = 'EIGEN2'
        obj.target = 'test2'
        obj.uselib_local = 'sferes2'

we can then edit the source code for our new experiment: test2.cpp.

### Add the relevant headers

Unlike the previous tutorial, we are going to include headers that wrap the previous neural network code with functions to generate random topologies and evolve them.

    #include <modules/nn2/gen_dnn.hpp>
    #include <modules/nn2/phen_dnn.hpp>

    using namespace sferes::gen::dnn;

If you look at these files, you will see that a few methods have been added to evolve the topology of neural networks. In the genotype, you will find the following methods:

- random() initializes a genotype randomly (using the parameters from Params).

- mutate() randomly changes the network by adding, removing or changing nodes and connections ( using the probabilities stated in params).

- cross(parent1, parent2) initializes the network genotype as a random of one of the two parents.

the phenotype file provides the following methods:

- develop() must be called to initialize a genotype (fill in the parameters) and is mandatory before using a network.

- show(ostream out) dumps the network topology to a dot file.

- nn() returns a reference to the neural network in the phenotype.

Finally, we need the dnn namespace.

### Define the necessary parameters

Again, we need to define the required parameters for our networks, the main addition concerns the mutation rates:

      struct evo_float
      {
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.1f;
        static const mutation_t mutation_type = polynomial;
        static const cross_over_t cross_over_type = sbx;
        static const float eta_m = 15.0f;
        static const float eta_c = 15.0f;
      };
      struct parameters
      {
        // maximum value of parameters
        static const float min = -5.0f;
        // minimum value
        static const float max = 5.0f;
      };
      struct dnn
      {
        static const size_t nb_inputs = 2;
        static const size_t nb_outputs  = 1;
        static const size_t min_nb_neurons  = 3;
        static const size_t max_nb_neurons  = 10;
        static const size_t min_nb_conns  = 100;
        static const size_t max_nb_conns  = 101;

        static const float m_rate_add_conn  = 1.0f;
        static const float m_rate_del_conn  = 1.0f;
        static const float m_rate_change_conn = 1.0f;
        static const float m_rate_add_neuron  = 1.0f;
        static const float m_rate_del_neuron  = 1.0f;

        static const int io_param_evolving = true;
    //    static const init_t init = ff;
        static const init_t init = random_topology;
      };

As previously, you need an evo_float struct which provides the mutation rate of the parameters (bias_t) and for the weights (weight_t). If you need different values for the two, you can create multiple Params classes and pass them as template arguments in your typedefs in the next paragraph.

The second struct, parameters, is used to define the scaling of the evo-float values (evo_float are between 0 and 1 normally). Again, if you require different scaling for the neuron parameters and weights, you can provide separate params.

The last struct, dnn, defines several new elements:

- the number of intputs and outputs of the evolved network (it must be constant)

- the minimum and maximum number of neurons in any evolved network

- the minimum and maximum number of connections in the network

- probabilities to (in order)

  - add a connection

  - remove a connection

  - change the source or target of a connection

  - add a new neuron (replaces an existing connection)

  - remove a neuron

The probability of changing a parameter is defined by the values in the evo_float struct. The evolution of parameters of the output nodes can be enabled or disabled through the io_param_evolving constant. Finally, the init constant is used to define if the initial random network have a feedforward topology with no hidden nodes ("ff" value) like NEAT or a random topology (random_topology).

### Declare the type of our input/outputs, weights, connections and neurons

Similarly to the previous tutorial, we need to define the template parameters of our neural network in the main function. However, here, the io_t and weight_t are replaced by evo_float which has the mutate, random and cross methods (bias_t is the internal bias of the neuron and has the same type as all the other inputs io_t). If you use a neuron with more than one parameter, you should increase the bias_t size). Finally, we need to set the fitness as the algorithm fitness.

    typedef phen::Parameters<gen::EvoFloat<1, Params>, fit::FitDummy<>, Params> weight_t;
    typedef phen::Parameters<gen::EvoFloat<1, Params>, fit::FitDummy<>, Params> bias_t;
    typedef PfWSum<weight_t> pf_t;
    typedef AfTanh<bias_t> af_t;
    typedef Neuron<pf_t, af_t > neuron_t;
    typedef Connection<weight_t> connection_t;

    typedef sferes::gen::Dnn<neuron_t,  connection_t, Params> gen_t;
    typedef phen::Dnn<gen_t, FitTest<Params>, Params> phen_t;

### Using the parallel evaluation

If your sferes has been compiled with tbb, you can evaluate your individuals in parallel. This works quite well on HyperThreaded processors by the way. However, this is only useful for significant time evaluation. To do so, just replace the eval::Eval by eval::Parallel in the main typedefs. This divided the runtime of this example from 18,023 seconds to 8,313 seconds using a three processor machine.

    typedef eval::Parallel<Params> eval_t;

### add code to initialise the neural network and run it

We are going to use the neural network in a way similar to the previous tutorial, to approximate a function. To do so, we must modify the eval function in the fitness class.

    template<typename Indiv>
        void eval(Indiv& ind)
      {
        //initialize network (set weights and parameters, reset accumulators and set the weight buffers if pf)
        ind.develop();
        float v = 0;
        float tmp_val = 0;
        for (size_t i = 0; i < Params::fit::x_pos_size(); ++i)
        {
          for (size_t j = 0; j < Params::fit::y_pos_size(); ++j)
          {
            //set the network inputs
            std::vector<float> inputs(0);
            inputs.push_back(Params::fit::x_pos(i));
            inputs.push_back(Params::fit::y_pos(j));
           //iterate the network 5 times (if the network is recurrent, additionnal tests could be necessary to test the stability)
           size_t max_iterations = 5;
           for (size_t k = 0;k < max_iterations; ++k)
           {
             ind.nn().step(inputs);
           }
           tmp_val=(_gaussian(inputs[0],inputs[1])-ind.nn().get_outf()[0]);
           v+=tmp_val*tmp_val;
          }
        }
        if (this->mode() == fit::mode::view)
        {
          std::ofstream ofs(std::string("graph.dot").c_str());
          ind.nn().write(ofs);
        }
        this->_value = -v;
      }

The first modification from the previous version are that we increased the number of step done by the network. We also call only the develop method from the individual's phenotype and we then use ind.nn() to access its network.

we also must add the positions to test in the parameters:

    struct fit
     {
       static const size_t nb_hidden_neurons = 3 ;
       SFERES_ARRAY(float,x_pos, 0.69, 0.52, 0.80, 0.95, 0.97, 0.82, 0.11, 0.13, 0.27, 0.59);
       SFERES_ARRAY(float,y_pos, 0.24, 0.30, 0.63, 0.30, 0.81, 0.50, 0.47, 0.53, 0.51, 0.66);
     };

and finally the function to approximate:

    protected:
     float _gaussian(float x, float y)
     {
       return exp(-(x*x+y*y)/2);
     }

You can now compile the new experiment (view the previous tutorial) and obtain networks such as the following one :

### using feedforward networks only

The dnn genotype can create recursive networks. In order to generate only feedforward networks, you can use the gen_dnn_ff.hpp include which is a derivative class from gen_dnn.hpp.

As we want to create both an experiment using dnn and dnn_ff with the same codebase, we are going to use a feature from sferes2, called variants to generate two different binaries from a single source code.

First, we need to replace the previosu wscript to tell the compiler to create two binaries

    import sferes

    def build(bld):
      uselib_ = 'EIGEN2 GSL BOOST_UNIT_TEST_FRAMEWORK'
      cxxflags_kdtree = \
          bld.get_env()['CXXFLAGS']
      sferes.create_variants(bld,
                              source = 'test2.cpp',
                              uselib_local = 'sferes2',
                              uselib = uselib_,
                              target = '',
                              json = '',
                              cxxflags=cxxflags_kdtree,
                              variants = ['DNN','DNN FEEDF'])

The important part of this code is the variants which define the variants of the binary to compile. The first flag is generic (you need at least one flag to compile a binary) and the second one, FEEDF, will create a variant which has a #define FF at the beginning of the file (do not use FF as it is already defined in BOOST code).

we can then modify the include to add the gen_dnn_ff.hpp genotype instead of the previous gen_dnn.hpp if FF is defined:

    #ifdef FEEDF
    #include <modules/nn2/gen_dnn_ff.hpp>
    #else
    #include <modules/nn2/gen_dnn.hpp>
    #endif

The next step is to change the network template parameters to use feedforward networks:

    #ifdef FEEDF
      typedef sferes::gen::DnnFF<neuron_t,  connection_t, Params> gen_t;
    #else
      typedef sferes::gen::Dnn<neuron_t,  connection_t, Params> gen_t;
    #endif

The next necessary step is to define the initial topology as a feedforward simple network, as a random topology may be recursive:

    #ifdef FEEDF
        static const init_t init = ff;
    #else
        static const init_t init = random_topology;
    #endif

As a bonus, if you use the DnnFF instead of Dnn, the number of steps required before convergence of the network is equal to the network's depth, which can be computed by the get_depth function.

    ifdef FEEDF
      size_t max_iterations = ind.gen().get_depth();
    #else
      size_t max_iterations = 5;
    #endif

the compilation should the provide you with two binary files :

    test2_dnn
    test2_dnn_feedf

### Complete source code for the experiment

    #include <iostream>
    #include <sferes/phen/parameters.hpp>
    #include <sferes/gen/evo_float.hpp>
    #include <sferes/ea/rank_simple.hpp>
    #include <sferes/eval/eval.hpp>
    #include <sferes/stat/best_fit.hpp>
    #include <sferes/stat/mean_fit.hpp>
    #include <sferes/modif/dummy.hpp>
    #include <sferes/run.hpp>

    #include <modules/nn2/gen_dnn.hpp>

    #ifdef FEEDF
    #include <modules/nn2/gen_dnn_ff.hpp>
    #endif

    #include <modules/nn2/phen_dnn.hpp>

    using namespace sferes;
    using namespace sferes::gen::evo_float;
    using namespace sferes::gen::dnn;
    using namespace nn;

    struct Params
    {
     struct evo_float
      {
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.1f;
        static const mutation_t mutation_type = polynomial;
        static const cross_over_t cross_over_type = sbx;
        static const float eta_m = 15.0f;
        static const float eta_c = 15.0f;
      };
      struct dnn
      {
        static const size_t nb_inputs = 2;
        static const size_t nb_outputs  = 1;
        static const size_t min_nb_neurons  = 3;
        static const size_t max_nb_neurons  = 8;
        static const size_t min_nb_conns  = 100;
        static const size_t max_nb_conns  = 101;

        static const float m_rate_add_conn  = 1.0f;
        static const float m_rate_del_conn  = 1.0f;
        static const float m_rate_change_conn = 1.0f;
        static const float m_rate_add_neuron  = 1.0f;
        static const float m_rate_del_neuron  = 1.0f;

        static const int io_param_evolving = true;
    #ifdef FEEDF
        static const init_t init = ff;
    #else
        static const init_t init = random_topology;
    #endif
      };
      struct pop
      {
        static const unsigned size = 200;
        static const unsigned nb_gen = 2000;
        static const int dump_period = 5;
        static const int initial_aleat = 1;
        static const float coeff = 1.1f;
        static const float keep_rate = 0.6f;
      };
      struct parameters
      {
        static const float min = -5.0f;
        static const float max = 5.0f;
      };
      struct fit
      {
        static const size_t nb_hidden_neurons = 3 ;
        SFERES_ARRAY(float,x_pos, 0.69, 0.52, 0.80, 0.95, 0.97, 0.82, 0.11, 0.13, 0.27, 0.59);
        SFERES_ARRAY(float,y_pos, 0.24, 0.30, 0.63, 0.30, 0.81, 0.50, 0.47, 0.53, 0.51, 0.66);
      };

    };

    SFERES_FITNESS(FitTest, sferes::fit::Fitness)
    {
     public:
      // indiv will have the type defined in the main (phen_t)
      template<typename Indiv>
        void eval(Indiv& ind)
      {
        //initialize network (set weights and parameters, reset accumulators and set the weight buffers if pf)
        ind.develop();
        float v = 0;
        float tmp_val = 0;
    #ifdef FEEDF
        size_t max_iterations = ind.gen().get_depth();
    #else
        size_t max_iterations = 5;
    #endif
        for (size_t i = 0; i < Params::fit::x_pos_size(); ++i)
        {
          for (size_t j = 0; j < Params::fit::y_pos_size(); ++j)
          {
            //set the network inputs
            std::vector<float> inputs(0);
            inputs.push_back(Params::fit::x_pos(i));
            inputs.push_back(Params::fit::y_pos(j));
            //iterate the network 5 times (if the network is recurrent, additionnal tests could be necessary to test the stability)
            for (size_t k = 0;k < max_iterations; ++k)
            {
              ind.nn().step(inputs);
            }
            tmp_val=(_gaussian(inputs[0],inputs[1])-ind.nn().get_outf()[0]);
            v+=tmp_val*tmp_val;
          }
        }
        if (this->mode() == fit::mode::view)
        {
          std::ofstream ofs(std::string("graph.dot").c_str());
          ind.nn().write(ofs);
        }
        this->_value = -v;
      }
     protected:
      float _gaussian(float x, float y)
      {
        return exp(-(x*x+y*y)/2);
      }

    };

    int main(int argc, char **argv)
    {
      typedef phen::Parameters<gen::EvoFloat<1, Params>, fit::FitDummy<>, Params> weight_t;
      typedef phen::Parameters<gen::EvoFloat<1, Params>, fit::FitDummy<>, Params> bias_t;
      typedef PfWSum<weight_t> pf_t;
      typedef AfTanh<bias_t> af_t;
      typedef Neuron<pf_t, af_t > neuron_t;
      typedef Connection<weight_t> connection_t;

    #ifdef FEEDF
      typedef sferes::gen::DnnFF<neuron_t,  connection_t, Params> gen_t;
    #else
      typedef sferes::gen::Dnn<neuron_t,  connection_t, Params> gen_t;
    #endif

      typedef phen::Dnn<gen_t, FitTest<Params>, Params> phen_t;
      typedef eval::Parallel<Params> eval_t;
      typedef boost::fusion::vector<stat::BestFit<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
      typedef modif::Dummy<> modifier_t;
      typedef ea::RankSimple<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;
      ea_t ea;
      run_ea(argc, argv, ea);
      //
      return 0;
    }

## Neurobiology Tutorial III

This third tutorial will guide you through the process of evolving the topology and parameters of maps, using the evoneuro module. We will test both the evolution of parameters first and then the evolution of a complete topology.

This tutorial will evolve networks to solve the task taken from neurosciences. With inputs corresponding to possible actions, the network must select the output corresponding to the maximum input and have a minimal output on this node. On the contrary, all the other outputs must be maximized. This a task similar to a winner-takes-all, but where the winner has a minimal output and losers have a maximal output.

Disclaimer : If you do not intend to evolve the topology of the network, you might instead use the first tutorial to create by hand any fixed network topology instead of learning to use this tool. The main advantage of the EvoNeuro module is the automatic generation of neural maps and its ability to evolve them.

### Presentation of the EvoNeuro module

The evoneuro module in sferes is composed of both a genotype and a phenotype class. The genotype class evolves a dnn network similar to the one used in dnn (appart from some additionnal evolvable parameters in each node and connection). The topology, weights and parameters can be evolved as in a standard neural network.

The specificity of the EvoNeuro module comes from the phenotype class: it creates a new network of neuron maps from the topology of the genotype network. During the develop() function (viewed in the second tutorial) each neurons from the genotype can be transformed into:

- a map of neurons with similar properties as the original neuron;

- a single neuron (no transformation).

This depends on an evolved parameter for each neuron. Each connection is similarly transformed into:

- a one-to-all connectivity pattern (if either the pre- and/or post-synaptic node is a single neuron, this is the only possibility);

- a on-to-one connectivity pattern between two maps;

- a on-to-all connectivity pattern between two mapos with gaussian weights (the connection between corresponding elements is at the maximum weights and the weights gets smaller when the index difference increases).

The development process is shown in the following figure.

evoneuro

### Creating a new experiment which uses the evoneuro module

Like previously, we are going to start from the sample experiment we used previously. The first thing to modify is the params. We need to declare the parameters for dnn (used by evoneuro) and evoneuro itself:

    struct dnn
     {
       static const size_t nb_inputs = 1;
       static const size_t nb_outputs  = 1;

       static const size_t min_nb_neurons	= 3;
       static const size_t max_nb_neurons	= 8;
       static const size_t min_nb_conns	= 2;
       static const size_t max_nb_conns	= 15;

       static const float m_rate_add_conn	= 0.05f;
       static const float m_rate_del_conn	= 0.03f;
       static const float m_rate_change_conn = 0.07f;
       static const float m_rate_add_neuron  = 0.02f;
       static const float m_rate_del_neuron  = 0.01f;

       static const int io_param_evolving  = 1;

       static const init_t init = random_topology;
     };
     struct evo_neuro
     {
       typedef nn::Neuron<nn::PfWSum<>, nn::AfLpds<> >  neuron_t;
       typedef nn::Connection<> connection_t;
       static const size_t map_size  = 5;
       static const float  max_weight  = 5.0f;
     };

First, nb_inputs and nb_outputs are set to one, as there are usually a single input and a single output map. The maximum and minimum values of neurons and connections will only apply to the genotype graph and not to the developped network.

The evo_neuro struct contains the type of neurons and connections that will be used by the develop() function to create the phenotype network as they are different from the genotype one (contrary to dnn). For the sake of simplicity, we will keep the AfLpds function used which is a neuron with internal variables. However, you can easily modify the phen_evoneuro.hpp file to use other types of neurons.

We must then define the typedefs for our network in the main function:

    typedef phen::Parameters<gen::EvoFloat<4, Params>, fit::FitDummy<>, Params> node_label_t;
    typedef phen::Parameters<gen::EvoFloat<3, Params>, fit::FitDummy<>, Params> conn_label_t;
    typedef gen::EvoNeuro<node_label_t, conn_label_t, Params> gen_t;
    typedef phen::EvoNeuro<gen_t, FitTest<Params>, Params> phen_t;

The first two lines are the nodes and connection types of neurons and connexions.

- a node should contain the number of parameters for your af (your pf is considered to have no internal parameter) plus 1 parameter (defining if the node is a map or a single neuron)

- a connection should contain 3 parameters:

  - one for the type of connection (1-1 or 1-all)

  - one for the weight pattern (constant or gaussian)

  - one for the weight of the connection

The two other lines are constant for use with evoneuro appart from the fitness definition

### Defining an adapted fitness function

Contrary to the previous tutorial, the EvoNeuro module is ill-adapted to calculating a simple mathematical function. We instead try to emulate the results of the paper described in this paper. We want our network to evolve a contrast augmenting function : it should maximize the max from the incoming signals while reducing the value of other inputs channels in the output.

contrast

### Using the NSGA II algorithms instead of simple sorting

As sferes2 provides NSGA2 (a multi-objective state of the art algorithm), we will optimize the two objectives separately.

To change the evaluation algorithm, we just need to add the relevant headers. The first one is the evolutionary algorithm, the second one the statistics used to dump the fitness of the individuals on the pareto front.

    //#include <sferes/ea/rank_simple.hpp>
    #include <sferes/ea/nsga2.hpp>
    //#include <sferes/stat/best_fit.hpp>
    #include <sferes/stat/pareto_front.hpp>

We also need to change the corresponding typedefs in the main function:

    //typedef boost::fusion::vector<stat::BestFit<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
    typedef boost::fusion::vector<stat::ParetoFront<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
    //typedef ea::RankSimple2<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;
    typedef ea::Nsga2<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;

The next step is to initialize the objective vector at the beginning of our eval function (NSGA II uses a vector of objectives instead of a single value):

    this->_objs.resize(2);
    for (size_t i=0;i<this->_objs.size();++i)
      this->_objs[i]=0;

### Defining the random inputs sets

The evaluation function will need to test the different networks on a set of random inputs, and, for each set, have only the output corresponding to the maximum input at zero, while the other outputs are maxed out. This can be easily done throught two objectives:

- minimize the output corresponding to the max input

- maximize the sum of the other outputs

The first step is therefore to create random inputs sets in the fitness function. These sets are created here as a vector of vector of floats with values between 0 and 1. The sets are created at the initialisation of the FitTest function.

    public:
      FitTest():
        inputs_list(0)
      {
       for (size_t test = 0; test < Params::fit::nb_tests; ++test )
       {
         inputs_list.push_back(std::vector<float>(Params::evo_neuro::map_size));
         for (size_t i = 0; i < inputs_list.back().size(); ++i )
         {
           inputs_list.back()[i] = misc::rand<float>();
         }
       }
      }

and

    protected:
      std::vector<std::vector< float> > inputs_list;

### Defining the eval function

We are going to use the following eval function:

    template<typename Indiv>
      void eval(Indiv& ind)
    {
      // resize the objective vector
      this->_objs.resize(2);
      for (size_t i=0;i<this->_objs.size();++i)
        this->_objs[i]=0.0f;
      // generate the phenotype network
      ind.develop();
      // if we are looking at the results, dump the genotype topology to graph.dot
      // the simplify function remove all non-useful parts
      if (this->mode() == fit::mode::view)
      {
        std::ofstream ofs(std::string("graph.dot").c_str());
        ind.gen().simplify();
        ind.gen().name_io();
        ind.gen().write(ofs);
      }
      // initialise the phenotype network
      ind.nn().init();
      // test all the possible input sets
      for (size_t test = 0; test < Params::fit::nb_tests; ++test )
      {
        //find the maximum input in the set
        size_t max = 0;
        for (size_t i = 0; i < inputs_list[test].size(); ++i )
        {
          if (inputs_list[test][i] > inputs_list[test][max])
            max = i;
        }
        //loop the network
        for (size_t cycles = 0; cycles < Params::fit::max_cycles; ++cycles)
        {
          ind.nn().step(inputs_list[test]);
        }
        //set the first objective (minimise output with the same index as the maximal input)
        this->_objs[0] += 1.0f-fabs(ind.nn().get_outf()[max]);
        //set the second objective as the average value of other outputs
        for (size_t i = 0; i < ind.nn().get_outf().size(); ++i )
        {
          if (i != max)
            this->_objs[1] += fabs(ind.nn().get_outf()[i])/(float)(ind.nn().get_outf().size()-1.0f);
        }
      }
      //average the objectives with the number of tests
      for (size_t i=0;i<this->_objs.size();++i)
        this->_objs[i]/=(float)(Params::fit::nb_tests);
    }

### Sample solution network

If you launch an experiment with this code, you should find networks like the following graph (the maps have not been developped):

graph

### Analysing the results

After compiling the code and executing the code, you can analyse the results by browsing the pareto.dat file or launching the runs with the load command.

### Complete source code

    #include <iostream>
    #include <sferes/phen/parameters.hpp>
    #include <sferes/gen/evo_float.hpp>
    #include <sferes/eval/eval.hpp>
    #include <sferes/ea/nsga2.hpp>
    //#include <sferes/stat/best_fit.hpp>
    #include <sferes/stat/pareto_front.hpp>
    #include <sferes/stat/mean_fit.hpp>
    #include <sferes/modif/dummy.hpp>
    #include <sferes/run.hpp>

    #include <modules/evoneuro2/gen_evo_neuro.hpp>
    //#include "phen_evo_neuro.hpp"
    #include <modules/evoneuro2/phen_evo_neuro.hpp>
    #include <modules/evoneuro2/lpds.hpp>

    using namespace sferes;
    using namespace sferes::gen::evo_float;
    using namespace sferes::gen::dnn;
    using namespace nn;

    struct Params
    {
      struct evo_float
      {
        static const mutation_t mutation_type = polynomial;
        static const cross_over_t cross_over_type = sbx;
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.5f;
        static const float eta_m = 15.0f;
        static const float eta_c = 10.0f;
      };
      struct pop
      {
        static const unsigned size = 200;
        static const unsigned nb_gen = 15000;
        static const int dump_period = 200;
        static const int initial_aleat = 1;
        static const float coeff = 1.1f;
        static const float keep_rate = 0.6f;
      };
      struct dnn
      {
        static const size_t nb_inputs	= 1;
        static const size_t nb_outputs	= 1;

        static const size_t min_nb_neurons	= 3;
        static const size_t max_nb_neurons	= 8;
        static const size_t min_nb_conns	= 2;
        static const size_t max_nb_conns	= 15;

        static const float m_rate_add_conn	= 0.05f;
        static const float m_rate_del_conn	= 0.03f;
        static const float m_rate_change_conn = 0.07f;
        static const float m_rate_add_neuron  = 0.02f;
        static const float m_rate_del_neuron  = 0.01f;

        static const int io_param_evolving  = 1;

        static const init_t init = random_topology;
      };
      struct evo_neuro
      {
        typedef nn::Neuron<nn::PfWSum<>, nn::AfLpds<> >  neuron_t;
    //    typedef nn::Neuron<nn::PfWSum<>, nn::AfTanh<float> >  neuron_t;
        typedef nn::Connection<> connection_t;
        static const size_t map_size	= 5;
        static const float  max_weight	= 5.0f;
      };

      struct parameters
      {
        static const float min = 0.0f;
        static const float max = 1.0f;
      };
      struct fit
      {
        static const size_t max_cycles = 200 ;
        static const size_t nb_tests = 50 ;
      };

    };

    SFERES_FITNESS(FitTest, sferes::fit::Fitness)
    {
     public:
       FitTest():
         inputs_list(0)
       {
        for (size_t test = 0; test < Params::fit::nb_tests; ++test )
        {
          inputs_list.push_back(std::vector<float>(Params::evo_neuro::map_size));
          for (size_t i = 0; i < inputs_list.back().size(); ++i )
          {
            inputs_list.back()[i] = misc::rand<float>();
          }
        }
       }
      template<typename Indiv>
        void eval(Indiv& ind)
      {
        // resize the objective vector
        this->_objs.resize(2);
        for (size_t i=0;i<this->_objs.size();++i)
          this->_objs[i]=0.0f;
        ind.develop();
        if (this->mode() == fit::mode::view)
        {
          std::ofstream ofs(std::string("graph.dot").c_str());
          ind.gen().simplify();
          ind.gen().name_io();
          ind.gen().write(ofs);
        }
        ind.nn().init();
        for (size_t test = 0; test < Params::fit::nb_tests; ++test )
        {
          size_t max = 0;
          for (size_t i = 0; i < inputs_list[test].size(); ++i )
          {
            if (inputs_list[test][i] > inputs_list[test][max])
              max = i;
          }
          for (size_t cycles = 0; cycles < Params::fit::max_cycles; ++cycles)
          {
            ind.nn().step(inputs_list[test]);
          }
          this->_objs[0] += 1.0f-fabs(ind.nn().get_outf()[max]);
          for (size_t i = 0; i < ind.nn().get_outf().size(); ++i )
          {
            if (i != max)
              this->_objs[1] += fabs(ind.nn().get_outf()[i])/(float)(ind.nn().get_outf().size()-1.0f);
          }
        }
        for (size_t i=0;i<this->_objs.size();++i)
          this->_objs[i]/=(float)(Params::fit::nb_tests);
      }
     protected:
      std::vector<std::vector< float> > inputs_list;
    };

    int main(int argc, char **argv)
    {
      int a(time(0));
      if (argc>1)
        a=atoi(argv[1]);
      srand(a);
      typedef phen::Parameters<gen::EvoFloat<4, Params>, fit::FitDummy<>, Params> node_label_t;
      typedef phen::Parameters<gen::EvoFloat<3, Params>, fit::FitDummy<>, Params> conn_label_t;
      typedef gen::EvoNeuro<node_label_t, conn_label_t, Params> gen_t;
      typedef phen::EvoNeuro<gen_t, FitTest<Params>, Params> phen_t;

      typedef eval::Parallel<Params> eval_t;
      typedef boost::fusion::vector<stat::ParetoFront<phen_t, Params>, stat::MeanFit<Params> >  stat_t;
      typedef modif::Dummy<> modifier_t;
      typedef ea::Nsga2<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;
      ea_t ea;
      run_ea(argc, argv, ea);
      return 0;
    }

## Reference

### The Sferes2 framework

#### Main concepts

##### Static object-oriented paradigm

Object-oriented programming (OOP) has many advantages which are desirable for complex high-level code. However, classic OOP (e.g. in C++) heavily relies on virtual methods to implement abstraction and polymorphism. Such an approach has a significant run-time overhead:

- selecting the good method to call adds an indirection (the software have to check the virtual table)

- abstract/virtual methods cannot be inlined, resulting in a overhead for very simple methods (e.g. setters/getters).

In a few word, we want the main benefits of OOP (abstraction, re-usability) and the efficiency of low-level languages.

One solution in C++ is to use the "curiously recurring template pattern" (http://en.wikipedia.org/wiki/Curiously_Recurring_Template_Pattern[http://en.wikipedia.org/wiki/Curiously_Recurring_Template_Pattern]). Here is the idea:

    template <class Derived> struct Base
    {
        void interface()
        {
            static_cast<Derived*>(this)->implementation();
        }
        void f()
        {
           for(int i = 0; i < 3; ++i) interface();
        }
    };

    struct Derived : Base<Derived>
    {
        void implementation() { std::cout<<"impl"<<std::endl; }
    };

This code mimics a virtual call of an abstract method: in the abstract class (Base), a generic algorithm can use methods which are only defined in the derived classes. Extending this pattern to draw a full hierarchy of classes requires more work; sferes2 relies on the method described in the following paper:

Nicolas Burrus, Alexandre Duret-Lutz, Thierry Géraud, David Lesage and Raphaël Poss. A Static C++ Object-Oriented Programming (SCOOP) Paradigm Mixing Benefits of Traditional OOP and Generic Programming. In Proceedings of the Workshop on Multiple Paradigm with OO Languages (MPOOL'03), 2003. http://www.lrde.epita.fr/dload/papers/mpool03.pdf

This mechanics is hidden in sferes2 behind some macros (see section [[macros]])

##### Static parameters

Evolutionary algorithms have a lot of parameters (e.g. population size, mutation rate, etc.) and we need a convenient way to set them. A configuration file (e.g. an XML file) is often used; however, such a method has two drawbacks:

- some code to read the files has to be written and kept synchronized with the objects;

- parameters are unknown at compile time so some checks (e.g. if (mutation_type = x) {} else { } ...) have to be done many times whereas they are useless.

In sferes2, parameters are defined at compile time using a structure which contains only constants. This structure is passed to all sferes2 classes so they can access to the parameters. This method allows to avoid to write read/write code for parameters. It also allows the compiler to propagate constants and settings in the whole source code, resulting in an executable optimized for the specific parameters.

##### Macros and special functions

1. <<macros>> To simplify the writing of "static virtual functions" and the definition of static parameters, sferes2 provides several macros which are defined in [stc.hpp] .

##### Classes

- SFERES_CLASS(Name): defines the base class Name ; this class uses the "static" inheritance scheme and takes a Param template parameter.

- SFERES_CLASS_D(Name, Parent): derives the class Name from Parent

- SFERES_PARENT(Name, Parent)::method(): calls method() in the parent class

- stc::exact(this)->method(): calls an abstract method() which is defined in the derived classes

Moreover, sferes2 provides special macros to define specific classes (SFERES_FITNESS, SFERES_INDIV, SFERES_EA, etc.). See ref.

Example:

    SFERES_CLASS(X)
    {
      X() { std::cout<<"X"<<std::endl; }
      void test() { stc::exact(this)->blah(); }
      void test2() { std::cout<<"test2"<<std::endl; }
    };
    // Y derives from X
    SFERES_CLASS_D(Y, X)
    {
      Y() { std::cout<<"Y"<<std::endl; }
      void blah()
      {
        std::cout<<"param::test::x"<<Param::test::x<<std::endl;
        SFERES_PARENT(Y, X)::test2();
      }
    };
    // Z derives from Y
    SFERES_CLASS_D(Z, Y)
    {
      Z() { std::cout<<"Z"<<std::endl; }
      void blah()
      {
        std::cout<<"Z"<<std::endl;
      }
    };
    // sample parameter (see next subsection)
    struct Params_test
    {
      struct test
      {
        static const float x = 0.1f;
      };
    };
    // to instantiate Y and Z (X is abstract):
    int main()
    {
      // Y uses a parameter
      Y<Param_test> y;
      // Z doesn't use any parameter, these two forms are possible:
      Z<> z1;
      Z<Param_test> z2;
      return 0;
    }

##### Parameter handling

The basic parameter format employs static const members in nested structures. For instance:

    struct Param
    {
      struct pop
      {
        static const size_t size = 100;
        static const float x = 0.2f;
      };
    };

However, c++ does not allow to define static const string or arrays in these structures. To solve this problem, sferes2 provides two macros:

- SFERES_ARRAY(type, name, val1, val2, ...)

- SFERES_STRING(name, "string")

    struct Param
    {
      struct test
      {
        SFERES_ARRAY(float, my_array, 1.0f, 2.0f, 3.0f);
        SFERES_STRING(my_string, "hello world");
      };
    };

These parameters can then be retrieved in the source code in the following way:

    ...
    // size
    size_t s = Params::test::my_array_size();
    // 2nd element
    float v = Param::test::my_array(2);
    // string
    const char* c = Params::test::my_string();

#### Mastering boost

Sferes2 heavily relies on boost libraries (see http://www.boost.org). To implement your own algorithms in the sferes2 framework, you should be familiar enough with the following libraries:

- boost::shared_ptr: http://www.boost.org/doc/libs/1_40_0/libs/smart_ptr/shared_ptr.htm

- boost::serialization (useful to define your own genotypes and statistics): http://www.boost.org/doc/libs/1_40_0/libs/serialization/doc/index.html

The following libraries are used in sferes2 but you might never notice them:

- BOOST_FOREACH (to simplify loops): http://www.boost.org/doc/libs/1_40_0/doc/html/foreach.html

- boost::filesystem (to create directories, etc.): http://www.boost.org/doc/libs/1_40_0/libs/filesystem/doc/index.htm

- boost::fusion (list of statistics): http://www.boost.org/doc/libs/1_40_0/libs/fusion/doc/html/index.html

- boost::mpi (MPI evaluator): http://www.boost.org/doc/libs/1_40_0/doc/html/mpi.html

#### UML Diagram

Here is a "conceptual" UML diagram of the sferes2 framework (modules are not represented). It's "conceptual" because some abstract classes are not implemented (because they are useless from an implementation point of view) but should exist in the mind of the user. Moreover, each class is parametrized by the Param class (see previous section), which is not represented on the diagram.

sferes

#### Overview of the call graph

call_graph

On this simplified call graph:

- ea_t : type of your EA (e.g. ea::Nsga2)

- eval_t : type of your evaluator (e.g. eval::Parallel)

- gen_t : type of your phenotype (e.g gen::EvoFloat)

- modifier_t : type of your modifier (e.g. modif::Dummy)

- phen_t : type of your phenotype (e.g. phen::Parameters)

- stat_t : type of your statistics vector (e.g. boost::fusion::vector<stat::BestFit<phen_t> > )

Time flows from left to right (i.e. random_pop() is called before epoch() and so on). Methods with a double bar are called for each individual or each new individual.

### Main classes and files

#### Naming conventions

- Names representing classes must be in mixed case starting with upper case: MyClass

- Variable and method names must be in lower case, using underscores to separate words: my_variable, my_method()

- Names of protected and private members must start with an underscore: _my_private_member, _my_private_method()

- File names must be in lower case, using underscores to separate words. A file which contains a class MyClass should be put in a file my_class.hpp

- File structure mirrors namespace structure. For instance gen::MyClass is in the file gen/my_class.hpp

- Named constants (including enumeration values) must be all uppercase using underscores to separate words

- Getters should have the name of the attribute. For instance, this->_objs should be accessed using this->objs()

- Setters should start with "set_" followed by the name of the attribute. For instance: set_objs(const std::vector<float>& ov)

- The public section should be the first section of a class

- Type names defined using typedefs should end with _t (e.g. iterator_t)

Full example:

    // filename: sferes/fit/my_test.hpp
    namespace sferes
    {
      namespace fit
      {
        class SFERES_CLASS(MyTest)
        {
        public:
          typedef float f_t;
          MyTest() {}
          void my_method() { _my_impl(); }
          float my_attr() const { return _my_attr;}
          float set_my_attr(float a) { _my_attr = v; }
        protected:
          float _my_attr;
          void _my_impl() {}
        };
      }
    }

#### Fitness

- File: [[file:~/svn/sferes2/trunk/sferes/fit/fitness.hpp][sferes/fit/fitness.hpp]]

- Description: Fitness is the base class of fitness functions in sferes2. It has two main attributes:

  - float _value (setter/getter Fitness::value()) -> the value of the fitness for single-objective optimization

  - std::vector<float> _objs (setter/getter Fitness::objs()) -> the value of each objective, for multiobjective optimization

- This class is abstract

##### Defining your own fitness

In most sferes2 experiments, you will have to design your own fitness. A fitness should:

- be defined using SFERES_FITNESS(MyFitness, sferes::fit::Fitness)

- define the eval() method, which takes an individual

- attribute a fitness value in this->_value (single objective optimization) or this->_objs (multiobjective optimization)

    A fitness is a "normal" class and consequently you can
     add other methods or attributes to suit your needs.

Example:

    // single objective fitess
    SFERES_FITNESS(FitnessSingle, sferes::fit::fitness)
    {
      // (optional) constructor
      FitnessSingle()
        {
          this->_value = -1;
        }
      // evaluation
      template<typename Indiv>
        void eval(Indiv& indiv)
      {
        this->_value = -42;
      }
    };

    // two-objective fitness
    SFERES_FITNESS(FitnessMulti, sferes::fit::fitness)
    {
      // constructor
      FitnessMulti()
        {
          this->_objs.resize(2);
        }
      // evaluation
      template<typename Indiv>
        void eval(Indiv& indiv)
      {
        this->_objs[0] = -42;
        this->_objs[1] = 42;
      }
    };

##### View mode

When loading a result file, it is often useful to slightly change the fitness function, for instance to display/log data about the behavior of the individual. This can be easily done in sferes2 using the boolean Fitness::mode() than can takes two values:

    * fit::mode::view when you are viewing an individual via --load
    * fit::mode::eval when you are evaluting individuals (during the
      evolutionary process)

Example:

    // single objective fitess
    SFERES_FITNESS(FitnessSingle, sferes::fit::fitness)
    {
      // evaluation
      template<typename Indiv>
        void eval(Indiv& indiv)
      {
        if (this->mode() == sferes::fit::mode::view)
          {
          std::ofstream ofs("fit.dat");
          ofs<<"this is a log file !"<<std::endl;
          }
        this->_value = -42;
      }
    };

#### Evolutionary algorithms

- Warning* Evolutionary algorithms maximize the fitness (whereas most optimization algorithms minimize the cost function).

##### Shared parameters

All evolutionary algorithms (EA) use the following parameters:

- static const size_t Params::pop::size = population size

- static const float Params::pop::initial_aleat = during the random generation, Params::pop::initial_aleat \times Params::pop::size individuals are generated and only the Params::pop::size best ones are kept

- static const int Params::pop::dump_period: the period between each write of results, examples: - 1: statistics are written at each generation (gen_0, gen_1, gen_2, ...) - -1: statistics are never written (useful for unit tests) - 10: statistics are written each 10 generations (gen_0, gen_10, ...)

##### RankSimple

- File: [[file:~/svn/sferes2/trunk/sferes/ea/rank_simple.hpp][sferes/ea/rank_simple.hpp]]

- Unit test: [[file:~/svn/sferes2/trunk/tests/ea/rank_simple.cpp][tests/ea/rank_simple.cpp]]

- Typical typename::

    typedef sferes::ea::RankSimple<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;

- Description: A rank-based single-objective evolutionary algorithm. Individuals are selected using the following formula:

  - n = k_r \times p

  - \kappa = c^{n + 1} - 1

  - f = n / (\kappa + 1)

  - i = p - f \times log(r \times \kappa + 1)

  - where - c = Params::pop::coeff - k_r = Params::pop::keep_rate - p = Params::pop::size - r = a random number in [0,1[ - i = index of the selected individual

- parameters:

  - static const float Params::pop::coeff = a internal parameter (see the previous formula) ; typical value : 1.1

  - static const float Params::pop::keep_rate = proportion of individuals kept from a generation to the next one ; typical value : 0.2 to 0.6

##### Nsga2

- File: [[../../sferes/ea/nsga2.hpp][sferes/ea/nsga2.hpp]]

- Unit test: [[file:~/svn/sferes2/trunk/tests/ea/nsga2.cpp][tests/ea/nsga2.cpp]]

- Example: [[file:~/svn/sferes2/trunk/examples/ex_nsga2.cpp][examples/ex_nsga2.cpp]]

- Typical typename::

    typedef sferes::ea::Nsga2<phen_t, eval_t, stat_t, modifier_t, Params> ea_t;

- Description: Implementation of the NSGA-II multiobjective evolutionary algorithm. See:

  - Deb, K. (2001). Multi-Objective Optimization Using Evolutionary Algorithms. Wiley.

  - Deb, K., Pratap, A., Agarwal, S. and Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. In IEEE transactions on evolutionary computation, 6:2:182-197

- Parameters: pop_size must be divisible by 4

- Note: NSGA-II can be efficient for single-objective optimization as it corresponds to a classic tournament-based evolutionary algorithm. In sferes2, a 1-sized _objs vector in the fitness can be employed (don't use _value).

##### EpsMOEA

- File: [[../../sferes/ea/eps_moea.hpp][sferes/ea/eps_moea.hpp]]

- Unit test: [[file:~/svn/sferes2/trunk/tests/ea/eps_moea.cpp][tests/ea/eps_moea.cpp]]

- Typical typename::

    typedef sferes::ea::EpsMOEA<phen_t, eval_t, stat_t, modifier_t, Params> ea_t

- Description: Implementation of the \varepsilon-MOEA multiobjective evolutionary algorithm. See:

  - Deb, K., Mohan, M. and Mishra, S. (2005). Evaluating the \varepsilon-domination based multi-objective evolutionary algorithm for a quick computation of pareto-optimal solutions. In Evolutionary Computation, 13:4:501-525.

- Parameters:

  - SFERES_ARRAY(float, eps, 0.0075f, 0.0075f) = values of \varepsilon for each objective. There should be as many values as objectives.

  - SFERES_ARRAY(float, min_fit, 0.0f, 0.0f) = minimum value of fitness for each objective.There should as many values as objectives

  - static const size_t grain = \varepsilon-MOEA is a steady*state algorithm which adds individuals one by one. A basic implementation would prevent the parallelization of individuals' evaluation. The parameter grain allows to generate and evaluate grain indidividuals in paralell at each generation. Typical value is size / 4.

- Notes:

  - \varepsilon-MOEA employs \varepsilon-dominance to sort individuals. This allows users to tune the precision of the pareto front for each objective and often results in better performance than NSGA-II.

  - EpsMOEA should not be used with a modifier (because it uses an archive). You should always use fit::ModifierDummy

##### Defining your own EA

To define your own EA, you should use the macro SFERES_EA(Class, Parent). Mandatory methods:

- random_pop(): fill this->_pop with random individuals

- epoch(): main loop of the algorithm

Defined types (these types will be available in your class):

- Phen: phenotype

- Eval: evaluator

- Stat: statistics vector

- FitModifier: modifier

- Params: parameters

    SFERES_EA(Test, Ea)
    {
     public:
     Test() {}
     void random_pop()
     { /* your code */ }
     void epoch()
     { /* your code */ }
    };

Multiobjective EA must also define a pareto_front() method, see [ParetoFront[ParetoFront]].

#### Statistics

Statistics should be combined in a boost::fusion::vector (see: boost fusion) before being passed to a EA. For instance:

    typedef boost::fusion::vector<stat::BestFit<phen_t, Params>, stat::MeanFit<Params> >  stat_t;

(yes, it is a vector of typenames). You can put in the vector as many statistics types as you desire.

##### BestFit

- File: [[file:~/svn/sferes2/trunk/sferes/stat/best_fit.hpp][sferes/stat/best_fit.hpp]]

- Typical typename:

    typedef boost::fusion::vector<sferes::ea::BestFit<phen_t, Params> > stat_t;

- Description:

  - stores the individual with the highest fitness in _best (getter: BestFit::best() ). This assumes that the population is sorted and consequently this statistics stores the first individual of the population.

  - This class is designed for single objective optimization and consequently works using Fitness::_value (and not Fitness::_objs)

  - writes the value of the best fitness in the file <result_directory>/bestfit.dat ; this file should be easily plotted with gnuplot or R.

##### MeanFit

- File: [[file:~/svn/sferes2/trunk/sferes/stat/mean_fit.hpp][sferes/stat/mean_fit.hpp]]

- Typical typename:

    typedef boost::fusion::vector<sferes::ea::MeanFit<phen_t, Params> > stat_t;

- Description: computes the mean fitness (single objective, based on Fitness::_value). This class is mainly aimed at providing a simple example of statistics.

##### ParetoFront

- File: [[file:~/svn/sferes2/trunk/sferes/stat/pareto_front.hpp][sferes/stat/pareto_front.hpp]]

- Typical typename:

    typedef boost::fusion::vector<sferes::ea::ParetoFront<phen_t, Params> > stat_t;

- Description: stores the set of non-dominated individuals by copying the list returned by ea_t::pareto_front(). ParetoFront does not compute the pareto front; this class assumes that the EA computed it. This implies that multiobjective EA classes should define the following method:

    SFERES_EA(MyEA, Ea)
    {
     public:
      std:vector<Indiv> &pareto_front() const { /* your code */ }
    };

- Note: since this statistic can store many individuals, to load a result file you must give the identifier (the rank in the Pareto front) to your executable. For instance, assuming that ParetoFront in the first (0th) statistics in the stat vector, loading the 10th individual should look like:

    ./my_exp --load gen_100 -s 0 -n 10 -o output

##### Defining your own statistics

    SFERES_STAT(MyStat, Stat)
    {
      // examines the population (via the EA) to update the statistics
      template<typename E> void refresh(const E& ea) { /* your code */ }
      // writes the content of the statistics in the stream; k is the number
      // of the individual in the statistics (e.g. in a Pareto front, the
      // statistics contains many individuals)
      void show(std::ostream& os, size_t k) const { /* your code */ }
      // serializes the statistics using boost::serialization
      template<class Archive> void serialize(Archive & ar, const unsigned int version)
      { /* your code */ }
    }

#### Genotypes

##### BitString

- File: [[file:~/svn/sferes2/trunk/sferes/gen/bit_string.hpp][sferes/gen/bit_string.hpp]]

- Unit test: [[../../tests/gen/bit_string.hpp][sferes/gen/bit_string.hpp]]

- Description: vector of bitstrings which can be used to represent a vector of integer parameters or a vector of discrete parameters. Cross-over is a simple one point cross-over (it always happens). The data can be easily converted to int (in the range 0..2^b, where b is the number of bits for each bitstring) via BitString<>::int_data(i) or to float (in the range 0..1) via BitString<>::data(i)

- Template parameter: the number of bitstrings

- Parameters (struct bit\_string):

  - static const size_t nb_bits -> number of bits for each bitstring

  - static const float mutation_rate -> the mutation rate for a bitstring (the bitstring i is mutated if a random real is belows mutation_rate)

  - static const float mutation_rate_bit -> the mutation rate for each bit. In pseudo-code, here is the mutation procedure:

    foreach(bitstring b in genotype)
      if (rand(0, 1) < Params::bit_string::mutation_rate)
        foreach(bit i in b)
          if (rand(0, 1) < Params::bit_string::mutation_bit)
            flip(i)

- Typical parameters:

    struct Params
    {
      struct bit_string
      {
        static const size_t nb_bits = 8;
        static const float mutate_rate = 0.1f;
        static const float mutate_rate_bit = 0.1f;
      };
    };

- Typical typename:

    typedef BitString<10, Params> gen_t;

##### EvoFloat

- File: [[file:~/svn/sferes2/trunk/sferes/gen/evo_float.hpp][sferes/gen/evo_float.hpp]]

- Unit test: [[file:~/svn/sferes2/trunk/tests/gen/evo_float.cpp][sferes/tests/gen/evo_float.cpp]]

- Description: This genotype is a n-sized vector of real (float) numbers. It is designed for real optimization.

- Three mutation types are available:

  - polynomial: polynomial mutation, as defined in Deb(200)1, p 124

  - gaussian: adds a gaussian noise to the current value; the variance is a user-specified parameter

  - uniform: adds a uniform noise to the current value; the range is a user-specified parameter

- Two cross-over types are available (you can disable cross-over by using a cross_rate equals to 0)

  - recombination:

    - a number k is randomly chosen

    - the first child is made of the [0, k[ values of the first parent and of the [k, n-1[ values of the second parent

    - the second child is made of the [0, k[ values of the second parent and of the [k, n-1] values of the first parent

  - SBX (simulated binary cross-over, cf Deb (2001), p 113). Warning: the current implementation is copied from Deb's code and is slightly different from the original paper.

- Parameters:

  - Any variant:

    - (mutation_t and cross_over_t are defined in the namespace sferes::gen::evo_float)

    - static const mutation_t mutation_type: polynomial, gaussian or uniform;

    - static const cross_over_t cross_over_type -> sbx or recombination;

    - static const float mutation_rate -> the rate of mutation of one member of the vector

    - static const float cross_rate -> the rate of cross-over (if no cross-over is applied, parents are copied to children)

  - polynomial mutation:

    - static const float eta_m -> \eta_m is a parameter which controls the polynomial distribution. See Deb's paper or book.

  - gaussian mutation:

    - static const float sigma -> the standard deviation of the gaussian

  - uniform mutation:

    - static const float max -> maximum change of the current value

  - sbx:

    - static const float eta_c -> see Deb's paper or book.

- Typical typename:

    typedef sferes::gen::EvoFloat<10, Params1> gen_t;

- Typical parameters:

    using namespace sferes::gen::evo_float;

    struct Params1
    {
      struct evo_float
      {
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.1f;
        static const mutation_t mutation_type = polynomial;
        static const cross_over_t cross_over_type = sbx;
        static const float eta_m = 15.0f;
        static const float eta_c = 15.0f;
      };
    };

    struct Params2
    {
      struct evo_float
      {
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.1f;
        static const mutation_t mutation_type = gaussian;
        static const cross_over_t cross_over_type = recombination;
        static const float sigma = 0.3f;
      };
    };

    struct Params3
    {
      struct evo_float
      {
        static const float mutation_rate = 0.1f;
        static const float cross_rate = 0.1f;
        static const mutation_t mutation_type = uniform;
        static const cross_over_t cross_over_type = sbx;
        static const float max = 0.3f;
        static const float eta_c = 15.0f;
      };
    };

##### Defining your own genotype

    #include <sferes/stc/stc.hpp>
    namespace sferes
    {
      namespace gen
      {
        SFERES_CLASS(MyGenotype)
        {
        public:
          // generates a random genotype
          void random()
          {
          // your code
          }
          // mutates an individual
          void mutate()
          {
          // your code to mutate (you have to handle your own mutation_rate)
          }
          // WARNING: the arguments of this method are of the same type
          // than your genotype
          void cross(const MyGenotype& o, MyGenotype& c1, MyGenotype& c2)
          {
          // if you don't know what to do, simply use:
          if (misc::flip_coin())
            {
              c1 = *this;
              c2 = o;
              }
            else
              {
                c2 = *this;
                c1 = o;
              }
          }
        protected:
          // your data here (your genotype)
        };
      }
    }

#### Phenotypes (Individuals)

Phenotypes are the expression of the genotypes: it is often useful to develop a genotype into a structure which is easier to evaluate. For instance, a neural network can be encoded with a compact encoding (genotype, e.g. a modular encoding which can repeat the same module several times), then developped into a standard neural network (phenotype).

Phenotypes are also the bridge between the fitness function and the genotype. In sferes2, a phenotype is the composition of a fit_t and a gen_t.

##### Indiv

- File: [[file:~/svn/sferes2/trunk/sferes/indiv.hpp][sferes/phen/indiv.hpp]]

- Description: The abstract base class for phenotypes.

- Main public methods:

    // returns the fitness
    Fit& fit();
    const Fit& fit() const;
    // returns the genotype
    Gen& gen()  { return _gen; }
    const Gen& gen() const { return _gen; }
    // develops the genotype, called by the evalutar before evaluating
    // the individual
    void develop();
    // you can also override the following methods (although in most
    // cases, you should override their equivalent in the genotype)
    void mutate();
    void random();
    void cross(const boost::shared_ptr<Exact> i2,
             boost::shared_ptr<Exact>& o1,
             boost::shared_ptr<Exact>& o2);
    // this method is not implemented in Indiv but it must be in all the
    // concrete genotypes. It should describe this particular individual
    // in the stream os (see the Parameter source code for an example)
    void show(std::ostream& os) const

- Defined types:

  - Fit -> the fitness functor

  - Gen -> genotype

##### Parameters

- File: [[file:~/svn/sferes2/trunk/sferes/phen/parameters.hpp][sferes/gen/parameters.hpp]]

- Unit test: [[file:~/svn/sferes2/trunk/tests/phen/parameters.cpp][sferes/tests/phen/parameters.cpp]]

- Description: This phenotype transforms a list of parameters in [0, 1[ (typically a gen::Bitstring or a gen::EvoFloat) to a list of parameter in [min, max[. The genotype class must define a method:

    cont std::vector<float>& data() const ();

- Typical typename:

    typedef eval::Parameters<gen_t, fit_t, Params> phen_t;

##### Defining your own phenotype

    #include <sferes/phen/indiv.hpp>
    namespace sferes
    {
      namespace phen
      {
        SFERES_INDIV(MyPhenotype, Indiv)
        {
        public:
          // constructor
          MyPhenotype() { /* your code */ }
          // develop the genotype to whatever you want
          void develop(){ /* your code */ }
          // describe this individual
          void show(std::ostream& os) const { /* your code */ }
        protected:
          // your data
        };
      }
    }

#### Evaluators

Evaluators are in charge of evalutating a vector of individuals (i.e. running phen_t::fit::eval() for each individual).

##### Eval

- File: [[file:~/svn/sferes2/trunk/sferes/eval/eval.hpp][sferes/eval/eval.hpp]]

- A basic evaluator (no multiprocessing, no MPI, etc.), i.e. a basic loop.

- Typical typename:

    typedef eval::Eval<Params> eval_t;

##### Parallel

- File: [[file:~/svn/sferes2/trunk/sferes/eval/parallel.hpp][sferes/eval/parallel.hpp]]

- Description: A SMP/multicore evaluator based on TBB ([[http://www.threadingbuildingblocks.org/]]). Your fitness should be thread-safe!

- Typical typename:

    typedef eval::Parallel<Params> eval_t;

##### Mpi

- File: [[file:sferes/eval/mpi.hpp][sferes/eval/mpi.hpp]]

- Description: A MPI evaluator based on boost::mpi. It works on clusters (we use it on a JS22 cluster with IBM MPI).

- Typical typename:

      typedef eval::Mpi<Params> eval_t;

##### Defining your own evaluator

The only required method is eval. It should call phen_t::develop() then phen_t::fit::eval() for each individual.

    namespace sferes
    {
      namespace eval
      {
        SFERES_EVAL(Eval)
        {
        public:
          template<typename Phen>
            void eval(std::vector<boost::shared_ptr<Phen> >& pop, size_t begin, size_t end)
          {
            for (size_t i = begin; i < end; ++i)
              {
                pop[i]->develop();
                pop[i]->fit().eval(*pop[i]);
              }
          }
        protected:
        };
      }
    }

#### Modifiers

Modifiers are run once all the individuals have been evaluated but before any sorting. They are designed to allow to modify the fitness values to implement niching strategies, diversity preservation mechanisms, etc.

The only predefined modifier available in sferes2 is modif::Dummy, which does nothing.

#### Dummy

- File: [[file:~/svn/sferes2/trunk/sferes/modif/dummy.hpp][sferes/modif/dummy.hpp]]

- Description: basic modifier which does nothing. [endsect] [section Misc] The misc namespace contains useful small functions for:

- system access (e.g getpid())

- random number generation

#### Modules

##### NN

##### Fastsim

##### Python