Prev: Generation of novel invariant features (CHIMPs)
Next: Learning what to eat


In social insects there is a clear division of labour between the queen and the workers. Looking at the differentiation of labour in honey bee colonies, Whitfield et al. (2006) found the following from a "genomic dissection":
  • Gene expression is age dependent
  • Variance in expression is already seen at low dimensions (as found in a Principle Component Analysis (PCA))
  • Only the 3rd PC showed differentiation between species (differences between species were smaller than those within species!)
  • There were similar patterns across species
  • The new borns differ the most from the other individuals
  • There is differentiation, but some become foragers sooner than other: TODO dependent?
  • There is no one to one mapping of gene expression, age and behavioural patterns

This illustrates that there are many factors - both interior and exterior - involved with differentiation and division of labour. Thus, even if we can explain things with the TODO mechanims, there is probably a lot of feedback from what is done: reinforcement of the behaviour. In the next example we study the effects of reinforcement on behaviour.


Here we look at a model of bumble bees and their social interactions, made to study the lifecycle of bumble bee colonies. Some remarkable features of this life cycle are:
  • Bumble bees are not truly eusocial, in the sense that the queen can and under some circumstances does still leave the nest to forage herself.
  • Only the queen survives the winter and starts a new nest on her own early in spring.
  • The first brood gives rise to workers.
  • When there are enough workers, the queen stays in the nest and only lays eggs.
  • At the end of the season, the queen lays special eggs that when fertilized produce queens, while unfertilized eggs produce drones (males).
  • However, around the same time the workers rebel and expel or kill the queen.
  • After this rebellion, some workers lay eggs (the so called elite workers).
  • These eggs are unfertilized and produce drones.
  • new queens and drones mate to start the cycle again.

This cycle of course raises many questions, for instance: who are these elite workers which lay eggs? It would seem that if this were a heritable trait, all workers should lay eggs! To study these issues van Honk and Hogeweg (1981) and Hogeweg and Hesper (1983, 1985) studied the social interactions patterns in real (in the lab!) and artificial (see next section) bumble bee nests. In the nest, bumble bees have pair-wise interactions:
  • When 2 bumble bees meet they antennate (make contact via their antennae). After the interaction one goes straight (dominant) and one gives way (subordinate). The queen always goes straight.
  • Hence there is some assesment of other bumble bees through a dominance interaction.

How to study these dominance relations: the interaction matrix of all individuals is only sparsely filled. Moreover, can we average over the whole time period, or do the interactions change over time? van Honk and Hogeweg used a cluster analysis in order to reveal the underlying social structure:
  • They obtained a matrix of interactions relative to how individuals behave to all others, not just to each other (cf human elites which vary to each other, but relative to rest are quite distinct).
  • They considered two bumble bees to be similar if they interact with all other bumble bees in a similar way.
  • They furthermore subdivided their data sets into periods (while the composition of nest stays more or less similar).

Results showed that for 3 time periods interesting patterns were observed (FIG 3):

  1. Early on: the colony is characterized by a dominant queen and a group of workers (top)
  2. Middle: Next to the queen now a group of elite workers has emerged and becomes more accentuated over time.
    • once an individual is in the elite cluster it remains there (consistent over time)
    • these are not just the oldest, so being elite is not age dependent
    • individuals can join the elite quite late in life
  3. Queen killed: a pseudo-queen emerges which together with other elites start to lay eggs
(Note that PCA makes even more perfect groups).

Thus we discovered a social distinction in bumble bees: there is a distinct group of elite workers that cluster with the queen. Furthermore, it is this elite which lay eggs at the end of the season.

A weak point of this study, however, is that these were data of only one (lab) nest. When this experiment was repeated in Germany (van Doorn 1986 REF!), the division between elite and common workers was less clear and less consistent over time. These nests also grew faster and became bigger. Next to repeating this experiment in vivo, we can also use simulations to study how this social structure can arise (see below).

Model: BUMBLEs

Hogeweg and Hepser (1983) formulated a BUMBLE model to study how the social structure in bumble bees can arise. They modelled the interactions between BUMBLEs via the hypothesized DODOM rule:
  • There is a winner-loser effect: once an individual wins an encounter this increases its propensity to win following encounters.
  • This works via a damped positive feedback, which maintains sensitivity to external conditions.
  • Individuals have an internal dominance parameter D. When two BUMBLEs meet, they can observe the value of the other's D.
  • They then have a dominance interaction (Do Dominance) where the probability to win: p_win = D_me / (D_me + D_you)
  • If you win the encounter (RANDOMNUMBER < p_win) then K =1 else K = 0
  • Your update dominance parameter is then: D_me' = D_me + a(K - D_me / (D_me + D_you) )
  • Thus the update depends on the difference in dominance (cf chess ratings): you get a high boost from an unlikely win, while only a small boost if you were likely to win. Hence: a damped positive feedback.

The other assumptions of the model were as follows:
  • The population dynamics of bumble bees was taken as a given.
  • All workers were created equal with no propensity to become elite.
  • The TODO was as follows: feeding, foraging, egg-laying on nest
  • The environment was given by the nest. The NEST had a center and a periphery, including a food pot. (Note that there was no real space in the model, but only pseudo-space through these compartments, Fig 1)
  • The model was event-based and actions take time.
  • Social interaction were basically random via DODOM (queen versus workers, and workers versus workers)
    • the Queen initially gets D = 7.5
    • workers initially get D = 1
  • The D parameter determines the location and activity of the BUMBLE (periphery vs center)
  • The queen is killed when she looses a number of dominance interactions in a row
  • The model was then observed in the same way as the real nests.

This model gave rise to the following results:
  • Elite workers arise in the model, and they are relatively constant in time.
  • All workers try to lay eggs.
  • The queen is expelled at a certain moment, and the timing of this event is the same in all replica simulations.
Hence, our first conclusion should be that a heritable predisposition to become an elite worker is not necessary to explain the social structure in bumble bees. Rather, we observe emergent division of labour.

As suggested from the results of the in vivo nests in Germany, the consistency of the elites seems to be dependent on group (nest) size. In the model, we can study nests with different size by varying the egg laying speed.
  • For differently sized nests the queen is still killed on about the same day!
  • Slow small nests have more differentiated D's (and thus a clearer division between elites and normal workers), fast nest less.
  • As a consequence the criterion for killing the queen is reached at same time (in small nests there are less, but more dominant, elite workers).
  • This result depends on how workers develop dominance.
  • This self-stabilizes on nest of certain age.

Next to explaining why the results were less clear in the bigger nests in Germany (van Doorn 1986 REF), this also gives us insight into another question: How does a bumble bee nests tune itself relative to the season? In terms of producing generative offspring (queens and drones) it shouldn't be too soon or too late. Do they use an external cue for this? Probably not, because the same process occurs in artificial labs without cues. How then to tune the timing of killing the queen? Even queen age is unlikely to be used as cue because old queens can still be successfully introduced into a young nest. The results of the BUMBLE model suggest that there could be a socially regulated clock. Nests that grow slowly switch (kill queen) soon in the model and in vivo! This also explains the bumble bee nests in germany, where there is fast growth, less social differentiation due to influx of new individuals which disrupt the system and keeps it out of balance.

DODOM, TODO and side-effects

What we observe in the BUMBLE model is social differentiation. This is dependent on nest structure and growth rate (without the periphery for inactive bees to settle the results are not obtained). Thus we obtain elite workers that lay eggs, but these properties are not heritable: all start equally. Thus the problem of the elites and their reproduction is a pseudo-problem. Instead we see that they play an integral part in a socially regulated clock, which is needed for the life-cycle of bumble bee colonies.

Finally, we note that there were some extra observations in the BUMBLE model in terms of adaptability, namely compensatory feeding. This was observed in vivo nests where if workers were taken away, then the remaining workers would spend more time feeding. In the model this happens simply through TODO, because there is more feeding to do! This is an example of automatic adaptation.

Next: Learning what to eat

**Whitfield et al.** (2006) Genomic dissection of behavioral maturation in the honey bee. PNAS vol. 103 no. 44 16068-16075
Van de Honk, C. and P. Hogeweg (1981) The ontogeny of the social structure in an captive Bombus terrestris colony. Behav. Ecol. and Sociobiol. 9, 111-119.
Hogeweg, P. and B. Hesper (1983) The ontogeny of the interaction structure in BumbleBee colonies: a MIRROR model. Behav. Ecol. Sociobiol. 12, 271-283
Hogeweg, P. and B. Hesper (1985) Socioinformatic processes, a MIRROR modelling methodology. J. Theor. Biol. 113, 311-330.

DoDom: Do dominance interaction

DoDom is a damped self-reinforcing function which models social dominance interactions between individuals.
The functions has two forms:
1) Without the mental DoDom and 2) with the mental DoDom.

DoDom allows differentiation of equally dominant individuals through the winner-loser effect which has been observed in many animals. This happens through positive reinforcement of initial stochastic events. An initial winner is likely to become dominant, and thus reduces the probability of losing subsequent interactions, although that can happen.
In this way social structure emerges from repeated interactions between individuals through positive reinforcement (akin to bootstrapping). I.e. generating divergence between individuals.

Interestingly such emergent structure feedsback on other behaviour such as spatial position in the groups (centrality of dominants).

Work on bees social hierarchy

COURSE NOTES 2006-2007

Differentiation of labour in BEE colonies: Whitfield et al PNAS 2006: (Genomic dissection)
- age dependent gene expression
- PCA: variance in expression already at low dimensions
- different species do same things!
- new born most different
- differentiation, but some who become foragers sooner than others, i.e. TODO dependent?
- not one to one mapping gene expression and age and behaviour patterns
- even if we can explain with TODO: there is probably a lot of feedback from what is done: REINFORCEMENT
(- only 3rd PC is separation between species: i.e. differences between species smaller than within species.)

Here we study the effects of REINFORCEMENT of behaviour

- winner-loser effect: once won propensity to win increases
- damped positive feedback: maintains 'sensitivity' to external conditions
- internal variable D: when two BUMBLES meet observe value of other's D
- dominance interaction: Pwin

Dm / (Dm + Dy) if RAND <

Pwin K=1 elso K=0
- D'm = Dm + a( K - Dm / (Dm + Dy))
- boost depends on dominance difference (cf chess ratings): damped positive feedback
(Note: if add up all updates they average out to zero if don't actually update)

About bumble bees:
- not truly eusocial (i.e. Queen bee never out foraging)
- Only Queen survives winter: starts nest on her own early spring from scratch
- first brood gives workers
- when enough workers Queen stays in nest and only lays eggs
- at end of season workers rebel and kill Queen and some lay eggs (elite workers)
- these are non-fertilized and produce males
- also: the last eggs the Queen lays: fertilized lead to Queens, while unfertilized lead to Drones (males)
- new Queens and Drones (male) mate to start the cycle all over

Q: So who are the ELITE workers that lay eggs?
- it would seem that if this were inheritable, all workers should lay eggs!

van Honk + Hogeweg (1981, 83, 85): when bumbles meet they antennate, one goes straight and Queen goes straight.
- some assesment of other bumble: DOMINANCE INTERACTION
How to study dominance relation?
- sparse filled interaction matrix
- however, is it good to average over the whole time period?
Therefore: used cluster analysis in order to know social structure
- matrix of interactions: relative to how they behave to ALL others, not just to each other (cf human elites which vary to each other, but relative to rest are quite distinct)
- and subdivided data sets into periods (composition nest stays more or less similar)

Results In time (FIGS):
1) Queen + workers
2) Queen + elite + workers: get more accentuated over time
- once someone in cluster close to Queen remains in that cluster
- not just oldest; they are in both groups
- can join elites quite late in life
3) when Queen killed: pseudo-Queen and other elites lay eggs

(PCA analyses makes even more perfect grouping!)

- discovered social distinction in bumbles
- + elites lay eggs at end of season
- weak point: just one nest

- can look at more nests -or-
- simulation study to understand how this can happen.

(Note: In Germany nest were bigger: more messy results, more back and forth of dominance, elite less good predictor of egg laying!)

MODEL: Hogeweg + Hesper: DODOM + TODO: BUMBLES
- population dynamics of BUMBLE: lets mimic that and take that as given
- all workers are CREATED EQUAL: no propopensity to become ELITE
- TODO: feed, forage, egg-laying on nest
- NEST: center: periphery, pot food (Note: not real space, pseudo)
- Event-based: actions take time
- Random social interactions: DODOM (Q-worker, worker-worker, Q gets D

7.5, worker get D

- then observe as real nests
- maintenance parameters: constraint for 'realness'
- STEPDOM: increment of dominance update
- kill Queen when she looses number of dominance interactions in a row

- ELITES arise in model: and they are relatively constant in time
- everyone tries to lay egg, later others also aly eggs

BUT: in vivo studies have shown that ELITES are dependent on group size
So: vary nest size: EGG LAYING SPEED
- in slower nests: Queen killed on about same day! So it is not just numbers ...
- slow nest: more differentiated D's
- fast nest: less differentiated
- therefore criterium for killing queens happens at the same time
- depends on how workers develop dominance
- self-stabilizes on nest of certain age

Q: how does bumble bee nest tune itself relative to season?
- not too late / early : costs in terms of offspring
- external cue? could be but also happens in lab without cue!
- how to tune switch of killing Queen?
- Queen age: not the cue because old Queens can be successfully lead a new nest
- Model suggests: socially regulated clock

(Note that nests that don't grow switch too soon, just like in vivo!)

The German setting therefore: fast growth, little social differentiation .....
- the influx of young animals:
- not differentiated, new bees disrupt this, system kept out of balance


social differentiation (nb nest structure + growth rate: doesn't work without periphery for inactive bees to settle)
- elite lay eggs + non-inherited: all equal start, problem eilte is PSEUDO-PROBLEM
- socially regulated clock
extra observations:
- adaptability: compensatory feeding; if take adults away: the remain spend more time feeding, but this is simply TODO increase! automatic adaptation.