Ticket #11 (closed feature: implemented)

Opened 5 years ago

Last modified 4 years ago

Memory Management

Reported by: tim Owned by: tim
Priority: major Milestone: 0.3
Component: Fawkes Version:
Keywords: memory management, blackboard Cc:
Git Branch:

Description

We need some kind of memory management to share information between threads. We want this to be a generalized system and not Plugin-individual data. We are going to re-use a few ideas from the old BlackBoard (OBB) but re-implement it from scratch. The new component will be called Fawkes BlackBoard (BB or FBB)

Requirements


The new BlackBoard has to :

  • provide storage for data
  • guard access to storage with locks
  • allow access to storage via defined interfaces
  • be as lightweight as possible
  • have minimum locking times, lock only per interface, use read/write locks
  • not use individual referenced values but chunks of memory per interface (C struct)

The data stored in the FBB has to be versioned. This means that each interface carries a version number (s. logging below)

Optional features


Optional features are:

  • Guard acccess to have only a single writer per interface
  • event messages if an interface has been changed (not sure if that is needed with current pipeline design, may come later)

Logging


There has to be a logging facility to record each change in the BB. We do not want a log writer process that comes every n msec and stores the data which is in the BB at that time. This could possibly loose data. Rather we want the data to be written to the log if it was just written via the interface. By putting the logging facility in the BB we can guarantee that no data is missed.

All data written must of course carry precise times, especially precise time offsets to the data written before so that the data is restored at the right time when it really was written to the BB (of course this cannot be guaranteed as readers may have aquired the lock at the exact time when the log data has to be restored, but we should be as close as possible with writer preference in the read/write lock).

The written data has to be versioned (and thus the interfaces have to be versioned). If the interface is changed later on the old log must still be usable by the log player. To reduce the amount of code interfaces have to be designed carefully such that these changes are not needed later on. An alternative might be to have one log-converter that converts log from any format to the current format (better option, faster during runtime).

The data is written byte-efficient, not textual representation is written. This is done for several reasons:

  • It is faster in writing, writing just the C-struct is much faster than creating and writing a textual representation (which is likely to eat more bytes)
  • It is faster in reading. The struct can just be copied from disk to local memory without any parsing from the file.
  • The files are likely to be smaller.

One of the major shortcomings here is the dependency of the endianess of the underlying system. A file created on a big-endian system will most likely not be readable on a little-endian system. Since we only use machines of the same endianess (robots and developer machines are all i386 PCs) this does not matter to us at the moment. If the times shows that this is needed a converter can easily be written.

Change History

Changed 5 years ago by tim

  • owner changed from AllemaniACs to tim
  • status changed from new to assigned

Changed 5 years ago by tim

Basic new BlackBoard has been implemented. Interfaces can be filled with data and messages can be sent. Events and logging still missing.

Changed 4 years ago by tim

  • status changed from assigned to closed
  • resolution set to implemented
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Subscriber, Participant

  • Fawkes Trac List(Always)
  • Tim Niemueller(Reporter, Owner, Participant)