While doing some scaling optimization the Zeitgeist team decided to give Berkeley DB a try. But porting from SQLite to Berkeley is not easy. However thanks to the Berkeley SQL API porting one can use the SQLite queries. All we did was change the Makefile and add a new vapi (which is a straight copy of the sqlite3.vapi but with dbsql.h as a referenced header instead of sqlite3.h).
The results are not good though. But maybe its because we need to optimize the queries a bit more. Nevertheless you will find the code on freedesktop under the branchname berkeley tonight.
If you like a challenge, try to optimize the Berkeley optimization yourself, but be aware this is not for beginners and it may break you DB.
To build it make sure you get Berkeley db (on Debian and Ubuntu):
- sudo apt-get install libdb-sql-dev
- git clone git://anongit.freedesktop.org/zeitgeist/zeitgeist
- git checkout -b “testing-berkeley”
- git am –signoff < /path/to/patch
Download the following vapi and place it into:
- cp /path/to/vapi /usr/share/vala/vapi/dbsql.vapi
- sh autogen.sh –prefix=/usr/ –disable-fts && make
Now run it and make sure you are NOT using the default datapath:
- ZEITGEIST_DATA_PATH=/home/seif/.local/share/zeitgeist-bdb/ src/zeitgeist-daemon –replace –log-level=debug –no-datahub
you will find a generate_events.py script in the zeitgeist tools folder as well as other benchmarking scripts. Feel free to poke around.





[...] source: Go to Source Feed source: http://planet.debian.org/deriv/rss20.xml License: The original licenses are retained [...]
The SQL stuff is just an afterthought in Berkeley DB, basically an sqlite3 storage backend using berkeley DB. To get real performance, you obviously have to work with Berkeley DB itself, not the extra sqlite layer.
Awesome work Seif! (and you’re a real git hacker now – what’s up with that!?!+11 ;-P)
@Julian: I think Seif’s work on this is mostly a spike to map out strategies if we want to bring Zeitgeist to be able to manage millions of events in one db. Taking the SQLite wrapper is just a first step to test the waters. Fully utilizing the key/value store directly will be a truckload of work, but not impossible.
every blog post featuring graphs in this style has made me cringe….
[...] Seif Lotfy: Zeitgeist: Playing with Berkeley DB (seilo.geekyogre.com) [...]