During UDS we changed the DB a bit to store the applications that were used for the activities. allowing us to bookmark applications, and detemrining most used ones.
Well here is the current Zeitgeist DB (pretty simple and stupid)
Natan has suggested improving the tag table. After some thought i decided to give it a shot on hopefully the rest of the Zeitgeist team agrees with me on this table. Working with INTEGERS instead of STRINGS should improve performance as well as disk space. But take a look first.
Here are the major features the new design would bring with it.
- The timetable should be able to differentiate between event types: Activities (things the user does) Notifications (Other peoples activities (sending you an email) that could trigger your attention and lead to new activities (Replying the mail or sending a tweet)) Maybe we could use the notify-osd to capture notficaitons of interest.
- Giving every registered application its own table to store specific info about a a uri (example position of the item in UNR) or Shell specific rankings.
- Allowing opening data from a timestamp with the Application that was used back then.
Any suggestions now are welcome. We will be developing it during next week to be done on saturday. I will then write a little API Doc for applications to use Zeitgeist. Current functions will not be replaced we will just add new ones.
Do you have a plan to be able to migrate your schema as the application changes? I’ve yet to find an application that didn’t have to adjust it’s sql schema as they add new features and refactor over time. It’s why many still consider using basic b-trees and multiple indexes.
Eager to use it, Cheers
@Christian Hergert
The idea is that the applications will be sending its data
so when the application changes it has to change its plugin to send the data in the format we accept
Obviously you have a plugin contract, but what happens when you find that you need to change your internal storage schema? How are you going to handle upgrading the database running on users desktops to your new schema without deleting all of there existing information?
I guess to clarify, this is where using a serialization format such as JSON or pickle and shoving in a b-tree really helps. Because you can just add a version tag to the object. And then have object migrations that translate from a ver2 to a ver3 object and vice-versa.
@Christian Hergert
agree we will need to have a migration schema! Using JSON to transfer from ver 2 to ver 3 is a good deal
I’m not sure I understand why you are “giving every registered application its own table”. If they all have the same schema as described by your diagram, why not just have a single table with foreign keys to both the “app” and “data” tables?
Also, you might want to consider using an ORM to make access to the database easier. Storm might be a good choice (disclaimer: I am a Storm developer).
A few comments on the schema itself:
1. It is common to just name the primary key of a table just “id”, and foreign keys refering to that table as “$table_id” or just “$table”.
2. Use a consistent naming scheme for tables. I’d suggest the singular form of what rows represent. For linkage tables, use a combination of the two table names. So that’d involve renaming “tag_ids” to “tag”, “tags” to “data_tag” and perhaps “timetable” to “event” and “data” to “uri”.
3. In some cases where you’ve specified “{KEY}” for multiple fields it is clear that you are using a compound primary key (e.g. the “tags” table), but in others you don’t (e.g. for the “data” table you probably don’t want two rows with the same “uri” value but different “uriid”). Perhaps specify “{UNIQUE}” or similar where you just mean that a field value is unique but not part of the primary key?
4. As you seem to be using sqlite for storage, you probably want a simple integer primary key for every table. Sqlite always includes such a key, and will try to name it to match a field in your schema. So add a primary key to “tags” (you don’t want it to treat “tags.tagid” alone as the primary key) and “timetable”.
For what I really understand, you design sounds good to me. A few details though:
- to identify applications, we just agreed yesterday on #gnome-shell that we should use the full path to a .desktop file – rather than the executable. (Jason Smith is going to add put code into gnome-settings-daemon to add a window manager hint on every window that would store that information. But more on that on IRC if you want.)
- to identify documents, I guess you could use a hash in addition to the path. I’m not sure whether indexers do that already, but that allows you not to lose track of a renamed/moved file, which would be silly. Though I’m not a specialist, I let you weight the pros and cons.
- any reason why timestamps are float in your model? To me, there are always integers…
- while giving every app its own table if needed can help integration, I think you should avoid as much as possible to do so. You could just see with UNR what they need, and create the DB for that, so that Zeitgeist itself knows how to deal with this data. And you could also list the cases where external apps would need their own table, and try to give them what they want in your original design.
And I’m particularly happy since my desktop contexts can now be considered as tags, which makes the whole thing cleaner!
@Milan Bouchet-Valat
We already intend to use the .desktop files for applications as it already contains the executable as well as other info.
renamed/moved files will be monitored by a new process i will introduce to zeitgeist that will update the info manually thus i wanted to give the data ID’s instead of using their path.
I think i will use one table for all apps to work out the issues.
We should go through the DB again. Let us talk online