Archive for August, 2009

A better sample of connect the dots! (Zeitgeist data relevancy monitor)

August 28th, 2009

After my last blog post here is a more real life explained sample of data relevancy over context and usage provided by the Zeitgeist Framework:

First some explanations:

  • The Cricles represent URIs such as “http://youtube.com” or “/home/tom/test.doc”. The number you see are ids for these URIs
  • The Arrows represent a focus switch from a URI to another (we now can log window activity). One can read it ( 0 –> 1 ) as “focus from URI 0 to 1″ in other words focus from “http://youtube.com” to “/home/tom/test.doc”. The numberon the arrows represent how many times the focus was switched form one to another.

So now for a real life sample of what was happening today around “http://git.codethink.co.uk/” ( To be clear this is an actual graph of what happened directly around the URI)

First he URI to id list:

  • 0 = “http://git.codethink.co.uk/”
  • 1 = “http://google.com”
  • 2 = “pidgin://Facebook/Karl Latimer”
  • 3 = “pidgin://Facebook/Philip Van Hoof”
  • 4 = “pidgin://Co-Workers/juergbi”
  • 5 = “pidgin://Co-Workers/einalex”
  • 6 = “tomboy://Today: Friday, August 28 2009″

codethink

For a wider neighbourhood here is some more sweetness however I won’t write down the URI list but the previous ones still applies:

codethink2

Connect the dots! (Unleashing some Zeitgeist Framework sweetness)

August 28th, 2009

There are 3 types of relationships between items:

  1. Content: Lets say two documents have the same content “Hello World” thus there is a probabilty that they are about the same topic. This could be done using indexers and keywords extractors like Tracker Indexer.
  2. Metadata: Tags, mimetypes, etc… are all data that describe the uniqueness of an URI. Data sharing the same tags or “Artist name” or even exist in the same folder are related over these attributes. This area is very well covered by Tracker as well as Organise FW’s “Path Projection”.
  3. Context: Well there are some things where content is hard to compare such as Websites and Videos. One can extract the metadata, but what if they show no relationships? However the context of the usage they were used in is somehow related. Let’s assume I visited Will Smith’s profile on IMDB (y)  and somehow then started watching Prince of Bel Air (x). There are a lot of instances where I also watched PoBA(x) too such as work while editing work.py (z). Now to be able to determine which of my activities lead to watching PoBA(x) I need something that can look at PoBA as a single node and look for URI activities that lead to using it as well as URI activities that were initiated by the usage of PoBA. The intensity of the relationship can be determined with the number of times these activities were undertaken in any given time period. This set of nodes(URIs) around PoBA we called activity neighborhoods.

The rest of this post will cover point 3.

BTW the following images are actually plotted by a Zeitgeist framework extension developed by Alexander Gabriel and me in an attempt to implement related items for Gnome Shell and Gnome Zeitgeist.

Activity neighborhoods can be of any degree. By degree I mean the sequence of nodes(URIs) that could possibly lead to the acitviy(x). A relationship of N0(x)  can be seen as the set of items that directly lead to the usage x as well as URIs acitvites initiated by the usage of x.

10:01 edited y

10:04 opened x

10:05 watched z

10:06 edited x

The neighbourhood N0(x) = { -: [y,z], +:[z] }. This shows that z was used once before and after x and y was only used before x making z more relevant to x for now.

“-” stands for incoming and “+” stands for outgoing

N0(z) = { -: [x], +:[x]} we cat see the relationship over 0 nodes.  -N0(z) are the incoming URIs and +N0(z) are the outgoing URIs.

but what if we say N2(z) ??? With this we want all items that

  • either directly lead to or the items that lead to the items that lead to z
  • the items were initiated by z or items that initiated to the usage of the items that lead to the usage of z

It’s very easy

N1(z) = N0(z) + ( N0(e)  ) where e is an element in N1(z)

=====>          N1(z) = { -:[-N0(z) + y] , +:[N0(z)]}

=====>          N1(z) = { -:[x, y] , +:[x]}

So enough theory lets connect the dots. The following images are plot of trees and graphs mapping my current Zeitgeist DB.

The nodes in the plots represent URIs and the arrows represent absolute events on URIs before or following the node attached to.

So first lets demo No(“Bad Boys”), here bad boys is presented by “0″:

badboysD1

This shows us that items 1-10 followed the usage of 0 while 11-12 were followed by 0. However 2,7,8 and 10 were also again used before 0.

Now for N1(“Bad Boys”):

badboysD2

You can see here that everything leads over maximum 1 Node to “0″. And only 12 of them lead directly to 0.

The numbers in the plots don’t really match since the traversing differs from N0 to N1, however if you take a good look the amount of directly incoming and outgoing arrows to/from “0″ is still the same! 10 outgoing and 12 incoming!

Currently the arrows have a low weight because the history of my current netbook is less than a week plus we only cover open/save events. Tomorrow we will implement window focus events to make things more dynamic. Once we can set weights of the arrows with the duration of the event to start or the lifetime of the items we can filter and prioritize the related nodes.

Right now all items in a small cycle with high weightend arrows are considered definite relationships!

Now for a more bad ass plot of my todays activities :)

graph




GNOME Zeitgeist almost there

August 17th, 2009

Now that the 0.2.1 version of the engine has been released I took the liberty to rewrite the whole UI from scratch in 4 days. Because of my f*cked up pc I enhanced performance quite a bit on the account of some memory around 20 MB (Natan will be doing his magic then) for the UI. And these are some early pics of it.

Here you have the day view:

Screenshot-zeitgeist-journal

Here you have the standard and default 3 day view:

Screenshot-zeitgeist-journal-1

and there is the week view:

Screenshot-zeitgeist-journal-2

there is a month view but you wont see any difference to the week view.

We are waiting for the OK from Kalle Persson and some testing: We are missing the bookmarks view, searching as well as browsing tags and editing them. Once done then we are almost there. I think 3 days is more than enough!

You can check it out at bzr branch lp:~gnome-zeitgeist/gnome-zeitgeist/new-interface

Exporting Zeitgeist events of interests into calendar applications

August 11th, 2009

So to cheer myself up a bit i decided to hack on a nice application that exports events from Zeitgeist to ics files that can be imported by every calendar application!

The basic idea is to reuse current calendar applications such as Evolution or Google Calendar to view activities that belong to a certain topic! Next is line is making evolution update and import the ics files automatically on changes!

Now for the pictures:

EVOLUTION:

Screenshot-Calendars - Evolution

Screenshot-Calendars - Evolution-1

Screenshot-Calendars - Evolution-2

GOOGLE CALENDAR:

Screenshot-Google Calendar - Chromium

Screenshot-Google Calendar - Chromium-1

Hello NEPOMUK !!!

August 11th, 2009

Having so much to blog about (being unemployed and over enthusiastic), I think this specific decision is worth sharing with the world!

Today after some discussion with the rest of the team we pulled it through! Zeitgeist is using the NEPOMUK namespace to support as well as profit from more cross desktop work!

I started the implementation today and did the first commit! There are some talks about creating a migration script! But I think its not necessary if we keep digging out history!

In combination with our new flexible backend module we can easily use tracker storage now!

These features will be part of our 0.2.1 this week or 0.3 release by the end of the month!

GNOME say “Hello NEPOMUK” !!!

Parental Control for GNOME 3.0

August 10th, 2009

So if everything goes according to plan Zeitgeist engine could be included as an optional module with the GNOME 2.28 release to enhance the Shell experience thanks to Siegfried Gevatter(RainCT).

If it is so then I will also rework some of the Parental Control concepts I presented during GCDS! I see it as a very good marketing point (not for hackers).

The concept is simple: Parental Control subscribes to events from Zeitgeist. Since we deliver the source (Application) as well as the subject (Document, Website, Video, etc…), we trigger some keyword extractor as well as metadata extractor on the subject looking for specific strings that we have in a blacklist. If found in the blacklist we freeze source application and ask for password! If password is wrong then kill the application or do whatever (we can set a dialog box or scripts that could be executed). Tracker could be used in this application!

The application should be simple and not really fancy schmancy as on UI wise… If you like the idea please contact the Zeitgeist team so we can work on it!

GNOME Shell: Zeitgeist integration

August 7th, 2009

I think the work of my GSoC student deserves a post on its own! Not only did he work it off all by himself. I barely gave him adivse maybe once or twice directions! He was responsible for the whole engine seperation and now the Shell integration. Watch this and that. And if possbile write a”Thank you Siegfried” comment!

Zeitgeist Project related stuff!

August 7th, 2009

After a long break the team is somehow back on track! After going through Jono’s “Art of Community” (which I recommend everyone involved in open source to read), I realized i was burned out. When I wanted to start hacking again my Laptop let me down and broke on me. Thanks to the guys at Novell I should be getting a new one soon. Now for the more important part here are some points being tackled:

  1. After Mikkel’s blog post, we are constructing the 0.3 release. Things were not so pesimistic we just needed to set some goals straight and at last we reached an excellent ontology for event representation (I will blog about that later).  If 0.3 works out well we will propose our ontology to tracker and Nepomuk thus taking away the storage pain and pushing for more cross desktop development. Now we need to finish some bugs and 3 blueprints :)
  2. Concerning storage Markus Korn implemented a flexible backend module that made it into trunk that will allow the user to choose which kind of database he/she wants to use for the storage.
  3. Hopefully I can introduce the idea of “expiring tags” in this release, thus automatically setting temporary associations between files, e.g: Imagine using google.com while working on Project A. This way google.com gets tags from Project A. Once you stop using them together for a little while the tags that were mapped form Project A to google.com will be deleted
  4. GNOME Zeitgeist (the journal UI) should be released soon too (forget what you see in trunk the performance and layout will be much better and detailed, just need a new lappy to work on it).
  5. Siegfried my enslaved GSoC student did some kick ass work with the Shell integration. Watch this and that.

Other interesting things happening in the zeitgeist universe would be:

  1. According to Markus Zeitgeist FS should be released soon.
  2. We have a little prototype of Feedgeist where we ignore the whole files and folders system but rather introduce a new way of dynamic file managment. We use topics where topics have criterias for files to be associated with it. For this we will be using Zeitgeist as well as Tracker. As in “GCDS” topic takes everything tagged or containing guadec, gcds, akademy, etc… and relates it to the topic. So topics are dynamic and its content increase and decrease dynamically. Natan Yellin will be leading the development of this project.
  3. I kinda gave some people more cookie monster aka Teamgeist (A Collabora + Tracker + Zeitgeist hack) peaks and was happy with the positive feedback, especially Jorge Castro but I think it should stay censored :P . Also I will be meeting up with Frank Karlitschek from KDE sometime within the next 2 weeks to discuss a possible  Teamgeist cooperation or integration with KDE’s social desktop. After the 0.3 release it should get a lot of attention.
  4. Alex Gabriel will be working on a little extension for zeitgeist that exports events into a ical file so it coudl be easily viewd within any calendar application such as google’s or evolution.

So its bedtime got me, cheers and good night!

Everything broke!!!

August 3rd, 2009

1. Laptop: My “F’ed up piece of SH*T” of a laptop (DELL XPS 1330) constantly overheated and had the nvidia malfunction. During GCDS my notebook stopped charging and was on low performance all the time + hot as hell! My brothers netbook benchmarks 2x faster than my shitty laptop! And to top it off Eclipse + firefox + pidgin + Do +banshee makes it freeze then crash! WARRANTY EXPIRED

2. Desktop: While plugging in an external HDD it just turned off and wont turn on again! :(

3. iPhone:….. The screen is constantly white with some stripes! turned it off and on and even restored its setting but can see crap! While writing here the whole screen turned red!

4. TV: won’t work! won’t turn on!

I am feeling pretty naked at the moment! Pretty crappy situation! Right now I am using my brothers netbook! Thinking of getting one myself!