So @Zeitgeist we took the liberty of experimenting of monitoring the document(window) focus of the user.

This is something I have been working on for  long long time. To cut it short. When monitoring the focus of items (items being websites, documents, notes, mostly anything), with time one can detect patters of document activities. We have some kind of directed adjacency matrix. If you are familiar with graph theories this is how it is calculated.  Every document is a node while every switch of focus becomes a vertex. The vertex holds the timestamp (possible also the GPS location) of the focus switch from a document to another. This matrix will be an essential part of the 0.3 version of the engine.

The benefits of this feature are:

1. Calculating “Focus life” of a document/application and tags: Think of instead of most used as in (how many times opened) you can now as for most focused (how long was it in the main focus). This feature could be used to sort search results for Tracker and other stuff. The focus lifetime of a document will be calculated by taking the absolute value of  “the timestamps of incoming vertices and subtracting them from timestamp of the outgoing”.
2. Determining Contextual relevancies of Documents: So there are somethings that a users usually uses together in context over time. Let us say every time I work on tutorial.c I switch focus between it and http://library.gnome.org/devel/gtk-tutorial/2.17/. Over time some habits can be detected and connections between URIs can be set. Right now I can ask “get me all documents that i switched from or to while working on X” this results into a little graph. We can also ask “get me all documents that i  indirectly switched from or to (over max. n nodes) while working on X” which will allow me to widen the radius of activities around a documents thus resulting into more potentially related files. This feature might be also used for GNOME Do. Another idea is to revive the old Dashboard.

Currently a vertex with one time focus switch between documents is ignored for filtering issues. Later we will use the lifetime of the documents to calculate if the documents contextual relationship could be considered spam or not. This can be relaized in a scenario where while working on my paper i changed the music. The music in now contextually related to the paper, however we can filter it out if we notice that there was a quick switch of focus back to the document. making the focus on the music only 1 second or 2. There will be other means of filtering the contextual relevancies using meta-data provided by Tracker.

The prototype is done and now needs to be cleaned up for the integration into the framework. Rob Taylor from Codethink encouraged us to build a UI that I am using to display the results. To test this feature I deleted all my history from every applications to benchmark how the little hack will learn. Here are some results of URIs that are contextually related over 1 week.

(+ is the currently focused document and  – are the most used switched from/to documents)

—
+ http://seilo.geekyogre.com
- http://seilo.geekyogre.com/wp-admin/post.php
- http://seilo.geekyogre.com/wp-admin/
- http://mail.google.com
—
+ http://facebook.com
- https://login.facebook.com/login.php?login_attempt1
- http://www.facebook.com/home.php?
- http://seilo.geekyogre.com
—
+ http://twitter.com
- http://twitter.com/direct_messages/sent
- http://twitter.com/#sent
- http://twitter.com/seiflotfy
—
+ https://blueprints.edge.launchpad.net/zeitgeist
- https://blueprints.edge.launchpad.net/~zeitgeist
- https://blueprints.edge.launchpad.net/zeitgeist/+spec/set-filter-results
- https://blueprints.launchpad.net/zeitgeist
—
+ http://www.youtube.com
- http://www.youtube.com/seiflotfy

For a little display here is the graph generated for the first example. 0 being http://seilo.geekyogre.com and the nodes form/to the 3 vertices with the highest weights pointing to from it are displayed in the results.

So we are experimenting a lot and I would like to know you opinion on this research. So keep in mind that this is jut 1 week history.

I moved my blog from here to there.

If you don’t know yet we are got a hackfest thanks too GNOME, TIS and Canonical. From the 9th to the 12th of November Zeitgeist and GNOME Activity Journal developers will meet up for 4 days of pure hacking and planning in Bolzano.

Right now from the engine side alot of planning has happened and topics of the hackfest have been 100% set (by vote).

• Re-factor DB and start using Tracker for Item and Annotations storage (not events), thanks to Codethink for allowing some of his employees to help us with this topic.
• Try to deploy events in a CouchDB (this could be helpful for generating temporary filtered databases to query)
• Get in contact with the people of the sugar project to learn about their experiences and discuss a possible engine deployment on their platform. ( We set a date for that already
• Discussing a roadmap for Zeitgeist (this will be relevant to the GNOME community)

I am full confident that these points will be fully developed into a good testing state. Allowing us to just fix bugs afterwards.

To make this possible we plan to finish blueprints discussed by the team before the hackfest. Also I encourage engine all developers to hack a dataprovider for their favorite application.

The UI and deployment side tracks will be set by Federico and Thorsten, since I need to devote a lot of time to the engine side and my UI skills are pretty limited (just a quick Gtk hacker not much)

A current problem I fear is coordinating minimum 14 attendees. My first suggestion was to split the developers into groups according to their area contribution, UI and engine. However it is not as easy as it seems, since some developers work on both engine and UI. I might lean to how UDS organized its sprints. Thus in the moring we discuss the tracks. And after that we split and hack on the topics.

Any suggestions?

One thing that makes FLOSS special is being able to credit people for their work. I would like to do so concerning Zeitgeist and GNOME Activity Journal (known as GNOME Zeitgeist).

A lot of people usually see me as the founder of the project. This is not all so true. I worked on a similar project at Fraunhofer under the supervision of Thorsten Prante. Back then I also worked with Alex Gabriel on Alex Graveley’s Gimmie from which we forked Mayanna. When I stumbled upon Federico’s GUADEC presentation about a journal I decided to give it a quick hack on the 10th of October during the boston hackfest based on the old Gimmie/Mayanna code (Mayanna at that point being now a whole different project with other goals). When Marina managed to convince me to show it to Federico I went for it and then he blogged about it. This gave me a big boost and motivated me to work it out more. I contacted Thorsten since he was the person I knew who had dealt with a journal issue for several years and planned out my graduation thesis. One day later Natan Yellin contacted me and pushed the code into launchpad for everyone to look at. Now Zeitgeist was accessible to a big audience. I proposed the name GNOME Zeitgeist for the project, and Natan and I started hacking like idiots using the prototype from Boston. We set up an IRC channel an soon enough Dave Richards as well as Ketil W. Aanson settled in to set the first requirements for there usage.

Federico made me get in contact with Sebastian Faubel from the Organise FW Project with whom I am working now on some secret stuff

Some time in March I got a little cute mail from “Siegfried Gevatter” who wanted to use Zeitgeist engine (back then we didn’t have the process split only code split) for his GSoC. Before it even started he had wrote a whole DBus interface for it. GNOME Activity Journal remained being hacked by me and Natan, while Siegfried was tampering around making the engine more stable and efficient. We decided to rewrite the engine before UDS. While I maintained contact with Alex Gabriel sharing some ideas, for example how to find contextual relevancy, during that time over the phone because of his work on Mayanna. We attempted a first merge between the two projects that failed for organizational reasons.

Now at UDS is where the magic happened. I met Siegfried and we started walking along the halls trying to promote our work. Meeting up with  the GNOME Do team (Jason Smith, Alex Launi and David Siegel) ideas came floating in for the UI and especially the engine (back then i had a crazy way to calculate the contextual relevancies which i will put in again). Siegfried and I got in contact with many other individuals who supported our ideas and suggested new features like Neil J. Patel, David Barth, Rick Spencer, Jorge Castro, Jono Bacon, Mark Shuttleworth, Ken VanDine, Martin Owens and the guys form the UX team. By the end of UDS the independent pre-Zeitgeist engine was born.

To meet the requirements of those specs we wanted to redesign the DB.

Then came the 2 Developers who actually in my opinion made Zeitgeist a real project. Mikkel Kamstrup and Markus Korn. These two developers took Zeitgeist engine to a whole new plateau. This made a team of 5 developers busting their @sses. Now Mikkel started redrafting the whole database while Markus started fixing and hacking code in almost every part of the Engine. The base and foundation of Zeitgeist engine of how it is now and the future was then set. This is for me when Zeitgeist was actually founded. Right before GCDS we had the whole engine code rewritten form scratch (That is when I was ripped off the liberty of hacking into trunk directly after some big ugly commits form my side) and started discovering new cool features to set.

GCDS was then a where we had the opportunity to present Zeitgeist. We then got a lot of feedback especially from very valuable members of the GNOME and KDE community such as Og Maciel, Diego Escalante, Sebastian Trueg, Frank Karlitschek, David Schelsinger, Daniel Siegel (The guy behind the Zeitgeist hackfest), Owen Taylor, John McCann, Robert McQueen, Robert Taylor, John Carr and the rest of the Collabora and Codethink team. By the end of GCDS we released the 0.2 version of the engine.

Things starting gaining momentum at the GCDS. Robert McQueen prototyped the Teamgeist concepts for us and now Youness Alaoui (aMSN) and Sumana Harihareswara are managing the Teamgeist project (also working on the 0.3 release of Zeitgeist). Siegfried was working on Shell integration, Mikkel was writing the specs for a 0.2.1 and and Markus went crazy rewriting all dataproviders again and developing Zeigeist-Filesystem. Sebastain Faubel helped me with the parental control which is being developed by Shane Fagan.

After GCDS my old partner in crime Alex Gabriel and I started again talking Zeitgeist and Mayanna. The vision of Mayanna being a platform to distribute data of interest, I convinced him to join the team and look at some of the contextual graphs prototypes I had. 2 days later after long long nights of hacking we were done.

After this intense period we took some time off and right before openSUSE conference Federico convinced me to rehack the UI. Sebastian Faubel and Moritz Eberl joined the crew at the conference and together with them and Alex Gabriel we recreated the whole UI. Later with the help of Kalle Persson, Hylke Bons, Andreas Nilson, Jason Smith, David Siegel and Karl Lattimer it kept becoming better and better. But before I forget Vincent Untz offered us a lot of organizational consultancy during the conference.

Stephane Lauriere form Mandriva invited us to talk at the OWF. And with the help of Rob Taylor there we got in contact with very good people to demonstrate Zeitgeist.

Right now the team is growing and more or less some each spec for the 0.3 release is being reviewed by at least 5 at a time.

Zeitgeist is just beginning to look really good. We learned alot in this one year journey and we expect to do some really sweet things at the hackfest and afterwards. Being referred to as the guy behind Zeitgeist is very flattering but not true in my opinion. It was a community effort. I just connected the people and the dots. (I wouldn’t mind being called initiator though ). I try to keep the momentum up, report the work the developers are doing, do some general spanking (as Federico likes to call it) and Blueprint driving. The developers are constantly working and do alot. I think everyone involved by now should blog about his work for zeitgeist or contributions.

I think it is now that the Zeitgeist project has started!

Thanks guys for an awesome year of development.

**I might have missed some people that deserve credit please contact me.

5 years after deploying “Ubuntu 4.10 (Warty Warthog)” on my laptop back in November 2004, I have become an Ubuntu member. Thanks to all the awesome people who cheered for me and pimped up my wiki.This is more motivation to hack. Hopefully I will get more involved with community (especially in my country) as much as upstream.

Before reading please understand that this is my personal opinion and experience and those of other middle eastern FLOSS users and non-users. Although the post is called “FLOSS in Egypt” I think alot of the issues also occur in other Arabic countries as well as third world countries (not all). Here I am pointing out problems and hoping to find solutions in a sprint during UDS or GUADEC.

One of the biggest benefits of FLOSS is that it is for free and open source. Which a lot of developed countries learned how to make use of, decrease costs and generate profit. It has become a trend that more and more nations are adopting in sense of deployment and contribution. Yet where I come from people are not catching up on the trend. I sat down with a couple of friends of mine (mostly Egyptians) and tried to analyze why is the middle east so ignorant towards FLOSS compared to other nations. I will list down the problems that FLOSS is facing to be deployed in the middle east and their impact on deployment and contribution

It starts off with Education. Basically this is the most essential problem that affects the existence of FLOSS in the middle east. I can’t speak for all Arabs but at least in my country (Egypt) the average kid gets exposed to computers at the age of 17. I remember my first day at an Egyptian university there was a workshop for all first year students to introduce them to computers (mouse, keyboard, etc…). And those who are exposed to them at a young age usually use the computer for video games and nonsense. At the uni I did not find one Linux or Solaris workstation anywhere. All computers were windows and all homework was to be submitted in Microsoft Office. Introduction to programming was in the second year and they were introduced to .NET (I left by then). So basically a whole year went by teaching students what computers are etc… Their first encounter is with computers was all closed source. Most FLOSS contributers encounter opensource at the same age most of Egyptians even encounter computers. Which really lowers the expectations of contribution. The Egyptian universities not even providing any FLOSS in their facilities makes the odds of ever really dealing with FLOSS even lower.

Let us assume that education went well. One encounters a new obstacle Society. In the middle east society has a way of controlling your life and how things are supposed to be done. As soon as you are in university your family pressures you to be done as soon as possible but this is still OK. After university one can not  take a quarter year off to find himself. You are pressured to find a job quickly and get married ASAP. Here is the deal. Usually one develops his last hobbies in the mid 20s. But the pressure of work and succeeding quickly to be able to get married etc, doesn’t leave you with space to find yourself and explore new hobbies. And once you are married and pressured into getting kids, I don’t think it gets better at all. Job+Family leaves very little space for exploring FLOSS if you were not really exposed to it before.

Now let us assume one is lucky and education and society were not an obstacle. Ladies and gentlemen meet the FLOSS Community in Egypt. Where should I start. Well there is a community but before I state my opinion on it let me explain how it functioned the last time I checked…. It is just like the regional policy “1 BIG guy” and everyone follows, and some others trying to overthrow him, not to provide a better option but rather to occupy the position of a leader. Activities of communities are almost only “Installation fests” and once a year at some university a weekend (in which no one really would show up due to their educational and society problems) they try to hold a little speech about FLOSS. Sometimes they give out CDs too. The idea of actually convincing small companies to switch to Linux is not in the communities agenda. So basically although there is an initiative also there is a lack of innovation in standing out as a community. The frequent disputes in the communities and the mentality of being the alpha male of the gang stands in the way of developing a fully functional community. Lately however EGLUG has taken in it up a notch IMHO and I am very proud of that, but not enough though. What worries me the most is the Ubuntu LoCo Team. It is hard to figure out who to contact when they are reachable online. And also although improving lately I don’t really get to see their presence in the Ubuntu world. The presence of a community is very important for deployment into bigger institutions and companies. As a LoCo one becomes a central point to refer to for FLOSS consultancy and help. Without that FLOSS will be ignored by local companies etc… Almost every organization I got in contact with when I explained to them FLOSS and why they should deploy it they responded “open source” means people who know the software can write viruses. Sadly by the time we get ot that point I lose my patience which is also an issue of the community. Also upstream communities don’t really exist, and when they do, then they don’t exist for long since again the crediting issue is a big deal and distributing responsibilities are usually minimal since the “leader” usually is afraid of losing control :/

Other issues that are affecting FLOSS in Egypt would be the Mentality. In a poor country people tend to be distrustful and also develop the complex of evaluating things by their costs. Example take a Linux CD and a Windows CD. Go to a random person and tell him here is Linux and here is Windows and both will cost u nothing although Windows actually would be valued XX dollars. What the person would think is “ah it costs money then it must be better”. Simple as that. And when you try to explain how FLOSS could be better well again u will stomp into the education issue. Corruption believe it or not is also an issue. I remember in Egypt the police going to all Internet cafes to check for pirated Windows. If found guilty the cafe was closed. I think this operation took place for 2 weeks. Next thing you know prices of Windows XP CDs drop down to 20 dollars. Guess who made money form behind this operation . This brings us to the Profit issue. Now as a Company in Egypt one faces all obstacles above to make money out of FLOSS by providing support etc… This lowers the interest of individuals as well as organizations to contribute to open source since money is a big issue in developing countries

All these effect the support for FLOSS too. One of the biggest disappointments to me this year was 2nd International Symposium on Computers and Arabic Language. RMS was invited to attend and after contacting Stormy to get in contact with some Arabs within the GNOME community to attend, we were then surprised by the their sudden disinterest. To quote the mail we got from RMS:

My only contact with them is *****.  Or rather, was *****.  He said
that the backers were interested in funding free software projects,
and then changed their minds completely.  ***** said that when he
heard this, he was so disgusted that he resigned all connection with
the activity
It is disappointing, but I think there is nothing left for us to do

I think this is actually the result of all the issues i summed up there.

I am not saying there are no contributors from the middle east. There are just too few. And those who are there are doing a heck of a job. Phillip Khalaf (Collabora), Youness Alaoui (aMSN), Khaled Hosny (GNOME Arabic translation), and a few more… We need more people though…

I honestly did not find a way of explicitly solving the problem since it is kind of a loop. But definitely one should start with the issues Education (projects such as OLPC and GSoC) and Community (inviting aspiring LoCo teams to conferences to boost their egos and momentum) .

Society and Mentality will change a bit when the Education and Community improve. Yet those are also affected by other factors such as politics, tradition, etc… and we don’t want to get into that.

Maybe this could be discussed  at UDS by some LoCo teams to find ways to give the Egyptian and other Arabic LoCo teams more momentum. Maybe have a LoCo sprint where 2 members of aspiring LoCo teams are invited to discuss local and global LoCo issues.

“Are you a FLOSS user in the Middle East? What do you think would help?”

So yet another blog post about Zeitgeist.

During GCDS I went looking for some Telepathy hackers. I got Rob McQueen, John Carr, Federico and others in one room brainstorming a collaboartion platform using Zeitgeist + Telepathy. By the end of the night the prototype worked.

Now Collabora has put Youness Alaoui from aMSN fame and Sumana Harihareswara to lead the project management and development. It is being developed very close to our 0.3 release due less than a month. But let me just explain a bit of how it will be and some use cases:

So the very high-level use-case is: you’re working on a project with a group (one could say a team) and you basically want everyone to have some indication of what other people are doing which is relevant to the project you’re working on.

You would give teamgeist some search query which would match zeitgeist events that are relevant to the project. Then the various users join a room which has a d-bus tube and the teamgeist/zeitgeist combination would forward the various events to this room so other people can keep track of them. (The UI for example could show a timeline of what people are/were working on).

Another interesting thing would be that when there are events referring to certain files it would be easy for the other users to get the updated version of the file (or it would automagically be downloaded). Or for users to add a local file to the project, such that teamgeist would update its filters to signal future events on that file to others.

This would also allow workflow management for teams as well as statistics of the work such as which were the most used files in this team etc…

We are looking forward to implement a more simple API which will be mostly proposed by Jason Smith and Youness. Let us see how far we can get with this. You can read the specs here.

Here are the first attempts of Jason Smith to use Zeitgeist in Docky!
Watch this docky-zg

** PLEASE THANK JASON SMITH FOR HIS AWESOME WORK

One of the reasons why i don’t hang out on facebook much is my whole family is there…
My mom then posted a picture of me as a baby.. compared to now…

Looks wise: Not much has changed just grew teeth and got the distribution of hair has changed in my face. Basically losing them on the head and growing them on the cheeks. Also form a round head it transformed to an egg shaped one.
Character wise: Nothing has changed still a big baby.

It has been almost a year since Zeitgeist started. Last year on october 10th I hacked up a little timeline, based on Federico’s mockups and the old gimmie/mayanna code during the Boston UI summit. And ever since alot has happened. From a UI we developed an engine, and the project kept growing while the ideas just kept flowing in. We had 2 releases for the engine and working on a big 3rd one. From a GNOME project we became a cross desktop project. We finally found our use cases and working on them allowing other applications to make use of it. Beside a 0.3 release we are planning to infect apps to make use of the zeitgeist engine. Here are some current use cases and deployments of an event aggregation fraemwork.

• GNOME Activity Journal: What triggered the whole project. Many iterations no release yet but to come soon

• GNOME Shell: Thanks to GSoC we were blessed with the “wunderkind” RainCT

• Chronosoup aka (Buttbuddy): A little sidebar app to be reimplemented in Docky2 (and Shell if they don’t mind). It was inspired while chatting with Mikkel Kamstrup and can be regarded as a reviving of Gimmie.

• Docky: Using some complex queries Jason Smith intends to push Zeitgeist to its limits soon. Basically on right clicking on an app you get “The most used documents within the last week of mimetypes supported by the applicaiton clicked”. There is more to come though…

• Zeitgeist FS: Makrus Korn came up with this brilliant idea and to be honest this is my favorite app using zeitgeist engine.

• GNOME Parental Control: This could be a good way to market GNOME 3. We will be needing some help from the tracker guys on that one. We forward events of interest to parental control while extracting metadata from tracker to analyze semantically for PG content.

• We can also provide any app over dbus most used documents/apps/or tags  with some set of filter

These are already semi functional usages of the zeitgeist engine. Now there is still more to come. Let me now list projects worth looking at:

• Teamgeist: imagine cross distributing events in teams. This will allow a Team Activity Journal as well as better workflow management. Thanks to Rob McQueen for hacking a prototype at GCDS now Collabora put Youness Alaoui and Sumana on the Project to make it happen. THANK YOU COLLABORA
• Tracker+Zeitgeist collaboration: We will be working alot with Tracker in the future, based on the fact that they have same interests as we do. Introducing GNOME to the semantic era. Tracker will handle “What is my data about and how are they related over metadata” While we are focusing on “How is my data used and how are they  contextually related”. Zeitgeist hackfest ftw.
• Contextual Relevancy analysis: Our kinda secret weapon, Alex Gabriel and me are putting lots of energy into it. Basically we monitor hwo the user switches his focus between documents and generate trees describing the contextually relevant data to a URI, it is pretty accurate for a first run. We will be exposing it over dbus for the GNOME Do crew to have a look at. It should be out with the next release. Here is a little example:

0 = “http://git.codethink.co.uk/”

1 = “http://google.com”

2 = “pidgin://Facebook/Karl Latimer”

3 = “pidgin://Facebook/Philip Van Hoof”

4 = “pidgin://Co-Workers/juergbi”

5 = “pidgin://Co-Workers/einalex”

6 = “tomboy://Today: Friday, August 28 2009″

• Ubuntu one storage management: Natan Yellin will be hacking on a dynamic storage management for ubuntu one allowing us to populate the storage dynamically according to most used/ recently used/ and bookmarks.

There is still alot to come and the team kinda grew dynamically currently around 12 developers have been providing awesome contributions and feedback to the engine (not the UI):

First the fulltime hackers:

1. Siegfried Gevatter: Splitting the engine form the UI, Shell integration, engine maintenance, and making me feel clueless and redundant.
2. Mikkel Kamstrup: Rewriting and planning what is now be to considered the stable fundamentals of Zeitgeist. Also taking the time to think of future plans and deigns of zeitgeist.
3. Markus Korn: the mystery guy who appeared form no where rewriting all of our dataproviders and adding awesome features such as an exchangeable DB module (Tracker guys should thank him). Also maintaining the engine
4. Natan Yellin: Putting the project on launchpad. Hacking the first concepts of zeitgeist and the Journal.
5. Alexander Gabriel: Taking time form mayanna to actually hack the context relevancy graphs with me. Kinda my partner in crime when we hack.
6. Me (Seif Lotfy): Uhm basically pissing the rest off. Getting into trouble and being a jerk. Thanks for not killing me yet guys, for my aweful code.

Other notable engine contributers would be:

1. Jason Smith
2. Youness Alaoui
3. Sumana Harihareswara
4. Federico Mena
5. Thorsten Prante
6. Rob Taylor
7. Sebastian Faubel
8. Moritz Eberl

This was an awesome year. Let us hope it gets better. We will start talks with KDE for KDE deployment