GNOME Software, The Application Installer We Have All Been Waiting For

For years now there have been talks about a proper application installer in Fedora. This cycle, Matthias Clasen and Richard Hughes have finally made it a reality.

In Fedora 20, GNOME Software will be available as a technical preview. In the future, it will replace gnome-packagekit as the default app installer. This will also allow us to make gnome-packagekit more geeky for people who are interested in packages and not in apps.

Meet GNOME Software

Based on the design by the amazing GNOME Design Team, GNOME Software is indeed extremely easy to use.

The main GNOME Software screen

The main GNOME Software screen

The main screen shows a big banner for a “Featured” application. In the future it would probably be cycling carousel-style, but for now (it’s still a tech-preview after all) it’s static and hardcoded. Below that, a list of “picks” is displayed. They would change dynamically in the future as well, allowing us to feature new or interesting applications to help the users discover them easier.

Categories view

Categories view

In addition to traditional searching, we allow users to discover apps by navigating categories. The categories we have now are taken directly from the freedesktop menu spec, which is suboptimal for our usecase and as Matthias said, “Very much 90s”. Therefor, in the future, we will find a better, more sane set of categories.

 

The Graphics category

The Graphics category

The Strategy Games category

The Strategy Games category

Application Information view

Application Information view

The application information is taken directly from the appdata.xml files. If you want your application to have detailed description, a link to your website (and, in the future, screenshots), you should add an appdata file! check out the specification and Richard’s post on fedora-devel-list for more information. A lot of upstream projects have added appdata files in the past few weeks. If you know of a project that didn’t hear about it, let them know!

Another application information screen

Another application information screen

List of installed applications

List of installed applications

 

 

Cross Distribution DE-Agnostic Awesomeness

yay for standards

While GNOME Software is designed according to the GNOME world-view, the metadata is not. While testing GNOME Software in Fedora, you will notice apps from other desktops as well. We don’t hide them. In addition to that, the metadata is standard and a subset of the AppStream Specification which was developed collaboratively by people from Fedora, OpenSUSE, KDE and Ubuntu. Ubuntu and SUSE already have the metadata in their infrastructure, and Fedora will have it soon.

Since all application descriptions are submitted upstream, and since we use PackageKit as a backend, any distribution who wishes to will be able to integrate gnome-software.

On the Fedora side of things, the plan is to generate the metadata (that means two tarballs, one containing app icons and the other containing appdata xml with application descriptions) in koji, and then make it available on the mirrors metadata. The metadata will be generated by using Richard’s little script. This script handles extracting descriptions and icons from packages, as well as blacklisting some specific packages we don’t want to show in the application installer (stuff that have desktop launchers, but are not actually apps).

Other desktop environments will be able to implement their own Application Installers using the same metadata, if they wish to.

Not Only For Packages

Due to the pluggable nature of GNOME Software, packages are not the only way of obtaining software that it will support. We already have someone working to add support for Firefox web applications.In addition to that, it would be possible to support installing shell extensions (from extensions.gnome.org) directly from within GNOME Software!

I’m also trying to convince Valve to expose some APIs to us so we can make a plugin that will integrate Steam (if installed) with GNOME Software.

In the future, the plan is to support bundles such as glick and listaller bundles, to make it easy for 3rd parties to distribute apps.

Work In Progress

As always, and especially with a tech-preview, your testing is appreciated. if you are already running Fedora 20, all you have to do to give GNOME Software a go is to type yum install gnome-software.  Please file bugs if you find any!

Upgrading CyanogenMod on an encrypted device

This is a quick tip post.

The first thing I noticed after enabling full-disk encryption on my Android device is that the CyanogenMod updater stopped working. That makes sense, the recovery UI is too simplistic to be able to mount encrypted storage on which the update file is stored.
TL;DR? Upgrading Android is not for those who fear command line.

Still, updating your phone is useful, so if you want to do it on your encrypted device, here’s how:

  1. Connect the phone to your computer using a USB cable
  2. If you haven’t already, install the “android-tools” package
  3. Download the CyanogenMod archive you want to install, from get.cm.
  4. Reboot your phone into recovery mode.
  5. Once recovery is up, open a terminal on your computer and type “adb shell”
  6. In the adb shell, you need to mount /sdcard as a tmpfs: mount -t tmpfs -o size=500m tmpfs /sdcard
  7. Now close the shell by typing “exit”, and then move your downloaded archive into the phone like this: adb push cm-10.1-20130430-NIGHTLY-i9100.zip /sdcard/update.zip
  8. Navigate to “install update from sdcard” (note: navigating through recovery differs between devices. on my Galaxy S2 it is done by moving the cursor up and down with the volume button, and selecting an item with the power button).
  9. Select “apply /sdcard/update.zip”
  10. Reboot your phone when this process is complete

RTL (Right to Left) Text Editing: Comparison of Implementations

Recently I had a very productive talk with GNOME designers and developers in relation to RTL (Right to Left) text editing in GTK.

For those of you who don’t know, RTL languages such as Hebrew, Arabic, Farsi and others are different than most languages in a very simple way: the line of text starts from the right side of the page, and goes to the left. You read it from the right (first letter is the rightmost) and finish it on the left (last letter is leftmost) On Roman (or other LTR – Left to Right) languages,   it goes the other way around. This concept is hard to grasp for most people, especially if they don’t read any RTL language but want to develop software that supports it.

So how well does RTL text editing and displaying works on our platform? I’d say “pretty well”, but there’s some room for improvement. We’ll get to it in a second, but first let’s look on how RTL editing is done on other platforms.

On Windows, RTL support works in the “dumb” way, I’m not trying to say it’s stupid (although it kinda is), I’m just trying to say they have no smart text directions algorithm at all. When you write software for Windows, you get to decide the default text direction of every widget. But what’s if it’s a user-editable entry?

On most text inputs, the user might type in any language they choose to, and altough the default text direction is the one of the app’s UI, it usually isn’t what the user wants when they type. So what does Windows do to solve this problem? Nothing. Instead, they make the users pick the direction themselves. This is sometimes better than most “smart” text direction algorithms, but isn’t optimal. To switch directions in windows, you use lctrl+lshift for LTR or rctrl+rshift for RTL, (relevant note: if you use Firefox, ctrl+shift+x toggles the direction of the text input across all platforms). And if you are editing a multi-line file you are in trouble, because there’s one direction setting for the entire file, instead of one per line.

In GTK, things are done a bit differently. For starters, every line gets its own direction, which is great when you write a multi-line message or text document. GTK also tries to be smart about the text direction, and sets the direction of the line according to the first “strong” characters in the line. “strong” characters are characters which has a defined direction in the unicode specification, so a latin/roman character (like the ones you are reading now) are “strong” left, and Hebrew and Arabic characters are “strong” right. Numbers are “weak”, meaning they don’t have any explicit direction defined (in most RTL languages numbers are read from left to right, yes, I know, this is confusing for those of you who don’t speak any RTL language).

While smart direction guessing is useful and a nice thing to have, GTK’s method does have few flaws, for example, if I highlight someone on IRC (IRC nicknames are always made of latin characters, which are LTR) when I chat in Hebrew, the line will be aligned to the wrong direction. There are many other (more common) cases in which you might want to start your sentence with an English word but continue in an RTL language – this is simply the first one I could think of (the other one that pops to my mind is instant messaging communication, but I can’t explain how this could happen outside IRC without boring you with rules of Hebrew grammar).

So how do others deal with this problem? to answer this question, I’m going to look away from desktop platforms and look at websites. Facebook does exactly what GTK does, apparently, so I’m going to go ahead and ignore it. Twitter and Google (across various products, including gmail, Google Talk and Google+) do things a bit differently and (in my opinion) it’s better: They appear to count the amount of strong characters in the line, and decide whether it will be RTL or LTR according to which number is higher. Sounds simple, right? closer examination shows that they do a bit more than just character counting, because “aaaa אאא” aligns the line to the right instead of the left.

According to their github repository where they host the code that does that (yay FOSS), they set the direction to RTL if the number of RTL characters in a line divided by the total number of characters (after stripping hashtags, URLs and mentions) is bigger than a set threshold (currently 0.3).  I do think that makes sense. Google, unfortunately, did not release the code that they use to do that, so we can’t see how they implemented it.

If I already mentioned Google, it’ll be interesting to add that on Google Talk, the input field does all these fancy auto alignment things, but then the messages are displayed as LTR. Sucks.

So what should we do in GTK? Our BiDi (bi-directional text) support is pretty good, in my opinion, but we can make it better. I have two simple suggestions: first, we should do what twitter does (and it’s open source, so we also know how they do it), and use character threshold instead of the first strong character. The second thing is doing what Windows and Firefox do: sometimes even the smartest automatic direction guessing algorithm fails, and it would be extremely useful to be able to manually toggle the line’s direction, especially when writing code.

One final word, for those of you who never touched BiDi before: I mentioned RTL text displayed as LTR. This (in most cases) doesn’t mean the text is mirrored and impossible to read. It does, however, means that the text is left aligned and the position of “weak” characters (punctuation and numbers, mostly) in the text is wrong. You might notice the same thing if you read LTR text displayed as RTL:

(this bit is in the brackets) and this one isn’t, and ends with a question mark?

Ugly and confusing (also try selecting parts of that line as well). Especially when the text is longer than just one line.

So what do you think? Is my suggestion for GTK seems reasonable?

Fedora ARM vs. Raspbian on the Raspberry Pi: Benchmarks

Using the Fedora ARM remix on the raspberry Pi, I couldn’t help but noticing how slow it was. I thought it was due to the Pi’s weak hardware, but since I accidentally corrupted my root filesystem on the Pi, I decided to try Raspbian for  a change.

I was surprised to discover Raspbian seems way more responsive and, well, faster than Fedora, so I decided to investigate.

In this post I will benchmark Raspbian and Fedora ARM Raspberry Pi Remix using various benchmarking suits. The setup is a single Raspberry Pi (not overclocked), a class 10 SD card for the root filesystem, with the 240MB memory split (240MB for RAM, all the rest for the GPU that I don’t use).

The first benchmark: sysbench CPU benchmark

On Raspbian:

 sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 10000

Test execution summary:
    total time:                          498.8142s
    total number of events:              10000
    total time taken by event execution: 498.7676
    per-request statistics:
         min:                                 49.70ms
         avg:                                 49.88ms
         max:                                 74.58ms
         approx.  95 percentile:              50.81ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   498.7676/0.00

on Fedora:

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 10000

Test execution summary:
    total time:                          869.8395s
    total number of events:              10000
    total time taken by event execution: 869.7930
    per-request statistics:
         min:                                 86.74ms
         avg:                                 86.98ms
         max:                                205.24ms
         approx.  95 percentile:              86.91ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   869.7930/0.00

Second benchmark: extracting the linux kernel mainline source code archive with tar and bzip2.

On Raspbian:

pi@raspberrypi ~ $ time tar -xf linux-3.6-rc7.tar.bz2 

real    5m49.174s
user    4m13.450s
sys    0m40.020s

On Fedora:

 [elad@raspi ~]$ time tar -xf linux-3.6-rc7.tar.bz2 

real    8m25.091s
user    4m25.350s
sys    0m50.400s

Third benchmark: writing 500MB of random data to the SD card using dd. I do not expect any major difference here, but testing anyway doesn’t harm anyone.

On Raspbian:

pi@raspberrypi ~ $ time dd if=/dev/urandom bs=1M count=500 of=somefile
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 466.911 s, 1.1 MB/s

real    7m46.929s
user    0m0.000s
sys    7m29.870s

On Fedora:

[elad@raspi ~]$ time dd if=/dev/urandom bs=1M count=500 of=somefile
500+0 records in
500+0 records out
524288000 bytes (524 MB) copied, 482.279 s, 1.1 MB/s

real    8m2.334s
user    0m0.000s
sys    7m52.840s

And now for some graphs: I have run PyBench and the Apache benchmark using the phoronix-test-suit.

System spec table from the phoronix test suiteI wonder why the test suite running on Fedora sees only 229MB of RAM while the test suite running on Raspbian sees 232MB…

Let’s take a look at the results:

PyBench output graph

On PyBench, Raspbian seems to be 1.37x faster

Apache benchmark result graphs

Again, Raspbian seems to be 1.25x faster.

Well, I don’t know much about ARM, but it seems to me Raspbian is just more optimized for the Pi, which is bad for me because I feel more at home managing a Fedora machine and not a Debian machine.

 

August Penguin 2012

I don’t usually wake up before the sun rises, but when I do, I have a very good reason, such as August Penguin, the annual conference of the Free and Open Source Software community in Israel.

This year, August Penguin was hosted at Haifa Auditorium, Haifa, so I woke up early, got ready, and took the train

Petach Tikva Train Station

Petach Tikva train station, early in the morning

Getting to Haifa was easier than I thought, and I got to the conference just in time. This year, there seemed to be much more people than last year, which is great.

Those who read this post from Planet Fedora probably don’t know, so I should point out that usually August Penguin is held in central Israel, but this year it was located way north (~100km). Of course, some people have complained about the “inaccessible” location, but I think the different location is what caused this conference to be, or at least to seem (I don’t have exact attendance numbers) bigger than last year.

We (me and Adir Shemesh) arrived to the conference just in time for the kenynote, so we delayed the deployment of the Fedora stand to the break after the first round of talks.

Lior Kaplan giving the keynote

Lior Kaplan giving the keynote

Lior Kaplan, from Hamakor NPO (an Israeli FOSS NPO) board made the keynote, in which he talked about events Hamakor organized during the past year, and the raising adaptation of open source solutions in the Israeli government.

That, of course, happened after the ~20 minutes delay we are used to due to various technical problems, although Moshe and Nadav, who organized the conference this year, got there hours before to make sure everything works. The theory accepted by most of the attendees is that in a place with so many techie people, things must break, otherwise it’s not a proper tech conference.

So, during the break, Adir and I deployed the Fedora stand, equipped with two laptops, an unofficial tablecloth, hundreds of Fedora 17 DVDs, and, (thanks to Christoph Wickert) Fedora logo stickers.

Since Ubuntu Israel are inactive recently, we were the only Linux distribution with a stand in August Penguin this year, so the DVDs were snatched like hot buns.

Apart from our stand, there was the usual Mozilla Israel stand, manned by

Me in the Fedora stand

Me in the Fedora stand (photo by Tomer Cohen)

Amir Aharoni, Tomer Cohen, Amiad Bareli and I. Since I’m a contributor to both Fedora and Mozilla, I was hopping between the two stands, manning them both at the same time. I think that, in total, I spent most of my time in the Mozilla stand, since the local FOSS community doesn’t need much explanation about Fedora, and Adir was handling it quite well.

One of the comments we got in the Fedora stand, more than once (and not only on this conference), is that the Fedora logo looks too much like the Facebook logo, and explaining we were first isn’t helping, Facebook’s brand is too strong. But apart from that, most of the feedback we got was quite positive, and I will list all of it at the end of this post.

Amiad, Amir and Tomer at the Mozilla stand

(from left to right:) Amiad, Amir and Tomer at the Mozilla stand

On the Mozilla stand, I showed Tomer GNOME Web and asked him who I need to talk with so that Firefox will act like GNOME Web and hide the titlebar when maximized on GNOME 3. According to Florian Müllner, who replied to my post about it in Google+ in March (wow, I’ve been postponing taking care of it for too long), it is only an X11 property that needs to be set, so the patch should be simple. Now I need to start searching Mozilla’s bugzilla for a bug about this, and if there isn’t one, file it, and maybe attempt to put a patch together (if someone could tell me which part of Firefox’s source code I need to tackle and link me to documentation of the said X11 property, it would be much helpful), I’m going to try and get this fixed over this weekend.

And, of course, I did not forget to take a decent amount of Firefox stickers and buttons for our FOSS booth in the next Icon conference, which will be just in few months.

Dor, Yaron and Yair

Dor (with the green shirt and back to the camera), the former leader of the local Ubuntu community.
Yaron Shahrabani (with the Ubuntu shirt), a well known GNOME translator, who practically maintains all of GNOME’s Hebrew translation himself, and
Yair Podemasky (rightmost)

Back on the Fedora front, I decided it’s about time Yair Podemasky, who is using Fedora for quite a while now, and helped us a lot in the FOSS stand at Icon and Olamot scifi and fantasy conferences this year, and is very good in explaining Fedora and Open Source to the average Joe, should become a Fedora Ambassador, to make his contribution to the project and to our local community more official. In the next few days I will try to make becoming a Fedora ambassador easy for him, especially since he already proved capable in the practical side.

Later on, I talked with Amir about the new (or rather, revised) Hebrew keyboard layout standard, si1452, which improves typing of Niqqud and Cantillation, as well as other minor improvements, such as using proper Hebrew quotation marks and maqaf (hyphen).

Me showing xkeyboard-config git to Amir

Me showing xkeyboard-config git to Amir
(photo by tomer cohen)

This revised standard will be included with Windows 8, and also certain touchscreen keyboards, and we agreed that we should get it to all Linux distributions as soon as possible. The proper way to do that, of course, would be getting the relevant config file into freedesktop’s xkeyboard-config. If we are quick enough, we might be able to get this in Fedora 18, and maybe, if update policy allows, as an update for Fedora 17 and 16.

Some of the feedback we got, and other minor notes:

  • Hardware support improved in recent Fedora releases. Not that we were unaware of that, but people started to notice. Some say Fedora is the only distro that manages to run all their hardware out of the box
  • SELinux was causing problems to more than one person, and instead of just disabling it, they wanted to learn more about how to use it and configure it, but I did not know where to send them. Do we have some organized place for all SELinux user documentation?
  • We need more events (Mozilla and Fedora both)
  • Many people thought Mozilla completely killed Thunderbird, which is not the case, I think there was a communication failure between Mozilla and the end-users somewhere along the way
  • Amiad claims Mozilla are fixing RTL bugs faster than GNOME does, I asked him for a full list of RTL bugs with GNOME so I could get the right people to notice them, if needed.

More photos

Re: Rethinking the Linux distribution

This is a response to Alexander’s post “Rethinking the Linux distibution“.

Well, Alexander is trying to solve a problem, or to be exact, some problems. I think he should focus on each problem separately.

I imagine a system where the OS is a well defined set of non-optional core libraries, services and apps. The OS is shipped as a read-only image that gets loopback mounted at / during early boot. So, not only does everyone have the same files, they are using (and testing) *exactly* the same bits. We can do semi-regular updates by replacing the image (keeping the old one for easy rollback), and we can do security hot-fixes by bind-mounting over individual files.

This approach is problematic. What would happen if the user need a 3rd party driver that we can’t ship due to licensing issues? You can’t deny that there are all kinds of hardware that we can’t support right now, certain broadcom wireless chips for example.

The core OS is separated into two distinct parts. Lets call it the platform and the desktop. The platform is a small set of highly ABI stable and reliable core packages. It would have things like libc, coreutils, libz, libX11, libGL, dbus, libpng, Gtk+, Qt, and bash. Enough unix to run typical scripts and some core libraries that are supportable and that lots of apps need.

The desktop part is a runtime that lets you work with the computer. It has the services needed to be able to start and log into a desktop UI, including things like login manager, window manager, desktop shell, and the core desktop utilities. By necessity there will some libraries needed in the desktop that are not in the platform, these are considered internal details and we don’t ship with header files for them or support third party binaries using them.

Another case you should consider, gstreamer plugins. Right now, we can’t play mp3 files or MPEG4 files out of the box, due to patent issues. Right now, users can install gstreamer plugins for those formats via a 3rd party repository. But with your solution, it is not possible. Not without rebuilding the core OS read only disk image, which is not a task for the average user that just want his mp3s to play.

All applications are shipped as bundles, single files that contain everything (libraries, files, tools, etc) the application depends on. Except they can (optionally) depend on things from the OS platform. Bundles are self-contained, so they don’t interact with other bundles that are installed. This means that if a bundle works once it will always keep working, as long as the platform is ABI stable as guaranteed. Running new apps is as easy as downloading and clicking a file. Installing them is as easy as dropping them in a known directory.

Well, it seems that you want to replace package management with bundles. Bundles are a good solution to allow 3rd party (proprietary, mostly) to ship Linux application easily. But this is not a good solution for FOSS applications.

You say that bundles are self contained. So what happens to, say OpenShot, that calls blender to create animated titles? It would need to ship the entire blender in it’s bundle. That doesn’t sound like a good idea. And there are more examples for Linux applications that interacts with each other.

Also, downloading applications from the web? Really? Well, that’s just a bad idea. What the bundle solution really need is, live side by side with package management, and an Application Store/Market/Center (whatever you want to call it) that will allow you to download and install bundles and packages easily, from the same place. There would still be an option to download bundles from the web, but the primary way for getting applications would be the “store”, in which all bundles will be verified, and there will be an infrastructure to donate for FOSS applications or pay for commercial ones easily. Such store should be shared between all distributions that’ll support bundles. Users won’t see any difference between packaged applications and bundled applications, except from a small line saying “source: $APP_DEVELOPERS (bundle)” or “source: $APP_DEVELOPERS (packaged by $DISTRO)”.

Anyway, the first step in this direction should be designing and developing a GNOME application installer (PackageKit UI) and having application information metadata in fedora.

Bundles are a great idea, actually, to solve the problem of 3rd party software developers (mostly proprietary) saying “I won’t develop for Linux because packaging for each distro is too much work”. But having a read only file system, and killing existing package management is just not the right solution. Making bundles the only way to install software destroys all advantages of a package management system.

I assume (mainly because I saw some gnome designers oppose to package management) that this is going to be the way you install software in GNOME OS. Am I right? Well, I hope at least you’ll develop it as a freedesktop spec and not only in GNOME.

Zif as Default PackageKit Backend – a suggested Fedora 17 feature

Zif is a package management backend for PackageKit, designed to do things right. It preforms much better than yum with PackageKit UIs, making the UI output results much faster. As yum is more tested and used than zif, we should keep yum for CLI by default.”

(from the feature page).

With zif, PackageKit UIs (such as gnome-packagekit) are much faster and more responsive. For example, search result appears in less than half a second when using the zif backend, while it sometimes take more then a minute with the yum backend.

More information in the feature page.

Fedora at August Penguin 2011

August Penguin is the largest FOSS event in Israel, and today (Friday) Hamakor NPO held the 10th August Penguin, at Shenkar College of Art Design and Engineering, Ramat Gan.

This year, we had a Fedora booth, and it was a great success.

We gave away almost all of the media, with only 4 Live DVDs left!

 

More pictures (my camera died after the 7th picture, sorry)

Even more pictures (by Shlomi).