Installer working fine on the Kurobox Pro

Per Andersson ported the Debian installer to the Kurobox Pro this summer as part of a Google Summer of Code project. Along with Riku Voipio, I acted as Per's mentor and gave him advice while he was trying to figure out all the details that were needed to get Debian running on the device. Since I spent the summer in Israel and didn't have my Kurobox Pro with me, I never performed an installation on my own though.

Yesterday I finally found time to play with my Kurobox Pro. Per did a great job and the installation worked without any problems. I also investigated how the recovery mode works and added various new information to my Debian on the Kurobox Pro pages. The Kurobox Pro seems like a nice machine, but I hope we'll add full support for the Linkstation Pro and Live soon since these devices are much more easily available. It shouldn't take too much work since these devices are quite similar to the Kurobox Pro.

Debian installer coming to the D-Link DNS-323

My posting about receiving a Conceptronic CH3SNAS a month ago prompted Matt Palmer to work on the installer. Matt did a fantastic job and had patches for everything within a few days: he added support for the revision B of the DNS-323 to the kernel, added code to the kernel so the MAC address is parsed from flash, and added DNS-323 support to various components of the installer. Finally, he wrote a really good porting guide that will hopefully inspire other people to port the installer to more NAS devices. Porting to another Orion based device usually doesn't take much now that we have the infrastructure in place.

While Matt's patches came too late for inclusion into the initial release of lenny, the release team seems agreeable to the idea of adding support for the D-Link DNS-323 and Conceptronic CH3SNAS to a stable update of lenny. To this end, the dns323-firmware-tools package which is needed to create proper firmware images for the DNS-323 was accepted for lenny a few days ago. Alan Fletcher also kindly agreed to make a serial cable for me, so hopefully I'll get a chance to test Matt's patches soon and integrate them into the installer. In the meantime, I created a few simple pages about Debian on the D-Link DNS-323 which I will extend once support has been integrated into the installer.

Case stories of good and bad community interaction

It is often argued that companies have to work with the FOSS community and there are good reasons for doing so. I've tried to collect a number of case stories of good and bad community interaction that may help as a starting point for further exploration of this topic:

• NVidia is probably the best example of a company that is not interested in providing open source drivers for their graphics chips. There are lots of folks who won't buy hardware with NVidia chips for this reason. Intel is the number 1 choice at the moment because they work closely with the community. Related to this, the Linux Foundation and kernel community released a statement regarding open drivers.
• When Sun finally made Java available as open source, they argued that it would help inclusion and distribution of Java (e.g. inclusion into Debian). Sun also felt that the GPL provides the strongest protection "against misbehaviour by monopolists in the Java platform".
• The Novell/Microsoft deal was not perceived well by the community, which led to web sites like http://boycottnovell.com
• Nokia recently had an insightful blog posting about the problems when nobody knows about the open source contributions you make: for example, you cannot influence decisions in a project as easily.
• Sun is often criticized for dominating all open source projects they run. They can be used as a good example that you have to give up some control in order to get outside participation.
• The fact that Xen is not in the mainline kernel is a key reason why a number of companies (e.g. IBM, Red Hat) are looking at KVM (which is in the mainline kernel and actively developed in the community). In fact, "looking" is probably not the best word. IBM is actively working on KVM to make it a viable solution and Red Hat recently acquired Qumranet, the company behind KVM.
• Oracle got fairly bad press for copying the Red Hat distribution and changing Red Hat to Oracle, without engaging in the Red Hat community.
• There's a paper that looks at "empirical evidence on the incentives of firms that engage in OS activities." They look at 146 Italian companies supplying open source solutions, but unfortunately the study doesn't include any major company.

Can you think of other examples?

(Originally published on FOSSBazaar where comments are possible)

Tracking GCC 4.4 related build errors

I started building the Debian archive with GCC 4.4 ten days ago to report build errors. I've completed the archive build now and reported about 220 bugs (the majority with patches). There are roughly 30 build failures left that I haven't analyzed yet. There are also about 35 packages that fail because the boost headers don't work with GCC 4.4. I'll try to build them when the boost headers get fixed.

The majority of GCC 4.4 build errors are trivial. The large majority of failures is because of missing #include statements. There are also about 20 build errors because of improved preprocessor checks.

I'll try to do another archive build when the gcc-4.4 package is in Debian.

Debian installer improvements for NAS devices in lenny RC1

There have been a number of improvements to the Debian installer on Network Attached Storage (NAS) devices in the last few months and they are available to users now that RC1 has been released.

We have added support for Marvell's Orion platform. Orion is a system on a chip (SoC) that integrates an ARM CPU, Ethernet, SATA, USB, and other functionality in one chip. The majority of NAS devices that have come out within the last two years are based on Orion chips, so it's exciting to have support for this significant platform in Debian. Specifically, we support the following devices based on Orion: QNAP Turbo Station (TS-109, TS-209, TS-409), HP Media Vault mv2120, and Buffalo Kurobox Pro (with more to come later).

Here are some other newsworthy improvements and changes:

• When a static IP address is set in the original system that is the default from the vendor, this address will now be used. In the past, the installer would use DHCP in this case but users found this confusing. This affects the Linksys NSLU2 and Thecus N2100.
• If a static IP configuration is used in the original system but this configuration is incomplete (i.e. IP address or DNS are missing), then the installer will use DHCP (instead of hanging during start). This affects the Linksys NSLU2 and Thecus N2100.
• On the Linksys NSLU2 and Cobalt devices, all required installer modules are automatically installed now and no longer have to be selected manually from a list.
• initramfs will use MODULES=dep rather than MODULES=most. In other words, only modules that are actually needed on your system will be included in the ramdisk.
• /etc/default/rcS will be configured on supported NAS devices to contain FSCKFIX=y, so e2fsck will attempt to repair the filesystem without prompting the user for input. This is useful on NAS devices since they are usually headless.

Finally, the armel port is recommended for new installations now. Debian lenny has two different and incompatible ARM ports: the old ABI port (arm) and the new EABI port (armel). Debian lenny is the last release with support for the arm port and future releases will only support the armel port. It's therefore recommended to use armel for new installations of lenny.

GCC 4.4 related build problems: missing #include

C++ header dependencies got cleaned up in GCC 4.3, which broke a lot of code which relied on headers to be included indirectly through some other headers. I found some new build failures with GCC 4.4 related to missing #includes; in particular, #include <cstdio> is missing in a lot of code.

Typical errors look like these:

error: 'sscanf' was not declared in this scope
error: 'EOF' was not declared in this scope
error: '::fseek' has not been declared
error: 'uint32_t' does not name a type

Below is a table showing which header needs to be included for a number of common functions:

Functions and defines	Header
fopen, fwrite, fclose	cstdio
fread, fseek, feof	cstdio
sscanf, sprintf	cstdio
rename, open	cstdio
EOF	cstdio
memset, memcpy, strdup	cstring
va_list	cstdarg
int16_t, uint32_t	stdint.h

GCC 4.4 and better preprocessor checks

GCC 4.4 will show more preprocessor errors when compiling code, partly because some new checks have been added and partly because the full preprocessor directives are now checked. So let's review some common problems. Let's take some code like this:

#ifdef A
#elif
#endif

This will now give:

t.c:2:6: error: #elif with no expression

The reason is that an #elif always needs an argument. Looking at the code, it's obvious that a #else should be used.

Another common issue is this one:

#define B
#ifdef A
#elif B
#endif

The problem here is that you're trying to check whether B is defined, so the third line has to be:

#elif defined(B)

Finally, the following code:

#ifdef A
#elif define(B)
#endif

will give:

t.c:2:13: error: missing binary operator before token "("

It took me a minute to see what was wrong with the code... but define is a typo and what you really want is defined.

Please test debian-installer rc1 images on ARM and MIPS

debian-installer rc1 is going to be officially announced next week. However, the netboot images are already available. I'm now looking for people who are interested in testing these installer images on ARM and MIPS based machines.

If you have a spare ARM or MIPS machine or want to reinstall your machine for some reason (for example to move from arm to armel), please see my messages on debian-arm and debian-mips for more information.

I've used this opportunity to update all my installation instructions to lenny. See here for Debian on Linksys NSLU2, Thecus N2100, QNAP Turbo Station, HP mv2120 and Cobalt.

European Commission publishes guidelines on the procurement of FOSS

The Open Source Repository and Observatory (OSOR), a new site sponsored by the European Commission to foster the exchange of FOSS related information and software among European public administrations, recently published guidelines on the procurement of open source software. Public administrations in Europe have to follow public tender procedures and the new guidelines give practical and legal advice on how open source software and related services can be incorporated into the procurement process.

Rishab Ghosh, who presented the guidelines at the Open Source World Conference in Malaga, argued that the procurement guidelines were needed because of two reasons. First, they studied recent tenders and found that many explicitly mentioned proprietary applications. 16% of 3615 software tenders explicitly asked for products from top 10 software vendors, such as Microsoft, SAP and Oracle. This practice may be illegal because public tenders generally have to describe functional requirements in a general way instead of specifying specific products. Second, many public administrations don't have any experience with the procurement of FOSS. In fact, they often don't know whether or under which circumstances they are allowed to adopt and ask for FOSS solutions. The guidelines are specifically designed in order to clearly and simply explain how public administrations can acquire open source and they don't assume that a country has adopted a specific policy regarding open source.

The guidelines include a long section about open standards, open source and how they relate. Both open standards and open source align very well with the needs of public administrations who have an "obligation to support interoperability, transparency and flexibility, as well as economical use of public funds". The guidelines argue that the exit cost, i.e. the cost incurred in moving to another IT system, is also an important consideration but one that is often neglected. The adoption of a proprietary solution without open standards may limit the future choice, thereby increasing the long-time costs and giving the proprietary vendor an unfair advantage in future tenders.

The procurement guidelines describe two ways of acquiring FOSS: it is possible to go the usual route of publishing a tender for the supply of software (possibly with related services). However, in the case of FOSS, it is also possible to download the software directly from the Internet. This is possible because the software is not only free of charge but comes with no contractual obligations. If there were any obligations involved with the download (such as fees, the agreement to an EULA or the requirement to purchase services in the future), software download is not an allowed method. What I like about the guidelines is that they explicitly say that downloading software has to be part of the formal procurement process. You have to think about your requirements, look at various alternatives, and so on, and not just blindly download something from the Internet.

When it comes to the procurement of FOSS, the guidelines don't suggest that tenders should explicitly ask for FOSS. Instead, they should describe the functional requirements of the software as well as certain properties. For example, a tender could specify that the public administration as well as third parties must have the right to study, distribute and modify the software. In a sense, the guidelines suggest that tenders should include the principles of the Free Software Definition along with justifications for these requirements.

Personally, I think there is a great need for these procurement guidelines. There are many public administrations that don't know how to acquire FOSS and these guidelines offer clear advice. Furthermore, I find the guidelines very balanced. They don't recommend that you should always ask for FOSS but incorporate FOSS principles into tenders where it makes sense. It remains to be seen whether the procurement guidelines will have an impact on the FOSS adoption in Europe, but I surely hope so.

(Originally published on FOSSBazaar)

IOP32x 2.6.26 kernel with Intel DMA engine patches

I've produced a 2.6.26 based kernel with the DMA engine patches from Intel which I intend to support throughout the lifetime of Debian lenny. The Thecus N2100 has considerably better hard drive performance with this kernel. However, the patches are experimental and not in the mainline kernel (some background on this issue). I haven't seen any data corruption with the current version of the patches, but please watch out for problems if you decide to install this kernel. Detailed instructions are available.

Amazing figures on the open source adoption in Finland

I attended Openmind last week, an interesting conference organized by the Finnish Centre for Open Source Solutions (COSS) to bring together open source professionals, community members and academics in Finland. In the session about business aspects of open source, in which I gave a talk about FOSS Governance, Nina Helander and Mikko Rönkkö presented the preliminary results of the National Software Industry Survey 2008.

I found the results incredibly interesting. In particular, they found that 75% of responding firms use open source software in some way. This figure is up from approximately 13% in the year 2000, which is a major increase in just a few years. The study also found that that there was no statistically significant difference of company size when it came to the use of open source. Finally, companies that have open source components in their offering rate themselves as more innovative.

Roberto Galoppini, who moderated the session, commented that Finland is where Gartner predicted that companies would be in 3 years. Indeed, I think everyone in the room was pleasantly surprised by the amazing numbers from the survey. We should not forget that the study was with software companies and that the use of open source will certainly be lower in other industries, but nevertheless the study shows that Finland is a leading country when it comes to the adoption of FOSS.

(Originally published on FOSSBazaar)

Debian on the QNAP TS-409U

Christmas came early for me this year: after receiving a Conceptronic CH3SNAS yesterday, a QNAP TS-409U showed up today. QNAP sent it to me a week ago to ensure that Debian works on it but it was stuck in customs for a few days. The QNAP TS-409U is similar to the TS-409 but comes in a rack case (and with more RAM). It seems that QNAP have also changed the layout of the mainboard but essentially it's still a TS-409. This means that the installer works just fine. Another difference to the TS-409 is that the TS-409U has two fans but qcontrol works without any problems on both machines.

Conceptronic CH3SNAS and the D-Link DNS-323 revision B1

Conceptronic kindly sent me a CH3SNAS for my Debian porting efforts. The Conceptronic CH3SNAS is reported to be basically the same hardware as the D-Link DNS-323 but it costs slightly less. There were some reports that the CH3SNAS uses a 88F5182 chip whereas the D-Link DNS-323 uses a 88F5181 but nobody was able to confirm this for sure. As it turns out, there are two revisions of the DNS-323. Revision A1 uses a 88F5181 chip along with a separate SATA chip. Revision B1 on the other hand uses a 88F5182 which integrates SATA into the SoC itself. The CH3SNAS is equivalent to a D-Link DNS-323 revision B1. In fact, the mainboard of the CH3SNAS even says DNS-323 rev B1.

The Linux kernel from mainline only works on the DNS-323 revision A1 at the moment. Adding support for revision B1 should be trivial since you basically just need to initialize the SATA chip on the SoC and possibly adapt some other values. Unfortunately, the DNS-323 uses a very small serial connector so none of my serial cables work. If I can find someone who will make a serial cable for me, I'll fix up the kernel.

Differences between paid and volunteer FOSS contributors

There's a lot of debate these days about the impact of the increasing number of paid developers in FOSS communities that started as volunteer efforts and still have significant numbers of volunteers. Evangelia Berdou's PhD thesis "Managing the Bazaar: Commercialization and peripheral participation in mature, community-led Free/Open source software projects" contains a contains a wealth of information and insights about this topic.

Berdou conducted interviews with members of the GNOME and KDE projects. She found that paid developers are often identified with the core developer group which is responsible for key infrastructure and often make a large number of commits. Furthermore, she suggested that the groups may have different priorities: "whereas [paid] developers focus on technical excellence, peripheral contributors are more interested in access and practical use".

Based on these interviews, she formulated the following hypotheses which she subsequently analyzed in more detail:

1. Paid developers are more likely to contribute to critical parts of the code base.
2. Paid developers are more likely to maintain critical parts of the code base.
3. Volunteer contributors are more likely to participate in aspects of the project that are geared towards the end-user.
4. Programmers and peripheral contributors are not likely to participate equally in major community events.

Berdou found all hypotheses to be true for GNOME but only hypothesis two and four were confirmed for KDE.

In the case of GNOME, Berdou found that hired developers contribute to the most critical parts of the project, that they maintained most modules in core areas and that they maintained a larger number modules than volunteers. Two important differences were found in KDE: paid developers attend more conferences and they maintain more modules.

Berdou's research contains a number of important insights:

• Corporate contributions are important because paid developers contribute a lot of changes, and they maintain core modules and code.
• While it's clear that the involvement of paid contributors is influenced by the strategy of their company, Berdou wonders whether another reason why they often contribute to core code is because they "develop their technical skills and their understanding of the code base to a greater extent than volunteers who usually contribute in their free time". It's therefore important that projects provide good documentation and other help so volunteers can get up to speed quickly.
• Since many volunteers cannot afford to attend community events, projects should provide travel funds. This is something I see more and more: for example, Debian funds some developers to attend Debian conference and the Linux Foundation has a grant program to allow developers to attend events.
• Paid developers often maintain modules they are not paid to directly contribute to. A reason for this is that they continue to maintain modules in their spare time when their company tells them to work on other parts of the code.

Personally, I'm also wondering why there are some significant differences between GNOME and KDE. For example, about 67% of contributors where paid to work on GNOME whereas only 38% were paid to work on KDE. Why do some projects attract more commercial involvement whereas other projects continue to be mostly volunteer based? [Update: it was pointed out that I made a mistake with these numbers. In reality, 37% of GNOME developers are paid to work on GNOME or GNOME and other FOSS according to the PhD thesis. The number for KDE was correct (38%). So according to this PhD there is not a lot of difference in the percentage of paid developers between GNOME and KDE. My original question still stands though because there are many projects that don't attract much commercial involvement at all.]

(Originally published on FOSSBazaar)

Choosing language during NAS installations

When the Debian installer starts, it will normally ask you what language you want to install in and gives you a really long list of languages to choose from. Unfortunately, that's not how it works when you install on a headless NAS device. The reason for this is that installations on NAS devices are done via SSH and the installer normally brings the network up after asking the user for their language. So we'd simply skip the language question and go with English.

When Timo Jyrinki recently mentioned that he couldn't install in Finnish, Frans Pop pointed out that localechooser has changed a lot since etch and that it should now be possible to have the language question after setting up the network. This turned out to be correct and I managed to choose a different language but the installer still showed everything in English. Frans reminded me that the installer drops all translations that are not necessary but unfortunately this happens too early in our case. He pointed me to a variable that determines whether translations will be dropped. So in lenny translations will not be dropped on NAS devices that have enough RAM and users will be asked when they connect to the installer via SSH what language they'd like to install in.

Required installer modules installed automatically on NSLU2 now

One thing that's annoying about the Debian installer on the NSLU2, as Joey Hess pointed out a few months ago when he reinstalled his NSLU2, is that you have to manually select some installer modules. The reason for this is that the NSLU2 only has 32 MB of RAM and thus Debian installer runs in lowmem mode in which less functionality is enabled by default. As a result of this, you have to select partman-auto, partman-ext3 and usb-storage-modules from a list of additional installer modules to load so the installer will recognize your USB disk, offer guided partitioning and format the disk with ext3.

When Joey submitted his installation report, Frans Pop suggested that we could put a list of installer modules we want to have installed automatically in /var/cache/anna/include. I finally got around to playing around with this yesterday and while this specific solution won't work on lowmem systems we can use anna-install to automatically queue installer modules for installation. Since anna-install works, I put some functionality into the installer today to automatically load required installer modules on the Linksys NSLU2 and on Cobalt machines.

This change is good for users because they don't have to select some modules during the installation. I'm a little bit worried that existing users of the installer on the NSLU2 will be confused when they don't see the modules in the list anymore, but I'll update the documentation accordingly.

Status of GCC 4.4 on Debian

I started testing GCC 4.4 ten days ago and the results are in. In total, I filed 28 new bugs and ran into 7 known issues. More than half of the bugs I filed have already been fixed and many of those that are still open have already received some attention (debugging, suggested patches). I sent a complete report to the GCC list, although I forgot to mention that I set optimization to -O3 to catch more problems.

I know that it's quite popular to complain about GCC, but I'd say that most impressions people have about the GCC community no longer hold true. The GCC community is very active these days, bugs tend to get a lot of attention and there are major infrastructure changes underway to improve the compiler.

More robust oldsys-preseed uploaded

To follow-up on my recent posting about oldsys-preseed: I uploaded a new version of netcfg yesterday that can be told via preseeding not to use a gateway. I also uploaded a new version of oldsys-preseed that will properly handle static IP configs without a netmask or gateway. Additionally, oldsys-preseed will now use DHCP when the static IP config is incomplete (i.e. when either the IP address or DNS are missing). This makes oldsys-preseed much more robust and will hopefully get rid of many problems users ran into with etch.

Visiting Marvell in Yoqne'am

Since I'm spending the summer in Israel, I used the opportunity yesterday to visit the Marvell site in Yoqne'am where the majority of the NAS software engineering team is located. They first gave me a tour through the hardware labs which I found incredibly interesting. It's amazing what equipment those folks use to perform QA tests of new chips, how they can reverse engineer chips and even make some modifications to test their theories about hardware bugs. I found it really amazing and at the same time felt glad that changing things in software is so easy.

Later they showed me the software lab where they had an exhibition of various devices based on the Orion chip, including NAS devices, but also printers, wifi APs and game machines. They also told me some details of their product roadmap but unfortunately I cannot share that information. The only thing I can say is that we'll see a lot of new devices to which it would be cool to port Debian.

The European Open Source Observatory and Repository (OSOR)

I attended the Open Nordic Conference in Norway in June, a conference that brought together people from Norway, Sweden, Finland and Denmark. Attending the conference allowed me to find out what's going on with FOSS in Northern Europe. What I found interesting is that there was a lot of talk about using FOSS in the public sector. A number of countries are working on repositories to exchange software, in particular for public administration.

One example is the Software Exchange (softwareborsen.dk), a project by the Danish National Software Knowledge Centre to promote the use of FOSS in Danish public administration. The Norwegian software exchange (delingsbazaren.no) plays a similar role in Norway.

It seems as if every country is working on their own software repository to share FOSS. As such, it's great to see that the European Commission is taking a step towards bringing everyone together by introducing the Open Source Observatory and Repository (OSOR) for European public administrations. The goal of OSOR is to provide a platform for the exchange of information, experiences and FOSS code for the use in public administrations.

OSOR will officially launch at the Open Source World Conference in Malaga in October. I've always thought that there is not enough cooperation and communication between European countries, so I have high hopes for OSOR.

(Originally published on FOSSBazaar)