Saturday, November 24, 2007

Week 4: Source Code Reading Lab

Bookmarks > Bookmark This Page...

Search Using LXR:
I selected the "Bookmarks > Bookmark This Page..." top-level browser action to search and trace through the code to better understand where the code is located and how it works. I started with a text search at http://lxr.mozilla.org/mozilla/ . My search produced the following resuls:

/browser/locales/en-US/chrome/browser/browser.dtd, line 58
<!ENTITY addCurPageAsCmd.label "Bookmark This Page...">

/browser/locales/en-US/chrome/browser/browser.dtd, line 234
<!ENTITY bookmarkPageCmd.label "Bookmark This Page...">

/browser/locales/en-US/chrome/help/menu_reference.xhtml, line 303
<h3 id="add_to_bookmarks">Bookmark This Page...</h3>

/browser/locales/en-US/chrome/help/firebird-toc.rdf, line 360
<rdf:li><rdf:Description ID="menu-bookmarks-add-to-bookmarks" nc:name="Bookmark This Page..." nc:link="menu_reference.xhtml#add_to_bookmarks"/> </rdf:li>

The first two results are entity references within the browser.dtd file of the Mozilla browser and appear to be more relevant to the "Bookmark This Page.." functionality of the browser. The third result is an XHTML file containing help information from the browser's help menu. The fourth result appears to be more relevant to Firebird.

Next, I searched using the term "bookmarkPageCmd.label". One of the results was in the browser directory:

/browser/base/content/browser-context.inc, line 127
label="&bookmarkPageCmd.label;"

I found that the oncommand attribute of the <menuitem> called a function named bookmarkThisPage(), so I did a search of this function. I found a JavaScript file (/browser/base/content/nsContextMenu.js, line 1142) with the following code:

bookmarkThisPage: function CM_bookmarkThisPage() {
window.top.PlacesCommandHook.bookmarkPage(this.browser,
PlacesUtils.bookmarksMenuFolderId, true);
},

I saw that this function called another function named bookmarkPage() so I searched for that function and I found a JavaScript file (/browser/base/content/browser-places.js, line 173) with the following function definition:

/**
* Adds a bookmark to the page loaded in the given browser.
*
* @param aBrowser
* a element.
* @param [optional] aParent
* The folder in which to create a new bookmark if the page loaded in
* aBrowser isn't bookmarked yet, defaults to the unfiled root.
* @param [optional] aShowEditUI
* whether or not to show the edit-bookmark UI for the bookmark item
*/
bookmarkPage: function PCH_bookmarkPage(aBrowser, aParent, aShowEditUI) {
var uri = aBrowser.currentURI;
var itemId = PlacesUtils.getMostRecentBookmarkForURI(uri);
if (itemId == -1) {
// Copied over from addBookmarkForBrowser:
// Bug 52536: We obtain the URL and title from the nsIWebNavigation
// associated with a rather than from a DOMWindow.
// This is because when a full page plugin is loaded, there is
// no DOMWindow (?) but information about the loaded document
// may still be obtained from the webNavigation.
var webNav = aBrowser.webNavigation;
var url = webNav.currentURI;
var title;
var description;
try {
title = webNav.document.title;
description = PlacesUtils.getDescriptionFromDocument(webNav.document);
}
catch (e) { }

var parent = aParent != undefined ?
aParent : PlacesUtils.unfiledBookmarksFolderId;
var descAnno = { name: DESCRIPTION_ANNO, value: description };
var txn = PlacesUtils.ptm.createItem(uri, parent, -1,
title, null, [descAnno]);
PlacesUtils.ptm.commitTransaction(txn);
if (aShowEditUI)
itemId = PlacesUtils.getMostRecentBookmarkForURI(uri);
}

if (aShowEditUI) {
// dock the panel to the star icon when possible, otherwise dock
// it to the content area
if (aBrowser.contentWindow == window.content) {
var starIcon = aBrowser.ownerDocument.getElementById("star-button");
if (starIcon && isElementVisible(starIcon)) {
this.showEditBookmarkPopup(itemId, starIcon, "after_start");
return;
}
}
this.showEditBookmarkPopup(itemId, aBrowser, "overlap");
}
},

Note about LXR: It provides an option for regular expression searches.

Search Using MXR:
I conducted a similar text search for the text "Bookmark This Page..." using the Firefox branch at MXR (http://mxr.mozilla.org/firefox/). I followed a similar progression during my search through files that were the same as that of my previous search. I found this search mechanism to be more helpful than LXR because it showed a summary of where the bookmarkPage() function is defined and in what files it is referenced:

Defined as a function in: Referenced (in 3 files total) in:
Like LXR, MXR provides an option for regular expression searches and it allows for case sensitive searches. MXR also enables users to limit the search results to certain types of files, specific output patterns and to specify what tree to search.

Search Using MXR-test:
This search yielded very similar results as the MXR search. I used the Mozilla starting point (http://mxr-test.landfill.bugzilla.org/mozilla/).

Defined as a function in:Referenced in: MXR-test enables users to narrow their search results similarly to MXR but it provides more file-types to select for refining the search and it also allows limiting the number of matches per file.

Sunday, November 18, 2007

FSOSS 2007 Keynote: Applying Open Source Concepts to Non-Software Industries

The keynote speaker on Day 1 of FSOSS 2007 was Bob Young, Co-founder of RedHat and Founder/CEO of Lulu.com. The topic intrigued me and I found it interesting to consider how industries outside of software development, to varying degrees, implement concepts that are similar to those of the Open Source Development Model and those of the Proprietary Software Development Model.

Among various other topics in his talk, Bob Young discussed how some sports, particularly North American football, may be viewed as having been developed using a form of open source model because it was derived conceptually from the British sport of rugby. He also talked about the legal system and the way that lawyers use open source concepts to freely construct arguments based upon the work of other lawyers and legal precedents.

Bob Young's address to the audience was fairly informal and entertaining. I had the expectation that he would have gone more in-depth into the subject matter and I had hoped to have gained more information and knowledge from his talk. Nonetheless, it gave me a new perspective on non-software industries and how they relate conceptually to the open source - proprietary software development continuum.

FSOSS 2007: Open Commercial Development

I attended an interesting presentation titled, "Open Commercial Development", on Day 1 of the Free Software & Open Source Symposium (FSOSS) held at Seneca College in Toronto. The title of the presentation alone piqued my curiosity and prompted me to attend this discussion. It was presented by Lawrence Mandel and Jeffrey Liu, both of whom are software developers involved with IBM Rational Software at the IBM Toronto Lab. They described Open Commercial Development as a new hybrid model that incorporates some characteristics of the Open Source Model and some characteristics of the proprietary software development model.

They gave their presentation in two parts. In the first part of the presentation, Mandel reviewed the main criteria for Open Source Software Development and he discussed "what is" open source as outlined by the Open Source Initiative and others.

Part two of the discussion was presented by Liu wherein he attempted to describe the Open Commercial Development Model to the audience. Some concepts pertaining to the model were unclear to me. My understanding of their model is that Open Commercial Development is an attempt to build communities around software projects through direct contributions by the software's users. My sense is that they want users of the software to provide feedback about the software, particularly when encountering bugs, and suggest fixes that would potentially be integrated back into the software by IBM's developers.

The presenters said that Open Commercial Development is NOT Open Source Development and Liu gave an analogy of it being like an open-kitchen style of restaurant where customers can see the ingredients of the menu items and see how the food is prepared, but they are not directly involved in the cooking of the food. In the case of Open Commercial Development, customers, or users of the software would have access to the code and they would be able to submit software code and bug fixes for review by IBM's developers. However, my understanding is that people who submit code to IBM would not receive any commercial or financial benefit from doing this, and IBM would retain proprietary ownership of the code.

Some of this appeared to be contradicted by a few of the misconceptions that the presenters mentioned about their software development model. They said that it was not the intent of the Open Commercial Development Model to get contributors to work for free, to steal the intellectual property of contributors, or to make money from code contributed by the open source community. Despite these reassurances, there was evidence of skepticism from audience members about the Open Commercial Development Model's intent and its feasibility during the audience discussion that followed the presentation.

A couple of IBM's Open Source Development Projects that were briefly mentioned are Project Zero and IBM's Jazz.

Friday, November 16, 2007

Week 3: Managing and Building Large Source Trees

Debug Mozilla Build

I was not able to do a debug build of Firefox on my home PC because it did not meet the necessary hardware requirements for a Windows Build that are listed at the mozilla developer center's web site. However, I was able to obtain a small, but powerful laptop through my instructor for OSD600 and the Office of Research and Innovation (ORI) at Seneca College. The laptop is a Dell XPS M1210 with an Intel Core 2 2.0 GHz CPU, 1 GB of RAM and a 150 GB hard drive. This laptop had a somewhat buggy installation of Windows XP Professional on it that seemed to be related to Windows Explorer.

In order to do the build in Windows XP Pro, I installed Microsoft's Visual Studio 2005 as part of the Software Requirements and the Mozilla Build 1.1 Package, which contains all the other software prerequisites necessary for building Mozilla. I carefully followed the instructions to checkout the latest Firefox source using CVS and the instructions to build Mozilla according to the Mozilla Developer Center (MDC). However, when I tried using the make command to checkout and build Firefox, a sh.exe.stackdump file was generated with a STATUS_ACCESS_VIOLATION exception that appeared to be related to MSYS. This occurred on several attempts to build Firefox. I did not retain a copy of the file, otherwise I would display its contents here.

Hats off to Fedora 7!

After a number of unsuccessful attempts to build Firefox in Windows XP Pro and not knowing how to solve the problem with MSYS generating a sh.exe.stackdump STATUS_ACCESS_VIOLATION exception, I decided to try an alternate build platform, Fedora 7. I received assistance from my instructor, Chris Tyler, to partition my hard drive using a copy of GParted-LiveCD. Firstly, I tried installing the 32-bit Live CD version of Fedora 7 but there were problems with the installation so Chris installed the Fedora 7 Live x86_64 version instead. Everything appeared to be alright with this installation of Fedora 7 but I received error messages again when I tried to build Firefox.

Eventually, out of frustration I decided to download, burn and install the full Fedora 7 x86_64 DVD version for 64-bit PCs. Again, I followed the checkout and build instructions for a debug build of Firefox and this time the build was successful. Below are the contents of my .mozconfig file and the build details for Firefox (Minefield):

.mozconfig
#
# See http://www.mozilla.org/build/ for build instructions.
#
# Options for client.mk.
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-debug
mk_add_options MOZ_MAKE_FLAGS=-j4

# Options for 'configure' (same as command-line options).
ac_add_options --enable-application=browser
ac_add_options --enable-pango

# Debug options for 'configure'
ac_add_options --enable-debug
ac_add_options --disable-optimize

about:
Copyright © 1998-2007 by contributors to the Mozilla Project.
Read the licensing information for this product.
Read the release notes for this version.
See the build configuration used for this version.
Build identifier: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b2pre) Gecko/2007111515 Minefield/3.0b2pre

about:buildconfig

Build platform

target
x86_64-unknown-linux-gnu

Build-tools

Compiler
gcc
Version
gcc version 4.1.2 20070925 (Red Hat 4.1.2-27)
Compiler flags
-Wall -W -Wno-unused -Wpointer-arith -Wcast-align -Wno-long-long -pedantic -pthread -pipe

Compiler
c++
Compiler
gcc version 4.1.2 20070925 (Red Hat 4.1.2-27)
Compiler flags
-fno-rtti -fno-exceptions -Wall -Wconversion -Wpointer-arith -Wcast-align -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wno-long-long -pedantic -fshort-wchar -pthread -pipe

Configure arguments
--enable-application=browser --enable-pango --enable-debug --disable-optimize

The elapsed times for the debug build of Firefox on Fedora 7 x86_64 are shown below:

real 18m 40.126s

user 15m 15.208s

sys 3m 8.559s

Back to Windows XP Professional

I was dissatisfied with my earlier unsuccessful attempts to build Firefox on Windows XP Pro so I decided to do a clean installation of the operating system and Visual Studio 2005. However, again I was unsuccessful at building Firefox and a sh.exe.stackdump file with a STATUS_ACCESS_VIOLATION exception was still being generated. Finally, I decided to do a Google search for this exception and I found some helpful information at the following web site:

http://www.mingw.org/MinGWiki/index.php/FAQ


Why does make often crash creating a sh.exe.stackdump file when I try to compile my source code?
If you experience random crashes with make as well as with the msysinfo command
(i.e., repeating the command often allows to compile the source code succesfully), the issue might be caused by the Logitech QuickCam software. Here is provided a solution to this issue:
type the command services.msc from a command prompt (you will need administrator privileges to do this), then find the "Logitech Process Monitor" service, and change the "Startup type" from "Auto" to "Manual". Do the same with the "LVSrvLauncher" service. type the command regedit from the command prompt (you will need administrator privileges
to do this), then find the key
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run. Among the other REG_SZ keys, there should be three of them related to the Logitech QuickCam, namely: "LogitechCommunicationsManager", "LogitechQuickCamRibbon", "LVCOMSX". Make a backup of the registry, then remove them.
reboot your PC
In this way the Logitech QuickCam deamon won't be available anymore. Nonetheless, the camera will keep working just ok with 3rd party software (e.g., Skype). If you want a less drastic solution, just kill the lvprcsrv.exe process whenever you want to use MSYS by typing at the command prompt: pskill -t lvprcsrv.exe (you will need PsTools and administrator privileges to do this). However, as soon as you will plug in the USB cable of your Logitech QuickCam, the deamon will be restarted, while with the first procedure it won't.

The laptop computer that I've been using for the builds has a built-in web cam and a Logitech QuickCam driver and software installed on it. Using this new information, I decided to uninstall the Logitech QuickCam driver and software. After doing this, I followed the checkout and build instructions for a debug build of Firefox and this time the build was successful. Below are the contents of my .mozconfig file and the about: details for Firefox (Minefield):

.mozconfig file
#
# See http://www.mozilla.org/build/ for build instructions.
#
# Options for client.mk.
mk_add_options MOZ_CO_PROJECT=browser
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-debug
mk_add_options MOZ_MAKE_FLAGS=-j4

# Options for 'configure' (same as command-line options).
ac_add_options --enable-application=browser

# Debug options for 'configure'
ac_add_options --enable-debug
ac_add_options --disable-optimize


about:
Copyright © 1998-2007 by contributors to the Mozilla Project.
Read the licensing information for this product.
Read the release notes for this version.
See the build configuration used for this version.
Build identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b2pre) Gecko/2007111512 Minefield/3.0b2pre

about:buildconfig

Build platform

target
i686-pc-mingw32

Build tools
Compiler

cl
Version
14.00.50727
Compiler flags
-TC -nologo -W3 -Gy -Fd$(PDBFILE)

Compiler
cl
Version
14.00.50727
Compiler flags
-GR- -TP -nologo -Zc:wchar_t- -W3 -Gy -Fd$(PDBFILE)

Configure arguments

--enable-application=browser --enable-debug --disable-optimize


The elapsed times for the debug build of Firefox on Windows XP are shown below:

real 38m 47.610s
user 13m 6.165s
sys 9m 26.050s

Monday, November 5, 2007

Week 2: Reaction to "The Cathedral and the Bazaar"

In the article, "The Cathedral and the Bazaar" by Eric Raymond, the author discusses how he used his own open-source project, fetchmail, in an attempt to parallel Linux's development history and to test some theories about software engineering. He discusses these theories from two distinct perspectives, one that he calls the "cathedral" model, which represents the commercial world of software development, and a dichotomous view that he refers to as the "bazaar" model, representing the Linux world of software development. Raymond argues that these opposing models come from two very different assumptions about the software debugging process.

I agree that these two models approach software debugging from different perspectives. The bazaar model views software bugs as "shallow" and software problems to be transparent, whereas the cathedral-building model views bugs as "deep" and difficult to trace. There appears to be a number of socio-economic forces that influence the reasons for these two opposing perspectives.

The cathedral-building style of commercial software development is typically characterized by small to medium sized teams working on a large project in terms of a structured software development process, e.g., analysis, design, development, testing, debugging, documentation, and deployment. Often in large scale projects, these tasks are separated among different work units. In this model, the leadership style of the project manager is usually authoritative and tasks are delegated to subordinates. Software developers often work on projects where they have little or no interest in the end result. In a cathedral-building scenario, software is released infrequently with the goal being "perfect" software. This stems from the broader business goals of profit maximization and customer satisfaction through "bug-free" software.

In contrast, the mantra of the bazaar model is "release early, release often". This strategy serves two important purposes. Firstly, it reduces the duplication of debugging work because bug "fixes" are quickly propagated back into the next release. Secondly, it perpetuates interest and developer motivation in the project because code contributors are able to see the results of their efforts and they receive recognition from their peers within the community through feedback. Within this model, developers tend to be motivated and committed to the project because they have a vested interest in the outcome; the software serves an important functional and practical purpose to them. In many instances, hackers are also users of the software too.

The bazaar model loosely adopts a sociological approach known as the Delphi Method and incorporates it in its approach to debugging. Raymond refers to this as Linus' Law and says that "Given enough eyeballs, all bugs are shallow." In effect, the bazaar model harnesses the expertise of a software project's large community to fix bugs and resolve software issues.

The leadership style in the bazaar model is that of a project coordinator. Rather than delegate tasks and set time-lines for project targets, the project coordinator attracts the interest of software users and developers by effectively communicating the project's potential and by fostering a sense of community around a project.

Each model appears to be sound given the appropriate context. I think that a software development process, such as an Agile Software Development framework for software engineering is a better model for commercial software development in instances when requirements are newly emerging and rapidly changing rather than the cathedral-building model described by Raymond. Some of the principles of agile software development include customer satisfaction through rapid, continuous delivery of useful software, close cooperation between business people and developers, projects built around motivated individuals, the existence of self-organizing teams, and regular adaptation to changing circumstances.

The bazaar model appears to require certain preconditions to exist to be successful in an open source software development context. One precondition is a large pool of software users and developers with the desire to collectively solve software problems that are of interest to them. Another important precondition is the existence of a coordinator who can form a sustainable community by rewarding the egos of developers and by capturing and maintaining interest in the project through the effective communication of their vision.