Kerry's Open Source@Seneca Blog

Friday, February 15, 2008

Mozilla Project Update: Release v.03

Mozilla Project: Localized Search in Firefox Search Box - Release v.03

My objective for this release was to dynamically "Remove" a search engine that Firefox 3 (i.e. Minefield) dynamically "Added" from a web site with a search plugin. I achieved this by building on my work from Release v.02, which dynamically "Adds" a search engine when Firefox 3 loads a web page with a search plugin.

The dynamic addition and removal of search engines presents some challenges when considering the user interface design. A few things that I considered were how to change the functionality of the searchbar while keeping it helpful and intuitive to use. I believe that it is important to keep features relatively simple and easy to use while providing some good utilitarian value to the user's web browsing experience, and not making the feature too obtrusive to the user. There are probably many Firefox users who are not aware of the searchbar's current ability to auto-detect OpenSearch plugins. Hopefully, by extending the searchbar's functionality, it will enable users to gain more benefit from it.

I arrived at a release that dynamically "Adds" a search engine when Firefox 3 navigates to a site with an OpenSearch search plugin. The newly added engine appears as the current engine on the searchbar and it can be used to search the web site. Click on images below to enlarge them.



If still on the site with the search plugin, the search engine is placed on the list of available engines as an "Add <Search Engine>" searchbar menu item when the user opens the searchbar popup.



When the user navigates to a different web site, the search engine will be dynamically removed and will not appear on the searchbar menu.



The user can navigate to a web site with an OpenSearch plugin and manually "Add" the search engine.



The manually added engine will not be dynamically removed from the installed engines list on the searchbar when the user navigates to a different web site.



The user may "Remove" the search engine as usual.



The removed engine appears as an "Add <Search Engine>" searchbar menu item when the user opens the searchbar popup.



The removed search engine will not appear on the searchbar menu after the user navigates to a different web site.




Excerpts of JavaScript code from the patch file are discussed and shown below.

Patch File Code Discussion - Release v.03

File: browser.js

addEngine() function:

If the auto-detected search plugin is NOT on the search service's list of engines, then dynamically "Add" the search engine and assign it to the addedEngine local variable. Set the search service's dynamicEngine attribute to the value of the newly added engine.

var searchService = Cc["@mozilla.org/browser/search-service;1"].
getService(Ci.nsIBrowserSearchService);
if (searchService.getEngineByName(engine.title))
hidden = true;
else {
// Dynamically "Add" the web site's search engine plugin.
var addedEngine = searchService.addEngine(engine.href, Components.interfaces.nsISearchEngine.DATA_XML, iconURL, false);
if (addedEngine) {
searchService.dynamicEngine = addedEngine;
hidden = true;
}
}

startDocumentLoad() function:

When the user navigates to a different web site, get an instance of nsIBrowserSearchService and assign it to the variable searchService. Get the search service's dynamicEngine attribute and assign it to the dynamicEngine local variable. Then, if dynamicEngine has a non-null value, use the search service's removeEngine() function to remove it from the list.

var searchService = Cc["@mozilla.org/browser/search-service;1"].
getService(Ci.nsIBrowserSearchService);
var dynamicEngine = searchService.dynamicEngine;

// Remove the dynamically added search engine if it is on the list.
if (dynamicEngine)
searchService.removeEngine(dynamicEngine);

File: nsIBrowserSearchService.idl

addEngine() Interface:

Change the return type of the interface from void to an nsISearchEngine object.

* @returns the created engine.
*/

nsISearchEngine addEngine(in AString engineURL, in long dataType, in AString iconURL,
in boolean confirm);

Create an attribute named dynamicEngine of the type nsISearchEngine to hold the value of a dynamically added search engine that can be set to null when the engine is removed.

/** The dynamically added search engine. Set to null when
* the dynamically added engine is removed.
*/
attribute nsISearchEngine dynamicEngine;

File: nsSearchService.js

Create a field named _dynamicEngine and initialize it to null.

SearchService.prototype = {
_engines: { },
_sortedEngines: null,
// Whether or not we need to write the order of engines on shutdown. This
// needs to happen anytime _sortedEngines is modified after initial startup.
_needToSetOrderPrefs: false,

// The dynamically added engine.
_dynamicEngine: null,

addEngine() function:

Return an nsISearchEngine object.

return engine;
},
removeEngine() function:

Set the _dynamicEngine field's value to null when an engine is removed.

this._dynamicEngine = null;

Get and set the value of the dynamicEngine attribute's value.

// Get the dynamically added engine.
get dynamicEngine() {
return this._dynamicEngine;
},
// Set the dynamically added engine.
set dynamicEngine(val) {
ENSURE_ARG(val instanceof Ci.nsISearchEngine,
"Invalid argument passed to dynamicEngine setter");
this._dynamicEngine = val;
},

File: search.xml

Create the dynamicEngine property to get the dynamic engine's value from the search service.

<!-- Returns the dynamicEngine from the search service. -->
<property name="dynamicEngine" readonly="true">
<getter><![CDATA[
var dynamicEngine = this.searchService.dynamicEngine;
return dynamicEngine;
]]></getter>
</property>

rebuildPopupDynamic() method:

When navigating to a site with an OpenSearch plugin and the user clicks the searchbar's button to display the engine list, remove the dynamically added search engine if it is on the list so that it will display as a searchbar "Add <Search Engine>" menu item. Use the search service removeEngine() function to remove the dynamically added search engine if it has a non-null value.

// Remove the dynamically added search engine if it is on the list when
// the popup's menu items are first displayed. The user may then choose
// to add the "Add " item to the list of installed engines.
if (this.dynamicEngine)
this.searchService.removeEngine(this.dynamicEngine);

Wednesday, February 13, 2008

Mozilla Project Update: Request for Project Contributions

Mozilla Project: Localized Search in Firefox Search Box - How to Contribute

I require individuals to add links to to my project's wiki page for web sites with OpenSearch plugins that can be "Added" to Firefox's searcbar list of search engines. Web sites with OpenSearch plugins cause Firefox's searchbar drop-down list button to turn blue in color when the browser "auto-discovers" a site with this type of search engine plugin. See the web site list on my project's wiki page for examples of these types of sites.

A web site that supports auto-discovery of a search plugin has a <link> tag in the <head> section of the web page that uses the following format:

<link rel="search" type="application/opensearchdescription+xml" title="searchTitle" href="pluginURL">

Thanks for your contribution to my project!

Monday, February 11, 2008

Mozilla Project Update: Release v.02

Mozilla Project: Localized Search in Firefox Search Box - Release v.02

My objectives for this release were to dynamically "Add" a search engine plugin when Firefox 3 (i.e. Minefield) loads a web page with a search plugin, and then propagate the search engine to the top of the searchbar's menu as the current engine. The propagation of a newly added search engine as the current engine in the searchbar's menu was already a behavioral feature of Firefox 3 so there were no changes required for this part of my release target. Determining how to dynamically "Add" an available search engine plugin was challenging to achieve, but in the end, it only required a few minor modifications to the code in the browser.js file.

Source Code Description and Modification

As I mentioned in my previous discussion of the source code for searchbar events, when Firefox 3 loads a web page containing a <link> element, a "DOMLinkAdded" event fires. The browser has an event listener for "DOMLinkAdded" events and it employs an event handler named DOMLinkHandler. This event handler calls its onLinkAdded() function, which creates a generic engine object if the page's <link> element's attributes have valid values. Subsequently, the engine object is passed to the BrowserSearch object's addEngine() function.

During the normal flow of code execution, the addEngine() function receives a reference to an engine object as one of its parameters. The function uses searchService, an nsIBrowserSearchService object, to determine if the search engine is already on the list of engines. If an nsISearchEngine object with a name value matching the generic engine object's title property is already on the list, then it is considered to be a "hidden" engine and it is "pushed" onto the browser.hiddenEngines array. This array is used by the search.xml file to determine how to populate and display the search engines on the searchbar menu. If the search engine is not on the list, it gets pushed onto the "non-hidden", browser.engines array. In this case, the search engine would then be displayed as an "Add <Search Engine>" item on the searchbar menu and the searchbar button's background color would be changed to blue after a call to the updateSearchButton() function.

My code modifications change the BrowserSearch addEngine() function by adding the new search plugin if it is NOT found on the existing list of engines. I dynamically "Add" the search plugin by calling the addEngine() function defined in the nsSearchService.js file. This function accomplishes the important step of creating a new nsISearchEngine object. It is this type of object that is required by methods in search.xml, such as observe(), offerNewEngine() and hideNewEngine(). The remainder of the code in the BrowserSearch addEngine() function follows the normal paths of execution.

My additions to the code in the browser.js file are denoted with plus signs in the patch file shown below:

? localsearchpatch_v02.txt
? nohup.out
? objdir-ff-debug
Index: browser/base/content/browser.js
===================================================================
RCS file: /cvsroot/mozilla/browser/base/content/browser.js,v
retrieving revision 1.961
diff -u -8 -p -r1.961 browser.js
--- browser/base/content/browser.js 10 Feb 2008 06:57:05 -0000 1.961
+++ browser/base/content/browser.js 11 Feb 2008 19:57:26 -0000
@@ -2780,16 +2780,19 @@ const BrowserSearch = {
// If this engine (identified by title) is already in the list, add it
// to the list of hidden engines rather than to the main list.
// XXX This will need to be changed when engines are identified by URL;
// see bug 335102.
var searchService = Cc["@mozilla.org/browser/search-service;1"].
getService(Ci.nsIBrowserSearchService);
if (searchService.getEngineByName(engine.title))
hidden = true;
+ else
+ // Dynamically "Add" the web site's search engine plugin.
+ searchService.addEngine(engine.href, Components.interfaces.nsISearchEngine.DATA_XML, iconURL, false);

var engines = (hidden ? browser.hiddenEngines : browser.engines) || [];

engines.push({ uri: engine.href,
title: engine.title,
icon: iconURL });

if (hidden)
See my project wiki page for information about how to apply and use this patch file.

Difficulties and Lessons Learned

Although the solution was straightforward for what I was essentially attempting to achieve with this release, I had some difficulties understanding how to arrive at it. From the outset, I understood that I would likely need to make changes to BrowserSearch's addEngine() function. My first thoughts were to pass the generic engine object to the searchbar's hideNewEngine() or observe() functions as shown in the following code examples:

Example 1

if (hidden)
browser.hiddenEngines = engines;
else {
browser.engines = engines;
this.searchBar.hideNewEngine(engine);
if (browser == gBrowser || browser == gBrowser.mCurrentBrowser)
this.updateSearchButton();
}

Example 2

if (hidden)
browser.hiddenEngines = engines;
else {
browser.engines = engines;
this.searchBar.observe(engine, "browser-search-engine-modified", "engine-added");
if (browser == gBrowser || browser == gBrowser.mCurrentBrowser)
this.updateSearchButton();
}
The results of these code modifications were that the search plugins were not dynamically added and they remained as "Add <Search Engine>" items on the searchbar menu. The following JavaScript error message was produced from the above code changes:

************************************************************
* Call to xpconnect wrapped JSObject produced this error: *
[Exception... "'[JavaScript Error: "aEngine.wrappedJSObject is undefined" {file: "chrome://browser/content/search/search.xml" line: 256}]' when calling method: [nsIDOMEventListener::handleEvent]" nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "<unknown>" data: yes]
************************************************************

I later learned from Gavin on the #seneca irc channel that I was attempting to pass generic engine objects to the functions in search.xml. However, these functions work with nsISearchEngine objects and it was ineffective to use a searchBar object (i.e. this.searchBar) to pass them to the functions in search.xml. I needed to use an nsIBrowserSearchService object to "Add" the search engine to the search service's list because the search service only deals with nsISearchEngine objects.

I also made unsuccessful attempts to dynamically load the search plugin by simply adding the generic engine object to the browser.hiddenEngines array in BrowserSearch's addEngine() function and by removing any code that would add it to the browser.engines array. However, these code changes and similar other ones did not produce the desired results for this release.

Saturday, February 2, 2008

Mozilla Project Update: Release v.01

Mozilla Project: Localized Search in Firefox Search Box - Release v.01
My main objective for this initial release is to demonstrate a basic understanding of how some of the search box (searchbar) features currently work in the Firefox browser by examining the source code. I attempt to achieve this by placing JavaScript dump() statements within the existing code. These statements output diagnostic messages to the terminal window when running a DEBUG build of Firefox 3 (i.e. Minefield). The messages contain information about what methods or functions are being called and a brief explanation about what is happening in the source code when certain searchbar events occur. These events are listed here and the more pertinent ones to my project are discussed below:
  • The browser auto-detects a site that has a search plugin available.
  • The user selects the button to display the search engine list when a search plugin is available.
  • The user selects the option to "Add <EngineName>".
  • The user changes the current search engine by selecting it from the search bar drop-down list (menu).
  • The user moves the position of a search engine using the Search Engine Manager.
  • The user removes an auto-detected search engine from the list of "installed" search engines with the Search Manager.
I created a patch that inserts the dump() statements in the code. You may preview the patch at the following location: Patch File: localsearchpatch_v01.txt . You can also get information about how to download and apply the patch on my project wiki page.

Discussion of Source Code for SearchBar Events:
When the browser detects that a web site has a search engine plugin, there are a few things that happen in the browser.js code. The browser has an event listener named "DOMLinkAdded" that employs an event handler named DOMLinkHandler. My understanding of the code is that when Firefox loads a page containing a <link> element, it triggers a DOMLinkAdded type of event. The DOMLinkHandler calls its function, onLinkAdded(event), which determines the rel attribute value of the <link> element. If it has a value of search and its type, title and href attributes have valid values, an engine object is created and passed to the BrowserSearch object's addEngine(engine, targetDoc) function. This function first checks if the engine object is already in the browser's array of "hidden" engines, and if it is, then it's pushed to the end of the browser's hiddenEngines array. If the engine object is not in the hiddenEngines array, then it is added to the browser's engines array.

When a web page has a search plugin and the user clicks the button to display the searchbar menu, an event handler named "popupshowing" calls the rebuildPopupDynamic() method in search.xml. If the main popup menu items have not been added yet, rebuildPopupDynamic() calls the rebuildPopup() method, which rebuilds the list of visible search engines in the menu. Next, the rebuildPopupDynamic() method clears any existing "Add <EngineName>" menu items from the popup menu and assigns the browser's engines array to the addengines array. Then, it inserts the elements of the addengines array as menu items onto the searchbar menu as "Add <EngineName>" menu items.

Selecting the "Add <EngineName>" menu notifies the searchbar's observe(aEngine, aTopic, aVerb) method and it calls the hideNewEngine(aEngine) method, passing it a reference to the search engine object. The hideNewEngine(aEngine) method moves the engine object to each browser's "hidden" array so it is no longer offered to be added if the engine was auto-detected on the web page. This method iterates through all of the browser's open tabs and finds the selected engine in the browser's engines array. Then, it adds the search engine to the browser's hiddenEngines array and removes it from the browser's engines array. As a further explanation, browser.js fills two arrays of auto-detected search engines (browser.engines and browser.hiddenEngines). Those arrays contain unnamed JS objects that the searchbar uses to determine whether to show any "Add <EngineName>" menu items in the drop-down list.

When the user opens the Search Engine Manager Dialog Box, selects an auto-detected search engine from the "installed" list of engines and clicks the "Remove" button, the dialog's remove() function gets called in the engineManager.js file. The remove() function then passes a reference of the selected engine to an EngineView object's removeEngine(aEngine) function. This function removes the selected engine from the array of "Visible" engines using the index of the selected engine. It also adds an EngineRemoveOp object to an array. It is this object's commit() function that gets called when the user clicks the "OK" button in the Search Engine Manager Dialog. The commit() function uses an object reference to call the removeEngine(aEngine) function defined in nsSearchService.js . This function completes the search engine removal process by removing the engine from its internal store and then removes it from disk if the file can be removed.

Removing an auto-detected search engine also notifies search.xml's observe(aEngine, aTopic, aVerb) method, which calls the offerNewEngine(aEngine) method, passing it a reference to the search engine object. This method moves the search engine to each browser's active list so that it will be available to be added again if the engine that was just removed from the searchbar menu was auto-detected on the web page. Similar to the hideNewEngine(aEngine) method, offerNewEngine(aEngine) iterates through all of the browser's open tabs and finds the engine in the browser's hiddenEngines array. Then, it adds the search engine to the browser's engines array and removes it from the browser's hiddenEngines array.

Files:
browser.js
engineManager.js
nsIBrowserSearchService.idl
nsSearchService.js
search.xml

Saturday, January 26, 2008

Mozilla Project Update: Determining Direction - Part II

Mozilla Project: Localized Search in Firefox's Search Box
After giving a few ideas some lengthy consideration, I've decided to take the approach to extend Firefox's search box so that it automatically adds search engine plugins to the search bar's drop-down list and then propagates search plugins to the top of the list when the browser encounters a web site that offers a search engine plugin and an OpenSearch description file. I've decided on this approach because the scope of the project appears to fit with the time constraints with which I'm working, it challenges my knowledge and abilities, and much of the code exists for what I'm attempting to achieve with this project.

Wednesday, January 23, 2008

Mozilla Project Update: Determining Direction - Part I

Mozilla Project: Localized Search in Firefox's Search Box
After doing some initial research, I was uncertain about what direction to take with my project for enhancing, or extending Firefox's existing search box. To help me determine a direction for my project, I searched Bugzilla@Mozilla where I found a number of bugs that were related to my project. I've added some of the more relevant bugs to my project's wiki page. I also spoke with Chris Tyler, Dave Humphrey and consulted with Gavin Sharp through e-mail and on IRC to get some ideas for my project's direction and to determine the scope of my project. From these discussions and communication exchanges, a few ideas emerged.
One idea that emerged was to extend Firefox's search box so that it automatically adds a search engine plugin to the search bar's drop-down list and then propagates the search plugin to the top of the list when the browser encounters a web site that offers a search engine plugin and an OpenSearch description file. From reading some of the bugs at Bugzilla@Mozilla and from personal experience, it appears that most users have a preferred search engine to use as their default search engine so it would be necessary to a) reset the original default search engine on the drop-down list and b) give the user the option to retain the newly added search if it was not originally on the drop-down list. Some further investigation is required to determine if it would be possible to toggle between different search engines on the list if the user has two or more tabs open for different web sites that offer different search engine plugins.
A second idea that came from Gavin was to prototype the feature described in Bug 335448 - Autodetect repeated use of a search field, and ask the user if they want that engine added to the browser search box. This enhancement bug proposes that when Firefox detects that a user has used a particular web site's search box n number of times, the user should be presented with the option to add the web site's search feature to the browser's search bar drop-down list and auto-generate a search engine plugin based on the web page's form elements. Gavin suggested that this would mostly involve implementing an Automatic Detection algorithm and combining it with the search engine addition code from the non-landed patch in Bug 261124 - Textbox context menu should have "Add to Search Bar..." The patch adds search engines to Firefox's search bar given an input field and its associated form element. Potential drawbacks to this enhancement are being able to accurately and consistently auto-generate a search plugin based on a web page's form elements if the names or id's of the elements change, and being able to maintain an accurate account of search box usage if the user clears the browser's private data or the user does not accept cookies from sites.
The third idea was to have the browser detect web site search elements based on names, id's and class names that are commonly used for search input elements, and then use this metadata to generate and add search engine plugins to Firefox's search bar. This approach is very dynamic and fluid but it presents the major problem of being able to accurately and consistently detect search fields based on this type of widely diverse and random metadata.

Mozilla Project Update: Initial Research

Mozilla Project: Localized Search in Firefox's Search Box
During the initial research for my project, I discovered that Firefox 2.0 detects search engine plugins that may be manually added by the user to Firefox's search engine drop-down list. The user may then decide to select a particular search engine as the default for the browser's built-in search box. When Firefox detects a search plugin that has not yet been "Added" to the browser's list of search engines, the search engine icon and the drop-down list arrow of the browser's search box turn blue in colour.
Firefox allows JavaScript code to install search engine plugins, and supports two search engine plugin formats: OpenSearch and Sherlock. Firefox supports autodiscovery of search plugins , which enables a web site that offers a search plugin to advertise it, so that Firefox users can easily download and install the plugin. Any web site offering a search plugin must provide an OpenSearch description file, which is an XML file describing the search engine to the browser. To support autodiscovery of a search plugin, HTML code similar to the following line needs to be added to the <head> section of the web page:

<link rel="search" type="application/opensearchdescription+xml" title="searchTitle" href="pluginURL">