Features: Internet Search Indexes and the Palestinian Voice

By Nigel Parry, 22 July 1999

GUIDE-LED SEARCH INDEXES - A NEW PLAYER MAKING OLD MISTAKES

Follow this link to an article on "The Six Day War" in the 'Israeli Culture' section of About.com, a new guide-led search index that is being heavily promoted by adverts, at least in my Midwest neck of the US. The article buys very heavily into the "Israeli victory as miracle rather than a result of superior firepower" myths of 1967. To understand how this kind of bias appears to have transferred seemlessly from its previous home in the traditional corporate media to its new home in the corporate projects of the new media, we need to understand a little more about some Internet specifics.

About.com's previous incarnation was as The Mining Co., "We mine the Web for you", which sums up very concisely the vision of this type of Internet search site. The Internet's a big place. There's a lot of garbage out there. So why not use a search site where people have already filtered out the garbage for you? On the surface it makes a lot of sense.

The current rise of these guide-led search indexes on the Web is partly a result of the popularity of the first manifestation of the first major guide site, Yahoo!. The main difference between a site like Yahoo! and a traditional search engine such as Excite or AltaVista is in the information architecture.

About.com and Netscape's Open Directory Project have attracted my attention because they take the Yahoo! model even further. Whereas Yahoo! is at least in theory aiming to index as much of the Internet as possible, About.com is "mining the Web for you", presumably to come up with the gems and not waste your time with the dross. Problems surface with this kind of approach as the quality of the information is entirely dependent on the person making the selection.

This practice differs radically from the process of getting a website listed in Excite or other traditional search engines. To do that a webmaster will submit the the address of that site (its "URL") via a form interface. The URL is transmitted to a queue in Excite's database and, when it reaches the top, Excite 's software will visit your site, follow the links to the various pages, extract all the text, find the main keywords, and store it in its publically searchable database. The whole process is automated and does not require any human intervention to decide whether your website is "worthy" of being part of Excite's index. Anyone who makes a search on Excite's page, therefore, and uses keywords which closely match those Excite earlier extracted from your site, will find your site listed among the search results.

In a guide-led search site, results of searches are usually not taken from keywords extracted from the text of your site using software following a simple series of mathematical categorisation rules. Rather it is humans who make the first and final decision to link to a site.

In my observation of traditional media bias in Palestine, I have concluded that a key part of the reason bias happens is not the result of any conspiracy, corporate interests, or pressure from the pro-Israel lobby. Much bias simply flows naturally from the organisation of the news gathering infrastructure. If you want to understand how major news networks seem to report so much of the Israeli point of view, just ask them where their correspondents live, where their local offices are based. Ninety-nine correspondents out of every one hundred covering the Israeli-Palestinian Conflict will be based in Israeli-controlled areas, making it no surprise that the Palestinian perspective is absent. What I am finding about the Internet's most popular media, search engines, is therefore not much different. And make no mistake, these sites are media.

The selection process of guides in About.com and Netscape's Open Directory Project is a serious issue that both they and we need to address, as these sites are "read" by far more people than newspapers, the traditional target of both sides' activists in the Israeli-Palestinian Conflict. Often, guides in these types of websites would seem to be unqualified to properly cover their subject area, having been picked on a "first come, first served" basis.

In commercial guide sites such as About.com, which actually offers some financial compensation to each of its guides, management must clearly make decisions that certain subject areas do not have enough priority to merit spending money on a guide. There is no Palestinian Culture section or any Palestinian section at all on About.com [http://www.about.com/].Whether this is a result of a financial issue or whether it is just a case of About.com not being sensitive to the particular and historical need for balance in this specific conflict, only they can tell us.

In an attempt to find out, I wrote several times to both About.com and Open Directory concerning this whole issue, offering myself as a guide citing my experiences at Birzeit building The Complete Guide to Palestine's Websites, my paper for the UN on Palestinian Internet development, and the like - figuring that I had a pretty good chance with that kind of experience - but was blown off with the standard "About.com has no plans to have a section on this subject at this time" or "Open Directory is not currently accepting new guides for this section".

E-mails to both search sites in reply to their turning down of my applications to become a guide/suggestions to create Palestinian areas, trying to explain the biases implicit in ignoring/badly maintained Palestinian sections while having fairly good to very good Israeli sections, went unanswered. Any developments to this state of affairs will be posted in an update paragraph at the end of this report.

WHAT TO DO?

Suggestions? I definitely think Freedom and other Palestinian discussion lists around the world should be alerted, as well as local US branches of the ADC and other Arab American organisations, and academics.

Palestinians and Arab American organisations need to start to consider that the Internet is a publishing medium accessed by far more people than newspapers, and the biases found in newspapers are usually structurally transmitted to Web published material as well. It is additionally a relatively untouched area of protest.

As some Israelis will be reading this feature I feel the need to state the otherwise unnecessary: I have no objection to the Israeli side in the conflict having free access to media, including the Internet. But when, as a result of corporate structural ommissions and a lack of effort put into balance, there are no complementary Palestinian sections or when the ones that do exist are poorly maintained, we find ourselves back in the kind of one-sided media reality that we had in the 1980s, which benefits no one.

The people that run these resources need to see there is a need for these sections as much as anything, and my suspicion is that very few people write to complain about the lack of Palestinian sections or coverage. The voices and resources of both sides need to be heard.

As most of you know, I lobbied Yahoo! and CNN for literally years to add Palestinian sections, both ultimately with success [See http://www.nigelparry.com/diary/media/cnnsites.html for the story]. However, these guide-led indexes are new players on the scene, making the same old mistakes. And it would be a mistake for us not to address them.

Even the current status quo at Yahoo! requires further focus so let me finish with a detailed breakdown of how Yahoo! (which also has humans manually adding its listed sites) covers the Palestinian side of town.

THE STATUS QUO AT YAHOO!

In May 1999, Yahoo!'s "Palestinian Authority" regional category in its "Countries and Regions" section had the claim that 166 Web resources are listed. This is compared to 205 local Palestinian sites that the best available resource - Birzeit University's The Complete Guide to Palestine's Websites [at http://www.birzeit.edu/links/] - had listed at the time. I figured that this wasn't too bad at the time, until I began to look closer.

Despite performing a thorough search of the many sub and cross-sections of Yahoo!'s Palestinian Authority category in May 1999, to find the 166 Web resources, I found only 108.

Of these 108 resources, 27 were produced outside the region by a mixture of Arab and non Arab sources, and three were produced by Jewish organisations or individuals. Of the latter resources, two were produced by Jewish settlers in Hebron and one by the right-wing Israeli Women in Green movement. None of the three Jewish resources are produced by individuals who have ever been resident in Palestinian Authority-controlled parts of Hebron, making their listing in this Yahoo! regional category erroneous.

In other words only 78 of the 108 links listed lead to truly local resources.

Although a section of a larger website can be considered to be a unique website if its content is distinct from the rest of the larger website (e.g. "The Ramallah Online Travel Guide" within Birzeit University's website), as many as 7 of these 78 links actually led to an individual web page, not a multi-page site.

Of the remaining 71 resources that I whittled it down to, a total of 4 links appeared to be dead, leaving us to note that only 67 of the existing 108 listed, out of a claimed 166 total sites, were truly unique, local, websites that deserved to be in a Palestinian Authority regional category.

In May 1999, The Complete Guide to Palestine's Websites, which includes an easy-to-find page for errant search engines that lists all the sites "at a glance" [See http://www.birzeit.edu/links/glance.html], listed 205 local Palestinian websites, underlining that Yahoo! is not only listing erroneously listing 30 resources outside the region in a regional category, but is 138 resources short of the actual number.

67 sites out of a possible 205 in May 1999 puts Yahoo!'s Palestinian coverage at just under one-third of potential. Quite a large margin of omission, no? Whether this is the omission of Yahoo! or a failure of Palestinian webmasters to register with Yahoo! is not clear. Although I have had some success with the registration of sites with Yahoo!, others I submitted just never made it onto the index, so perhaps it's a mixture of both. Apportioning blame here should not be the issue, but educating Palestinian webmasters to submit material to this popular index, while simultaneously targetting Yahoo! with polite queries might cause the situation to improve.