bestkungfu weblog

Amanda Stent reported on her work on a second-generation audio Web browser. HearSay is a prototype of an audio browser for users with visual disabilities.

Existing audio browsers have “several shortcomings”: they’re read out with little or no user-driven selection or ordering of content. Or they’re targeted to low-vision users, or use a walled-garden approach to get to a small number of sites.

HearSay’s approach is to determine function based on the structure of a document. It works well on template-based sites. It does a structural analysis of the document to discover semantic structures. (Cool.) It has been determined that semantically-related items exhibit consistency in presentation style and spatial locality. (And then the speaker showed mathematical equations, and I napped.)

If HearSay can find stuff in the DOM tree that it can use for semantics, it does that. If not, it looks for heuristics, and then annotates that portion of the DOM tree as a place to look. It then separates things into “partitions,” something which has been done successfully on a number of news and e-commerce sites.

When this was tested, it was found that task completion was even with a traditional browser. However, it took 4 times longer than a visual interface, because of rate of speed and mode errors. Users preferred it over JAWS, but would like non-speech input methods.

There are additional features being worked on like bookmarking items within a document (apparently including those without IDs) with a voice command. (Annotation is something the W3C has been working on for years. It’s good to see a decent application of this being requested by users.)