User Tools

Site Tools


Grass Roots Peer Review

Grass Roots Peer Review is the name for a system designed by [ Erowid] in 2000 which is a type of collaborative filtering oriented towards using networks of relationships and categorized ratings to create an evolutionary environment for information.

Is the project ready for release? No. It is still vaporware as of July 2004.

Who is working on it currently? Currently erowid's two main developers are working on the system. Mindlace joined the project and is (Aug 2004) currently working to refactor the database interface and move the system to using postgres instead of MySQL.

What is the expected release date? Because of other constraints, it is impossible to say what the release date on the first working version might be. Prototypes for elements of the system are working, but there still isn't a whole lot to show as of Aug 2004.

Why is it taking so long to have a prototype? I work on the project whenever I have free time, which isn't often. Although I had the help of another coder for about 50 hours in the last year, there still isn't much to show publicly.

d What is the purpose of the project? The need for the system is rooted in the fact that there is a lot of information to keep track of and sorting out what is good information and what is not-as-good information is often a process involving discussion with other people. For factual information, the use is very obvious. Facts are not valid based on democratic voting, experts in fields are often more likely to be right about an issue than non-experts. The goal of the system is to provide tools for people to be able to share expertise by delegating or assigning value to the opinions of trusted others and using that networked assignement of 'factual trust' to do filtering and to record opinions, critiques, and praise in a standardized format.

Erowid currently uses a system of informal peer review. [ Experience Vaults] are the closest thing that currently runs on erowid which implements a tiny part of the review system. Experiences are submitted and it takes two review crew to delete a submission and two reviewers need to check an experience before it is considered final. The next design of the Experience Vaults is underway and uses what we call a Triage system where people with much less training than our main reviewers rate and categorize incoming reports. Reviewers are then chosen from the pool of active Triagers.

For other types of incoming factual documents, we have a non-systematic queue of documents which we read and decide whether to push to our editors. We find experts in the correct knowledge domain and these experts give feedback & critiques which we then send to the author. Our editors take the resulting back and forth and suggest changes which we then run by both the Expert Reviewers and the author. Once a draft is ready, we send that to our Content Review List which includes both experts and non experts (membership is open) and ask for any feedback. Another round of editing is usually required and then the document goes public. Once public, we still consider all documents to be part of an evolutionary process where, over time, errors get noted or new advances in knowledge obviate or anachronize a given piece of information. Although we believe most documents should never be deleted, we know it is important that out of date documents get marked as such.

That is one of the main UI targets for GrassRootsPeerReview: to provide a consistent interface by which documents can easily be checked for their currency, their review status, who has reviewed them and what ratings were given, etc.


Please see [ documentation on customizing the interface] and the [ User's Guide] for usage and configuration help.

start.txt · Last modified: 2008/10/08 05:56 by earth