Capturing the Great FOAM

There has been an ongoing discussion among a number of FOAM participants recently about creating a Journal of FOAM (this discussion is summarised best by BoringEM). This discussion has largely arisen as a result of critics saying FOAM lacks peer review.

However, while I think formal peer review is great in some circumstances, for one I thing I think it’s not what FOAM is necessarily about, two I think it largely happens anyway (e.g. retweets, feedback on twitter, word of mouth, etc), three there is a larger question about how we capture and collate high quality FOAM.

Why do we want to capture the good FOAM?
This may seem like a simple question, but I get the impression that people would have slightly different answers and it’s a crucial starting point.

In my opinion, the obvious reasons we want to capture and highlight the good FOAM, are that; it is going to improve our professional development, challenge our thinking or alter our clinical practice in a positive manner. Ultimately, we want access to the most correct information, in a timely fashion that suit our learning approach.

Strategies to evaluate, highlight and find good FOAM
There are a number of strategies that others have suggested or put in place to capture the good FOAM. I’ve outlined some of the strategies below, and potential pros and cons I see with each of them.

Twitter Feedback

Used by many.
How: Give feedback directly to the author and share online with others (e.g. through the #FOAMed hashtag).
Pros: Quick, easy, open too many participants, promotes ongoing discussion.
Cons: Comments are ultimately lost after a period of time. Feedback is also limited to 140 characters.

Peer Feedback directly on the blog

Michelle Lin from Academic Life in Emergency Medicine, recently started to setup/trial a peer-review feature on her blog.
How: Created a peer review section at the bottom of each post, containing metrics such as accuracy, valid for practice, true to literature, overall quality. Rater is anonymous. Metrics are collated and viewable on each post.
Pros: Receives a peer review.
Cons: Is viewable on the blog only. This is useful for those viewing the blog post and as feedback to author. However, this data (as far as I’m aware) is not available beyond that direct level.

Search Engines

Examples: Google, Bing, EMGoogle
Pros: Provide a quick means of usually finding relevant material based on keywords, page views and site rankings.
Cons: The above metrics don’t always correspond with quality. New or low ranking sites can have their posts further down search results, and subsequently a great post may be overlooked. Your search results are only as good as the data you put in. No written feedback generally available to author or viewer.

RSS Feeds and Feed Readers

How: Depends on which feed reader use, but a number have options to vote or comment on posts. Subscribe to the FOAMEM feed (http://www.foamem.com/feed/) for a full suite of Emergency Medicine FOAM feeds.
Pros: can capture large amounts of material/resources. Readers can discover material typically based on popularity (e.g. page views).
Cons: Takes time to setup, subject to your own selection bias. Comments are not shared with the larger community.

Blog Reviews/Grand Rounds

Example: LifeintheFastLane.com Reviews
Pros: Collation of a number of FOAM posts. Ideal for those with limited time, as less time is spent looking for quality resources.
Cons: At the end of the day a review is only as good as the author who selected the posts. It is susceptible to selection bias. Typically only highlights the author’s views of what constitutes good posts.

Journal of FOAM

Not in existence as of yet.
Pros: Brings peer review to some FOAM Content. More pros can be viewed at BoringEM’s post on Journal of FOAM.
Cons: Creates essentially a closed feedback network. Selection of quality FOAM resources is determined by a few. Likely to have significant time requirements on any editors/reviewers. Again see BoringEM’s post for some more cons.

Global Medical Education Project

www.gmep.org
How: GMEP already captures a large portion of FOAM such as images, videos, questions and even podcasts.
Pros: Free access. Allows users to tag, vote up/down, comment, share, add to folders.
Cons: Requires authors to host their material on GMEP (some may not like this as it takes away page views from their site). Requires viewers/reviewers to sign-up to GMEP. Only captures feedback posted on GMEP.

Out of all the strategies I’m least keen on having a Journal of FOAM. This is for several reasons including;

Twitter provides quick dissemination of material/feedback, is open to many and promotes discussion. So is ideal (provided there are people to drive it) in getting a relative idea of the quality of FOAM resources.
I believe reviews like the once-a-week LITFL Review already do a good job of collating quality material, but in an informal matter. So if anything I would like to see more reviews started up by different authors.
I believe that there are better options available, which fit within what I like to think of as the principles of FOAM.

Currently, I feel that GMEP with a few modifications could potentially offer the best method of capturing and reviewing FOAM resources.

My dream FOAM Capturing Machine.
My dream FOAM capturing machine would have the following components (there are obviously limitations with my model, but one can dream);

Chiefly it would be a web portal that functions as an online directory. This differs from GMEP which I view as an online repository.
Material is still posted on individual blogs, websites, podcasts, etc. So the author still has control over their material.
A peer review (widget/plugin) similar to Michelle Lin’s, that can be attached to each blog post on the author’s blog, with the data feeding back into the directory.
A tracker on each blog post, which captures metrics such as page views, length on page and so on (similar to Google Analytics), which feeds back into the directory and helps determine resource ranking.
A system to capture tweets relating to an author’s content.
Online Directory Features

Search
Best blogs, Podcasts,etc/new blogs, etc
Most popular/most recent/highest quality/and so on FOAM posts.
Tags and categories
Resource type (blog, podcast, etc)
Level of Education/What level is it pitched at (e.g. medical students, consultants)
Discussion/Comments section
Ability to curate reviews on the site (e.g. select a number of posts, that then get posted as a review). Again have ranking of the reviews.
Ability to favourite items for own personal viewing later.

So essentially it would be a system that collates a large percentage of the subjective and objective data around a particular FOAM resource.

The Wrap-up
There are currently a number of concepts, coming through the FOAMed movement on ways of evaluating and highlighting FOAM resources. The selection of strategies I’ve mentioned above, are merely just a few of them.

I’m keen to see FOAM remain largely open-access and crowd driven, rather than delegate the responsibility of curating quality resources to a few. I believe the community already does a reasonable job of this, and rather we need a way of capturing these resources & feedback for the long-term.

If you have any thoughts on anything I’ve mentioned, please leave your feedback here or flick me a message at @IVLine. It is always welcome, and hopefully will promote a solution that works best for the majority of the FOAMed community.