First we had peer review – and now patient review!

~ Carolyn Thomas

by Carolyn Thomas  ♥  @HeartSisters

You already know that research papers submitted for publication in medical journals are first subject to peer review before a final decision to accept the paper is made. Peer review is a time-honoured way to evaluate scientific or academic papers by others working in the same field.  But The British Medical Journal, the world’s oldest, has launched a unique initiative to include patient review of submitted studies as well. Here’s how BMJ explains this project:

“The BMJ has committed to improve the patient centredness of its research, education, and analysis articles by asking patients to comment on them. Patient peer review is a new initiative for The BMJ. We are taking the lead here, and hope other publishers will follow.“

Tessa Richards, a senior editor at the BMJ, adds that patients tend to raise different and often insightful points.  With over 300 patient reviewers already in place, she reminds physicians that improving healthcare depends on health professionals having a better understanding of the burdens of illness and treatment and of the difficulties and dangers that patients encounter while navigating fragmented delivery systems.

And it gets better!

Authors of research papers submitted for publication to this journal are now asked to state if, and how, they involved patients when choosing their research question, study design, and outcome measures and in implementing and disseminating study results. Tessa Richards concludes:

“In future we are likely to consider clinical research papers only if the authors can demonstrate partnership with patients in their study.”

My own BMJ debut as a brand new patient reviewer for heart-related studies happened last month!  I described that experience to BMJ readers like this:  

. 

Carolyn Thomas: My experience of patient peer review

Published in The British Medical Journal, October 16, 2014

“I’ve finally hit the Submit button on my patient review of a research paper submitted to The BMJ—and in time for its deadline. Hurray!

“This is the first project of this type I’ve ever been involved in, and at first blush I wondered if I would have anything at all meaningful to contribute—as a non-scientist who wouldn’t know a Hosmer-Lemeshow test from my left elbow at the best of times.

“Once I started reading the paper, the pressure to come up with something profound seemed to grow exponentially, especially given that The BMJ is among those journals leading the way in even considering the inclusion of patient reviewers in this process. I also knew that, unless a patient review is meaningful or useful, why bother? Would my clearly non-professional input be used to explain trashing the entire patient peer review initiative? (‘We tried patient reviewers, but they just didn’t pan out . . . ‘)

“And then I decided to carefully re-read The BMJ‘s patient reviewer guidelines. The first question of the guidelines is:

“Is this an issue that matters to you, other patients, and carers?”

“I was suddenly able to take a nice deep breath and remind myself that The BMJ already has real scientists and healthcare professionals to offer peer review input, so I needn’t lose sleep over methodology minutiae. This first question reminded me as well that the particular focus of the paper I was reviewing did matter to me. In fact, I’d already written – from a patient’s perspective – about the very same issues surrounding controversial cardiovascular risk calculators that were discussed in the paper up for review.

“This experience has turned out to be far less stressful than I’d first anticipated, thanks to the helpful patient reviewer guidelines.

“I appreciate The BMJ‘s brave and generous offer to grant patients the opportunity to participate in what matters to us.”

Carolyn Thomas is a heart attack survivor, women’s health activist, and blogger at Heart Sisters.
Competing interests: I declare that I have read and understood BMJ policy on declaration of interests and I have no relevant interests to declare.
Read Carolyn Thomas’s previous BMJ post:  Why physicians must stop saying: “We are all patients”

♥

Are you a patient, patient advocate or caregiver interested in volunteering to be a patient reviewer for The British Medical Journal? 

If so, read the patient reviewer guidelines that I mentioned above. Near the bottom of the guidelines are listed a number of published patient review examples (FYI, mine is #3). If you decide to sign up as a reviewer, you’ll be asked about medical conditions you have had or for which you are a patient advocate (for example, cancer, heart disease, diabetes).

You’ll also be asked if you have a special interest or expertise in topics like advancing patient partnership through shared decision-making, promoting self-management and patient leadership, etc.

Here’s more from BMJ editors about this unique opportunity:

“Don’t worry, you don’t need to have any medical or scientific training, because we’ll be asking you to consider a slightly different set of questions about the paper you’ll be reviewing than the traditional peer reviewers.

“As a patient reviewer, we’d like you to answer questions like:

  • Is this an issue that matters to you, other patients and carers?
  • Are there any areas relevant to patients and carers that are missing or should be highlighted?
  • If the article is a research paper looking at a new intervention of treatment, say if you think it will really work in practice? What challenges might patients face?
  • Are the outcomes measured and issues discussed in the article the ones that are important to patients? Are there others that should have been considered?
  • Do you have any suggestions which might help the author/s strengthen their paper to make it more useful for doctors to share and discuss with patients?

“By answering these questions, you will be giving the editors your perspective on the patient-focused aspects of selected manuscripts, drawing on your experience of a particular topic, condition or intervention.

“You will also gain a unique insight into how medical research is conducted and the way educational articles are published for doctors. It’s your opportunity to have a real voice in shaping the way researchers and clinicians act, and to further their understanding on what is most important to, and of benefit to patients.”

Intrigued?  Find out more about how you can get involved as a BMJ patient reviewer – and please forward this on to any other patients you know who might also be keen on getting involved like this.

 .

♥  Need a translator for complex cardiology terms?  Visit my Heart Sisters patient-friendly, no-jargon glossary.

 

.

Q: What do you make of The British Medical Journal taking this step towards “patient centredness”?

.

See also:

.

Published by Carolyn Thomas

25 thoughts on “First we had peer review – and now patient review!”

      1. another question if you don’t mind! say the editors don’t like the review for some reason.who has control over what is published.would any editing by BMJ editors be allowed?

        Like

        Reply

        1. I can only speak for the review I wrote. My understanding (based on being copied on an email directly to the study authors) is that my review was sent ‘as is’ along with those of three peer reviewers directly to the authors with no editing by the BMJ. But only the editors decide on whether the paper is ultimately published in the journal or not.

          Like

          Reply

  3. It was great to read your BMJ blog post today Carolyn. I’ve heard Fi Godlee, the editor of the BMJ, speak last year at the BMJ Evidence Conference where she said that the journal is committed to only publishing research which is of value to patients.

    I think that they are genuinely progressive and committed to this mission, and patient review makes complete logical sense in this context.

    If I have misquoted Fi then I hope someone will correct me!

    Best wishes,
    Anne Marie

    Liked by 1 person

    Reply

  4. Good idea – but are they really truthful by saying they will consider input from patients? I do not know who funds this journal – so if a patient’s opinion might upset the funders, will it still be considered? I hope it is not just a tactic to assure people.

    Liked by 1 person

    Reply

    1. That’s a fair question, Wilma, given how we see industry co-opting patient engagement, or token patients being invited to participate in boards, committees and conferences where their participation is seen merely as a tickbox on somebody’s checklist. In this case, I believe the BMJ editors who answer: “Same as all reviewers” when asked: “How much weight do patient reviews have?” As in all journals, papers are approved for publication by editors, not reviewers.

      Like

      Reply

      1. Carolyn, thank you so much for alerting me to Dr Groves’ reply. Pls run me through the steps that one has to take to partake in reviewing, if you don’t mind?

        Like

        Reply

        1. All I did was complete the guidelines/registration process on the BMJ site outlining my specific review interests (in my case, heart disease) Then several months later, I got an email asking if I were willing to review a cardiac paper that had been submitted for publication. Check out the BMJ site – will likely answer many of your questions.

          Like

          Reply

    2. Hi Wilma

      Editors at The BMJ always look first at the importance, originality, and relevance of the question that a study has tried to answer. Then we look at the study design to check if the methods were strong enough to give a reliable answer. If the question’s good and the design is right, we should be interested in the paper, whatever the answer. This approach stops us being biased against negative results.

      The BMJ’s sources of revenue are explained here where it says:

      “The BMJ receives revenue from a range of sources, to ensure wide and affordable access while maintaining high standards of quality and full editorial independence. The sources of income include subscriptions from institutions and individuals; classified advertising for jobs and courses; display advertising for pharmaceutical and non-pharmaceutical products; events (exhibitions, sponsorship, and visitor fees); sale of reprints, rights, and royalties; sponsorship; and open access publication fees.

      “Separation is maintained between the editorial team and the advertising and sponsorship sales teams. Where sponsorship has been obtained for any of The BMJ’s content — for example, as a result of an unrestricted educational grant — this is clearly indicated.”

      And here’s more detail on our peer review process:
      http://www.bmj.com/about-bmj/resources-authors/peer-review-process
      http://www.bmj.com/about-bmj/resources-reviewers
      http://www.bmj.com/about-bmj/resources-reviewers/guidance-patient-reviewers

      The BMJ was the first high profile medical journal to require all reviewers to sign their reports, a system we’ve used for 15 years or so. We’ve just introduced even greater openness. For all research submitted from September 2014 onwards, we’re using fully open peer review i.e. for all papers that we accept, we’ll post the signed reviews, editors’ report, and authors’ responses alongside at http://www.bmj.com/thebmj

      If Carolyn’s review was for a paper submitted before September 2014, we won’t be using this completely open form of review. However, when the editors make the final decision on the paper – whether to reject or accept – all reviewers, including Carolyn, will receive a copy of the editor’s letter to the authors and will be able to see all the reviewers’ reports for that paper.

      Hope this helps answer your questions.
      Trish

      Dr. Trish Groves
      Head of Research, British Medical Journal

      Liked by 1 person

      Reply

      2. Thank you for your reply, it is very informative. Can one write a review about research that is still being conducted? For example, I would like to write a review about Mobikids. Would that be allowed?

        Like

        Reply

        1. Hello again.
          Good question.

          It’s fine – and could be interesting and helpful – to write in public about research that’s being conducted, as long as you don’t leak any details that aren’t already in the public domain. So it’s okay to refer to details in a published study protocol or on a study register, or on the study’s website.

          But if your review disclosed details of methods or results that weren’t yet public, then it could make it hard for the authors to get papers published when the study’s finished. Journals aim to publish original papers, and some might turn down papers whose main results are already available on the web.

          I’m not sure where you’re thinking of putting in your review. I don’t know of a journal that would seek or post reviews of studies that they’re not handling. But there’s always the blogosphere, of course.

          Dr. Trish Groves
          Head of Research, British Medical Journal

          Liked by 1 person

          Reply

          1. Thank you Dr Groves, there is one specific study that is ongoing that I am very interested in, and I have concerns about it. Progress was published – that is how I came to know about it. Thank you so much for answering.

            Kind regards,
            Wilma Miles
            Cape Town, South Africa

            Like

            Reply

  5. Carolyn,
    Can we read the review you did? – I couldn’t find it.

    On first blush I thought what a great idea!

    My second reaction is that a lot of research may not make sense or be of importance to me as a patient but contributes to research/science in the long run in ways that I may not understand.

    The only focus that I clearly see without having read any reviews is that what may be helpful is the last: “Do you have any suggestions which might help the author/s strengthen their paper to make it more useful for doctors to share and discuss with patients?”

    I do believe that more articles from patients and that perspective is valuable and needed. I’m just not sure about patient-review in medical research journals.

    It will be interesting to read a few patient-reviews for me to better understand what the BMJ is hoping to accomplish.

    Liked by 1 person

    Reply

    1. CAROLYN’S UPDATE March 31, 2017: YES! Today, the BMJ updated its Patient Review guidelines and included six samples of published patient reviews (near the end, mine is #4!)

      Once the review is submitted to the BMJ online, it’s considered confidential and sent directly to the study authors. But I too would have loved to have seen a couple of sample patient reviews before diving in cold, just for reassurance.

      I had the same reaction you did at first – “Great idea!” Then when the BMJ editors actually emailed me the full study, lots of panic – “What was I thinking?” I couldn’t even force myself to open the file for a week, and when I did, it all seemed too overwhelming, with pages of indecipherable graphs, charts and tables. But that’s when I sat down and had a wee chat with myself – and I re-read those helpful patient review guidelines, which start off, wisely: “Don’t worry….”

      Perhaps editors need to beef up that “don’t worry” part for us patients. We are NOT statisticians or academics – but I can find my way around reading a medical journal paper (entirely self-taught because I’ve been so interested in cardiac research – particularly women’s cardiac research, such as it is). After I decided to focus step by step on the paper’s content, I started scribbling notes in the margins (more like questions, really). An unexpected bonus afterwards was that I got to read the peer reviews of the three health care professionals who had also been assigned this review – all three quite different in tone and commentary. All in all, the experience for me was not nearly as scary or time-consuming as I had worried it might be – and it certainly delivered what the editors promised: “a unique insight into the way educational articles are published for doctors.”

      Liked by 1 person

      Reply

  7. I also think it sounds promising. Thanks for posting this. I haven’t signed up but probably will after I’ve read all the info you’ve presented.

    Liked by 1 person

    Reply

    1. I hope you will, cave. You sign up to review only papers on the specific medical topics you are most informed about – they’re not sending me cancer or diabetes papers, for example. I felt quite nervous before I opened my first study, but just kept in mind those patient review guidelines listed above.

      Like

      Reply

Your opinion matters. What do you think?