Hi Sujit,
Thanks for posting this. As someone who is invested in both animal health and environmental sustainability, I will have a go at appraising this study.
1. Exposure: These are period prevalence figures, over the 'past 12 months'.
The denominator is the 2,536 survey respondents whose dogs were maintained on one of the 3 main diet types (conventional, raw meat, or vegan). This excluded 27 respondents who did not indicate their dogs' main diet (clearly necessary due to missing data). However, it also excluded 76 (3%) dogs fed other diet types. Though this latter is a small % and unlikely to have a large impact on overall results, I'm not sure the authors' justification for excluding these from this initial prevalence figure 'due to low numbers' is justified/statistically sound, beyond the fact that this is the denominator they used in subsequent analyses.
For the numerators, the authors report asking about the main components of the animals' diet over the past 12 months, excluding treat/scraps/supplements, or the main components before switching to a therapeutic diet. It is unclear, however, how they categorised respondents where the animal had switched between diet types within the past 12 months, and how long exactly they had to be on a diet before classifying it as their 'maintenance diet'.
2. Outcome definition:
'4 or more visits'- This outcome is only actually highlighted in the Guardian article, and the main paper reports its statistical analyses of 'more than one veterinary visit'. The latter definition, used by the authors, seems to be reasonably justified in that a single annual health check is routine for healthy animals, and any visits in excess of that indicate potential health problems. I suspect the guardian article used the '4 or more' definition as these were easy figures to read off of one of the graphs (fig 4.), whereas they would have had to combine percentages otherwise. This is perhaps a more arbitrary cut-off indicating health problems, but one could argue that the authors should have perhaps presented a graph corresponding to the binary anaysis they did for clarity (eg a bar graph of =<1 and >1 instead of split by exact visit number).
3. Study design:
- retrospective cohort study, administered as an online pet-guardian self-reported questionnaire.
- the strengths of this design are that a relatively large quantity of data can be gathered relatively quickly (vs a prospective cohort) and cheaply (vs an RCT), making this a 'large scale survey' (by veterinary standards, anyway), which was identified as a gap in the literature in the introduction. There are no losses to follow-up (vs a prospective cohort). The authors also claim that outcome reporting bias was minimised by asking about pet health indices before diet type (advantage over eg a case-control study). They also claim this survey design is a more accurate representation of the real-world situation compared to an RCT.
- Weaknesses: the retrospective nature leaves room for considerable recall bias, in both the exposure and outcomes. Furthermore, their cohort is fairly ill-defined, with a sampling method that is poorly representative of the pet-owning population, and seems to be a mix of random internet sampling with some targeting at specific vegan groups. It is, additionally, a lot harder to control for confounding in these observational study designs. While data was collected on potential confounders, like respondent demographics, the authors do not, in my opinion, do a good job of adjusting for potential confounders in their analyses.
4. Mediator hypothesis:
The mediator hypothesis is that vegan diets make dogs healthier, and thus they need fewer vet visits (though not explicitly stated by the authors).
5. Confounders:
It is noteworthy that while the authors discuss potential confounding factors for similar patterns of fewer vet visits in raw meat-fed dogs (eg younger dogs, reluctance to seek veterinary care by owners), they do not consider these for the vegan group, even though both these factors may be important confounders in the vegan group as well. Additionally, there are quite a large number of other potential confounding factors I can think of just from personal experience including dog breed and owner socioeconomic status. As mentioned above, although the authors collected data on pet and owner demographics, they did not report adjusting for any of these in their analyses.
6. Alternative Study design:
Knight is alluding to a Randomised Controlled [Diet] Trial here. It is clearly not feasible to do this on as large a scale as this online survey, but such trials have been done and are in fact the gold-standard for pet food companies when researching, for instance, specific therapeutic diets. They are therefore clearly feasible, although expensive.
I do think RCTs would be necessary to provide the strongest evidence on the safety or potential benefits of specific vegan diets before they can safely be recommended to the general pet population, as it is very difficult to control for all potential confounding factors, I
Importantly, I feel Knight's use of emotive language here ('locked up') reveals potential personal biases which are probably inappropriate for what is meant to be an objective, scientific platform. Based on personal communications with several researchers involved in such diet trials, they do not necessarily involve hundreds of beagles kept in lab cages.
7. Knight quote:
I do not agree with Knight. From the data presented on recruitment methods and respondent demographics, the external validity of the study is clearly questionable to poor at best. I would imagine that there is an equal distribution of male and female pet owners in the general population, although I have no data to back this up, nor do I have data on how pet dietary choices or health-seeking/reporting may differ with owner sex. It is conceivable that female pet owners may be systematically biased if they are more diligent in seeking veterinary care/recognising health problems when they are also more likely to respond to the online questionnaire, although this may have been an effect of the specific social media circles used for recruitment.
While I would also imagine that most western pets have similar lifestyle influences in the UK/Europe/USA/Australia, again, I have no evidence to back this up, so it may be more valid to limit the conclusions of this study to only UK pets. Climate factors, for example, may affect specific health outcomes.
While systematic bias is thus likely, more concerning would be differential reporting or response bias that differs between vegan and conventional diet feeding owners, as this would affect the internal validity, and therefore overall conclusions of the study as well.
Overall, although this study is important in highlighting the non-inferiority, or potential benefits of vegan diets, I feel the authors' conclusions, that 'the healthiest and least hazardous for health are vegan diets', are premature and unsupported by their level of evidence.
