Potential research malpractice by Surgisphere

[Agema]

Science | AAAS

www.sciencemag.org

A few studies published recently, including ones in major medical journals The Lancet and New England Journal of Medicine caused quite a stir; the one in The Lancet caused a pause in some studies on hydroxychloroquine. However, it turns out these studies originate from a small and little known company called Surgisphere, which claims to have access to a large amount of hospital data. However, all three papers are coming under heavy scrutiny for anomalies.

Under this scrutiny it turns out that Surgisphere appears to be a very small company founded in 2008, and given it's known employees likely doesn't have the manpower or expertise to conduct these studies, although it claims to be using "AI" and advanced "machine learning". It was founded as a medical education company, providing textbooks. However, there is some doubt about the quality of these textbooks, and there is evidence of fake reviews used to sell them. How this small company somehow changed direction to acquire / develop the major database it claims to have seems very uncertain. There is considerable concern that this "database" is a scam.

It's founder and CEO, Sapan Desai, was until recently a cardiovascular surgeon, resigning from active medical practice in Feb this year "for family reasons". There are currently three outstanding medical malpractice suits against him from 2019; a spokesperson for Desai says they cannot coment on ongoing cases, but they are unfounded. Surgisphere also founded a medical journal, Journal of Surgical Radiology, which ran from 2010-2013 and was by its own claims vastly successful (one million views per month, no less), before Desai closed it down as he claimed he was too busy to run it.

It's not that I'm saying this guy definitely is a con-artist and his company is a fraud, but... it's got all the hallmarks of it. Hustling at niches, dodgy adverstising scams, low verifiability of quality of products / services, deeply improbable boasts. I wonder if Surgisphere could have kept nibbling away in this manner for years, but the problem with the type of person I suspect Desai is (delusions of grandeur) they can't help themselves, often overreach and bury themselves. But won't it have given him his 15 minutes of fame.

 

[Tireseas]

I'm going to let peer review make the determination more than anything else, but any time I hear "AI" and "Machine learning" in terms of analyzing data, I get really really really skeptical because, as the saying goes, "garbage in, garbage out."

 

[Agema]

I'm going to let peer review make the determination more than anything else, but any time I hear "AI" and "Machine learning" in terms of analyzing data, I get really really really skeptical because, as the saying goes, "garbage in, garbage out."

I'd certainly wait until the evidence is in on the science to be sure.

But on the other hand, as I teach on a BSc., when I advise students on good papers to read I advise them to consider the reputation of the authors and their institutions. Whilst by no means a guarantee of quality, on balance eminent researchers and institutions tend to be eminent for a reason, and the bottom end is clogged up with a great deal of chancers and CV-padders.

 

[SilentPony]

Yeah "Little known company" and "Advanced AI and Machine Learning" are huge red flags. And although the cases are still on going, resigning with 3 malpractice suits against you and starting an AI machine learning company is a bad look.

 

[Kwak]

What was the gist of the research?

 

[tstorm823]

What was the gist of the research?

Do you recall the claims that Hydroxychloroquine actually lead to more people dying of covid-19? That's the one.

 

[Agema]

Do you recall the claims that Hydroxychloroquine actually lead to more people dying of covid-19? That's the one.

To be fair, it's not the only study that's reported that. It was the most notable thus far because of the much larger sample size.

 

[Agema]

Covid-19 studies based on flawed Surgisphere data force medical journals to review processes

New England Journal of Medicine and Lancet peer reviewers did not see raw data behind findings before publication

www.theguardian.com

So, this is an interesting one. Peer review is not designed to catch fundamental fraud: it works on the assumption data is genuine, and tests whether the data has been collected, analysed and discussed properly.

There's a thing about peer review: it's actually quite hard. About two-thirds of academic reserchers out there aren't very good peer reviewers: about a third are too lenient and a third too harsh - editors work out who these guys are stop giving them papers to review. Peer review is unpaid and thankless, particularly when you consider that research-active scientists tend to be phenomenally busy people without having to take on additional work. A fair number of researchers who can do peer review turn it down because they don't have the time (or just think they're too important for that sort of menial duty).

If peer review ends in having to sift through raw data regularly under the current system, it's utterly impractical. The system needs to be completely rebuilt. People don't have that kind of time, especially for a service that is completely unrewarded (in terms of pay, respect, honour, status, whatever else). I'm not against the system being redesigned, but who is going to oversee this and pay for it?

 

[Dreiko]

This whole thing is just so petty I get annoyed by it whenever I see it. The people who think this is a magic cure are idiots, the lady who drank fish tank cleaner is a moron, the people who are trying to debunk this like it's a big deal just to get Trump to have been wrong sound like they'd rather the virus not get cured if they can have their petty win. Nothing is good here. And now you add this company trying to confirmation bias something for who knows what reason. I don't care to even entertain conspiracy theories about their motives. They're all wastes of time.

 

[Agema]

And now you add this company trying to confirmation bias something for who knows what reason. I don't care to even entertain conspiracy theories about their motives. They're all wastes of time.

It's got very little to do with Trump, except perhaps that Trump has made this line of study unusually popular.

Surgisphere seems to have been around making claims on its data for several years across a fair number of topics, and via sub-companies, all with very dubious prominence. Its CEO has been publishing much longer, over 100 articles, all of which are now under scrutiny. An expert has already found likely evidence of fraudulent data (image manipulation, etc.) in a paper of his from 2005, and I'm sure plenty more will crop up.

If you take a look through the world's most prolific scientific fraudsters, some of them are truly incredible: hundreds of fraudulent publications. Usually a key to the top ones is that they've done relatively small-scale, boring studies. Lots of experiments in scientific papers are "wrong". They're not necessarily fraudulent - it's genuine data and genuine analysis - but imperfections in the methods or that they lack sufficient statistical power mean that they're not actually true. For instance, you'll see a typical "p < 0.05" to be statistically valid, i.e. more than 95% likely to be true, but with sample sizes under 10 this may be more in the realm of 50-50 in reality. Most science is little read and little cited, and there's a lot of, well, junk in there, so it's easy to hide fraud. This guy, Sapan Desai, is probably one of these career fraudsters. He's fallen foul here simply because he's waded into an area that has generated a lot of scrutiny of his work, and it has collapsed remarkably quickly. If he hadn't stepped into such a huge arena, he might have continued for years.