Weekly Roundup #2
The last week of January sees a wider range of fallacy scoring.
Happy Monday from the team at credAIble.
This week we used our chatbot to evaluate a larger stack of editorials and op-eds, with topics ranging from the New Hampshire primary fallout for Republican presidential candidate Nikki Haley, the apocalyptic incoming collapse of American democracy, and (of course) the unstoppable force of Donald Trump’s apparently destined candidacy. What else is there to talk about?
We received a comment during our launch to cover more of the real news — not just op-eds. So, this week includes some actual news articles as well as two editorials from various editorial boards, as opposed to focusing solely the works of columnists themselves. Why cover only the opinion pieces when bias exists in a more visibly impactful way in our purported most “objective” forms of front-page news? We agree with the premise, and are following the strategy of crawl-walk-run as we expand credAIble coverage.
The Editorial Boards
One of our first editorial review week is a daily newsletter selected at random from the 1440 Daily Briefing. More about 1440 later. We were delighted to see that their focus on facts earned an impressive CredAIbility score of 0.
1440 Daily Briefing
1440, January 26, 2024
Viewpoint: Neutral
Fallacies: None
Click for detailed analysis >
No other editorials hit the same standard. They tended to score average marks on our scale, earning 4s and 6s. Goes to show faulty reasoning happens to more people (entire editorial boards, even) than you’d think.
Washington Post — Strong Left, 6 fallacies for “N.H. Republicans should be honest about what backing Trump would mean”
New York Times — Strong Left, 4 fallacies for “The Responsibility of Republican Voters”
The Economist — Moderate Right, 4 fallacies for “How the border could cost Biden the election”
The 1440 story is an interesting one, named after the year that the printing press was invented, and the site was recommended by a longtime friend. Although the newsletter has 3+ million subscribers, we had never heard of it before last week, so are happy to share:
“1440 is the daily news briefing followed by millions of readers. But it began as an email we sent to 78 people. Every morning, we'd wake up to the same broken media landscape. Filler content. Talking heads. Opinions. Infotainment. But one day, instead of repeating the cycle, we started working on a solution: a comprehensive news source edited to be unbiased as humanly possible. A daily briefing—in one simple email. For more insight. And less outrage.”
The Problem with Libertarianism
We analyzed two posts by famous economists Bryan Caplan and Robin Hanson. Both scholars are considered libertarians with innovative viewpoints that simply aren’t measurable on the standard ideological left-right spectrum. We wanted to assess how well the chatbot handled their inputs, and the results indicate that it was difficult.
Robin’s essay “Why The Line” was assessed with 3 logical fallacies and a “Moderate Left” viewpoint. That’s actually a relatively low score, especially given that the essay in question was lengthy. Even so, it’s hard to see any ideological leaning in the essay which covers a wholly original topic with an astoundingly new development in a patently non-ideological, brainy manner. Bryan’s essay “Substack Versus the Slippery Slope” was assessed with 5 logical fallacies and a “Strong Right” viewpoint. Anyone who reads Bryan’s books and columns would find that laughable, but it goes to show the problem we face. We take this score as a sign that the AI algorithm needs some work, mainly so that it will consider strong but alternative viewpoints like these.
The Logifail Trophy
Bringing up the rear, here is the most egregious article of the week with a record 8 different fallacies:
Liberal cowardice, Trump, and the Constitution: Surrender is not an option Mike Lofgren, Salon, January 14, 2024
Viewpoint: Strong Left
Fallacies: Straw Man, Appeal to Emotion, Ad Hominem, Slippery Slope, False Dilemma, Biased Language, Hasty Generalization, Red Herring.
Click for detailed analysis >
That’s all from us this week! See you in February.
Want to recommend an article for evaluation?
You can find a submission box at the bottom of
https://www.credaible.com/
The categorization problem seems secondary & unrelated to the AI routine, and likely much easier. As long as your ideological measure is one-dimensional left/right, libertarians will be a problem. (Aren't they always?) Steve's two-dimensional model would improve this. But better still, I'd recommend a 3D categorization based on Arnold Kling's "Three languages of politics," which I think would align best with text-cat routines. To summarize: if your rhetoric stresses oppressor/oppressed differentials of power & influence, you're a "liberal." (Scare quotes here because IMO this is more "progressive.") If you stress civilization vs. barbarism , you're "conservative." (Perhaps amend as: stability around a set of rules vs. disorder?) If you stress freedom vs. coercion, you're "libertarian" (aka "liberal" IMO). I prefer how such categorization is more rooted in the fundamental language you use than how you fall on specific issues (as Tom suggests), which often invert based on transient partisan concerns. E.g., if you're anti-illegal-immigration, you're likely "conservative." Except if it's an influx of Cubans, in which case you're "liberal." Or if you're agitated about too many/few members of ethnicity X being admitted to elite universities, the rhetoric you use can help predict if X is Black/Hispanic or Asian/Jewish.
This seems like a GREAT idea. Less bias, from more honest evaluation OF bias, should help.
But: " It also evaluates political bias on a seven-point scale from Extreme Left to Extreme Right."
Get rid of "left - right" and label the actual positions. Pro-choice vs pro-life. Border control vs open border (where limited acceptable illegal immigration is "center"). One part, probably significant, is the lazy news/ pundit labeling of positions as L - R, to avoid saying Dem or Rep, while not honestly identifying the extremes of the issue. Or the reasonable extremes?
High tax - moderate tax - low tax - no tax. Far right is no-tax, but few argue that, seldom even those claim to be anarchists or anarcho-capitalists; often an extreme Libertarian.
'Responsibility of Rep Voters" - how is that responsibility different from Dem voters?
"SC must kick Trump off" -- nobody's been indicted, much less convicted, of insurrection.
"Founders against Mar-A-Lago Exec Branch" -- what about Delaware vacations?
Biden's been President 3 years, where is the actual news, NEW FACTS, about what the current government has done. There are no "facts", today, about the future. Only about the past, recent or far back. Yeah Trump looks unstoppable, but Biden & Dems are in power now -- what is the government actually doing? In FACT?
News stories should be rated to a large extent on the facts they present, if any are new.
Again, this seems like a great start. Related to the Fantasy Intellectual Teams idea of Arnold Kling, which I've long supported.