The Absurd Problems with Online Political Ideology Tests

Here’s a fun exercise to try: take the 36, 60, and 240-version questions of the 12 Axes ideology test and see how the results vary.

While our longtime readers certainly know this, our recent bursts of new subscribers (welcome, new friends!) likely do not:

My primary professional speciality for the last 23 years now has been political ideology.

Left, Right, Liberal, Conservative, Communist, Fascist, Islamist, Christian Nationalist — it has been years now of working with ex-Marxist activists, counter-terrorist investigators, and antisemitism researchers. So it’s not just a hobby, it’s becoming a profession. Being able to write knowledgeably about ideological movements has undergirded my writing career for 17 years now.

So when my close friend Jon Bishop sent me this 12 Axes ideology test, I felt intrigued and soon impressed when looking at the test’s approach.

They offer three versions of varying length. Each features mostly a series of values questions - strongly agree, agree, neutral (or it depends on), disagree, or strongly disagree.

Then at the conclusion the test asks a few broader questions about values and worldview.

After submitting, 12 Axes tells you which ideology most aligns with you as well as the others which come closest. It provides some intriguing analysis and suggests which figures you most resemble.

Jon told me that the test had told him he was a Reagan Conservative, which hit the axe on the bullseye. If anyone had to ask me — the ideology specialist — to describe my pal Jon, that would be what I’d tell them. So that impressed me. I was curious if the test would likewise put me in that camp too. I’ve outright declared myself a Reagan conservative all year and checked out dozens of Reagan books from the library as part of this silly research project. So surely if the test could identify Jon as such, then it would me. Right?

Today I put the test to the test to find out.

Could the test likewise give such a precise descripion of my values?

This is the best way to do journalism and research. Ask the question when you already know the correct answer, and then see if the person (or LLM chatbot program) responds with accuracy. If they do not, then that becomes data which opens up the simple question: why?

The 12 Axes test offers three version of varying lengths and includes a collection of 237 ideologies which you can read here. The list is very thorough and seems accurate to me. I’m very impressed.

So the 60-question version was recommended and I took it last night.

And I was tremendously impressed with the results:

This too also resonated:

That’s me! Very much a Thomas Sowell libertarian on economics but a Ronald Reagan hawk on militarism. And on “morality,” look at that? My position “sits almost exactly at the catalog median.”

Meaning: as extreme as I may sound at times about “moral values” and so forth, I’m actually very moderate and inclined toward compromise. On the so-called “social issues” around sex, gender, abortion, and culture war hot stoves with steaming kettles, I’m a boring moderate.

The next result from 12 Axes was cerainly accurate:

And then the “closest figures to your results” offered some apt choices:

The damn quiz correctly named Barry Goldwater, Geoge W. Bush, Ronald Reagan, and William. F. Buckley, Jr. as 85%-83%-aligned with me. (And BTW, this Kim Kataguiri guy is a Brazilian politician I had never heard of but will keep an eye on now. Brazilian politics have never been a priority for me to learn since they don’t cause any trouble on the global stage. They’re not one of the problem children in the class.)

The suggested reading is further absurd:

Both The Conscience of a Conservative and a pile of Buckley books are within arm’s reach right now, as is a book-length analysis of de Tocquevillle by the late Michael Ledeen — who I used to edit at Pajamas Media and whose views I’ve admittedly embraced on a number of subjects, unfortunately — titled Tocqueville on American Character: Why Tocqueville’s Brilliant Exploration of the American Spirit is as Vital and Important Today Asit Was Nearly Two Hundred Years Ago.

So that seems absurdly embedded in the bullseye. Books by and about these authors are on the table in front of me.

The final bit of analysis just confirmed the test’s accuracy:

Those all make sense. The difference between the “Libertarian Militarism” position closer to Barry Goldwater than the Reaganite Conservative position which emerged as the party’s unifying doctrine 16 years later, is sort of akin to asking whether you prefer your peanut butter crunchy (Goldwater) or smooth (Reagan.)

Quite taken aback by this stunning level of accuracy, I wondered if the test would yield more accurate results in its 240 question version or more bogus in its 36-question version. While “Libertarian Militarism” certainly fits like a shoe I could put on and then go run around the block, it still seems like “Reagan conservatism” is the better fit. “Libertarian militarism” is more a description of ideas rather than any sort of movement like mainstream conservatism. So would the longer or shorter quiz find that?

I’ll let you compare/contrast each category.

Here is the 240-question results (blue) followed by the 36-question results (green,) look at the huge difference:

Now things get strange. Both the longer quiz and the shorter quiz are much more inaccurate than the medium quiz.

While I have some sympathies and alignments with aspects of “interventionist democracy” it is not my primary ideology at all. I’m well aware of its shortcomings and wrote about them 13 years ago at Pajamas Media.

And this Quilombism thing I’ve never heard of before — it’s another Brazilian person or concept appearing. Whatever.

However, then things get weirder. While the 240 quiz again names Kataguiri as 87% in line with me, take a look who the simpler test sees me as closest to:

I have lived in California for all three elections when Gavin Newsom ran for governor —2010, 2018, and 2022 — and never voted for him.

I will absolutely never vote for him.

Open up a dictionary and Gavin Newsom’s face appears next to “Archetypal California Liberal Democrat Who Believes Absolutely Nothing Except the Need for Expensive Patrick Bateman Haircuts.”

This version of the quiz apparently regards me as having politics similar to what I did 18 years ago:

According to this 36-question version of the test, I am 99% economically aligned with George Soros, the leading left-wing activist patron of the 21st century:

I literally actively supported the 2009-2011 efforts by my late boss David Horowitz and his buddy Glenn Beck to cast Soros as the leading left-wing boogeyman. Yet, this test instead thinks that he and I are 99% in line on economics.

Compare this to the 240-question version where I am 99% in line with classical liberal economist David Ricardo. This is accurate because of how much Thomas Sowell I’ve absorbed. Sowell wote a whole book titled On Classical Economics which lays out Ricardo and other classical liberal thinkers. I still need to read it.

Soros and Ricardo are at opposite ends of the economics spectrum spectrum.

Meanwhile, the 240-question quiz seems to see me as more a Clinton than a Bush:

On the book recommendations, again, the short quiz suggests books that I read nearly 20 years ago:

I’ve never had any interest in Albert Camus, BTW.

The French are not good at philosophy and should generally stop embarrassing themselves further attempting it.

French philosophers have inflicted enough of their bad ideas on the world for centuries. The French should stick to their primary contributions to humanity: amazing wine, delicious pastries, Avant Garde films, and their stereotypically beautiful women.

(R.J.B. - Echoes of Curiosity, you get a pass on your philosophy Substack. Go forth and redeem your people.)

The 240 test at least gets closer on further reading with Barry Goldwater appearing:

Likewise, in identifying the adjacent ideologies, it names two of the three same ones as the 60-question one, and “Atlanticism” generally isn’t an ideology I tend to associate with — it’s known simply as “the status quo everybody serious accepts” — but I’ll take it, as I certainly am a strong NATO-backer.

Things again get weird with the shorter test’s results:

I guess you could say that I have general sympathies toward “Christian Anarchism” as defined while I certainly embrace “federalism” and “green liberalism,” the latter which I gather is just the center-left/center-right general common sense consensus on environmental issues.

Interestingly, while all three tests disagree about how to classify which ideology I’m closed to, they all agree on which one is furtherst away from me, and I ACTUALLY AGREE!

“Juche” Socialism = the Kim family’s Communist cult of personality in North Korea.

---

Just How Many Questions Do You Really Need to Ask to Understand Someone?

So this little exercise seems rather paradoxical doesn’t it?

On the one hand, it certainly makes sense why a 36-question test of someone’s beliefs could yield much weaker results than a 60-question test.

But how could it be that a test which collects four times as much data could end up with a less accurate result — Bill Clinton and Tony Blair instead of Ronald Reagan and Barry Goldwater — instead?

Well, it’s one of the strange paradoxes of research:

Just because you collect more data, that does not mean that you will necessarily gain a more accurate picture of reality. The reason is simple:

Signal and noise.

The more data you collect, the more “noise” you have to filter out. Just how much of this data is actually of a high enough quality? How much of it is even useful?

For me as a journalist, which is a better situation: an interview with someone where I have 15 minutes or 60 minutes?

Unless the interview subjct is somebody absolutly mind-blowing amazing then I’ll take the 15 minutes every time. Because then when the interview subject and I know we only have 15 minutes then we’re both going to do our best to get all the important information right in there. Whereas if we know we have our time then we’ll meander this way and that, leading to me having to spend so much more time going through the interview to try and see what’s really in there of value.

And that’s the same thing with people in real life too. When time is more limited then people tend to focus on what matters more. By asking me 240-questions, the quiz then struggled more to weigh which issues really mattered to me more, hence it somehow failed to realize that I’m a center-right guy, not a center-left guy.

And that’s why the middle-ground, 60-question test seemed to deliver the most accurate results. It knew the most important questions to ask. That test was an A+ in its answer. It literally recommended books on my table right now.

The 240-version at least landed in the neighborhood. It has some things really wrong, but others notably right, so we’ll split the difference and give it a C.

And, of course, the 36-question test is a flat-out F.

Heh. Gavin Newsom. Yeah, GTFO with that!

So now it’s your turn: take the quizzes and see which one most aligns with who you are. Maybe your results will be different from mine and the shorter or longer versions will be better than the middle one. You tell me.

Please share your results with me on Substack Notes and let me know your thoughts in the comments below or by responding to the email if you are reading this in your email client.

And a WARM WELCOME again to our recent wave of new subscribers. Your support has really meant a lot to me. You — and everyone on list — is always invited to reach out to me with questions, comments, or suggestions.

---