I don’t think your results are surprising due to an issue with your setup. You aren’t really running a Turing test.
Consider the statement, “I am a Republican.” Can you tell if the person who wrote that is republican or democrat? Of course not! Anyone can say those words.
How is writing a paragraph all that different?
I think what you need is to conversations, which is what the Turing test required. You’d need people to be able to ask questions - why do you think this, why do you think that, what about such and such evidence.
Hi @apxhard, I think this is definitely a valid concern. However, do you think this relates more to a flaw in the researchers’ methodology or American politics more broadly?
In the future directions section, Adam mentions this Turing Test could gain more credibility if it would be a 1,000 word prompt compared to 100 words. I want to say, “it’s BS all the way down.” I think it’s so easy to feign political allegiance; both sides have claims/principles/values that can be adopted quickly. It’s super memeable.
Instead, I wonder how these ideological or social Turing Tests could be adopted for other forms of identity. On one extreme, we have something like politics that can be faked. On another extreme, we have something like family. I sure hope nobody except my family can recount that Summer ‘09 vacation to Chicago. In the middle, we could have something like hobbies. For example, I stand no chance faking a D&D player because I’ve never played.
To this point, my hypothesis would be that we identify with groups where we share the same kind of language. A form of insider baseball. These would be the groups people might be best at correctly identifying the fakers. Hopefully there is a link between what you identify with and where you can identify the bullshit artist painting with too broad of a brush.
I think you’re onto something, but I have to insist the experimental setup is wrong: The Turing test has a specific definition: a conversation. Not a one off statement. It’s factually incorrect to say that people have passed a test they haven’t passed. I think you will get significantly different results if you ask people to try and pretend to be something they aren’t, not for one word or even a thousand words, but for a 10 minute conversation.
Having said this, I think your characterization fits with most people, but definitely not all. There’s another kind of conversation you can have, where you talk about reasons for believing what you believe. This is fundamentally a different kind of conversation, and I suspect most people can’t do this, without getting indignant. If you ask someone “why do you believe x”, they frequently interpret as a kind of “attack on x”, as opposed to curiosity or a desire to understand and connect.
The hard thing here is, I’ll be the results from mechanical Turk will be very different from results you’re get comparing elites. Vivek Ramaswamy could fool any democrat into think he’s one of them via text. I don’t think anyone except maybe Matt Yglesias could pull off the reverse. It’s impossible to be a white collar elite without being bombarded by messaging from the left, since being a democrat is practically a white collar professional norm. I would expect a valid experiment to capture this dichotomy, or at least have some chance of doing so. Any democrat can pretend to believe global warming is a scam, I don’t think most could make the argument that “policy attempts to mitigate climate change won’t actually fix the situation because they misunderstand the relationship between energy usage and human flourishing”, because they’ve almost never encountered this. When Vivek says it, my democratic peers can’t argue against it because they’re aghast that he’s not saying what they think all smart people will say.
I don't think any experimental project, can ever be perfect, or flawless. It will be better informed with some of the variant forms added to the program that some might have suggested.
But the real point here, or two points actually, one is the obvious that AI is not about credulity. The second point is that people may vote republican or democratic they only know they are republican or democrat. I am a registered independent considered too liberal by my conservative friends and family and too conservative y liberals. But I speak with both and neither can discern the truth about their own parties.
For instance most of those I know who support Trump don't believe Trump means what he says, but if you only speak about specific issues they are against what their own party or some in their party may stand for instance they all think abortion should be to some degree legal, but believe who claim they can't get treatment for a miscarriage are democratic fabrications.
They don't want any cuts in any social programs either, but will tell you that the budget needs to be cut so there is less national debt.
On the other hand the democrats will tell me that we can have unlimited growth and a vibrant middle class at the same time. Of course they don't understand if the economy grows and banks and businesses are unregulated the middle class shrinks. Well you can believe whether or not that is true for yourself but many democratic candidates do not. On the other hand most democratic candidates support a great deal of the programs for the needy, while my democratic acquaintances tend to believe the only reason the poor exist is because they choose to be poor and while they are disinclined to suggest any specific program they would like to see cut, and in general favor all of them, at the same time they think there are too many social welfare programs and recipients are just lazy and should work as hard as they do.
If people do not even understand what their parties stand for and basically don't even comprehend the meaning of liberal or conservative how can they understand whether a democratic or republican statement true or false.
The truth is neither party is a singular ideology in its constituents, although they have becoming more and more so in their representatives.
And the other truth is probably most of "news" is construed to represent the negative of the other and a very generalized positivity of their own party, And the candidates of both parties are frequently deceptive in both their own aspirations and the aspirations of the other,
The point I may be trying to make is perhaps we should be eliminating parties who can never be representative of issues but thrive by convincing people---that whatever their policies are the solutions to everyone's "problems" and so the "truth" doesn't really exist and perhaps that is why there is a great deal of difficulty in knowing the true from the false statements and no one I know who is strongly partisan really even understands their own party and certainly knows nothing about the other.
Well that is a personal survey, but my sample size is considerably larger and it is formed by direct interviews that some have suggested might be a failing.
My sampling is around 1200 people I have contact with but obviously some more than others. They are all "lower-middle" and a much higher percentage of republican than democrat so not an honest sampling in parity. But across generations from the 20's to my own age group (70's) but probably quite balanced amongst the age groups. It contains very few minorities, and as I mentioned, few white collar. There is a pretty distributive balance in educational levels, between religious affiliated and non-religious affiliated, but overall would not stand to any strict scrutinous test of being called a "fair sampling". But it might be a better cross-section than some who might have similar conservations if all of their discussions are primarily only with those whom they agree.
I had a similar thought. It seems like the experiment might just be testing whether people can tell if someone is lying via text.
Adam writes that "According to the Ideological Turing Test, both sides seem to understand each other about as well as they understand themselves." But that seems like a bold claim, and one I'm not sure would hold up if the participants were put in the same room.
When I took the test, a lot of the answers were pretty banal. So if all you have to do is articulate the other party's most surface-level features, you're going to get a lot of answers that are vague, and therefore plausible. Would the results change if the questions demanded deep knowledge of a subject? I haven't read the Yeomans study, but I wonder if that's what happened there.
It could be interesting to see if there's a relationship between the questions the Readers got right, whether the Writer was lying or not. (Sorry if that was in the post and I missed it.) If there is, I wonder if it has to do with how specific the Writer got.
Having just taken the vanntile test, one thing that struck me is that I identified one as Real not so much because I'm good at identifying Republicans or Democrats, but because that one had the characteristics of a true post that a fake one wouldn't have: it was very long and rambly with a very specific local anecdote. Didn't even have to read the full thing.
This would be amazing if it had featured an intellectual Turing Test, rather than a test of spotting plausible-sounding vague platitudes. Not even attempting to measure the right thing.
"We got 902 participants on Amazon Mechanical Turk, roughly split between Democrats and Republicans. (Sorry, Independents: to take the study, you had to identify with one side or the other.)"
Apparently the ITT team didn't know what -- or how much -- they were missing. Gallup has been taking party "affiliation" polls for quite some time now . . . and it's been true for the past dozen years and more that more people identify with neither Republicans nor Democrats than with either.
Maybe it's time for ITT-2, where the groups compared are self-identified supporters or non-supporters of -- well, the "traditional" or "establishment" parties? Or feel free to come up with some other neutral terms.
(BTW . . . yes, I see the follow-up question Gallup asks. It might be nice if there were a parallel follow-up for those who say they're Democrats or Republicans -- something no more "pushy" than the one they're pushing on independents . . . maybe something like this: "If you knew your one vote would decide the election, who would you vote for?")
There are any number of reasons to claim to be unaffiliated: I think in most cases it's some kind of half-baked, half-thought-out smish of "my privacy" and "they'll put me on Some Kind of List" and "I'm American and I run alone." Of these, I think the List is the most relevant: I've voted straight-ticket Democrat for 36 years and I identify as "independent" on forms. I'm currently just overwhelmed with blue fundraising texts, not completely tsunami-ed out like my more outspoken liberal friends. Fast and frugal heuristic, works for me.
I think it is important to acknowledge that partisans get their idea of what the other side believes from party leaders and activists, who are often very unrepresentative of regular party voters. Partisans strongly dislike what the leaders and activists of the opposite party say, but they do not realize that regular party voters also have misgivings about their own party.
So it is not so much misunderstanding as misattribution. The party leaders and activists are far more polarized than voters, but the voters are forced to choose between only two sides.
And this is what I came here to say! I know hardly anyone (anecdotal, I know) who agrees with the extreme rhetoric of their own party, although obviously there are some, but instead we’re all basically forced to pick one of two sides. Like I’m a die-hard socialist with a wide libertarian streak who therefore upon occasion has voted republican; so I’m reasonably sure I know what each party hypothetically stands for. So I do like an experiment that finds that of course every middle of the road person actually does have a pretty good handle on what a person of another party thinks
I wonder if for the ChatGPT result the poor performance of the model is due to it being trained to be trusting of humans in general. I'd be interested to know how well ChatGPT performs if you specifically fine-tune it to peform well (the OpenAI API allows for this).
Adding on to WD Lindberg's comment, I went and looked more closely at the graphs under "COULD WRITERS PREDICT THEIR (LACK OF) SUCCESS?" and noticed that some of the data points go above 100% or below 100% on the x or y axis. For the y axis, I don't think that's possible, and you should check the data analysis and visualization. For the x axis, there in one data point below 0% and several above 100%, which is also very strange. What was the data input method for that question in the survey?
I suspect the actual "reasonable hypothesis" here may have something to do with the ability to identify what's reasonable in a particular written text. ("Likely" is a huge component of what's "reasonable.")
Democrat, 7/10 on test (you oughta share the overall results, unless I missed 'em), used a "likely" heuristic that served me pretty well. I do read well over a million words a year (60-65 books, about half that much in news, articles, Experimental History, etc.). I've done fairly well on "is this real" sorts of tests before, and it may be correlated?
I'm also reminded of the successful intervention to cut down college drinking--all they did was make widely available how much students *actually* drank, which tended to be far less than the students assumed, and knowing this helped them cut down even further.
I really wish there had been some explanation for why some people *do* perform well. I have to imagine that raising the number of people that can pass this test would have some beneficial effects on society.
This reminds me of Tetlock’s research on superforecasters. Most people are not good at predicting the future, but some are and they have common features. I’d love to reverse engineer that for ideology detection.
My take on why Democrats thought more Republican statements were fake overall, whether they were real or not:
I'm a die-hard Democrat. Raised in a red state but now have lived my whole adulthood in blue states. My one anecdotal explanation was that as I was taking the test, a lot of the Republican statements felt too over-the-top for me. Like too extreme. I thought they were caricatures of what Democrats think Republicans sound like. But it turns out I guess that is what a lot of Republicans really think .... I think us Democrats are still just surprised at Republican beliefs overall?
I think these are really interesting results, but I'm not entirely sure it's the best way of measuring whether the two sides understand each other. As apxhard says, a really simple way for a Democrat to fool people into thinking she's a Republican is to write "I am a Republican". I think what you need to do is something a little more elaborate.
I'll explain - I'm going to describe this as though all the Writers were Democrats, but of course you'd run the experiment using a group from each side.
Half the Writer test group gets told "write something that will convince other Republicans, *and only them*, that you are a Republican. You will get $10 for each Republican who thinks you're a Republican, minus $10 for every Democrat who thinks you're a Republican."
The other half gets the opposite: "write something that will convince Democrats, *and only Democrats*, that you are a Republican. $10 for each Democrat convinced, minus $10 for each Republican convinced."
If the two sides really understand each other, those will be impossible tasks. There won't be any shibboleths that one side all knows and the other doesn't.
But if they don't understand each other at all, then you might get a Writer producing something like "I'm voting Trump because he's going to protect our Christian guns from demonic gay immigrants", because she sincerely thinks that's how Republicans think - and all the Democrats agreeing that she's definitely a Republican, while all the Republicans think "clearly a fake, any true Republican would know that we Republicans only care about import tariffs, the military, and disliking Hillary Clinton".
I'm curious if the act of sitting down and writing a paragraph from the other party's perspective itself influences people to form a better understanding of the other party than they would have otherwise. This intuitively feels wrong to me, but one comparison I have in mind is that apparently people who do Jordan Peterson's Future Authoring Program perform significantly better in life afterwards. I haven't confirmed the stats on that to be sure, but from that I get a sense that maybe some people don't actually ever stop to deeply consider certain ideas until someone else makes them write it out clearly.
I don’t think your results are surprising due to an issue with your setup. You aren’t really running a Turing test.
Consider the statement, “I am a Republican.” Can you tell if the person who wrote that is republican or democrat? Of course not! Anyone can say those words.
How is writing a paragraph all that different?
I think what you need is to conversations, which is what the Turing test required. You’d need people to be able to ask questions - why do you think this, why do you think that, what about such and such evidence.
Hi @apxhard, I think this is definitely a valid concern. However, do you think this relates more to a flaw in the researchers’ methodology or American politics more broadly?
In the future directions section, Adam mentions this Turing Test could gain more credibility if it would be a 1,000 word prompt compared to 100 words. I want to say, “it’s BS all the way down.” I think it’s so easy to feign political allegiance; both sides have claims/principles/values that can be adopted quickly. It’s super memeable.
Instead, I wonder how these ideological or social Turing Tests could be adopted for other forms of identity. On one extreme, we have something like politics that can be faked. On another extreme, we have something like family. I sure hope nobody except my family can recount that Summer ‘09 vacation to Chicago. In the middle, we could have something like hobbies. For example, I stand no chance faking a D&D player because I’ve never played.
To this point, my hypothesis would be that we identify with groups where we share the same kind of language. A form of insider baseball. These would be the groups people might be best at correctly identifying the fakers. Hopefully there is a link between what you identify with and where you can identify the bullshit artist painting with too broad of a brush.
I think you’re onto something, but I have to insist the experimental setup is wrong: The Turing test has a specific definition: a conversation. Not a one off statement. It’s factually incorrect to say that people have passed a test they haven’t passed. I think you will get significantly different results if you ask people to try and pretend to be something they aren’t, not for one word or even a thousand words, but for a 10 minute conversation.
Having said this, I think your characterization fits with most people, but definitely not all. There’s another kind of conversation you can have, where you talk about reasons for believing what you believe. This is fundamentally a different kind of conversation, and I suspect most people can’t do this, without getting indignant. If you ask someone “why do you believe x”, they frequently interpret as a kind of “attack on x”, as opposed to curiosity or a desire to understand and connect.
The hard thing here is, I’ll be the results from mechanical Turk will be very different from results you’re get comparing elites. Vivek Ramaswamy could fool any democrat into think he’s one of them via text. I don’t think anyone except maybe Matt Yglesias could pull off the reverse. It’s impossible to be a white collar elite without being bombarded by messaging from the left, since being a democrat is practically a white collar professional norm. I would expect a valid experiment to capture this dichotomy, or at least have some chance of doing so. Any democrat can pretend to believe global warming is a scam, I don’t think most could make the argument that “policy attempts to mitigate climate change won’t actually fix the situation because they misunderstand the relationship between energy usage and human flourishing”, because they’ve almost never encountered this. When Vivek says it, my democratic peers can’t argue against it because they’re aghast that he’s not saying what they think all smart people will say.
amen x 100
I don't think any experimental project, can ever be perfect, or flawless. It will be better informed with some of the variant forms added to the program that some might have suggested.
But the real point here, or two points actually, one is the obvious that AI is not about credulity. The second point is that people may vote republican or democratic they only know they are republican or democrat. I am a registered independent considered too liberal by my conservative friends and family and too conservative y liberals. But I speak with both and neither can discern the truth about their own parties.
For instance most of those I know who support Trump don't believe Trump means what he says, but if you only speak about specific issues they are against what their own party or some in their party may stand for instance they all think abortion should be to some degree legal, but believe who claim they can't get treatment for a miscarriage are democratic fabrications.
They don't want any cuts in any social programs either, but will tell you that the budget needs to be cut so there is less national debt.
On the other hand the democrats will tell me that we can have unlimited growth and a vibrant middle class at the same time. Of course they don't understand if the economy grows and banks and businesses are unregulated the middle class shrinks. Well you can believe whether or not that is true for yourself but many democratic candidates do not. On the other hand most democratic candidates support a great deal of the programs for the needy, while my democratic acquaintances tend to believe the only reason the poor exist is because they choose to be poor and while they are disinclined to suggest any specific program they would like to see cut, and in general favor all of them, at the same time they think there are too many social welfare programs and recipients are just lazy and should work as hard as they do.
If people do not even understand what their parties stand for and basically don't even comprehend the meaning of liberal or conservative how can they understand whether a democratic or republican statement true or false.
The truth is neither party is a singular ideology in its constituents, although they have becoming more and more so in their representatives.
And the other truth is probably most of "news" is construed to represent the negative of the other and a very generalized positivity of their own party, And the candidates of both parties are frequently deceptive in both their own aspirations and the aspirations of the other,
The point I may be trying to make is perhaps we should be eliminating parties who can never be representative of issues but thrive by convincing people---that whatever their policies are the solutions to everyone's "problems" and so the "truth" doesn't really exist and perhaps that is why there is a great deal of difficulty in knowing the true from the false statements and no one I know who is strongly partisan really even understands their own party and certainly knows nothing about the other.
Well that is a personal survey, but my sample size is considerably larger and it is formed by direct interviews that some have suggested might be a failing.
My sampling is around 1200 people I have contact with but obviously some more than others. They are all "lower-middle" and a much higher percentage of republican than democrat so not an honest sampling in parity. But across generations from the 20's to my own age group (70's) but probably quite balanced amongst the age groups. It contains very few minorities, and as I mentioned, few white collar. There is a pretty distributive balance in educational levels, between religious affiliated and non-religious affiliated, but overall would not stand to any strict scrutinous test of being called a "fair sampling". But it might be a better cross-section than some who might have similar conservations if all of their discussions are primarily only with those whom they agree.
I had a similar thought. It seems like the experiment might just be testing whether people can tell if someone is lying via text.
Adam writes that "According to the Ideological Turing Test, both sides seem to understand each other about as well as they understand themselves." But that seems like a bold claim, and one I'm not sure would hold up if the participants were put in the same room.
When I took the test, a lot of the answers were pretty banal. So if all you have to do is articulate the other party's most surface-level features, you're going to get a lot of answers that are vague, and therefore plausible. Would the results change if the questions demanded deep knowledge of a subject? I haven't read the Yeomans study, but I wonder if that's what happened there.
It could be interesting to see if there's a relationship between the questions the Readers got right, whether the Writer was lying or not. (Sorry if that was in the post and I missed it.) If there is, I wonder if it has to do with how specific the Writer got.
(I did poorly on the test, in case it matters.)
Having just taken the vanntile test, one thing that struck me is that I identified one as Real not so much because I'm good at identifying Republicans or Democrats, but because that one had the characteristics of a true post that a fake one wouldn't have: it was very long and rambly with a very specific local anecdote. Didn't even have to read the full thing.
This would be amazing if it had featured an intellectual Turing Test, rather than a test of spotting plausible-sounding vague platitudes. Not even attempting to measure the right thing.
"We got 902 participants on Amazon Mechanical Turk, roughly split between Democrats and Republicans. (Sorry, Independents: to take the study, you had to identify with one side or the other.)"
Apparently the ITT team didn't know what -- or how much -- they were missing. Gallup has been taking party "affiliation" polls for quite some time now . . . and it's been true for the past dozen years and more that more people identify with neither Republicans nor Democrats than with either.
https://news.gallup.com/poll/15370/Party-Affiliation.aspx
Maybe it's time for ITT-2, where the groups compared are self-identified supporters or non-supporters of -- well, the "traditional" or "establishment" parties? Or feel free to come up with some other neutral terms.
(BTW . . . yes, I see the follow-up question Gallup asks. It might be nice if there were a parallel follow-up for those who say they're Democrats or Republicans -- something no more "pushy" than the one they're pushing on independents . . . maybe something like this: "If you knew your one vote would decide the election, who would you vote for?")
There are any number of reasons to claim to be unaffiliated: I think in most cases it's some kind of half-baked, half-thought-out smish of "my privacy" and "they'll put me on Some Kind of List" and "I'm American and I run alone." Of these, I think the List is the most relevant: I've voted straight-ticket Democrat for 36 years and I identify as "independent" on forms. I'm currently just overwhelmed with blue fundraising texts, not completely tsunami-ed out like my more outspoken liberal friends. Fast and frugal heuristic, works for me.
This is an interesting experiment.
I think it is important to acknowledge that partisans get their idea of what the other side believes from party leaders and activists, who are often very unrepresentative of regular party voters. Partisans strongly dislike what the leaders and activists of the opposite party say, but they do not realize that regular party voters also have misgivings about their own party.
So it is not so much misunderstanding as misattribution. The party leaders and activists are far more polarized than voters, but the voters are forced to choose between only two sides.
And this is what I came here to say! I know hardly anyone (anecdotal, I know) who agrees with the extreme rhetoric of their own party, although obviously there are some, but instead we’re all basically forced to pick one of two sides. Like I’m a die-hard socialist with a wide libertarian streak who therefore upon occasion has voted republican; so I’m reasonably sure I know what each party hypothetically stands for. So I do like an experiment that finds that of course every middle of the road person actually does have a pretty good handle on what a person of another party thinks
I wonder if for the ChatGPT result the poor performance of the model is due to it being trained to be trusting of humans in general. I'd be interested to know how well ChatGPT performs if you specifically fine-tune it to peform well (the OpenAI API allows for this).
Adding on to WD Lindberg's comment, I went and looked more closely at the graphs under "COULD WRITERS PREDICT THEIR (LACK OF) SUCCESS?" and noticed that some of the data points go above 100% or below 100% on the x or y axis. For the y axis, I don't think that's possible, and you should check the data analysis and visualization. For the x axis, there in one data point below 0% and several above 100%, which is also very strange. What was the data input method for that question in the survey?
Very good, but 100 word essay without interaction wouldn't make possible to tell apart a human from LLM, let alone two different kind of humans
Important digestif to this is Devon Price’s most recent piece.
I suspect the actual "reasonable hypothesis" here may have something to do with the ability to identify what's reasonable in a particular written text. ("Likely" is a huge component of what's "reasonable.")
Democrat, 7/10 on test (you oughta share the overall results, unless I missed 'em), used a "likely" heuristic that served me pretty well. I do read well over a million words a year (60-65 books, about half that much in news, articles, Experimental History, etc.). I've done fairly well on "is this real" sorts of tests before, and it may be correlated?
I'm also reminded of the successful intervention to cut down college drinking--all they did was make widely available how much students *actually* drank, which tended to be far less than the students assumed, and knowing this helped them cut down even further.
Fascinating. I wonder how the data have changed during the 2024 election season.
Great article!
Is the data available somewhere? I'd like to do some of my own analysis
I really wish there had been some explanation for why some people *do* perform well. I have to imagine that raising the number of people that can pass this test would have some beneficial effects on society.
This reminds me of Tetlock’s research on superforecasters. Most people are not good at predicting the future, but some are and they have common features. I’d love to reverse engineer that for ideology detection.
My take on why Democrats thought more Republican statements were fake overall, whether they were real or not:
I'm a die-hard Democrat. Raised in a red state but now have lived my whole adulthood in blue states. My one anecdotal explanation was that as I was taking the test, a lot of the Republican statements felt too over-the-top for me. Like too extreme. I thought they were caricatures of what Democrats think Republicans sound like. But it turns out I guess that is what a lot of Republicans really think .... I think us Democrats are still just surprised at Republican beliefs overall?
I think these are really interesting results, but I'm not entirely sure it's the best way of measuring whether the two sides understand each other. As apxhard says, a really simple way for a Democrat to fool people into thinking she's a Republican is to write "I am a Republican". I think what you need to do is something a little more elaborate.
I'll explain - I'm going to describe this as though all the Writers were Democrats, but of course you'd run the experiment using a group from each side.
Half the Writer test group gets told "write something that will convince other Republicans, *and only them*, that you are a Republican. You will get $10 for each Republican who thinks you're a Republican, minus $10 for every Democrat who thinks you're a Republican."
The other half gets the opposite: "write something that will convince Democrats, *and only Democrats*, that you are a Republican. $10 for each Democrat convinced, minus $10 for each Republican convinced."
If the two sides really understand each other, those will be impossible tasks. There won't be any shibboleths that one side all knows and the other doesn't.
But if they don't understand each other at all, then you might get a Writer producing something like "I'm voting Trump because he's going to protect our Christian guns from demonic gay immigrants", because she sincerely thinks that's how Republicans think - and all the Democrats agreeing that she's definitely a Republican, while all the Republicans think "clearly a fake, any true Republican would know that we Republicans only care about import tariffs, the military, and disliking Hillary Clinton".
I'm curious if the act of sitting down and writing a paragraph from the other party's perspective itself influences people to form a better understanding of the other party than they would have otherwise. This intuitively feels wrong to me, but one comparison I have in mind is that apparently people who do Jordan Peterson's Future Authoring Program perform significantly better in life afterwards. I haven't confirmed the stats on that to be sure, but from that I get a sense that maybe some people don't actually ever stop to deeply consider certain ideas until someone else makes them write it out clearly.