A research team uses millions of tweets from the Arabic Middle East in 2014 and 2015 to uncover suspicions about the US and ISIS and a deep mistrust of entities wielding power in the region.
By Michelle Nicholasen
New research by David Romney, Amaney Jamal, Robert Keohane, and Dustin Tingley finds that Arabic Twitter participants who are negative toward ISIS are also more likely to hold negative views of the United States. This counter-intuitive finding is explored in their new International Quarterly Studies paper, “The Enemy of My Enemy Is Not My Friend: Arabic Twitter Sentiment toward ISIS and the United States.” We sat down with David Romney and Dustin Tingley to ask them how they managed to analyze such a large dataset of tweets and what the negative content might reveal about the enduring impact of US foreign policy in the region.
Q: What inspired you to do this analysis?
DAVID ROMNEY: We started this project back when ISIS was in its heyday. As we were finishing a paper on anti-Americanism for Perspectives in Politics, we came across lots of surprising anti-US content that was related to other entities, like ISIS or Iran, talking about the US being in cahoots with these entities. We were surprised by that and decided to take a step back and ask—what does the population of people who care about the US and ISIS look like? Are they negative toward both, or are they negative toward ISIS and positive about the US?
Q: Hence the “enemy of my enemy is my friend” scenario, which is also the title of your paper?
DUSTIN TINGLEY: We had thought that it could be the case that people who dislike ISIS might like the US, because the US is engaging in an intervention that seems targeted at removing ISIS. And there is a certain logic to this stemming from ideas around alliance politics in international relations. If a country is opposed to one’s foes, then this implies some degree of shared interest. Hence the “enemy of my enemy is a friend.” Obviously, that was not the case. It was more like the opposite, for reasons we try to explain in the paper.
Q: What was going on in 2014–2015?
DAVID ROMNEY: This is the period when ISIS was making its strongest gains in Iraq. In particular, they'd captured a couple very large cities in Iraq, and they had a strong presence in Syria due to the power vacuum that started with the attempted revolution in Syria—that turned into armed conflict between various groups and the Assad regime.
The US engaged in some limited intervention, mostly in terms of bombing particular targets and providing aid to some of the groups fighting against ISIS on the ground, not wanting to engage in a troops-on-the-ground type campaign, but wanting to engage somewhat to stop the spread of ISIS. The other thing to remember is that this period petered out fairly quickly; by 2016–2017 ISIS’s gains were mostly reversed.
Q: You had access to millions of Arabic texts from a data company called Brandwatch. Can you explain your relationship?
DAVID ROMNEY: We worked with Brandwatch, a data company that is really a merger between our original partner Crimson Hexagon and Brandwatch. I should mention that Gary King in the Department of Government at Harvard was one of the board members of Crimson Hexagon and helped found the company.
We were maybe an odd partner, as most of the people they work with are actually in the commercial sector, companies who are interested in evaluating what their brand social media presence looks like. To help companies do this, Brandwatch has traditionally focused on analyzing text from social media sites (like Facebook and Twitter) using algorithms. They are always expanding their offerings—you can now analyze other types of “unstructured data” like video and images, and they are constantly adding new sources of data. But our analysis in this paper focuses on posts from Twitter.
Q: So, the company scrapes data from public Twitter accounts?
DAVID ROMNEY: They don’t even have to scrape data, depending on the source. They are a special client of Twitter, so they have direct access to all of the tweets as they come in—Twitter just funnels it all to them.
One advantage this gave us is that we had access to tweets that had been deleted or censored by Twitter, which was important because starting around the time of our data, Twitter became more aggressive in removing posts by ISIS because of publicity surrounding ISIS’s social media presence. This is something you cannot do anymore because of changes in policy, but we had access for our project.
Q: How many tweets did you actually receive?
DAVID ROMNEY: If you're looking at the aggregate analysis, the first table in our paper, it refers to something like eighty or 100 million tweets; it's actually not a sample, it’s the entire universe of posts that matched our filtering criteria.
In our criteria, we said, we want all Arabic-language tweets from 2014–2015 that mention the US or ISIS. Then, to train an algorithm to categorize these tweets, the platform would randomly select ones for me and my colleagues to then place into categories based on what we were interested in, which was generally sentiment (positive, neutral, or negative) and sometimes other content.
The next step in our analysis was to correlate anti-US and anti-ISIS sentiment, and that's where things get a little more complicated, because then we had to look at individual-level data at that point.
Q: Going back to the universe of all Arabic tweets from 2014–2015, there existed negative and positive and neutral sentiment. What was the breakdown?
DAVID ROMNEY: For tweets mentioning the US, about 70 percent were negative—pretty overwhelmingly negative—and then about 30 percent neutral. For ISIS it was about 35 percent negative and the majority neutral at about 60 percent. There was some positive content for ISIS as well, about 8 percent.
We think this level of positive content toward ISIS is tied to the fact that ISIS had such a strong social media presence, especially during the early part of our time period of interest. As you get toward the end of our timeline, that positive content goes down quite a bit.
We think that has to do with Twitter banning ISIS accounts. But it also has to do with ISIS doing some things that really ticked off a lot of Sunni Muslims. For instance, you have the killing of the Jordanian pilot, whom they burned alive on video and was a huge news item in the Middle East. People in the Middle East were outraged at what ISIS was doing. You see a lot of negative sentiment for ISIS around events like that.
Q: But what you were really interested in were the tweets that linked the US and ISIS. How many of those did you find?
DAVID ROMNEY: We have two analyses in the paper that examine patterns linking the US and ISIS. The first is more broad: we look at about 250,000 people who posted something about ISIS and then use our categorized data I mentioned before to estimate the sentiment of what they post about the US. This let us determine that negativity toward ISIS and the US are correlated even when you look at individuals.
However, this doesn’t let us look closely at what people are saying when they talk about the US and ISIS. To dig more into that, we had to step away from using algorithms, because they just weren’t accurate enough. So instead, we randomly selected 100 users we estimated to be really negative toward ISIS, and we examined all of their tweets about the US.
We then categorized these users based on what they said. If they never connected the US and ISIS, we labeled them as “No Linkage.” If they linked the US and ISIS but never talked about a conspiracy between them, we labeled them as “Linkage–Non-Conspiratorial.” If they linked the US and ISIS and did at some point mention a conspiracy between them, we labeled them as “Linkage–Conspiratorial.”
Overall, about 50 percent of this sample linked the US and ISIS in some way, falling into one of these latter two categories.
Q: What did you consider to be non-conspiratorial?
DAVID ROMNEY: Non-conspiratorial meant that they linked them in the sense that US actions in some way are responsible for making room for ISIS. This is actually a somewhat common view among Sunni Arabs. There's a view that the US got rid of a stable regime that was friendly to Sunni Arabs—Saddam Hussein’s—and replaced it with a regime that is more friendly to Iran and to Shia Muslims. Sunni Arabs belong to the same sect of Islam as Saddam Hussein, so they view this as one of the negative ramifications of US involvement: to sort of open Iraq to Iran, and also destabilize Iraq and create a situation where groups like ISIS can come in and fill a power vacuum.
If you look at some of these tweets that were linking US and ISIS non-conspiratorially, they're very explicit in this. The first one we note in the paper says, “The Soviet Union's intervention in Afghanistan brought us al-Qaeda; America's intervention in Iraq brought us ISIS; what is the Russians’ intervention in Syria going to bring us?”
The idea is that foreign powers are meddling in the affairs of Middle Eastern states and this creates instability and opens the room for extremist groups to flourish, basically, and so that's what a lot of this non-conspiratorial content is about.
Q: So, In the non-conspiratorial case, the US is seen as taking actions that inadvertently lead to bad outcomes. Not so for the conspiratorial tweets. Can you explain the distinction?
DAVID ROMNEY: In the conspiratorial content there’s this added element of secret action, of secret motives. The US actually wanted ISIS to take power: its main goal was to create instability and to support groups like ISIS who can make Sunni Muslims look bad and help destabilize the region.
Q: Is the assumption that they are working together?
DAVID ROMNEY: Yeah, exactly right.
Q: Of the tweets that linked the US and ISIS, how many were conspiratorial/non-conspiratorial?
DAVID ROMNEY: In the sample of 100 people that we analyzed closely, there were forty-eight people (or about 50 percent) who linked the US and ISIS in some way. Out of those, forty-three of them did so in a conspiratorial manner.
Q: Was this finding surprising, it being completely opposite from the hunch you started with?
DAVID ROMNEY: Yes, it was surprising that the majority of people who linked the US and ISIS did so in a conspiratorial manner. The tweets claimed the US and ISIS were in cahoots, and sometimes involved a weird smorgasbord of different actors: like Israel and the US and Iran secretly creating ISIS to destabilize the region, that sort of thing.
For example, one tweet says, “America claims to be fighting ISIS! I believe the more correct expression would be: America is supporting ISIS.” And another says, “ISIS is a Zionist and American creation and it is apparent that (even) their passports are printed in Israel or the US.”
Q: How do you make sense of the outlandishness of the conspiratorial claims?
DUSTIN TINGLEY: It is hard to say. In separate work supported by the WCFIA, I looked at conspiracies around “chemtrails,” wherein it is claimed that commercial and military airplanes are spraying chemicals. Pretty weird stuff, but there is clearly a connection being made that sits on top of deep mistrust of powerful interests. In this case with ISIS, the historical experience of the Arab Middle East with Western intervention may have laid the foundation for motivated reasoning in the interpretation of current actions by the US in the area. With societies around the world awash in conspiracies, including in the US where extreme partisanship is leading to a variety of conspiracies, I think it is very important to understand the institutional and psychological foundations of conspiracy theories.
For instance, in research on the United States, a lot of work has been done on the relationship between partisanship and belief in conspiracy theories, as well as on the effect of things like education, extremism, and belief in supernatural forces, usually through survey and lab experiments.
Q: What do you mean by “motivated reasoning”?
DUSTIN TINGLEY: By motivated reasoning, we mean a process that happens when people encounter new information. In an ideal world, you might hope that someone who encounters new information would do their best to interpret that information accurately. However, a lot of research shows that we don't reason in this manner. We often begin with a desired conclusion—something we already want to believe—and then find a way to interpret new information that supports our desired conclusion. This is the "motivated" part of motivated reasoning: we are motivated by this goal of supporting our desired conclusions, even if the most accurate interpretation of the new information shouldn't bring us to that conclusion.
Q: How can this research inform those who make decisions about foreign policy?
DAVID ROMNEY: I think the important point here is that it says something important about the way that citizens interpret information about foreign entities.
At times, researchers have thought that people, when they receive new information, are going to weigh it in an accurate or rational manner. I think that’s what people believe sometimes, especially in international relations. This is the idea that the nation that you belong to has a set of interests that you're going to be thinking about when you ask yourself, “Okay, which one of these foreign entities do I feel warm toward or cool toward—or which of these do I favor?”
But I think what our analysis shows is that in countries that have had a lot of experience with foreign intervention and US intervention, their historical experience really paints the way that they interpret new information and new actions by entities. So, it’s not necessarily the case that if the US comes in and intervenes, even in a way that a large majority of the Arabic public might in some sense favor (because most people don't like ISIS), you won’t still have this historical experience that is going to paint the way they interpret those actions.
They are going to potentially assume some sort of ulterior motive and that's going to influence the extent to which they might believe misinformation about US intervention in the region.
DUSTIN TINGLEY: More broadly, it is important for foreign policy makers to consider and think through ways that the public will respond to their actions. Social media has the ability to lift the voice of individuals and organizations in ways that earlier eras did not. Time and time again we have seen how important it is to develop trust with local communities and institutions. But if we ignore or half-heartedly engage with them then we are just laying the groundwork for the sort of conspiratorial connections to proliferate.
Q: What do you think the results would be if you did this analysis today?
DAVID ROMNEY: I don’t think we would be able to, because I think ISIS is just not salient in conversations, to the point where it would be really hard to get content and try to estimate people’s sentiment. ISIS is not really in the news as much anymore.
There is also the fact that the US is decreasing its involvement in the region; we just withdrew from Afghanistan, for instance. I would hope this makes the specter of US intervention and its relationship to entities in the region a little bit less salient, that there’s less of a tendency to interpret things in a conspiratorial manner; but again, this is all speculation.
Q: You took measures to ensure you weren’t analyzing posts created by bots or by “angry uncle” users who always post negative content. This hints at the caveats of using social media in analysis. Overall, do you think using social media is a reliable way of assessing public opinion? Do you hope to do more research using these methods?
DUSTIN TINGLEY: There is a reason that a range of firms have emerged to tap into social media, including the company we worked with. It is reliable for assessing how interested populations will react to salient events. Notice I used the term “interested” as these are people who have already selected to speak up. In the aggregate, you can't use it to assess public opinion the way you would a poll, because people select into the platform and then select into posting/not posting about a particular topic. But public opinion polls face a different problem, where people respond to questions but really don’t have an opinion. Moving forward, there is a lot of interesting work to do in different areas, including on topics like climate change.
Weatherhead Center Faculty Associate Dustin Tingley is a professor of government at Harvard University. He is Deputy Vice Provost for Advances in Learning; faculty director for the Vice Provost for Advances in Learning Data Science and Technology Group (Harvard higher education data science group), and faculty director for the Harvard Initiative on Learning and Teaching. His research interests include international relations, international political economy, and experimental approaches to political science.
David Romney was the 2020–2021 Raphael Morrison Dorman Memorial Postdoctoral Fellow in the Weatherhead Scholars Program at the Weatherhead Center for International Affairs, and he is now assistant professor of political science at Brigham Young University. His research interests focus on the psychology of intergroup relations, social media and text analysis, and the Middle East and Southeast Asia.
Sampling process used to analyze the dataset of Arabic tweets, from the paper “The Enemy of My Enemy Is Not My Friend: Arabic Twitter Sentiment toward ISIS and the United States” in International Studies Quarterly. Credit: Kristin Caulfield
AMMAN, JORDAN - FEBRUARY 06, 2015: Thousands of Jordanians participate in a mass demonstration after Friday prayers near Al Hussein Mosque to express their solidarity with the pilot murdered by the Islamic State (IS) group earlier this week. Muath al-Kasaesbeh was captured by the terror group after crashing his plane near Raqqa in northern Syria, during a mission against IS in December. Credit: Jordan Pix/ Getty Images
Example of a conspiratorial tweet from Romney et al. dataset. “America claims to be fighting ISIS! I believe the more correct expression would be: America is supporting ISIS.” Credit: Josie O’Toole