Identifying Different Semantic Features of Public Engagement with Climate Change NGOs Using Semantic Network Analysis

By inergency On Feb 9, 2024

1. Introduction

The landscape of information dissemination and accessibility has undergone a profound transformation with the advent of social media, exerting a significant influence on public discourse and engagement across socio-cultural, political, and policy domains [1,2]. Ranging from simple likes, shares, and brief comments easily accessible to a range of public audiences [3] to profound discussions taking place within topic-oriented communities on social media platforms [4,5,6], these cost-effective communication tools empower activists and ordinary individuals to readily participate in discussions concerning various issues, thereby amplifying their voices effectively. With this influence, social media serves as a conduit for both online and offline activism, mobilizing ‘non-expert publics’ (hereafter, ‘publics’) (In current research, the term ‘non-expert publics’ refers to non-expert individuals who recognize a problem or opportunity, and take action to address it (in this case, climate change), as well as those who have built or can build relationships with relevant organizations (in this case, organizations active on climate change) [7,8,9,10]) and advocating for climate policies [11,12,13,14].

Among the paramount concerns that have garnered substantial traction on social media is the predicament of climate change—an issue of utmost significance confronting our global landscape. According to the 2018 United Nations’ Intergovernmental Panel on Climate Change (IPCC) report, countries worldwide are not doing enough to limit CO₂ emissions. The report highlights the urgent need to reduce global emissions by 45 percent by 2030 compared to 2010 levels [15]. Additionally, the IPCC, established in 1988, has consistently emphasized the severity of our current inaction and the future consequences of climate change [16,17]. In this context, social media serves as a means to express concerns about prominent climate change issues (e.g., extreme weather), share pertinent information, engage in discussions about climate science, and participate in climate politics [18].

Numerous non-governmental organizations (hereafter NGOs) are actively involved in advocating for climate change-related concerns, connecting global audiences and translating complex climate science for a range of public audiences [19]. Some climate change NGOs have been operating since the 1970s, with notable examples such as Greenpeace, which has been instrumental in influencing national and international discussions on environmental matters, including deforestation, since its establishment in 1971. For these NGOs, social media platforms have been valuable tools to address their diverse communication goals despite limited resources [20]. Specifically, social media has been identified as a crucial tool for climate-focused NGOs, enabling them to establish a direct information flow to a range of public audiences about climate issues, attract widespread media coverage, appeal to both internal and external audiences to take action on these issues, and establish direct communication channels with policymakers [19,21].

While the significance of social media for climate NGOs—key communicators on climate issues—has been acknowledged, we contend that there is a need for further exploration into the dynamics of engagement between these NGOs and their target audiences. While online platforms have been recognized as a means for deliberate scientific discourse with a range of public audiences [5], and dialogic two-way communication between NGOs and their audiences has been advocated for better engagement [22], existing research has primarily concentrated on the content and manner in which NGOs address climate change matters on social media, within one-way communication framework. This framework assumes that NGOs exert a unidirectional influence on publics, often relying on the metrics of public engagement on social media (e.g., number of likes) as the solo indicator of communicational success. In essence, there has been limited investigation into how various stakeholders, particularly publics, engage in discussions with climate NGOs, and consequently, there is a dearth of understanding regarding the organizations’ communication strategies and how these strategies relate to these interactions.

In light of the existing research gap, this study challenges conventional public engagement metrics on social media in two ways. First, it questions the notion that popularity, commitment, and virality, which are gauged through metrics such as likes, replies, and shares [23], should be considered indicators reflecting the identical success of communication strategies, as each type of public engagement may be linked to distinct aspects of interactions between the organization and its corresponding audiences [24]. Second, to gain a comprehensive understanding of ‘public engagement’, it advocates for an exploration not only of communications from the organization, but also of dialogues between the organizations and their publics, thus shedding light on the discourses facilitated by publics.

Therefore, guided by the two-way communication approach, our objective was to investigate the distinct characteristics of discussions related to climate change, with a particular focus on the themes and focal points in conversations initiated by both climate NGOs and publics across various types of public engagement on social media. Using semantic network analysis (SNA), we uncovered distinctive patterns and conversation themes between organizations and their audiences, which reflect the specific types of public engagement on social media platforms. Additionally, our study explored the extent to which these organizations and their specific target audiences maintain alignment in the subjects and central themes of their conversations across these different modes of engagement. This was accomplished through an analysis of the semantic similarity between the discourses of the organizations and their respective publics. We also examined the emotional alignments between organizations and their publics, as it could offer valuable insights into effective social media strategies. Through these explorations, this study enhances our understanding of the dynamics at play in the ‘two-way’ interactions between climate NGOs and their respective audiences and improves social media communication strategies tailored to the distinct engagement objectives of each organization.

3. Materials and Methods

To measure public engagement with the climate NGOs on social media, we adopted and revised three public engagement measures with organizational social media accounts from Bonsón and Ratkai [43] and Haro-de-Rosario et al. [23] (i.e., citizen engagement): popularity (i.e., popularity of messages [from the climate NGOs] in public engagement); commitment (i.e., commitment of public in the communication with the climate NGOs); and virality (i.e., virality of messages among publics’ communication). More specifically, popularity measures the frequency of affective reactions from the public to social media messages, while commitment indicates a higher and more sustained level of engagement [44]. Virality represents the breadth of a message’s reach [44]. The three dimensions of public engagement have been operationalized as shown in Table 1.

This study aimed to explore the relationships between (a) public engagement and (b) shared meaning between social media communications of climate NGOs and their corresponding public. To operationalize shared meaning within this research’s context, we utilized semantic similarity metrics [40]. These metrics enable the identification of similarities between terms or texts that convey the same meaning, even if they do not exhibit lexical similarity [45]. More specifically, we used two semantic similarity metrics: (a) Euclidean distance and (b) Levenshtein distance. Euclidean distance is a measure calculating the straight-line distance between the corresponding coordinates of two points in a multidimensional space [46]. Within the context of assessing semantic similarity between two texts, it quantifies the distance between their vectorized representations in a multidimensional space, ranging from 0 to positive infinity [47]. Levenshtein distance is a measurement for quantifying the dissimilarity between two strings, which calculates the number of single-character edits (e.g., insertions, deletions, or substitutions) needed to transform one string into another [48]. A lower score for the two measurements indicates a higher level of similarity between the two examined documents, possibly suggesting the presence of shared meaning as the organization and its public have common or at least similar themes or focal points in their discourses.

This study investigates the sentiment alignment between a climate NGO’s tweets and the replies the organization received. To identify sentiment alignment, we conducted a correlation analysis between the trends of the organization’s weekly sentiment score and the ones of the corresponding replies that we computed.

To identify the prevailing themes and focal points of climate NGOs’ Twitter posts and following public discourses (represented in public audience replies to the climate NGOs), we used SNA. The method allowed us to identify key concepts—used by the NGOs and publics—and their interpretive contexts, by analyzing the significance of specific words based on their frequency and centrality measure values, as well as their co-occurrences and clustering patterns within the text [49,50]. In practice, previous studies have identified salient themes and frames within texts in various contexts, such as the ESG policies in sustainability reports of corporations, and publics’ discussions on childhood vaccination and COVID-19 vaccines [49,51,52].

3.1. Data Collection

We collected Twitter posts (i.e., tweets) posted by ten climate-change NGOs and corresponding public replies that were sent to those organizations from 1 July 2020 to 30 June 2021. Twitter was selected as the representative social media platform for this research because it is suitable for organizations including climate NGOs to share information publicly and engage with publics [53,54,55]. This time range was selected to encompass various factors such as seasonal climate change issues (e.g., extreme weather conditions, flooding), U.S. national political or policy-making issues (e.g., presidential election, Keystone XL pipelines), and global climate change concerns [56]. The acquisition of Twitter data (i.e., tweets and replies) was accomplished by employing the data collection service offered by exportcomments.com (1 September 2023) [57], which enabled us to extract relevant data including the textual content of tweets and replies, the date and time of posting, the number of likes received by the post, and the number of retweets generated by the post.

The organizations were selected from a list of top NGOs working to stop climate change [58] and then screened based on (a) relevance of their posts to climate change issues and (b) comparable volumes of posts from their Twitter accounts. The finalized list of climate NGOs in this research and brief information about the accounts are available in Table 2.

3.2. Analytic Approach

For RQ1, we calculated the three public engagement metrics for each organization following the formulas suggested by Haro-de-Rosario et al. [23].

For RQ2 and subsequent research questions that involve sentiment analysis and SNA, we performed pre-processing on the textual content of organizational tweets and replies that we collected. More specifically, we removed URLs, stopwords, and non-contextual elements, which included punctuation and special characters, except for the ‘@’ symbol used to indicate mentioned accounts. Additionally, we excluded Twitter function words such as ‘replying to’ to focus on the relevant content for our analysis. We then lemmatized and tokenized for each corpus (e.g., a group of tweets from an organization, a group of replies sent to an organization) using packages such as spaCy [59] and TextBlob [60] on Python.

For RQ2, we computed two semantic similarity metrics, namely Euclidean distance and Levenshtein distance, between each organization’s corpus and its corresponding reply corpus. We utilized Scikit-learn [61] and scipy.spatial [62] libraries to calculate the semantic similarities for each organization. To assess the relationships between three types of public engagement and the two types of semantic similarity, we performed a correlation analysis.

To conduct sentiment analysis for RQ3, we employed the Azure sentiment analysis model, developed through Microsoft Azure machine learning (version: 5.2.0), to calculate weekly average sentiment scores [63]. The sentiment analysis model provides a “sentiment label” (positive, negative, neutral) along with a confidence score for each post, ranging from 0 (lower confidence) to 1 (higher confidence). Using the model, we identified sentiment label for each preprocessed tweet/reply. To quantify sentiments and compare changes within the two groups (i.e., tweets from an organization vs. replies to the organization), we devised a “weighted sentiment score” by assigning numeric values to sentiment labels (positive = 1, negative = −1, neutral = 0) and multiplying these values by the corresponding confidence score. Higher scores close to 1 indicated more significantly positive posts, while scores close to −1 indicated more significantly negative posts. Weekly sentiment scores were determined by calculating the average sentiment scores of tweets and replies published during each week. Next, we conducted an analysis to explore the correlations between the weekly sentiment score trends of an organization’s posts and the corresponding replies. Our aim was to identify organizations that exhibited a significant and positive correlation, indicating their success in maintaining alignment with the sentiments expressed by the reply public.

To address RQ4, we employed SNA, which involves examining the structure of a semantic network constructed from a large volume of unstructured text datasets [64]. In this method, each word (e.g., ‘climate’, ‘change’) is treated as a node within a network, and the analysis focuses on the co-occurrences of these words [65]. These co-occurrences, such as the words ‘climate’ and ‘change’ appearing together in a single post, are counted as instances of co-occurrence. Representing the links between nodes, these co-occurrences are crucial for calculating closeness among words (i.e., nodes). SNA allows for a spatial representation of language structure, enabling the visual grasp of relationships between main concepts (e.g., ‘climate change’ in the current context) originating from specific terms and their connections to other concepts derived from different terms (e.g., ‘conservation’ or ‘wildfire’ in the current context) [65]. The visualization capability of SNA empowers researchers and professionals to uncover insights that might not be immediately apparent through traditional quantitative or qualitative analysis, thereby enhancing our understanding of complex and contextual information underlying the text.

Using the preprocessed text from the previous stage, which included tokenization, lemmatization, and the removal of stop words, we first extracted the most frequent words from the corpus and converted these into a list, annotating each with the frequency of its co-occurrence with other words (i.e., weight). For example, we created a link-list showing how often words like “change”, “crisis”, or “disaster” co-occurred with “climate” in the same post within the corpus of the NGO’s posts or replies to the NGO. Subsequently, employing NeTxt [66], we transformed the processed text and this annotated link-list into a network. In this network, the words serve as nodes, and their co-occurrences become the ties. As a result, we generated a semantic network for each corpus, resulting in six networks, each featuring the top 150 frequent words. For a more detailed step-by-step explanation, you may refer to Segev [66,67].

The generated semantic networks were then exported as weighted edge lists and converted to a format suitable for analysis in Gephi [68], using Python. Subsequently, we imported the data into Gephi to visualize and explore the networks further. Within Gephi, we conducted modularity analysis to identify distinct clusters or themes within each network and calculated various network statistical indicators, such as degree and eigenvector centrality, as outlined by Segev [66] and Luo et al. [69]. These measures helped us determine the importance and prominence of specific keywords within the networks.

5. Discussion

The aim of this exploratory study was to enhance our understanding of public engagement on social media, focusing on the features of two-way communications on social media. In addition to traditional public engagement metrics, we employed sentiment analysis and examined shared meaning with semantic similarity measures in our research. Furthermore, by utilizing SNA, we identified and compared key themes and focal points within climate NGOs’ tweets and the corresponding replies they received. Through our explorations, we discovered several noteworthy findings.

First, we observed that climate NGOs developed unique themes and focuses within their discourses. This highlights the distinct characteristic of science/organizational communication on social media, where organizations can participate in or establish topic-driven communities [34]. Social media allows individuals and organizations to engage in focused discussions on particular topics. To effectively connect with and mobilize like-minded individuals who share climate-related interests, climate NGOs may strategically monitor and select topics and issues to allocate their attention and communication efforts. Our study revealed that strategically selecting topics and discussion foci may encourage specific types of public engagement. For instance, Climate Central, by aligning its topics and foci with those of publics in terms of scientific information about climate change (with aligned sentiments with publics (Table 5)), it maintained shared interests with the committed publics and generated a high level of public commitment. In other words, both organization and the engaged publics aligned their perspectives and sentiments on the issues [71], incorporating their discussed issues (This alignment is facilitated potentially by both inviting certain groups of publics who share interests and knowledge about the issues that the specific NGO advocates for, and by mutually shifting their agendas and frames to reflect each other’s viewpoints). For example, when the organization focused on the ‘extreme temperature’ issue, the committed public shared their knowledge and opinions with the organization as below.

Climate Central: “RT @afreedma: This graph helps explain why heat extremes are becoming so much more common/severe in a warming world. [link redacted]”

A public: “On average, July in Raleigh-Durham is 4 degrees hotter now than it was in the 1970s. (See all locations here: [link redacted]) #ClimateMatters”

Understanding the themes and focuses that resonate with the public may enable climate NGOs to tailor their communication strategies and content effectively, thereby enhancing engagement and mobilization.

This finding highlights the significance of effectively managing organizational discourses while simultaneously monitoring public interests and concerns, considering the specific types of public engagement being targeted. Similar to the GPU in our sample, when attempting to engage a broader audience like publics, it would be advisable to concentrate on uncontested climate change topics such as nature, rather than delving into specific issues like “fossil fuel” that may have been raised by more active segments of publics (See Table 7).

Second, our findings highlight the importance of avoiding a one-size-fits-all strategy focusing solely on increasing public engagement scores, such as likes and retweets, in the research context. Instead, we recommend that climate NGOs tailor their communication strategies based on the specific types of public engagement they are targeting on social media [24]. We found that organizations that succeeded in generating specific types of public engagement on social media did not necessarily succeed in generating other types. For example, Climate Central achieved a high level of commitment but had low virality. This could be attributed to the nature of the topics chosen by Climate Central, which focused on climate sciences and meteorology: topics requiring a high level of science literacy from publics. If Climate Central aimed to engage more informed and active participants in the discourse, the low popularity and virality might be an insignificant concern. Conversely, organizations like GPU, which focused on climate topics generally applicable to the wider public, achieved high popularity but avoided highlighting national or regional political issues (e.g., U.S. presidential election). In contrast, EDF achieved high virality by actively mentioning political figures and associating climate change issues with the responsibility of the government (e.g., Biden action, Biden administration). Although EDF’s approach might have appealed more to the U.S. public than international audiences, it generated intense virality as engaged individuals shared EDF’s messages with like-minded individuals on social media. These examples suggest that organizations need to establish specific objectives regarding the types of public engagement on social media. Accordingly, they should allocate attention and interest to specific issues, as each objective necessitates different communication strategies.

Third, our analysis revealed significant and positive correlations between the weekly sentiment scores of the three organizations, namely GPU, Climate Central, and EDF, and the corresponding sentiment scores of the replies they received. Remarkably, the three organizations demonstrated commendable performance in at least one public engagement metric, such as popularity, commitment, and virality. While the correlations based on weekly sentiment scores may not provide a complete depiction of how organizations align with publics’ interests, they do highlight the importance of organizations focusing on current issues that evoke a range of sentiments [72]. By aligning their interests and attitudes with publics on these issues, climate NGOs may enhance public engagement by leveraging the incorporated attentions and shared sentiment surrounding these topics [42].

Furthermore, the study uncovered counter-intuitive findings. Specifically, the semantic similarity between organizational posts and public replies did not have a positive relationship with public engagement on social media. In other words, when organizations generated higher engagement from publics in terms of popularity, commitment, and virality, the use of similar themes and focuses was either insignificant or low. This might be due to the fact that organizations with more engagement are likely to have more diverse audiences, as their posts are shared not only with like-minded individuals but also individuals with different perspectives and opinions on climate issues. Investigating the network attributes of organizations’ social media communities could confirm these counter-intuitive findings in future studies.

There are several limitations and opportunities for further development in this study. First, to fully understand the implications of specific themes and focuses in generating public engagement, one must delve further into the influence of specific words and frames used in posts or posts within shorter time periods, such as a few days or weeks. Since Twitter imposes a 280-character limit, we had to explore the semantic features of all posts and replies within a one-year period, which may not fully capture the ‘real-time’ dynamics of dialogues between organizations and publics. Second, as a case study, we only investigated a few organizations with successful public engagement generation. To gain a more comprehensive understanding, it is important to explore whether our findings are generally applicable to a wider range of organizations and other contexts. Additionally, to comprehend the implications of specific types of public engagement on social media, it would be beneficial to investigate the network characteristics of an organization’s communities. For example, understanding how posts become viral can be better understood by examining how these posts are shared with individuals outside an organization’s immediate network. By conducting larger-scale investigations and employing additional analytic approaches, we can gain deeper insights into public engagement within this context.

6. Conclusions

In conclusion, our study sheds light on the themes, focuses, and sentiments identified in social media discourses of climate NGOs and their publics, within the framework of public engagement. Recognizing the imperative for climate NGOs to maximize the ‘two-way’ communication capabilities of social media to educate, persuade, captivate, and understand their target audiences concerning climate change issues, we delved into the interactive dynamics of these discourses. Based on our exploration, we advocate for communication strategies that are more oriented toward the public audience’s understanding of and interest in climate issues. This involves:

Assessing public perceptions and understanding of climate topics, as exemplified by the challenges faced by Climate Central in making scientific discourses appealing to lay public audiences.
Exploring the depth and variety of climate-related issues that captivate publics’ interest, demonstrated by the case of GPU, which focused on broader climate issues.
Understanding how different publics associate different issues with climate change, such as the disparate linking of climate change with political and wildfire issues in the communications of EDF and its public audiences.

When these tailored strategies align with each organization’s specific communication objectives and target audiences (e.g., individuals with interests and knowledge in climate science, in the case of Climate Central), they are likely to contribute to more desirable public engagement in climate change discourses. These shifts in strategies from ‘delivering effective messages to the public audiences in order to educate them’ (e.g., the IPCC report) to ‘understanding and representing the interests and issues of the public audiences’ also resonate the call for the shift in first-order thinking to second- and third-order thinking in science communication [73].

Specifically, with the ‘two-way’ approach indicative of second-order thinking, climate NGOs can not only build consensus on climate change issues but also directly address the uncertainties and concerns of publics [73]. This approach aligns with the principles of second-order thinking by prioritizing dialogue, engagement, and building trust through a transparent and accountable communication style.

Furthermore, by embracing the diverse perspectives and agendas of different public audiences, which is a hallmark of third-order thinking, these organizations can situate climate change within a wider cultural, societal, and political context of the publics as in the case of EDF. This approach goes beyond organization-led initiatives, recognizing the importance of heterogeneity and constructive disagreement as valuable societal resources to address climate change issues [73]. This multidimensional perspective will enhance reflexivity and critical analysis in climate communication, which is crucial for addressing ‘wicked problem’ [74] like climate change.

We also acknowledge the additional limitations of our study in terms of application, including the need to explore the influence of specific words and frames within shorter time periods, expand the scope of organizations and contexts studied, and investigate network characteristics for a more comprehensive understanding. Continued research in this area can contribute to a better understanding of public engagement and communication strategies of climate NGOs’ social media presence.

The current study used SNA, following previous studies [75,76], and recognized the limitations of methods like topic modeling in comparing multiple corpora and identifying nuanced themes within a topic [77]. Newly designed methods, such as contrastive topic modeling [77] and qualitative approaches (e.g., Haupt et al. [78]), which address these limitations, may be considered in our future studies.

We conducted sentiment analysis using the Microsoft Azure API (version: 5.2.0), which offers a parsimonious and accessible analytic approach for communication professionals to capture current public sentiments and potential sentiment gaps between the organization’s communication and publics’ responses. We propose that the practitioners may monitor what leads to specific sentiments among public audiences during specific periods, with a more careful review of the original posts and replies from these publics. However, to develop more sophisticated sentiment analysis, practitioners need to consider dictionary-based approaches or develop their own trained models for such analysis to reflect their organization’s specific context.