Generating Diverse Negations from Affirmative Sentences

TL;DR

This work proposes NegVerse, a method that tackles the lack of negation datasets by producing a diverse range of negation types from affirmative sentences, including verbal, non-verbal, and affixal forms commonly found in English text.

Abstract

Despite the impressive performance of large language models across various tasks, they often struggle with reasoning under negated statements. Negations are important in real-world applications as they encode negative polarity in verb phrases, clauses, or other expressions. Nevertheless, they are underrepresented in current benchmarks, which mainly include basic negation forms and overlook more complex ones, resulting in insufficient data for training a language model. In this work, we propose NegVerse, a method that tackles the lack of negation datasets by producing a diverse range of negation types from affirmative sentences, including verbal, non-verbal, and affixal forms commonly found in English text. We provide new rules for masking parts of sentences where negations are most likely to occur, based on syntactic structure and use a frozen baseline LLM and prompt tuning to generate negated sentences. We also propose a filtering mechanism to identify negation cues and remove degenerate examples, producing a diverse range of meaningful perturbations. Our results show that NegVerse outperforms existing methods and generates negations with higher lexical similarity to the original sentences, better syntactic preservation and negation diversity. The code is available in https://github.com/DarianRodriguez/NegVerse

Figures (6)

• Figure 1: (Top): Overview of NegVerse steps. The input sentence $x$ is masked with blanks on specific positions based on structural rules. The masked sentences $\bm c$ and the original sentence $x$ are used to create prompts that are fed to a pretrained language model, which then generates $n$ candidate negations, $\hat{\mathcal{X}}_{gen}$. A filtering mechanism selects the most relevant negations from these candidates, producing the final set, $\hat{\mathcal{X}}_f$. (Bottom Left): (A) The input data is masked at different spans using the token [BLANK]. Masks cover different parts of the sentence, or the entire sentence (see sentence 3). (B) Training samples concatenate the input text with the masked sentence and the target words needed to fill the blanks. Each span is separated by the token [ANSWER], and [SEP] separates the context from the answers. During inference, the model accepts the sentence as input, masks the sentence, and predicts the words to fill the blanks, effectively negating the input text. (Bottom Right): Summary of our token selection rules for masking. Tokens are chosen based on Part-of-speech (POS) tags and dependency labels. NegVerse masks either the selected token or its entire subtree.
• Figure 2: Illustrative example of sentences that follow our proposed blank placement rules. Although some sentences comply with multiple rules, only the words matching the specific rule are highlighted in green for each case. Below each sentence, possible negations that can be introduced by filling in the blanks are provided. This example demonstrates how this placement strategy can produce diverse forms of negation. The arrow sign ($\rightarrow$) indicates that when the word is a determiner (DET), it masks the accompanying noun or adjective, allowing the model to generate richer negations.
• Figure 3: Dependency parse tree representing the grammatical structure of an example sentence. (A) The syntactic structure of the sentence, with arcs representing grammatical dependencies between words. Dependency labels (dep tags) are displayed on the arcs, and part-of-speech tags (POS) are shown under each word, illustrating the sentence's syntactic structure. (B) Tokens within the subtree rooted at the selected token are highlighted in yellow. The highlighted tokens are then masked with [BLANK] instead of just the individual token. If a verb is selected, all words dependent on it within the sentence are included in the subtree, resulting in the entire sentence being masked.
• Figure 4: An illustrative example of sentence masking. The masking function considers a maximum of two tokens per sentence, and six different masked sentences. Part (A) represents the masked sentence with Polyjuice automatic masking, where the main verb is masked in none of the options, nor were the adjectives with possible affixal negated forms like "insusceptible". Additionally, in Option $3$, a [BLANK] was inserted rather than replacing a token. Part (B) shows how the proposed approach masks a sentence. In particular, Option $6$ masks the adjective, Option $4$ masks the main verb, and the other options mask in places to produce non-verbal negations.
• Figure 5: Prompt template used to generate data with affixal negations by leveraging one-shot learning with an instruction-following LLM assistant using Llama-2-7b-Chat. The text in blue indicates where the new pair of words is inserted for inference.