Multifeature fusion for claim scope-aware litigation risk prediction for patent drafts

Patent scope indicators

Quantifying the scope of a patent is a longstanding challenge in intellectual property research. Numerous indicators have been proposed to estimate the breadth of legal protection and technological applicability offered by patents. These can be categorized into the following groups:

Citation-based indicators: Citation analysis has been extensively utilized in patent research, primarily through backward and forward citation metrics. The number of forward citations, originally proposed by Trajtenberg (1990), is widely used to assess a patent’s technological impact, with a higher number generally interpreted as reflecting broader scope. The number of backward citations indicates the extent of prior art reviewed, suggesting a wide technological foundation (Packalen & Bhattacharya, 2012). In addition, non-patent literature (NPL) citations indicate a broader research base supporting the invention, as noted by Narin, Hamilton & Olivastro (1997). However, forward citations are not available for early-stage or draft patents, limiting their practical utility during the drafting phase.

Patent classification-based indicators: Classification-based indicators assess technological breadth based on the number of categories assigned to a patent. Studies by Lerner (1994) and Harhoff, Scherer & Vopel (2003) have demonstrated that patents with a greater number of subclasses tend to span a wider array of technological fields, reflecting broader applications and scope.

Claim-based indicators: Claim-based indicators are among the most direct measures of patent scope and can be further divided into two subgroups: indicators based on claim quantity and those based on claim structure.

Claim quantity-related indicators focus on the number and types of claims included in the patent. The number of claims is widely recognized as a measure of scope, with a greater quantity generally suggesting broader protection (Lanjouw & Schankerman, 1997, 2001, 2004). Similarly, the number of independent claims is interpreted as reflecting wider coverage, since each independent claim typically represents a distinct technological aspect (Marco, Sarnoff & Charles, 2019; Graham & Mowery, 2003). In contrast, dependent claims, although providing specificity and detail, do not significantly contribute to a broader scope (Graham & Mowery, 2003).

Claim structure-related indicators evaluate the linguistic, syntactic, and logical organization of individual claims. Commonly used metrics include words per claim (Lerner, 1994; Osenga, 2011; Harhoff, 2016), independent claim length (Malackowski & Barney, 2008; Marco, Sarnoff & Charles, 2019), and first claim length (Harhoff, 2016; Wittfoth, 2019). These studies have suggested that shorter claims are generally broader in scope due to fewer embedded limitations. Okada, Naito & Nagaoka (2016) introduced character count as an alternative metric, particularly useful in languages without word spacing, arguing that longer character sequences correlate with narrower, more detailed claims. Additionally, claim dependency structure, as explored by Wittfoth (2019), plays a role in defining the hierarchical and interpretive relationship between independent and dependent claims, impacting how broadly a claim set may be interpreted.

Semantics-based indicators: In response to limitations of numeric and bibliographic features, recent studies have introduced semantics-driven approaches. Tanaka, Nakashio & Kajikawa (2018) proposed the use of semantic range of words to measure vocabulary diversity, enabling scope visualization through semantic hierarchies. Ragot (2023) introduced a novel textual metric called self-information, which quantifies the informativeness of individual claims. Their findings suggest that higher self-information scores correlate with broader conceptual scope.

The number of inventors has also been interpreted as an indirect scope metric. Chan, Mihm & Sosa (2021), highlighted that a higher number of inventors reflects greater collaboration and the non-decomposability of the invention. The scope tends to decrease with the number of inventors.

Table 1 summarizes existing scope indicators, outlining their theoretical bases and known limitations. While these metrics span a range of approaches, they predominantly rely on bibliographic data, numeric heuristics, or surface-level linguistic cues. Notably absent are robust, semantically informed indicators capable of evaluating the breadth of a patent claim based on its underlying meaning and hierarchical structure. Currently, no widely adopted method allows authors to quantify whether a claim is semantically broad or narrow. This gap hinders precise calibration of claim scope and increases the risk of either under-protecting the invention or inviting legal challenges due to overly broad claims.

Patent litigation prediction models

The prediction of patent litigation has evolved substantially, transitioning from traditional statistical models to sophisticated machine learning (ML) and deep learning (DL) frameworks. The existing literature can be organized into the following categories:

Classical machine learning approaches: Early work in this area focused on regression-based and tree-based models. Chien (2011) employed logistic regression (LR) to analyze how specific intrinsic and acquired patent traits influence the likelihood of litigation. Juranek & Otneim (2021) used the XGBoost algorithm with features drawn from united states patent and trademark office (USPTO) datasets, OECD patent quality indicators (PQI), and USPTO patent litigation docket reports, achieving high predictive performance (AUC up to 0.818). They identified that indicators related to patent value, internationality, and patent owner characteristics hold higher predictive power. However, their model’s reliance on post-grant information limits its applicability during the drafting phase. Similarly, Follesø & Kaminski (2020) utilized random forest (RF) classifiers trained on PQI-derived features to assess litigation risk.

Semantic and similarity-based approaches: Several researchers have explored textual content to infer litigation potential. Park, Yoon & Kim (2012) applied semantic similarity analysis based on Subject-Action-Object (SAO) patterns and clustering to identify potential infringement scenarios. Lee, Song & Park (2013) evaluated claim text similarity using keyword vector models and analysed inter-claim dependencies. While these methods incorporate both linguistic and structural elements, they often face limitations in scalability and generalizability, particularly across large or heterogeneous patent datasets. Although effective in identifying potential overlaps between pairs of patents, extending such analysis to all patent pairs for litigation prediction poses significant computational challenges.

Unsupervised and ensemble techniques: Several studies have integrated unsupervised learning and ensemble methods to enhance prediction accuracy. Wongchaisuwat, Klabjan & McGinnis (2017) combined K-means clustering with ensemble classification models to estimate the likelihood and timing of litigation jointly. Kim et al. (2022) applied principal component analysis (PCA) for dimensionality reduction and used Autoencoders in combination with K-nearest neighbors (K-NN) for classification, improving predictive performance by emphasizing the most informative features. Chen & Lai (2023) implemented an ensemble machine learning classifier leveraging USPTO examination and assignment data, achieving 79% accuracy and demonstrating the viability of ensemble methods for litigation risk assessment.

Deep learning models: Recent advances in deep learning have enabled the modeling of complex, multi-dimensional relationships present in patent litigation data. Liu et al. (2018) proposed a convolutional tensor factorization framework to identify high-risk patents based on textual and collaboration features. Wu et al. (2024) introduced the multi-aspect neural tensor factorization (MANTF) model to predict plaintiffs, defendants, and target patents jointly. Convolutional neural networks (CNNs) have also been utilized for one-to-many infringement detection (Liu & Pei, 2023), while Kim et al. (2021) employed random survival forests to model litigation risk over time.

The most recent and closely related work to the objectives of this study is by Juranek & Otneim (2024), who refined their XGBoost model to handle newly granted patents by minimizing reliance on post-grant features that are not available at the time of grant. In their study, the XGBoost algorithm was used for litigation prediction and achieved an AUC score of up to 0.822. However, this approach remains inapplicable to draft-stage documents due to its dependence on post-grant data.

Table 2 summarizes the prominent litigation prediction models and related studies, outlining their methodological foundations and known limitations. While these approaches span a range of machine learning and deep learning techniques, the majority rely on post-grant features such as forward citations, patent family size, assignment records, and other patent quality indicators. Models that assess litigation risk using only information available at the drafting stage are notably absent from the existing literature. In particular, semantic features embedded within patent claims, despite being central to legal interpretation and enforceability, remain largely underutilized in current predictive frameworks. Although some studies have applied semantic similarity analysis to identify potential overlaps or infringement between individual patent pairs, scaling such analyses across large patent datasets introduces significant computational challenges. Furthermore, no existing model provides a structured framework for predicting litigation risk at the draft stage using claim-level semantic features. This gap restricts the ability to conduct early-stage risk assessment and reduces the practical value of these models for inventors, legal professionals, and innovation strategists. The literature survey indicates that the proposed work is a pioneering effort for litigation prediction in patent drafts, and no comparable work for a one-to-one comparison is available.

Hyponym tree score calculation

In natural language processing (NLP), hyponyms and hypernyms represent hierarchical relationships between words, which are crucial for understanding semantics and building structured knowledge. A hypernym refers to a broader, more general term, while a hyponym refers to a narrower, more specific term that falls under the hypernym. For example, in a taxonomy, ‘vehicle’ represents a hypernym of ‘car’, and ‘car’ serves as a hyponym of ‘vehicle’. Similarly, the hypernym ‘fruit’ encompasses hyponyms such as ‘berry’, ‘banana’, and ‘mango’. These relationships are often modelled in NLP using resources like WordNet (Fellbaum, 1998), where hypernym-hyponym hierarchies are explicitly defined. Understanding such relationships enables NLP systems to infer broader or narrower meanings, which is essential for analyzing the scope of patent claim texts. Words with more hyponyms in a patent claim indicate the potential to create multiple restrictive versions of claims, which can lead to overlaps in scope representing potential infringement cases and litigation risks. Consequently, studying hyponyms within patent claim text is pivotal in devising a new scope indicator for patents.

Patent claims often employ varying levels of specificity, where claims with broader scope support a larger interchangeability of terms to protect a more extensive set of derived ideas (Cohen & Lemley, 2001). However, when a claim employs overly generic language, the claim scope increases drastically, potentially clashing with more specific claims in other patents, leading to increased litigation risks and legal uncertainties. Conversely, highly specific claims may reduce infringement risks but face challenges in enforcing their rights against variations and derivative innovations. Analyzing hypernym and hyponym characteristics within patent claim texts (Andersson et al., 2014) can potentially play a crucial role in claim scope quantification. In this context, a new HTS indicator is developed to represent the patent scope. The HTS indicator is derived by considering the hyponym count in claim sentences and their structural composition. When words in a patent claim text have more hyponyms, the possibility of interchangeability increases, broadening the claim scope. The new scope indicator will be validated by assessing its effectiveness in predicting patent litigation likelihood.

Mathematically, let the patent claim text be represented as a hyponym dependency tree, $T=(V,E)$, where $V=\{v_1,v_2,\dots ,v_n\}$ is the set of nodes corresponding to the words in the claim, and E is the set of directed edges that denote the syntactic or semantic dependency relations between these words. Each node $v_i\in V$ is associated with a degree $\mathrm{d}\mathrm{e}\mathrm{g}(v_i)$, representing the number of hyponyms (i.e., more specific terms) that can replace the corresponding word. The degree $\mathrm{d}\mathrm{e}\mathrm{g}(v_i)$ reflects the flexibility of the word within the claim text, where higher values represent greater possibilities for creating restrictive variations of the claim.

Given the tree structure, a cumulative score C for the entire set of claims as follows:

$$C(T)=\prod _{v_i\in V}(\mathrm{d}\mathrm{e}\mathrm{g}(v_i)+1),$$where $\mathrm{d}\mathrm{e}\mathrm{g}(v_i)$ is the degree of node $v_i$, representing the number of hyponyms for the word corresponding to node $v_i$ and the term $(\mathrm{d}\mathrm{e}\mathrm{g}(v_i)+1)$ accounts for the word itself (original term) and its associated hyponyms.

This cumulative score reflects the maximum number of specific or restrictive versions of the claims that could be generated from the given claim text. Each restricted version is a modified claim with a smaller scope, offering different legal interpretations and enforcement potentials. The cumulative score $C(T)$ indicates the scope of the original patent claims. A larger claim scope increases the likelihood of overlapping with other patents, a primary cause of litigation. Patents with higher cumulative scores are more prone to infringement due to the more significant number of possible interpretations and restrictive variations that could overlap with existing claims. Thus, $C(T)$ can quantify the scope or coverage of the patent claim and provide a theoretical foundation for predicting patent litigation risk based on hyponym analysis. Multiplicative $C(T)$, which calculates the number of sub-trees possible from the original tree, has a problem with the lengthy claims producing very large values, and the effect of smaller claims goes unnoticed, hence discarded.

Three tasks were carried out to calculate the HTS of claim text: claim dependency tree generation, dependency tree generation for each sentence in the claim, and hyponym extraction of the words in each sentence. Algorithm 1, the Text2Scope, was developed to compute the HTS value from the patent claim text. Claims are represented as a graph with individual claims as the nodes and the dependency among them as the edges. A claims text corpus is processed using Algorithm 1 (Text2Scope), and a tree structure of the claims is generated initially. In the tree, each node contains the text corresponding to a numbered claim. Algorithm 2 (Claim2Scope) is invoked from Text2Scope to calculate the score of a given patent claim. Claim2Scope invokes Algorithm 3 (Sentence2Scope) to calculate the score of each sentence of the given claim. Sentence2Scope algorithm involves dependency tree generation to extract the sentence structure and node weight assignment using the hyponym counts of each node(word). Then it computes the cumulative score calculation for that sentence. These algorithms return hyponym tree scores and weighted hyponym tree scores. Equation (1) represents the sentence level non-weighted score calculation. Equation (2) is used for weighted score calculation.

(1) $$\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{T}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{{\sum }_{i=1}^{c\mathrm{\_}node}\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}(w_i)}{c\mathrm{\_}node\times \mathrm{t}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{\_}\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}}$$

where:

• $\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}(w_i)$: The number of hyponyms for the $i$-th word in the dependency tree.

• $c\mathrm{\_}node$: The total number of nodes in the dependency tree.

• $\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{\_}\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}$: The maximum depth of the dependency tree.

(2) $$\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{T}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{S}\mathrm{c}\mathrm{o}\mathrm{r}{\mathrm{e}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{e}\mathrm{d}}={\displaystyle \frac{{\sum }_{i=1}^{c_{n}ode}\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}(w_i)\cdot NodeHeight(w_i)}{c\mathrm{\_}node\times \mathrm{t}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{\_}\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}}}$$

where:

• $\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{y}\mathrm{m}\mathrm{C}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{t}(w_i)$: The number of hyponyms for the $i$-th word.

• $\mathrm{N}\mathrm{o}\mathrm{d}\mathrm{e}\mathrm{H}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}(w_i)$: The height of the $i$-th word in the dependency tree.

• $c\mathrm{\_}node$: The total number of nodes in the dependency tree.

• $\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{e}\mathrm{\_}\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}$: The maximum depth of the dependency tree.

When considering the implementation options, the dependency tree of a sentence can be created using two popular NLP libraries, namely, Stanza (Qi et al., 2020) and SpaCy (Honnibal et al., 2020). It has been observed that the dependency tree representation for the same sentence differs between SpaCy and Stanza. Figures 3 and 4 shows the dependency trees created for a sample sentence using Stanza and SpaCy, respectively. The hyponym tree score is dependent upon the dependency tree structure. For this reason, the score calculation was evaluated using both libraries, and a third option was created by averaging the sentence-level scores generated by both libraries. Thus, six candidate hyponym tree scores were generated for further evaluation: three based on SpaCy, Stanza, and averaging, and the weighted versions of all three. Table 4 summarises the HTS candidates generated for evaluation. The difference between the options is primarily based on two factors: the NLP library used to generate the dependency structure of a sentence and whether the node-level score (hyponym count of that word) is multiplied by a weight. The weight corresponds to the height-based level value, where the leaf node is assigned to level 1, and the root node is assigned level N for a tree with N levels.

Hyponym tree score validation

Length-based indicators like ‘first claim length’ blindly treat lengthy claims as specific and short claims as broader, irrespective of the semantics. Ragot (2023) presented a set of fixed-length representative claims with varying scopes in section C1 of their work to study the claim scope. The same set of claims is used in this work to study the ability of the newly calculated HTS candidate values. The HTS candidate values are calculated with all the sample claims and presented in Table 5. The sentences are arranged in the descending order of their scope. Figure 5 is a normalized plot of the scope values generated for sample sentences using all the six HTS candidates under evaluation. As per the results, HTS candidates can show scope reduction, whereas the word count fails to represent any scope change. However, due to the close similarity between the results from all the HTS candidates under evaluation, a decision is made to generate all six HTS candidate scores for the entire dataset and to make the final HTS candidate selection only after a complete evaluation with the entire dataset.

Connecting HTS with litigation risk and claim scope

The preliminary validation of the relationship between HTS and claim scope (CS) is demonstrated in “Hyponym Tree Score Validation”. The results indicate that higher HTS values correspond to broader CS. Previous studies (Merges & Nelson, 1994; Arinas, 2012; Marco, Sarnoff & Charles, 2019) have established that broader claim scope increases the likelihood of litigation and legal events. By transitive reasoning, the relationship between HTS and patent litigation probability ( $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$) can be considered valid. However, not all patents with broad claim scope result in litigation, as litigation requires legal action to be pursued. This observation may weaken the relationship between HTS and litigation probability.

The evaluation model to study the connection between the HTS and CS is summarized as follows:

• Patents with high HTS are likely to have a broader CS.

• Broader CS increases the probability of litigation ( $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$).

• Patents with high HTS and high $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$ are indicative of broader CS.

• Observation: Not all patents with broad Claim Scope (CS) will result in litigation ( $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$).

Objective: A high HTS strongly predicts $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$, which indicates a broader CS. This provides a scientific basis for using HTS as a quantification method for claim scope and offers a robust framework for patent strategy formulation and risk assessment.

Predicate logic:

• Let HTS(x): Patent x has a high Hyponym Tree Score.

• Let CS(x): Patent x has a broad Claim Scope.

• Let P_lit(x): Patent x has a high probability of litigation.

Statements:

• 1.

  $\mathrm{\forall }x(\mathrm{H}\mathrm{T}\mathrm{S}(x)\to \mathrm{C}\mathrm{S}(x))$: High HTS implies broad CS.

• 2.

  $\mathrm{\forall }x(\mathrm{C}\mathrm{S}(x)\to P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x))$: Broad CS implies a high probability of litigation.

• 3.

  $\mathrm{\forall }x((P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x)\wedge \mathrm{H}\mathrm{T}\mathrm{S}(x))\to \mathrm{C}\mathrm{S}(x))$: High $P_{\mathrm{l}\mathrm{i}\mathrm{t}}$ and HTS imply broad CS.

• Observation $\mathrm{\exists }x(\mathrm{C}\mathrm{S}(x)\wedge \mathrm{\neg }{P}_{\mathrm{l}\mathrm{i}\mathrm{t}}(x))$: Not all patents with broad CS result in litigation.

Proof: Hypothesis: $\mathrm{\forall }x(\mathrm{H}\mathrm{T}\mathrm{S}(x)\to \mathrm{C}\mathrm{S}(x))$

• Proof. 1. Assume $\mathrm{H}\mathrm{T}\mathrm{S}(x)$ for an arbitrary patent $x$.

• 2. From statement 1, $\mathrm{H}\mathrm{T}\mathrm{S}(x)\to \mathrm{C}\mathrm{S}(x)$, so $\mathrm{C}\mathrm{S}(x)$ holds.

• 3. From statement 2, $\mathrm{C}\mathrm{S}(x)\to P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x)$, thus $P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x)$ holds.

• 4. From statement 3, $(P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x)\wedge \mathrm{H}\mathrm{T}\mathrm{S}(x))\to \mathrm{C}\mathrm{S}(x)$.

• 5. Given $P_{\mathrm{l}\mathrm{i}\mathrm{t}}(x)$ and $\mathrm{H}\mathrm{T}\mathrm{S}(x)$, concludes $\mathrm{C}\mathrm{S}(x)$.

Therefore, high HTS implies broad CS.

Selecting the HTS best candidate

The values of all the HTS candidates are calculated for the entire dataset. Table 6 documents the statistics of the different HTS candidates under evaluation. The distribution of the IPC sections in the dataset is shown in Fig. 1. Section G has the most samples in the dataset. Figure 6 shows the average HTS values for non-litigated and litigated patents belonging to each section. This justifies the Proof “Connecting HTS with Litigation Risk and Claim Scope”, on the IPC section level, litigated patents have a higher HTS value than the non-litigated patents and supports the connection between the Litigation probability and HTS value.

The most suitable candidate to represent the CS has to be selected from the six HTS candidates. This section presents seven experiments designed to assess the relative merit of the HTS candidate in litigation prediction. The difference between the experiments is only in the features used for the classification. Set of standard pre-grant features, termed as the baseline features, include ‘bwd_cits’, ‘npl_cits’, ‘claims_x’, ‘num_dependent_claims’, ‘num_independent_claims’, ‘assignee_pcount’, fc_word_count, avg_claim_length and ‘num_inventors’. Details of these features are documented in Table 3. Each experiment used random forest, XGBoost, support vector classifier (SVC) and balanced random forest (BRF) models to perform litigation prediction to assess the impact of including the HTS candidate feature with the baseline features.

Figure 7 presents the changes in the accuracy of the litigation prediction during each experiment with different prediction models. Experiment A (Exp-A) performs the prediction by using only the baseline features. Experiments B to G added each HTS candidate along with the baseline features. In all the experiments, XGBoost resulted in the best prediction results. Table 7 presented the features used in each experiment and the best prediction performance achieved with XGBoost. Table 8 shows the correlation between the HTS candidates and other existing patent scope or value indicators. A larger value of HTS indicated a higher litigation probability.

Figure 8 shows the variation of the metric from the experiment A results during each experiment. Accordingly, prediction accuracy improved marginally with the introduction of the HTS candidate features. Whenever the positive correlation between the HTS and Litigation probability is valid, the positive correlation between the HTS and Claim Scope is also valid as per the proof “Connecting HTS with Litigation Risk and Claim Scope”. Thus, the relationship between the HTS and Patent Scope is reconfirmed.

From the evaluation results, hts_stanza and hts_stanza_wtd are unambiguously ruled out. The hts_avg_wtd produced slightly better prediction results compared to hts_spacy and hts_spacy_wtd. During the experiments, it was observed that the creation of a stanza-based dependency graph failed for several sentences. Calculating the average scores requires both stanza and spacy-based dependency tree creation. The average scores were also discarded to avoid the dependency tree creation issues observed with Stanza. The remaining candidates are hts_spacy and hts_spacy_wtd, and experiments B and C evaluate the prediction with these features. When comparing the results between experiments B and C, hts_spacy_wtd produces slightly better results. Figure 9 represents the information gain of the candidate HTS features. As per the information gain of the candidates, hts_spacy shall be the candidate.

To overcome the ambiguity, a feature-based extremes evaluation was conducted to investigate the relationship between the candidate features and the litigation label and the results are presented in Fig. 10. Specifically, the top 100 and bottom 100 records based on the candidate feature’s values were identified, and their litigation labels were analyzed. The objective was to determine whether the top 100 records predominantly correspond to the positive class (litigated) and the bottom 100 records to the negative class (non-litigated). This analysis provided insights into the HTS candidate feature’s discriminative power, thereby offering a supplementary validation of the feature’s relevance in the litigation prediction task. Extremes analysis and correlation analysis resulted in favour of hts_spacy against hts_spacy_wtd. In these circumstances, hts_spacy was selected as the final candidate for the HTS. All further references to HTS will indicate the usage of hts_spacy as the indicator for the claim scope representation.

Proposed litigation prediction model

The proposed model, referred to as MAPRA (Multifeature BERT-Powered Fusion for Author-level Patent Litigation Risk Analysis), is specifically designed to assess litigation risks in patent drafts. The model’s architecture aims to capture both the semantic intricacies of claim text and the scope awareness conveyed by numerical features. The textual component is processed through a BERT-based encoder, which extracts semantic information essential for identifying litigation-prone claims. Concurrently, the numerical features provide supplementary insights, such as claim scope and other litigation-relevant factors, resulting in a holistic view of the data. Additionally, author and assignee details, often absent in the claim text, are incorporated as numerical features to enhance prediction accuracy. The ten numeric features used in this analysis are bwd_cits, npl_cits, claims_x, avg_claim_length, num_dependent_claims, num_independent_claims, assignee_pcount, num_inventors, hts_spacy, and fc_word_count.

Data preprocessing plays a pivotal role in ensuring the robustness of the model. Textual claims are tokenized and encoded using the BERT tokenizer, which standardizes the input by padding sequences to a fixed length. This ensures compatibility with the BERT encoder while preserving consistency in input dimensions. Simultaneously, numerical features are imputed, trimmed, normalized, and scaled to maintain uniformity across the dataset. Despite the 2% real-world prevalence of the positive class, coverage of positive samples is ensured through a 10% oversampling scheme, thereby guaranteeing that positive examples are included in each training epoch. The processed data are stratified into training (80%), validation (10%), and test (10%) sets, with both balanced (1:1) and realistic imbalanced ( $\approx 2\mathrm{\%}$ positives) splits created for evaluation. This separation is critical to mitigating overfitting and validating the model’s generalizability to unseen data.

Model architecture and workflow

The proposed MAPRA model integrates textual and numerical data modalities to predict a binary litigation outcome (litigated/not litigated). The MAPRA architecture is specifically designed to assess litigation risks, combining semantic intricacies from claim text with numeric indicators representing claim scope, inventor, and assignee attributes. Figure 11 illustrates the complete workflow of the model.

• 1.

  Input representation

• Let $x$ represent tokenized patent claim text and $\mathbf{n}=(n_1,n_2,\dots ,n_{10})$ represent the vector of associated numeric features.

• 2.

  Tokenization and embedding

  • Tokenization: Claim texts are tokenized and padded to length 512 offline using BERT tokenizer, yielding cached $input\mathrm{\_}ids$ and $attention\mathrm{\_}mask$.

  • BERT embedding: Text embeddings are generated by fine-tuning layers 8–11 of BERT while keeping layers 0–7 frozen. The output embeddings are: $$\mathbf{E}=({\mathbf{e}}_1,\dots ,{\mathbf{e}}_T),{\mathbf{e}}_i\in {\mathbb{R}}^d,d=768.$$

• 3.

  CLS token representation

• The embedding of the [CLS] token is extracted as a semantic representation of the claim: $${\mathbf{e}}_{\mathrm{C}\mathrm{L}\mathrm{S}}={\mathbf{E}}_{\mathrm{C}\mathrm{L}\mathrm{S}}\in {\mathbb{R}}^{768}.$$

• 4.

  Combining text and numeric features

  • Numeric features are subjected to median imputation, trimming, and Min-Max normalization to the range $[0,1]$.

  • Numeric embeddings ( ${\mathbf{e}}_{\mathrm{N}\mathrm{u}\mathrm{m}}\in {\mathbb{R}}^{48}$) are generated through a multilayer perceptron (MLP): $$\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(10\to 64)\to \mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\to \mathrm{D}\mathrm{r}\mathrm{o}\mathrm{p}\mathrm{o}\mathrm{u}\mathrm{t}(0.4)\to \mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(64\to 48)\to \mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}.$$

  • A learnable modality weighting vector $\mathbf{p}\in {\mathbb{R}}^2$, clamped and normalized via softmax, yields weights $w_0,w_1$, resulting in: $$\mathbf{f}=[w_0{\mathbf{e}}_{\mathrm{C}\mathrm{L}\mathrm{S}};w_1{\mathbf{e}}_{\mathrm{N}\mathrm{u}\mathrm{m}}]\in {\mathbb{R}}^{816}.$$

• 5.

  Classification layer

  • The combined feature vector $\mathbf{f}$ is passed through a feedforward neural network.

  • Dropout with probability 0.33 is used.

  • Fully connected layer, batch normalization (BN), and ReLU activation: $${\mathbf{h}}_1=\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}(\mathrm{B}\mathrm{N}({\mathbf{W}}_1\mathbf{f}+{\mathbf{b}}_1)),{\mathbf{W}}_1\in {\mathbb{R}}^{256\times 816}.$$

  • The logits for binary classification are computed as: $$\mathbf{z}={\mathbf{W}}_2{\mathbf{h}}_1+{\mathbf{b}}_2,{\mathbf{W}}_2\in {\mathbb{R}}^{2\times 256}.$$

  • The softmax function is applied to obtain the predicted probabilities: $$\hat{\mathbf{y}}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathbf{z})\in {\mathbb{R}}^2.$$where $\hat{\mathbf{y}}\in {\mathbb{R}}^2$ represents the probabilities for the two classes.

• 6.

  Loss function

• To effectively handle class imbalance during training, the MAPRA model employs a cost-sensitive variant of the binary cross-entropy loss known as the focal loss. The standard binary cross-entropy (CE) loss for binary classification, with true label $y\in \{0,1\}$ and predicted probability $\hat{p}$, is given by: $${\mathcal{L}}_{\mathrm{C}\mathrm{E}}=-[y\log (\hat{p})+(1-y)\log (1-\hat{p})].$$

• The focal loss extends this by emphasizing difficult-to-classify examples, and is defined as: $${\mathcal{L}}_{\mathrm{F}\mathrm{L}}=\alpha (1-p_t{)}^{\gamma }{\mathcal{L}}_{\mathrm{C}\mathrm{E}},$$

• where $p_t$ is the model’s predicted probability for the true class, $\gamma$ is the focusing parameter (set to $2$), and $\alpha$ balances the class weights (set to $1$). To further account for severe class imbalance, class-specific weights are employed as follows: $$w_{+}=\sqrt{\frac{1-{\pi }_{+}}{{\pi }_{+}}},w_{-}=1,{\pi }_{+}=\mathrm{0.02.}$$

• Thus, the final weighted focal loss for the dataset of size N is expressed as: $$\mathcal{L}=-\frac{1}{N}{\sum }_{i=1}^N{w}_{y_i}[\alpha (1-p_{t,i}{)}^{\gamma }(y_i\log ({\hat{p}}_i)+(1-y_i)\log (1-{\hat{p}}_i))].$$

• This combined loss function enhances the model’s sensitivity toward the minority class, significantly improving recall performance on rare litigated patents.

• 7.

  Training objective

  • The model parameters $\mathrm{\Theta }$ are optimized by minimizing the focal loss ( ${\mathcal{L}}_{FL}$) across the training dataset using the AdamW optimizer (learning rate $1.2\times {10}^{-5}$, weight decay $0.01$), cosine scheduler with a 5% warmup phase, gradient clipping (maximum norm of $1.0$), and early stopping based on validation area under the precision-recall curve (AUPRC): $$\underset{\mathrm{\Theta }}{min}\frac{1}{N}\sum _{i=1}^N{\mathcal{L}}_{FL}(y_i,{\hat{\mathbf{y}}}_i),$$where N denotes the total number of training samples, $y_i$ is the true label, and ${\hat{\mathbf{y}}}_i$ is the predicted probability vector for the $i$-th training example.

• 8.

  Evaluation under true class imbalance

  Given the true positive prevalence $${\pi }_{+}=\frac{N_{+}}{N}=0.02,$$where $N_{+}$ is the number of litigated patents in a test set of size N, the classifier produces scores ${\hat{p}}_i=P(y_i=1\mid x_i)$. Instead of relying solely on the default threshold $0.5$, the decision threshold is calibrated using multiple criteria on a 2%-positive validation set:

  • F1-optimal threshold: $$t_{F1}=\arg \underset{t}{max}{F}_1(t),F_1(t)=\frac{2P(t)R(t)}{P(t)+R(t)},$$where precision (P) and recall (R) at threshold $t$ are defined as: $$P(t)={\displaystyle \frac{{\sum }_{i=1}^N\mathbb{I}[{\hat{p}}_i\ge t]y_i}{{\sum }_{i=1}^N\mathbb{I}[{\hat{p}}_i\ge t]},R(t)=\frac{{\displaystyle {\sum }_{i=1}^N\mathbb{I}[{\hat{p}}_i\ge t]y_i}}{{\sum }_{i=1}^N{y}_i}.}$$

  • Accuracy-optimal threshold: $$t_{acc}=\arg \underset{t}{max}Acc(t).$$

  • Fixed 2% flag-rate threshold ( ${98}^{th}$ percentile): $$t_{2\mathrm{\%}}\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}\mathbb{P}[{\hat{p}}_i\ge t_{2\mathrm{\%}}]=\mathrm{0.02.}$$Ranking quality under extreme class imbalance is further evaluated using Precision@K and Recall@K: $$P@K=\frac{1}{K}{\sum }_{i=1}^K{y}_{(i)},R@K=\frac{1}{{\sum }_i{y}_i}{\sum }_{i=1}^K{y}_{(i)},$$where $y_{(i)}$ denotes the ground-truth label of the $i$-th highest-scoring example. Additionally, the Area Under the Precision–Recall Curve (AUPRC), $$AUPRC={\int }_0^{1}P(R)dR,$$is monitored and optimized, given its superior informativeness over area under the receiver operating characteristic curve (ROC-AUC) in highly imbalanced contexts ( ${\pi }_{+}\ll 0.5$).

    The model training explicitly uses a class-weighted focal loss: $${\mathcal{L}}_{FL}=-\frac{1}{N}{\sum }_{i=1}^N{w}_{y_i}\alpha (1-p_{t,i}{)}^{\gamma }[y_i\log {\hat{p}}_i+(1-y_i)\log (1-{\hat{p}}_i)],$$with class weights $$w_{+}=\sqrt{\frac{1-{\pi }_{+}}{{\pi }_{+}}}\approx 7,w_{-}=1,\alpha =1,\gamma =2.$$These class weights imply that false negatives incur significantly higher penalties due to the extreme class imbalance ( ${\pi }_{+}=0.02$), effectively enhancing recall for rare litigation-positive cases. Combined with offline positive-class oversampling (10%), comprehensive numeric preprocessing (imputation, trimming, scaling), multi-threshold calibration, and rigorous ranking metrics, this approach effectively maximizes recall for rare litigation events while controlling false-positive predictions.

• 9.

  Model inference

  • During inference, the class label is predicted by selecting the class with the highest probability: $$\hat{y}=\arg \underset{c}{max}{\hat{\mathbf{y}}}_c.$$Optionally, calibrated thresholds (e.g., F1-optimal, accuracy-optimal, or fixed flag-rate thresholds) may be applied to enhance inference quality under class imbalance.

Summary of the workflow

• 1.

  Offline tokenize and pad claim texts (length 512) using the BERT tokenizer.

• 2.

  Extract the [CLS] token embedding from fine-tuned BERT as the textual representation.

• 3.

  Preprocess numeric features (imputation, trimming, scaling) and encode them using a dedicated numeric MLP.

• 4.

  Combine textual and numeric embeddings using learnable modality clamped-weighting and concatenation.

• 5.

  Pass the weighted combined embedding through a feedforward neural network with dropout and batch normalization.

• 6.

  Compute class probabilities using softmax and minimize the focal loss during training.

• 7.

  Calibrate optimal decision thresholds on a validation set.

• 8.

  Predict class labels based on calibrated probabilities during inference.

Training and evaluation

Training involves minimizing the focal loss using the AdamW optimizer, gradient clipping, dropout regularization, and early stopping based on validation performance (AUPRC). Final evaluation on balanced and realistic imbalanced splits involves comprehensive metric computation and visualization of receiver operating characteristic curves, precision-recall curves, and confusion matrices. This architecture, which combines NLP techniques with numeric feature integration, provides a robust framework for predicting litigation risk in legal analytics.