The Patent Trials and Appeals Committee (PTAB) recently issued a rehearsal of the decision Application number 16/319,040 (Deepmind Technologies Ltd.; The Application) and all claims remained rejected under 35 USC §101 and §103. The PTAB denied the arguments presented in the rehearsal request and reaffirmed the new foundations of the §101 rejection brought during the initial decision on appeal.
This decision will help explain the board's approach to assessing technical improvements to artificial intelligence/machine learning inventions, taking into account testing, particularly for evaluating the eligibility of patent subject matter.
introduction
This decision is particularly relevant to technologies including artificial intelligence and machine learning, with a focus on continuous learning, transfer learning, and multitasking learning.
The applications originally submitted on January 18, 2019 are directed towards the “system.” [implemented in software] This trains a single machine learning model on multiple tasks without forgetting the previous tasks” (Specification, Para 6).
Its core includes a three-stage training routine.
- Estimating importance: For each parameter θᵢ, calculate the important weight ωᵢ after task A. For example, “by approximating the posterior distribution” (Claim 1/spec at Paraz 49–50).
- Sequential training: Get task B data and update the same set of parameters (claim 1).
- Constrained optimizations: To protect Task A performance, adjust the parameters using objective functions in penalty terms weighted with ω (claim 1).
fig. Reproduce application 1 below.
The claim was revised during the prosecution and more specifically recited training method procedures. For example, the revised Independent Claim 1 (an amendment of September 9, 2022) has added * emphasis.
“allocation, Use an approximationthe value for each of multiple parameters, the value is a measure of importance… and Approximate the probability The first value is… the correct value…”
The appellant demands and argues that there are certain technical improvements in the field of machine learning that integrates alleged abstract ideas into practical applications, and claims that rehearsals (emphasis from the rehearsal decision):
“The claimed invention provides specific technical improvements in the field of machine learning by allowing a single model to be trained sequentially on multiple tasks, while maintaining acceptable performance for each task.. This is achieved without the need to store or maintain separate models for each task, significantly reducing system complexity and storage requirements. Instead of requesting multiple sets of parameters per task, the system maintains a single parameter that is adjusted during training for a new task, using an objective function that incorporates penalty terms that reflect the importance of the parameters for previously learned tasks. This training strategy allows the model to maintain performance of previous tasks, even as it is learning new tasks, and directly address the technical issues of “catastrophic forgetting” in the ongoing learning system. ”
Rehearsal board survey results
The rehearsal request was primarily challenged (i) the interpretation of the board's “parameters”, (ii) the reliance on the Mehanna reference for the requested functions, and (iii) the analyses of the Alice/Mayo's eligibility. The panel (Bui, Khan, Curcuri) addressed each debate and found no basis for changing the previous decision.
35 Abnormality below USC 103
The board said the applicant “did not identify any issues that we misunderstood or overlooked.” The board repeated that Gordon teaches probabilistic “important” elements, as Gordon explains the calculation of the posterior distribution. P (w | d, h) Above parameters (Reh'g, p. Decision of 5, cited original decision on page 15). After this decision, the board found that the discussion about Mehanna was no longer related to trivial analysis. Judge Horvath's previous consent to distinguish between “functions” and “parameters” did not change the outcome.
35 Eligibility of Patent Subjects under USC 101
The USPTO's 2019 revised guidance states that the analysis is described in several steps.
- Step 1: Determines whether the claim is for a process, machine, manufacturing, or material composition (this step is rarely considered in detail)
- Step 2A: Determine if the claim is directed towards a judicial exception
- Prong One: Assess whether a claim recites a “judicial exception” (i.e., an abstract idea, a law of nature, or a natural phenomenon).
- Prong 2: If a judicial exception is being recited, the entire claim determines whether to integrate the exception into a practical application. This includes considering whether a claim applies, relies on, or uses judicial exceptions in a way that imposes meaningful restrictions on the exception.
- Step 2b: Assess whether a claim is added significantly more than a “judicial exception”
The board's framework tracked procedures based on the USPTO's 2019 revised guidance, and Step 2A considered the details of the rehearsal.
Step 2A – Prong 1 (Judicial Exception)
The board again characterized the claim “Mathematical concepts/calculations for training machine learning models” I have decided to configure this Abstract ideas (Decision of Reh'g, p. 6).
Step 2A – Prong 2 (Integration into Practical Applications)
The rehearsal request argued that the methodology “reduces storage requirements” and “addresses catastrophic forgetting.”
However, the panel did not find “technical improvements” because the specifications “do not include evidence of a particular means or method of solving problems in existing technical processes” (Id., p. 7).
In its original decision on appeal, the Board discovered that general computing elements (“one or more computers” and “one or more storage devices”) did not add much abstract ideas, and the committee concluded that the abstract ideas were not integrated into real applications.
In a rehearsal, the panel cited the previous Federal Circuit decision in the requirement for the conclusion that machine learning models will be either “repeatly trained” or dynamically adjusted in machine learning training. Therefore, it does not represent an improvement in technology, but cites that abstract ideas are not abstracted by using a technological environment using a particular field.
Procedure points
Because the §101 rejection was entered as a new frontier in the original decision, a request for rehearsal was required to “state along with singularity” points that were considered misunderstood or overlooked in §41.52(b)(2) of 37CFR. The Board held that although the appellant's request did not identify any particular oversight, it repeated the previously considered arguments.
Conclusion Comments
This decision can be related to innovations related to systems that aim to train models on multiple tasks, such as robotics, autonomous vehicles, natural language processing, and recommendation engines. The decision highlights the challenges of patenting innovations that address issues such as catastrophic forgetting. The model loses performance of previously learned tasks when trained on a new task.
While this application clearly describes technical problems and their corresponding technical solutions, the Board downplayed the technology of the present invention and made simplified analogies related to traditional machine learning training.
Developers of AI frameworks, edge computing solutions, and hardware accelerators for machine learning should note that improvements limited to algorithmic procedures and general computing environments may not meet patent eligibility thresholds.
Instead, patented applications in these fields should emphasize specific technical solutions that demonstrate concrete improvements to computer functions or systems architectures beyond abstract mathematical concepts. During drafts, technical improvements should be fixed in specifications and include more technical and potentially narrow fallback positions if there are difficult inspections.
