English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neuro-symbolic AI bridges the gap between purely symbolic and neuralapproaches to learning. This often requires maximizing the likelihood of asymbolic constraint w.r.t the neural network's output distribution. Such outputdistributions are typically assumed to be fully-factorized. This limits theapplicability of neuro-symbolic learning to the more expressive autoregressivedistributions, e.g., transformers. Under such distributions, computing thelikelihood of even simple constraints is #P-hard. Instead of attempting toenforce the constraint on the entire output distribution, we propose to do soon a random, local approximation thereof. More precisely, we optimize thelikelihood of the constraint under a pseudolikelihood-based approximationcentered around a model sample. Our approximation is factorized, allowing thereuse of solutions to sub-problems, a main tenet for efficiently computingneuro-symbolic losses. Moreover, it is a local, high-fidelity approximation ofthe likelihood, exhibiting low entropy and KL-divergence around the modelsample. We evaluate our approach on Sudoku and shortest-path prediction cast asautoregressive generation, and observe that we greatly improve upon the basemodel's ability to predict logically-consistent outputs. We also evaluate onthe task of detoxifying large language models. Using a simple constraintdisallowing a list of toxic words, we are able to steer the model's outputsaway from toxic generations, achieving SoTA detoxification compared to previousapproaches.

A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints