English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Data processing is one of the fundamental steps in machine learning pipelinesto ensure data quality. Majority of the applications consider the user-definedfunction (UDF) design pattern for data processing in databases. Although theUDF design pattern introduces flexibility, reusability and scalability, theincreasing demand on machine learning pipelines brings three new challenges tothis design pattern -- not low-code, not dependency-free and notknowledge-aware. To address these challenges, we propose a new design patternthat large language models (LLMs) could work as a generic data operator(LLM-GDO) for reliable data cleansing, transformation and modeling with theirhuman-compatible performance. In the LLM-GDO design pattern, user-definedprompts (UDPs) are used to represent the data processing logic rather thanimplementations with a specific programming language. LLMs can be centrallymaintained so users don't have to manage the dependencies at the run-time.Fine-tuning LLMs with domain-specific data could enhance the performance on thedomain-specific tasks which makes data processing knowledge-aware. Weillustrate these advantages with examples in different data processing tasks.Furthermore, we summarize the challenges and opportunities introduced by LLMsto provide a complete view of this design pattern for more discussions.

LLMs with User-defined Prompts as Generic Data Operators for Reliable Data Processing